AI for Researchers

It can be used for various tasks in the different phases of the research process ranging from formulating a research question, conducting a literature review, generating ideas for a research project, and data collection and analysis. Use of AI can be more or less risky in each of these tasks. The general rule of thumb is: the more substantial the use of the AI system, the more verification, control and accountability is needed to ensure it is used appropriately.

Below we explain some examples, and what we expect from researchers [1].

Brainstorming: eg help in generating new research ideas

The model generates new research ideas, which the researcher then incorporates into a self-written paper. We expect researchers to thoroughly check that the ideas generated come from existing sources, and add appropriate references if necessary. There is a good chance that the ideas will be generated based on existing work. If that is not quoted correctly, it is plagiarism. If the ideas generated by GenAI turn out to be innovative, then we expect researchers to mention the use of the generative language model as well.

Use as a search engine to learn more about a certain domain: eg help with conducting a literature study

Generative AI systems can be used to gather information about a particular domain or research topic, similar to the use of a regular search engine. We urge caution in such use. There are drawbacks to using generative language models as a search engine [2]. In most cases, it is impossible to trace the source of the information. In addition, there is still the real risk of inaccuracies and fictitious references. In any case, we expect researchers to thoroughly check the generated information for accuracy, and to thoroughly check and consult references. If researchers then write a text themselves, they do not have to mention the use of GenAI, but of course they do have to refer to the primary sources.

Text generation for existing knowledge: eg help with writing a literature review in a publication or project application

GenAI can be used to generate descriptions of existing, commonly known concepts. We expect researchers to check the generated texts for accuracy, and provide them with appropriate, accurate references. In addition, we expect researchers to be transparent about the use of the generated text in their own documents, and to mention the use of the generative language model (including reference).

Text generation with new research ideas: eg help with writing a project plan in a research proposal

The system is asked to generate text that concerns new research ideas. We expect researchers to thoroughly evaluate the generated text before incorporating it into their own documents. In any case, researchers remain responsible for the accuracy and coherence of the text, as well as possible hidden plagiarism. Transparent mention of the use of the model (including reference) is also necessary here.

Generation of programming code

A specific use of generative language models is the generation of programming code. The general warnings also apply here, in particular the risk of inaccuracies and hidden plagiarism. For example, it is possible that the generated code is executed without error messages, but in reality does not (fully) complete the intended task. It is therefore still necessary to thoroughly check the code for errors. In addition, it is possible that the model is based on pieces of code with a software license that does not correspond to your use. In any case, we expect transparency: the use of generative models for the generation of programming code should be explicitly mentioned (including reference).

Synthetic data generation

Another specific use of GenAI models is the generation of synthetic data that mimics existing datasets. In this way, limitations such as privacy issues and confidentiality of real datasets can be overcome. It is important to carefully evaluate the generated synthetic data for quality and possible bias, as the output is highly dependent on the quality of the data on which the models are trained. In any case, we expect transparency: the use of generative models for the generation of synthetic data should be explicitly mentioned (including reference).

Use in data analysis

We expect researchers to make explicit reference to the analytical methods used.

Visualisation of research results

 In the future, more and more multimodal models, which combine text and visual material, will see the light of day. Among other things, this offers opportunities to automatically visualise research results. We expect researchers to thoroughly check such visualisations for inaccuracies, and explicitly mention the use of the generative model (including reference).

Use as a language assistant: eg help with rewriting or improving your own text, such as a manuscript or a project application

A generative language model can be used to rewrite and improve (and translate) a text that you have written yourself. The model does not add any new content. When the model is only used to improve language use, it is used in the same way as the spelling and grammar checkers we already know today. If the model is only used as a language aid, this does not have to be disclosed.

Peer review of project applications and manuscripts

As a reviewer, we strongly advise against entering a project request or manuscript into ChatGPT or similar 'non-privacy-friendly' platforms, for example to get a summary and then assess it, as important aspects in the application could be missed, which could make the review inaccurate. In addition, such documents are confidential [3].

Keep track of AI's raw output

It's recommended to keep track of AI's raw output when you're using the model to generate new ideas or new text (and not when you're using it as a text-enhancing tool). You have a duty to keep an audit trail of how you came to something.

Use generative AI as a help and support and be critical of the output

Enter a human verification step. Fact-check the generated information, check that the generated output cannot be found online (for example, by a random sample via a search engine) and add corrections or the correct citation where necessary. Be aware that output is likely to follow certain historical biases about the topic and deal with them critically. As a researcher, you remain ultimately responsible for what you publish [4].

Think about what data you enter

Don't enter personal data [5] or confidential information (for example, in the context of review assignments, the development of original research ideas, or from a valorisation point of view) on the platforms that are not managed by the University of Edinburgh. If you introduce intellectual property that has not yet been protected (such as a new method, the description of a unique material, or another invention), there is a good chance that you will no longer be able to protect it. Do not disclose information about which a non-disclosure agreement has been signed, for example, in the context of contract research or a Masters thesis in collaboration with a company. The information entered is often kept by the owner of the AI tool, and it is unclear what happens to this information. Make sure you have the necessary permission or licence to enter copyrighted material into the AI application. If you are unsure about the confidential nature of the information, you can ask the provider of the information.

Take into account the vision of the journal/publisher/funder

Take into account the vision of the journal/publisher/funder regarding the use of GenAI. Although there is a certain consensus that the technology can be used to support the writing of scientific texts, there are also journals where the use of the technology is not (currently) allowed.

It goes without saying that generative AI should not be used to fabricate research data [6] [7] [8], or evade plagiarism detection through the use of paraphrasing tools. It is true that the development of detection tools is lagging behind, but just because the use is not easy to detect at the moment does not mean that it cannot be detected later.

The legislative framework for the use of the tools is still under development.

References

[1] Inspired by Association for Computational Linguistics ACL 2023, 10 January 2023. ACL'23 Policy on AI Writing Assistance. https://2023.aclweb.org/blog/ACL-2023-policy/

[2] Chirag Shah and Emily M. Bender. 2022. Situating Search. In Proceedings of the 2022 Conference on Human Information Interaction and Retrieval (CHIIR '22). Association for Computing Machinery, New York, NY, USA, 221–232. https://doi.org/10.1145/3498366.3505816

[3] Association for Computational Linguistics, 9 February 2023. ACL'23 Peer Review Policies Policy on AI Writing Assistance.

[4] University of Kent, 15 March 2023. Using artificial intelligence in your studies. https://www.kent.ac.uk/guides/using-artificial-intelligence-in-your-studies

[5] Personal data is a broad concept and includes any information relating to an identified or identifiable natural person. Identifiable means that you can still find out the identity of a natural person by means of the combination of different elements.

[6] Chang Qi, Jian Zhang, Peng Luo. Emerging Concern of Scientific Fraud: Deep Learning and Image Manipulation. bioRxiv 2020. https://doi.org/10.1101/2020.11.24.395319

[7] Liansheng Wang, Lianyu Zhou, Wenxian Yang, Rongshan Yu. Deepfakes: A new threat to image fabrication in scientific publications? Patterns Volume 3, Issue 5, 13 May 2022, 100509. https://doi.org/10.1016/j.patter.2022.100509

[8] Luka Posilović, Duje Medak, Marko Subašić, Marko Budimir, Sven Lončarić. Generating ultrasonic images indistinguishable from real images using Generative Adversarial Networks. Ultrasonics Volume 119, February 2022, 106610. https://doi.org/10.1016/j.ultras.2021.106610

AI Ethics for Researchers

The Commission, together with the European Research Area countries and stakeholders, has put forward a set of guidelines to support the European research community in their responsible use of generative artificial intelligence (AI). Building on the principles of research integrity, they offer guidance to researchers, research organisations and research funders to ensure a coherent approach across Europe.

• Download the guidelines
• Download the factsheet summarising the guidelines

The principles framing the new guidelines are based on existing frameworks such as the European Code of Conduct for Research Integrity and the guidelines on trustworthy AI.

• The European Code of Conduct for Research Integrity
• Ethics guidelines for trustworthy AI

Key takeaways from the guidelines include:

• Researchers refrain from using generative AI tools in sensitive activities such as peer reviews or evaluations and use generative AI respecting privacy, confidentiality, and intellectual property rights.
• Research organisations should facilitate the responsible use of generative AI and actively monitor how these tools are developed and used within their organisations.
• Funding organisations should support applicants in using generative AI transparently.