This article has been reviewed according to Science X's editorial process and policies. Editors have highlighted the following attributes while ensuring the content's credibility:

fact-checked

peer-reviewed publication

proofread

ChatGPT generates 'convincing' fake scientific article

AI unleashes a Pandora's box: ChatGPT generates convincingly fake scientific article
AI-generated image, in response to the request "pandoras box opened with a physician standing next to it. Oil painting Henry Matisse style", (Generator: DALL-E2/OpenAI, March 9, 2023, Requestor: Martin Májovský). Credit: Created with DALL-E2, an AI system by OpenAI

A new study published in the Journal of Medical Internet Research by Dr. Martin Májovský and colleagues has revealed that artificial intelligence (AI) language models such as ChatGPT (Chat Generative Pre-trained Transformer) can generate fraudulent scientific articles that appear remarkably authentic. This discovery raises critical concerns about the integrity of scientific research and the trustworthiness of published papers.

Researchers from Charles University, Czech Republic, aimed to investigate the capabilities of current AI language models in creating high-quality fraudulent medical articles. The team used the popular AI chatbot ChatGPT, which runs on the GPT-3 language model developed by OpenAI, to generate a completely fabricated scientific article in the field of neurosurgery. Questions and prompts were refined as ChatGPT generated responses, allowing the quality of the output to be iteratively improved.

The results of this proof-of-concept study were striking—the AI language model successfully produced a fraudulent article that closely resembled a genuine scientific paper in terms of word usage, sentence structure, and overall composition. The article included standard sections such as an abstract, introduction, methods, results, and discussion, as well as tables and other data. Surprisingly, the entire process of article creation took just one hour without any special training of the human user.

While the AI-generated article appeared sophisticated and flawless, upon closer examination expert readers were able to identify semantic inaccuracies and errors particularly in the references—some references were incorrect, while others were non-existent. This underscores the need for increased vigilance and enhanced detection methods to combat the potential misuse of AI in scientific research.

This study's findings emphasize the importance of developing ethical guidelines and for the use of AI language models in genuine scientific writing and research. Models like ChatGPT have the potential to enhance the efficiency and accuracy of document creation, result analysis, and language editing. By using these tools with care and responsibility, researchers can harness their power while minimizing the risk of misuse or abuse.

In a commentary on Dr. Májovský's article, Dr. Pedro Ballester discusses the need to prioritize the reproducibility and visibility of scientific works, as they serve as essential safeguards against the flourishing of fraudulent research.

As AI continues to advance, it becomes crucial for the scientific community to verify the accuracy and authenticity of content generated by these tools and to implement mechanisms for detecting and preventing fraud and misconduct. While both articles agree that there needs to be a better way to verify the accuracy and authenticity of AI-generated content, how this could be achieved is less clear.

"We should at least declare the extent to which AI has assisted the writing and analysis of a paper," suggests Dr. Ballester as a starting point. Another possible solution proposed by Majovsky and colleagues is making the submission of data sets mandatory.

The article "Artificial Intelligence Can Generate Fraudulent but Authentic-Looking Scientific Medical Articles: Pandora's Box Has Been Opened" was published in the Journal of Medical Internet Research.

More information: Martin Májovský et al, Artificial Intelligence Can Generate Fraudulent but Authentic-Looking Scientific Medical Articles: Pandora's Box Has Been Opened, Journal of Medical Internet Research (2023). DOI: 10.2196/46924

Pedro L Ballester, Open Science and Software Assistance: Commentary on "Artificial Intelligence Can Generate Fraudulent but Authentic-Looking Scientific Medical Articles: Pandora's Box Has Been Opened", Journal of Medical Internet Research (2023). DOI: 10.2196/49323

Provided by JMIR Publications
Citation: ChatGPT generates 'convincing' fake scientific article (2023, July 5) retrieved 3 March 2024 from https://medicalxpress.com/news/2023-07-chatgpt-generates-convincing-fake-scientific.html
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.

Explore further

AI-generated academic science writing can be identified with over 99% accuracy

68 shares

Feedback to editors