Personalized cancer medicine: Humans make better treatment decisions than AI, says study

Treating cancer is becoming increasingly complex, but also offers more and more possibilities. After all, the better a tumor's biology and genetic features are understood, the more treatment approaches there are. To be able to offer patients personalized therapies tailored to their disease, laborious and time-consuming analysis and interpretation of various data is required.

Researchers at Charité—Universitätsmedizin Berlin and Humboldt-Universität zu Berlin have now studied whether generative artificial intelligence (AI) tools such as ChatGPT can help with this step. This is one of many projects at Charité analyzing the opportunities unlocked by AI in patient care.

The findings are published in the journal JAMA Network Open.

If the body can no longer repair certain genetic mutations itself, cells begin to grow unchecked, producing a tumor. The crucial factor in this phenomenon is an imbalance of growth-inducing and growth-inhibiting factors, which can result from changes in oncogenes—genes with the potential to cause cancer—for example. Precision oncology, a specialized field of personalized medicine, leverages this knowledge by using specific treatments such as low-molecular-weight inhibitors and antibodies to target and disable hyperactive oncogenes.

The first step in identifying which genetic mutations are potential targets for treatment is to analyze the genetic makeup of the tumor tissue. The molecular variants of the tumor DNA that are necessary for precision diagnosis and treatment are determined. Then the doctors use this information to craft individual treatment recommendations. In especially complex cases, this requires knowledge from various fields of medicine.

At Charité, this is when the "molecular tumor board" (MTB) meets: Experts from the fields of pathology, molecular pathology, oncology, human genetics, and bioinformatics work together to analyze which treatments seem most promising based on the latest studies. It is quite an involved process, ultimately culminating in a personalized treatment recommendation.

Can artificial intelligence help with treatment decisions?

Dr. Damian Rieke, a doctor at Charité, Prof. Ulf Leser and Xing David Wang of Humboldt-Universität zu Berlin, and Dr. Manuela Benary, a bioinformatics specialist at Charité, wondered whether artificial intelligence might be able to help at this juncture. In their recently published study, they worked with other researchers to examine the possibilities and limitations of large language models such as ChatGPT in automatically scanning scientific literature with an eye to selecting personalized treatments.

"We prompted the models to identify personalized treatment options for fictitious cancer patients and then compared the results with the recommendations made by experts," Rieke explains. His conclusion: "AI models were able to identify personalized treatment options in principle—but they weren't even close to the abilities of human experts."

The team created ten molecular tumor profiles of fictitious patients for the experiment. A human physician specialist and four large language models were then tasked with identifying a personalized treatment option. These results were presented to the members of the MTB for assessment, without them knowing where which recommendation came from.

Improved AI models hold promise for future uses

"There were some surprisingly good treatment options identified by AI in isolated cases," Benary reports. "But large language models perform much worse than human experts." Beyond that, data protection, privacy, and reproducibility pose particular challenges in relation to the use of artificial intelligence with real-world patients, she notes.

Still, Rieke is fundamentally optimistic about the potential uses of AI in medicine: "In the study, we also showed that the performance of AI models is continuing to improve as the models advance. This could mean that AI can provide more support for even complex diagnostic and treatment processes in the future—as long as humans are the ones to check the results generated by AI and have the final say about treatment."