AI is already being used in health care, but not all of it is 'medical grade'

Artificial intelligence (AI) seems to be everywhere these days, and health care is no exception.

There are computer vision tools that can detect suspicious skin lesions as well as a specialist dermatologist can. Other tools can predict coronary artery disease from scans. There are also data-driven robots that guide minimally-invasive surgery.

To precisely diagnose diseases and guide treatment choices, AI is used to analyze patients' genomic and molecular data. For instance, machine learning has been applied to detect Alzheimer's disease and to help choose the best antidepressant medication for patients with major depression.

Deep learning methods have been used to model electronic health record data to predict health outcomes for patients and provide early estimates of treatment cost.

With new language-based generative AI technologies like ChatGPT, the clinical world is abuzz with talk of chatbots for answering patient questions, helping doctors take better notes, and even explaining a diagnosis to a concerned grandchild.

There is no doubt that in terms of patient health, workflows and system efficiency, AI will benefit the health system.

But there are legitimate concerns about the accuracy of such tools, including how well they work in new settings (such as a different country or even a different hospital from where they were created), and whether they "hallucinate"—or make things up.

Developing 'medical grade' tools

In our recent article in the Medical Journal of Australia, we argue using AI effectively in health care will require retraining of the workforce, retooling health services, and transforming workflows.

Critically, we also need to collect evidence AI tools are "medical grade" before we use them on patients.

Many claims made by the developers of medical AI may lack appropriate scientific rigor and evaluations of AI tools may suffer from a high risk of bias. This means the tests run to ensure their accuracy are too narrow.

AI tools can make errors, or stop working when the application context changes. Conversational agents such as chatbots may produce misleading medical information that may delay patients seeking care. They may also make inappropriate recommendations.

All this means we need standards for the AI tools that impact diagnosis and treatment of patients. Clinicians should be given training on how to critically assess AI applications to understand their readiness for routine care.

We should expect to be able to replicate the results from one context to another, under real-world conditions. For example, a tool developed using historical data from a hospital in New York should be carefully trialed with live patient data in Broome before we trust it.

Randomized controlled trials of AI tools, where these differences are controlled for, would represent a gold standard of evidence for their use.

We can't just copy what other countries do

It is important to carefully examine how AI tools are embedded into workflows to support clinical decisions. The benefits and risks of a tool will depend on precisely how the human clinician and the tool work together.

There's a view that all we need to do in Australia is adopt the best of what is produced internationally, and that we don't need deep sovereign capabilities.

Perhaps we can rely on the regulation of AI tools under way through the European Union's AI Act, or the United States Food and Drug Administration's processes for assessing Software as a Medical Device.

Nothing is further from the truth.

AI requires local customization to support local practices, and to reflect diverse populations or health service differences. We don't want to just export our clinical datasets and import back the models built with them without adapting to our contexts and workflows. We need to monitor the clinical deployments of AI tools into our settings.

Without some degree of algorithmic sovereignty—the capability to produce or modify AI in Australia—the nation is exposed to new risks and the benefits of the technology will be limited.

A roadmap for AI in Australian health care

The Australian Alliance for Artificial Intelligence in health care has produced a roadmap for future development.

It identifies gaps in Australia's capability to translate AI into effective and safe clinical services and provides guidance on key issues such as workforce, industry capability, implementation, regulation, and cybersecurity.

These recommendations offer a path toward an AI-enabled Australian health care system capable of delivering personalized and patient-focused health care, safely and ethically.

The plan also envisages a vibrant AI industry sector that creates jobs and exports to the world, working side by side with an AI-aware workforce and AI-savvy consumers.

AI has the potential to transform medicine. It can do so by harnessing computational power to discern subtle patterns in complex data spanning biology, images, sensory and experiential data, and more.

With care and strategic investment, innovations in AI will surely benefit clinicians and patients alike. Now is the time to act to ensure Australia is well-placed to benefit from one of the most significant industrial revolutions of our time.

This article is republished from The Conversation under a Creative Commons license. Read the original article.