What is the role of human decision-making regarding health care in an AI–driven future?

Australian researchers have established a set of protocols for a research project in JMIR Research Protocols that aims to explore whether humans will continue in meaningful decision-making roles in an AI-driven future. The researchers, Dr. Nandini Doreswamy and Dr. Louise Horstmanshof from Southern Cross University, found that there is a dearth of guidance in establishing whole sets of protocols for methods used in complex health research.

They have now established protocols that focus on non-clinical health services such as health regulation and health policy. These health services can be complex, the researchers say, and even more extensive in their scope and scale than clinical health care.

During the pandemic, these services formed a continuum of rules, laws, and public health measures that varied from country to country. An array of organizations, including private organizations, commercial concerns, and different levels of government may be involved in the oversight and control of these services.

"Right now, humans are the leaders, managers, and decision makers in these complex health services," Dr. Doreswamy said, "but AI is transforming this space. The time has come to ask whether humans will continue to have meaningful decision-making roles in this domain."

AI-driven decision-making may not be as accountable, unbiased, or transparent as required, the researchers say, but nevertheless, it is starting to dominate these complex domains. It may be prone to incorrect or unfair decisions, which is, perhaps, why AI is not currently allowed to make the final decisions in complex health services.

"Furthermore, rationality has long dominated this space," Dr. Horstmanshof said. "What role does intuition play?"

The researchers have established protocols for a research project that aims to explore these questions. The project will consist of four steps. The first is a review that will identify and map human attributes that influence decision-making in complex health services. The second is also a review, one that will identify and map AI attributes that influence decision-making in this context.

The third step is a comparative analysis of the two sets of attributes—human and AI—to find out whether humans have attributes that could influence decision-making for the better. The fourth and final step is a simulation study, where a virtual human and a virtual AI decision maker will be introduced into a simulated health policy environment.

"We want to explore, observe, and document how humans interact with AI, at least in a simulated environment," Dr. Doreswamy said. "It may show us whether humans are likely to compete, cooperate, or converge with AI, and thus remain relevant in the decision-making of the future, in complex health care."

Dr. Horstmanshof noted that the protocols not only provide a roadmap for the proposed research, but also provide an example of a 'protocol of protocols' for methods used in health research. This may be relevant and useful in other spheres of complex research, she said, such as human-AI interaction and health informatics. It may also offer an opportunity to further investigate the issue of bias, the dominance of rationality, and the likely influence of intuition.