Seeing eye to eye: Researchers train AI to copy gaze of clinical professionals

An artificial intelligence (AI) system, which mimics the gaze of radiologists reading medical images such as mammograms, has been developed by a team of scientists at Cardiff University.

The first-of-its-kind system improves the speed, accuracy and sensitivity of medical diagnostics and could lead to early detection of breast cancer, the researchers claim.

The team, from the University's School of Computer Science and Informatics, also hopes it will help address a U.K.-wide shortage of radiologists through training and education applications.

To build the system, they used a sophisticated algorithm known as a convolutional neural network which is designed to mimic neurons in the human brain and is modeled specifically on the visual cortex.

This type of algorithm is ideal for taking images and assigning importance to various objects or aspects within the image itself.

The research team developed the algorithm with radiologists from three NHS hospitals—Breast Test Wales, University Hospital of Wales (UHW) and Great Ormond Street Hospital.

Their findings, published in IEEE Transactions on Multimedia, show that the AI system can accurately predict the areas of an image where radiologists are most likely to look when preparing a diagnosis.

Dr. Hantao Liu, a Reader at Cardiff University's School of Computer Science and Informatics and one of the study's co-authors, said, "With all of the challenges facing the NHS, it is important that we look to data science and AI for possible solutions.

"This doesn't mean replacing people with robots but instead demonstrates how machine learning can support and augment the work of clinical professionals. In our study we've done just that, by developing a system which can work with radiologists to act like a critical friend or colleague during their medical diagnostic work and support decision-making arising from it," says Dr. Hantao Liu.

Dr. Liu, who has been awarded an honorary research position with NHS Wales, aims to develop a strategic roadmap for AI deployment across radiology departments in Wales.

Dr. Richard White, a consultant radiologist at UHW who participated in the study, said, "There's so much data involved in radiology that I think it's best we make use of it and the expertise available. All radiologists miss things. Often, we don't notice it for years because it can be very subtle—it may not even appear visible at all in some cases. But we know AI can spot things that we don't so, in that sense, this kind of system is going to enhance our work, reducing the risk of errors.

"We're also seeing how the system will help us prioritize patient referrals by finding abnormal scans and images so that these can be reported before ones that need less attention. I can see more of that happening in the future with this system, helping us to tackle what are often long waiting lists."

While their study focuses on gaze-prediction, the researchers from the Multimedia Computing Research Group at Cardiff University say systems which aid decision-making might be the next step in clinical applications of AI.

Zelei Yang, another radiologist at UHW who also participated in the study, added, "Effectively this system is bringing with it a greater humanization of AI and therefore a more realistic representation of what we do as radiologists. For these reasons it is capable of revolutionizing how we approach deep learning and its clinical applications because it is now more effective and more realistic in this setting rather than just a tool that has been developed for the sake of it."