Combining augmented reality with deep learning network to help pathologists spot cancer cells

April 17, 2018 by Bob Yirka, Medical Xpress report
Left: Schematic overview of the ARM. A digital camera captures the same field of view (FoV) as the user and passes the image to an attached compute unit capable of running real-time inference of a machine learning model. The results are fed back into a custom AR display which is inline with the ocular lens and projects the model output on the same plane as the slide. Right: A picture of our prototype which has been retrofitted into a typical clinical-grade light microscope. Credit: Google

A team of researchers at Google has developed a tool that combines augmented reality with a deep learning neural network to provide pathologists with help in spotting cancerous cells on slides under a microscope. Google has published a paper outlining the new tool, which they describe as an augmented reality microscope (ARM) platform, on their website. They also gave a presentation at this year's American Association for Cancer Research meeting, showing what it can do.

Performing a biopsy entails slicing tiny bits of tissue from a part of the body doctors suspect might be cancerous. That tissue is sliced thinly enough to view on a glass slide under a microscope by a highly trained pathologist looking for . The job of the pathologist, the Google team notes, is highly valued, but the work is also tedious, which can lead to fatigue and errors. To help the pathologist, the team at Google built on prior work at the company in which a team developed a network that was trained to spot breast cancer. The team then incorporated lessons learned from various projects surrounding to train their deep learning application to draw circles around suspected in a tissue sample. They interfaced their system with the type of microscope pathologist already use. The result is an add-on tool for the microscope. It works like this: Through the , the pathologist sees the cells of the tissue—moving the slide around allows them to see the whole sample. But then,if the ARM spots possible cells, it draws a ring around them, alerting the pathologist to take a closer look.

The ARM, Google reports, is now capable of assisting with detecting breast and prostate cancers, but they note that it could be trained to spot other cancers, as well. It could even be trained to look for some types of infectious diseases such as TB and malaria. They note also that it could also be programmed with other AR capabilities, such as posting labels over certain tissue sections, including arrows or other indicators pathologists might find useful.

Explore further: Smart software can diagnose prostate cancer as well as a pathologist

Related Stories

Smart software can diagnose prostate cancer as well as a pathologist

March 16, 2018
Chinese scientists and clinicians have developed a learning artificial intelligence system which can diagnose and identify cancerous prostate samples as accurately as any pathologist. This holds out the possibility of streamlining ...

Computer accurately identifies and delineates breast cancers on digital tissue slides

May 10, 2017
A deep-learning computer network developed through research led by Case Western Reserve University was 100 percent accurate in determining whether invasive forms of breast cancer were present in whole biopsy slides.

IU-invented microscope could make surgeries more effective, disease diagnoses timelier

March 22, 2018
In the not-so-distant future, surgeons could ensure the complete removal of malignant tumors, and pathologists could analyze tissue more efficiently, by using a device invented at the Indiana University School of Medicine.

Microscope can scan tumors during surgery and examine cancer biopsies in 3-D

June 26, 2017
When women undergo lumpectomies to remove breast cancer, doctors try to remove all the cancerous tissue while conserving as much of the healthy breast tissue as possible.

Will 'AI' be part of your health-care team?

December 12, 2017
(HealthDay)—Artificial intelligence is assuming a greater role in many walks of life, with research suggesting it may even help doctors diagnose disease.

Recommended for you

Combining three treatment strategies may significantly improve melanoma treatment

December 12, 2018
A study by a team led by a Massachusetts General Hospital (MGH) investigator finds evidence that combining three advanced treatment strategies for malignant melanoma—molecular targeted therapy, immune checkpoint blockade ...

Lethal combination: Drug cocktail turns off the juice to cancer cells

December 12, 2018
A widely used diabetes medication combined with an antihypertensive drug specifically inhibits tumor growth—this was discovered by researchers from the University of Basel's Biozentrum two years ago. In a follow-up study, ...

Researchers use computer model to predict prostate cancer progression

December 12, 2018
An international team of cancer researchers from Denmark and Germany have used cancer patient data to develop a computer model that can predict the progression of prostate cancer. The model is currently being implemented ...

An integrated approach to finding new treatments for breast cancer

December 12, 2018
Unraveling the complexity of cancer biology can lead to the identification new molecules involved in breast cancer and prompt new avenues for drug development. And proteogenomics, an integrated, multipronged approach, seems ...

New insight into stem cell behaviour highlights therapeutic target for cancer treatment

December 12, 2018
Research led by the University of Plymouth and Technische Universität Dresden has identified a new therapeutic target for cancer treatment and tissue regeneration – a protein called Prominin-1.

Pushing closer to a new cancer-fighting strategy

December 11, 2018
A molecular pathway that's frequently mutated in many different forms of cancer becomes active when cells push parts of their membranes outward into bulging protrusions, Johns Hopkins researchers report in a new study. The ...

0 comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.