Combining augmented reality with deep learning network to help pathologists spot cancer cells

April 17, 2018 by Bob Yirka, Medical Xpress report
Left: Schematic overview of the ARM. A digital camera captures the same field of view (FoV) as the user and passes the image to an attached compute unit capable of running real-time inference of a machine learning model. The results are fed back into a custom AR display which is inline with the ocular lens and projects the model output on the same plane as the slide. Right: A picture of our prototype which has been retrofitted into a typical clinical-grade light microscope. Credit: Google

A team of researchers at Google has developed a tool that combines augmented reality with a deep learning neural network to provide pathologists with help in spotting cancerous cells on slides under a microscope. Google has published a paper outlining the new tool, which they describe as an augmented reality microscope (ARM) platform, on their website. They also gave a presentation at this year's American Association for Cancer Research meeting, showing what it can do.

Performing a biopsy entails slicing tiny bits of tissue from a part of the body doctors suspect might be cancerous. That tissue is sliced thinly enough to view on a glass slide under a microscope by a highly trained pathologist looking for . The job of the pathologist, the Google team notes, is highly valued, but the work is also tedious, which can lead to fatigue and errors. To help the pathologist, the team at Google built on prior work at the company in which a team developed a network that was trained to spot breast cancer. The team then incorporated lessons learned from various projects surrounding to train their deep learning application to draw circles around suspected in a tissue sample. They interfaced their system with the type of microscope pathologist already use. The result is an add-on tool for the microscope. It works like this: Through the , the pathologist sees the cells of the tissue—moving the slide around allows them to see the whole sample. But then,if the ARM spots possible cells, it draws a ring around them, alerting the pathologist to take a closer look.

The ARM, Google reports, is now capable of assisting with detecting breast and prostate cancers, but they note that it could be trained to spot other cancers, as well. It could even be trained to look for some types of infectious diseases such as TB and malaria. They note also that it could also be programmed with other AR capabilities, such as posting labels over certain tissue sections, including arrows or other indicators pathologists might find useful.

Explore further: Smart software can diagnose prostate cancer as well as a pathologist

Related Stories

Smart software can diagnose prostate cancer as well as a pathologist

March 16, 2018
Chinese scientists and clinicians have developed a learning artificial intelligence system which can diagnose and identify cancerous prostate samples as accurately as any pathologist. This holds out the possibility of streamlining ...

Computer accurately identifies and delineates breast cancers on digital tissue slides

May 10, 2017
A deep-learning computer network developed through research led by Case Western Reserve University was 100 percent accurate in determining whether invasive forms of breast cancer were present in whole biopsy slides.

IU-invented microscope could make surgeries more effective, disease diagnoses timelier

March 22, 2018
In the not-so-distant future, surgeons could ensure the complete removal of malignant tumors, and pathologists could analyze tissue more efficiently, by using a device invented at the Indiana University School of Medicine.

Microscope can scan tumors during surgery and examine cancer biopsies in 3-D

June 26, 2017
When women undergo lumpectomies to remove breast cancer, doctors try to remove all the cancerous tissue while conserving as much of the healthy breast tissue as possible.

Will 'AI' be part of your health-care team?

December 12, 2017
(HealthDay)—Artificial intelligence is assuming a greater role in many walks of life, with research suggesting it may even help doctors diagnose disease.

Recommended for you

Aspirin could play valuable role as additional treatment for cancer

September 26, 2018
Regular use of aspirin could help in the treatment of some cancers, finds a new review of 71 medical studies.

Lung cancer drug could be repurposed to target 'zombie' proteins linked to leukemia

September 25, 2018
A new study by scientists at the University of Liverpool highlights how a clinically-approved lung cancer drug could potentially be 'repurposed' to design new treatments for future cancer therapies.

A protein called vaccinia-related kinase 1 may help cancer establish itself in new areas of the body during metastasis

September 25, 2018
Sometimes negative results can point researchers in the right direction.

Combo therapy of prostatectomy plus radiotherapy may improve survival in prostate cancer

September 25, 2018
High-risk prostate cancer, that which has continued to grow but not yet metastasized, is commonly treated with combination therapies. Each method has pros and cons, but there is little clarity whether one might be more effective ...

Brigatinib becomes potential new first-line option for ALK-positive non-small lung cancer

September 25, 2018
Results of a 275-patient, multi-national phase III clinical trial known as ALTA-1L published today in the New England Journal of Medicine and presented concurrently in the press program at the International Association for ...

Two studies describe improved approach to bone marrow transplant

September 25, 2018
Two recent studies in the journal Leukemia present a new approach for bone marrow donation and transplant that preclinical laboratory tests suggest could make the life-saving procedure safer and more effective for patients.

0 comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.