Combining augmented reality with deep learning network to help pathologists spot cancer cells

April 17, 2018 by Bob Yirka, Medical Xpress report
Left: Schematic overview of the ARM. A digital camera captures the same field of view (FoV) as the user and passes the image to an attached compute unit capable of running real-time inference of a machine learning model. The results are fed back into a custom AR display which is inline with the ocular lens and projects the model output on the same plane as the slide. Right: A picture of our prototype which has been retrofitted into a typical clinical-grade light microscope. Credit: Google

A team of researchers at Google has developed a tool that combines augmented reality with a deep learning neural network to provide pathologists with help in spotting cancerous cells on slides under a microscope. Google has published a paper outlining the new tool, which they describe as an augmented reality microscope (ARM) platform, on their website. They also gave a presentation at this year's American Association for Cancer Research meeting, showing what it can do.

Performing a biopsy entails slicing tiny bits of tissue from a part of the body doctors suspect might be cancerous. That tissue is sliced thinly enough to view on a glass slide under a microscope by a highly trained pathologist looking for . The job of the pathologist, the Google team notes, is highly valued, but the work is also tedious, which can lead to fatigue and errors. To help the pathologist, the team at Google built on prior work at the company in which a team developed a network that was trained to spot breast cancer. The team then incorporated lessons learned from various projects surrounding to train their deep learning application to draw circles around suspected in a tissue sample. They interfaced their system with the type of microscope pathologist already use. The result is an add-on tool for the microscope. It works like this: Through the , the pathologist sees the cells of the tissue—moving the slide around allows them to see the whole sample. But then,if the ARM spots possible cells, it draws a ring around them, alerting the pathologist to take a closer look.

The ARM, Google reports, is now capable of assisting with detecting breast and prostate cancers, but they note that it could be trained to spot other cancers, as well. It could even be trained to look for some types of infectious diseases such as TB and malaria. They note also that it could also be programmed with other AR capabilities, such as posting labels over certain tissue sections, including arrows or other indicators pathologists might find useful.

Explore further: Smart software can diagnose prostate cancer as well as a pathologist

Related Stories

Smart software can diagnose prostate cancer as well as a pathologist

March 16, 2018
Chinese scientists and clinicians have developed a learning artificial intelligence system which can diagnose and identify cancerous prostate samples as accurately as any pathologist. This holds out the possibility of streamlining ...

Computer accurately identifies and delineates breast cancers on digital tissue slides

May 10, 2017
A deep-learning computer network developed through research led by Case Western Reserve University was 100 percent accurate in determining whether invasive forms of breast cancer were present in whole biopsy slides.

IU-invented microscope could make surgeries more effective, disease diagnoses timelier

March 22, 2018
In the not-so-distant future, surgeons could ensure the complete removal of malignant tumors, and pathologists could analyze tissue more efficiently, by using a device invented at the Indiana University School of Medicine.

Microscope can scan tumors during surgery and examine cancer biopsies in 3-D

June 26, 2017
When women undergo lumpectomies to remove breast cancer, doctors try to remove all the cancerous tissue while conserving as much of the healthy breast tissue as possible.

Will 'AI' be part of your health-care team?

December 12, 2017
(HealthDay)—Artificial intelligence is assuming a greater role in many walks of life, with research suggesting it may even help doctors diagnose disease.

Recommended for you

Single-cell study in a childhood brain tumor affirms the importance of context

April 20, 2018
In defining the cellular context of diffuse midline gliomas, researchers find the cells fueling their growth and suggest a potential approach to treating them: forcing their cells to be more mature.

Aggressive breast cancer already has resistant tumour cells prior to chemotherapy

April 20, 2018
Difficult to treat and aggressive "triple-negative" breast cancer is chemoresistant even before chemotherapy begins, a new study by researchers from Karolinska Institutet and the University of Texas MD Anderson Cancer Center ...

Mechanism that drives development of liver cancer brought on by non-alcoholic fatty liver disease discovered

April 19, 2018
A team of researchers from several institutions in China has found a mechanism that appears to drive the development of a type of liver cancer not caused by alcohol consumption. In their paper published in the journal Science ...

Discovery adds to evidence that some children are predisposed to develop leukemia

April 19, 2018
St. Jude Children's Research Hospital researchers have made a discovery that expands the list of genes to include when screening individuals for possible increased susceptibility to childhood leukemia. The finding is reported ...

Scientists identify 170 potential lung cancer drug targets using unique cellular library

April 19, 2018
After testing more than 200,000 chemical compounds, UT Southwestern's Simmons Cancer Center researchers have identified 170 chemicals that are potential candidates for development into drug therapies for lung cancer.

Chip-based blood test for multiple myeloma could make bone biopsies a relic of the past

April 19, 2018
The diagnosis and treatment of multiple myeloma, a cancer affecting plasma cells, traditionally forces patients to suffer through a painful bone biopsy. During that procedure, doctors insert a bone-biopsy needle through an ...

0 comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.