Combining augmented reality with deep learning network to help pathologists spot cancer cells

April 17, 2018 by Bob Yirka, Medical Xpress report
Left: Schematic overview of the ARM. A digital camera captures the same field of view (FoV) as the user and passes the image to an attached compute unit capable of running real-time inference of a machine learning model. The results are fed back into a custom AR display which is inline with the ocular lens and projects the model output on the same plane as the slide. Right: A picture of our prototype which has been retrofitted into a typical clinical-grade light microscope. Credit: Google

A team of researchers at Google has developed a tool that combines augmented reality with a deep learning neural network to provide pathologists with help in spotting cancerous cells on slides under a microscope. Google has published a paper outlining the new tool, which they describe as an augmented reality microscope (ARM) platform, on their website. They also gave a presentation at this year's American Association for Cancer Research meeting, showing what it can do.

Performing a biopsy entails slicing tiny bits of tissue from a part of the body doctors suspect might be cancerous. That tissue is sliced thinly enough to view on a glass slide under a microscope by a highly trained pathologist looking for . The job of the pathologist, the Google team notes, is highly valued, but the work is also tedious, which can lead to fatigue and errors. To help the pathologist, the team at Google built on prior work at the company in which a team developed a network that was trained to spot breast cancer. The team then incorporated lessons learned from various projects surrounding to train their deep learning application to draw circles around suspected in a tissue sample. They interfaced their system with the type of microscope pathologist already use. The result is an add-on tool for the microscope. It works like this: Through the , the pathologist sees the cells of the tissue—moving the slide around allows them to see the whole sample. But then,if the ARM spots possible cells, it draws a ring around them, alerting the pathologist to take a closer look.

The ARM, Google reports, is now capable of assisting with detecting breast and prostate cancers, but they note that it could be trained to spot other cancers, as well. It could even be trained to look for some types of infectious diseases such as TB and malaria. They note also that it could also be programmed with other AR capabilities, such as posting labels over certain tissue sections, including arrows or other indicators pathologists might find useful.

Explore further: Smart software can diagnose prostate cancer as well as a pathologist

Related Stories

Smart software can diagnose prostate cancer as well as a pathologist

March 16, 2018
Chinese scientists and clinicians have developed a learning artificial intelligence system which can diagnose and identify cancerous prostate samples as accurately as any pathologist. This holds out the possibility of streamlining ...

Computer accurately identifies and delineates breast cancers on digital tissue slides

May 10, 2017
A deep-learning computer network developed through research led by Case Western Reserve University was 100 percent accurate in determining whether invasive forms of breast cancer were present in whole biopsy slides.

IU-invented microscope could make surgeries more effective, disease diagnoses timelier

March 22, 2018
In the not-so-distant future, surgeons could ensure the complete removal of malignant tumors, and pathologists could analyze tissue more efficiently, by using a device invented at the Indiana University School of Medicine.

Microscope can scan tumors during surgery and examine cancer biopsies in 3-D

June 26, 2017
When women undergo lumpectomies to remove breast cancer, doctors try to remove all the cancerous tissue while conserving as much of the healthy breast tissue as possible.

Will 'AI' be part of your health-care team?

December 12, 2017
(HealthDay)—Artificial intelligence is assuming a greater role in many walks of life, with research suggesting it may even help doctors diagnose disease.

Recommended for you

Magnetized wire could be used to detect cancer in people, scientists report

July 16, 2018
A magnetic wire used to snag scarce and hard-to-capture tumor cells could prove to be a swift and effective tactic for early cancer detection, according to a study by researchers at the Stanford University School of Medicine.

Researchers suggest new treatment for rare inherited cancers

July 16, 2018
Studying two rare inherited cancer syndromes, Yale Cancer Center (YCC) scientists have found the cancers are driven by a breakdown in how cells repair their DNA. The discovery, published today in Nature Genetics, suggests ...

Researchers map 'family trees' of acute myeloid leukemia

July 16, 2018
For the first time, a team of international researchers has mapped the family trees of cancer cells in acute myeloid leukaemia (AML) to understand how this blood cancer responds to a new drug, enasidenib. The work also explains ...

Looking at the urine and blood may be best in diagnosing myeloma

July 13, 2018
When it comes to diagnosing a condition in which the plasma cells that normally make antibodies to protect us instead become cancerous, it may be better to look at the urine as well as the serum of our blood for answers, ...

Massive genome havoc in breast cancer is revealed

July 12, 2018
In cancer cells, genetic errors wreak havoc. Misspelled genes, as well as structural variations—larger-scale rearrangements of DNA that can encompass large chunks of chromosomes—disturb carefully balanced mechanisms that ...

Study shows biomarker panel boosts lung cancer risk assessment for smokers

July 12, 2018
A four-protein biomarker blood test improves lung cancer risk assessment over existing guidelines that rely solely upon smoking history, capturing risk for people who have ever smoked, not only for heavy smokers, an international ...

0 comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.