Surgeons may use hand gestures to manipulate MRI images in OR

January 10, 2013 by Emil Venere, Purdue University
This table shows hand gestures surgeons might use in the operating room to browse and display medical images of the patient during an operation. Surgeons routinely need to review medical images and records during surgery, but stepping away from the operating table and touching a keyboard and mouse can delay the surgery and increase the risk of spreading infection-causing bacteria (Purdue University photo). Credit: Purdue University

Doctors may soon be using a system in the operating room that recognizes hand gestures as commands to tell a computer to browse and display medical images of the patient during a surgery.

routinely need to review medical images and records during surgery, but stepping away from the operating table and touching a keyboard and mouse can delay the procedure and increase the risk of spreading infection-causing bacteria, said Juan Pablo Wachs, an assistant professor of industrial engineering at Purdue University.

"One of the most ubiquitous pieces of equipment in U.S. surgical units is the computer workstation, which allows access to medical images before and during surgery," he said. "However, computers and their peripherals are difficult to sterilize, and keyboards and mice have been found to be a source of contamination. Also, when nurses or assistants operate the keyboard for the surgeon, the process of conveying information accurately has proven cumbersome and inefficient since spoken dialogue can be time-consuming and leads to frustration and delays in the surgery."

Researchers are creating a system that uses depth-sensing cameras and specialized algorithms to recognize as commands to manipulate MRI images on a large display. Recent research to develop the algorithms has been led by Mithun George Jacob.

Findings from the research were detailed in a paper published in December in the . The paper was written by Jacob, Wachs and Rebecca A. Packer, an associate professor of neurology and in Purdue's College of Veterinary Medicine.

The researchers validated the system, working with veterinary surgeons to collect a set of natural for clinicians and surgeons. The surgeons were asked to specify functions they perform with in typical surgeries and to suggest gestures for commands. Ten gestures were chosen: rotate clockwise and counterclockwise; browse left and right; up and down; increase and decrease brightness; and zoom in and out.

Critical to the system's accuracy is the use of "contextual information" in the —cameras observe the surgeon's torso and head—to determine and continuously monitor what the surgeon wants to do.

"A major challenge is to endow computers with the ability to understand the context in which gestures are made and to discriminate between intended gestures versus unintended gestures," Wachs said. "Surgeons will make many gestures during the course of a surgery to communicate with other doctors and nurses. The main challenge is to create algorithms capable of understanding the difference between these gestures and those specifically intended as commands to browse the image-viewing system. We can determine context by looking at the position of the torso and the orientation of the surgeon's gaze. Based on the direction of the gaze and the torso position we can assess whether the surgeon wants to access ."

The hand-gesture recognition system uses a camera developed by Microsoft, called Kinect, which senses three-dimensional space. The camera, found in consumer electronics games that can track a person's hands, maps the surgeon's body in 3-D. Findings showed that integrating context allows the algorithms to accurately distinguish image-browsing commands from unrelated gestures, reducing false positives from 20.8 percent to 2.3 percent.

"If you are getting false alarms 20 percent of the time, that's a big drawback," Wachs said. "So we've been able to greatly improve accuracy in distinguishing commands from other gestures."

The system also has been shown to have a mean accuracy of about 93 percent in translating gestures into specific commands, such as rotating and browsing images.

The algorithm takes into account what phase the surgery is in, which aids in determining the proper context for interpreting the gestures and reducing the browsing time.

"By observing the progress of the surgery we can tell what is the most likely image the surgeon will want to see next," Wachs said.

The researchers also are exploring context using a mock brain biopsy needle that can be tracked in the brain.

"The needle's location provides context, allowing the system to anticipate which images the surgeon will need to see next and reducing the number of gestures needed," Wachs said. "So instead of taking five minutes to browse, the surgeon gets there faster."

Sensors in the surgical needle reveal the position of its tip.

Explore further: Researchers link wild chimpanzee gestures to language evolution

More information: HAND-GESTURE-BASED STERILE INTERFACE FOR THE OPERATING ROOM USING CONTEXTUAL CUES FOR THE NAVIGATION OF RADIOLOGICAL IMAGES

ABSTRACT
This paper presents a method to improve the navigation and manipulation of radiological images through a sterile hand gesture recognition interface based on attentional contextual cues. Computer vision algorithms were developed to extract intention and attention cues from the surgeon's behavior and combine them with sensory data from a commodity depth camera. The developed interface was tested in a usability experiment to assess the effectiveness of the new interface. An image navigation and manipulation task was performed, and the gesture recognition accuracy, false positives and task completion times were computed to evaluate system performance. Experimental results show that gesture interaction and surgeon behavior analysis can be used to accurately navigate, manipulate and access MRI images, and therefore this modality could replace the use of keyboard and mice-based interfaces.

Related Stories

Researchers link wild chimpanzee gestures to language evolution

July 15, 2012
(Phys.org) -- A Stirling researcher has identified between 20 and 30 manual gestures used by a community of wild chimpanzees, used to communicate with others in a range of activities including nursing, feeding, sex, aggression ...

Recommended for you

Drug may help surgical patients stop opioids sooner

December 13, 2017
(HealthDay)—Opioid painkillers after surgery can be the first step toward addiction for some patients. But a common drug might cut the amount of narcotics that patients need, a new study finds.

Children best placed to explain facts of surgery to patients, say experts

December 13, 2017
Getting children to design patient information leaflets may improve patient understanding before they have surgery, finds an article in the Christmas issue of The BMJ.

Burn victim saved by skin grafts from identical twin (Update)

November 23, 2017
A man doomed to die after suffering burns across 95 percent of his body was saved by skin transplants from his identical twin in a world-first operation, French doctors said Thursday.

Is a common shoulder surgery useless?

November 21, 2017
(HealthDay)—New research casts doubt on the true effectiveness of a common type of surgery used to ease shoulder pain.

Study shows electric bandages can fight biofilm infection, antimicrobial resistance

November 6, 2017
Researchers at The Ohio State University Wexner Medical Center have shown - for the first time - that special bandages using weak electric fields to disrupt bacterial biofilm infection can prevent infections, combat antibiotic ...

Obesity increases incidence, severity, costs of knee dislocations

November 3, 2017
A new study of more than 19,000 knee dislocation cases in the U.S. between 2000 and 2012 provides a painful indication of how the nation's obesity epidemic is changing the risk, severity and cost of a traumatic injury.

0 comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.