Surgeons may use hand gestures to manipulate MRI images in OR
This table shows hand gestures surgeons might use in the operating room to browse and display medical images of the patient during an operation. Surgeons routinely need to review medical images and records during surgery, but stepping away from the operating table and touching a keyboard and mouse can delay the surgery and increase the risk of spreading infection-causing bacteria (Purdue University photo). Credit: Purdue University
Doctors may soon be using a system in the operating room that recognizes hand gestures as commands to tell a computer to browse and display medical images of the patient during a surgery.
Surgeons routinely need to review medical images and records during surgery, but stepping away from the operating table and touching a keyboard and mouse can delay the procedure and increase the risk of spreading infection-causing bacteria, said Juan Pablo Wachs, an assistant professor of industrial engineering at Purdue University.
"One of the most ubiquitous pieces of equipment in U.S. surgical units is the computer workstation, which allows access to medical images before and during surgery," he said. "However, computers and their peripherals are difficult to sterilize, and keyboards and mice have been found to be a source of contamination. Also, when nurses or assistants operate the keyboard for the surgeon, the process of conveying information accurately has proven cumbersome and inefficient since spoken dialogue can be time-consuming and leads to frustration and delays in the surgery."
Researchers are creating a system that uses depth-sensing cameras and specialized algorithms to recognize hand gestures as commands to manipulate MRI images on a large display. Recent research to develop the algorithms has been led by doctoral student Mithun George Jacob.
Findings from the research were detailed in a paper published in December in the Journal of the American Medical Informatics Association. The paper was written by Jacob, Wachs and Rebecca A. Packer, an associate professor of neurology and neurosurgery in Purdue's College of Veterinary Medicine.
The researchers validated the system, working with veterinary surgeons to collect a set of gestures natural for clinicians and surgeons. The surgeons were asked to specify functions they perform with MRI images in typical surgeries and to suggest gestures for commands. Ten gestures were chosen: rotate clockwise and counterclockwise; browse left and right; up and down; increase and decrease brightness; and zoom in and out.
Critical to the system's accuracy is the use of "contextual information" in the operating room—cameras observe the surgeon's torso and head—to determine and continuously monitor what the surgeon wants to do.
"A major challenge is to endow computers with the ability to understand the context in which gestures are made and to discriminate between intended gestures versus unintended gestures," Wachs said. "Surgeons will make many gestures during the course of a surgery to communicate with other doctors and nurses. The main challenge is to create algorithms capable of understanding the difference between these gestures and those specifically intended as commands to browse the image-viewing system. We can determine context by looking at the position of the torso and the orientation of the surgeon's gaze. Based on the direction of the gaze and the torso position we can assess whether the surgeon wants to access medical images."
This video is not supported by your browser at this time.The hand-gesture recognition system uses a camera developed by Microsoft, called Kinect, which senses three-dimensional space. The camera, found in consumer electronics games that can track a person's hands, maps the surgeon's body in 3-D. Findings showed that integrating context allows the algorithms to accurately distinguish image-browsing commands from unrelated gestures, reducing false positives from 20.8 percent to 2.3 percent.
"If you are getting false alarms 20 percent of the time, that's a big drawback," Wachs said. "So we've been able to greatly improve accuracy in distinguishing commands from other gestures."
The system also has been shown to have a mean accuracy of about 93 percent in translating gestures into specific commands, such as rotating and browsing images.
The algorithm takes into account what phase the surgery is in, which aids in determining the proper context for interpreting the gestures and reducing the browsing time.
"By observing the progress of the surgery we can tell what is the most likely image the surgeon will want to see next," Wachs said.
The researchers also are exploring context using a mock brain biopsy needle that can be tracked in the brain.
"The needle's location provides context, allowing the system to anticipate which images the surgeon will need to see next and reducing the number of gestures needed," Wachs said. "So instead of taking five minutes to browse, the surgeon gets there faster."
Sensors in the surgical needle reveal the position of its tip.
More information: HAND-GESTURE-BASED STERILE INTERFACE FOR THE OPERATING ROOM USING CONTEXTUAL CUES FOR THE NAVIGATION OF RADIOLOGICAL IMAGES
This paper presents a method to improve the navigation and manipulation of radiological images through a sterile hand gesture recognition interface based on attentional contextual cues. Computer vision algorithms were developed to extract intention and attention cues from the surgeon's behavior and combine them with sensory data from a commodity depth camera. The developed interface was tested in a usability experiment to assess the effectiveness of the new interface. An image navigation and manipulation task was performed, and the gesture recognition accuracy, false positives and task completion times were computed to evaluate system performance. Experimental results show that gesture interaction and surgeon behavior analysis can be used to accurately navigate, manipulate and access MRI images, and therefore this modality could replace the use of keyboard and mice-based interfaces.
Journal reference: Journal of the American Medical Informatics Association
Provided by Purdue University
- Future surgeons may use robotic nurse, 'gesture recognition' Feb 03, 2011 | not rated yet | 0
- Operating a computer by gesture only Mar 12, 2010 | not rated yet | 0
- Robotics: Gesturing for control May 24, 2012 | not rated yet | 0
- New interactive system detects touch and gestures on any surface (w/ Video) Oct 09, 2012 | not rated yet | 0
- Influencing others through gestures: misleading eyewitnesses Apr 24, 2012 | not rated yet | 0
- Motion perception revisited: High Phi effect challenges established motion perception assumptions Apr 23, 2013 | 3 / 5 (2) | 2
- Anything you can do I can do better: Neuromolecular foundations of the superiority illusion (Update) Apr 02, 2013 | 4.5 / 5 (11) | 5
- The visual system as economist: Neural resource allocation in visual adaptation Mar 30, 2013 | 5 / 5 (2) | 9
- Separate lives: Neuronal and organismal lifespans decoupled Mar 27, 2013 | 4.9 / 5 (8) | 0
- Sizing things up: The evolutionary neurobiology of scale invariance Feb 28, 2013 | 4.8 / 5 (10) | 14
Dipole term in multipole expansion
3 hours ago Hi. I'm having some difficult in understanding something about the dipole term in a multipole expansion. Griffiths writes the expansion as a sum of...
Bubbles in a Pre-Boiling/Boiling pot of water
4 hours ago How is it that bubbles form on the bottom of a surface of a pot of boiling water? I think that there is probably an elementary answer to this...
Assumptions of Griffith's fracture theory
15 hours ago Any experts on Griffith's fracture theory? I am studying the subject and I am having hard time finding out if the theory is valid for all possible...
Current leading voltage or vice versa concept
16 hours ago Hello, I was wondering if there is a conceptual explanation for when current leads voltage or vice versa for capacitors or inductors with AC...
Angular Frequency of AC voltage
19 hours ago Hello, I am wondering, what is the physical interpretation of the angular frequency of AC voltage? I don't see the physicality of what the angle...
Modeling Rigid Body - Unsure about Euler angles and angular velocity
20 hours ago I'm modeling a single 3D rigid body in preparation for some more complicated modeling in order to gain a better understanding of Euler angles, the...
- More from Physics Forums - Classical Physics
More news stories
Surgeons investigated sexual function in 62 patients, 50 years and older, who had received extensive spinal–pelvic instrumentation for spinal deformity at the University of Virginia Health Center. Based on their results, ...
Surgery 5 hours ago | not rated yet | 0
Physicians at Monash University and The Alfred Hospital in Melbourne, Australia describe the logistic, medical, and societal challenges faced in treating spine trauma in morbidly obese patients. Based on a case series of ...
Surgery 5 hours ago | not rated yet | 0
Arachnoid cysts are a common type of brain lesion that is usually harmless, but with a risk of rupture or bleeding. A new study identifies risk factors for rupture or bleeding in children with "incidentally" detected arachnoid ...
Surgery 5 hours ago | not rated yet | 0
Posterior fossa subdural hematoma (PFSDH) is a serious and rare condition in newborns, generally occurring after difficult deliveries. But with appropriate treatment, there's an excellent chance of good long-term outcomes ...
Surgery May 20, 2013 | not rated yet | 0
Many women in Texas who are found to have an abnormality on routine mammogram or discover a lump in one of their breasts end up having an old-fashioned surgical biopsy to find out whether the breast abnormality is malignant. ...
Surgery May 17, 2013 | not rated yet | 0
A drug commonly used to treat depression and anxiety may improve a stress-related heart condition in people with stable coronary heart disease, according to researchers at Duke Medicine.
59 minutes ago | not rated yet | 0 |
(HealthDay)—The overall health of Americans isn't improving much, with about six in 10 people either overweight or obese and large numbers engaging in unhealthy behaviors like smoking, heavy drinking or ...
1 hour ago | not rated yet | 0
(HealthDay)—In 2008 to 2010, the prevalence of key health behaviors among U.S. adults varied, with about one in five adults current smokers and 62.1 percent overweight or obese, according to a report presented ...
59 minutes ago | not rated yet | 0
(HealthDay)—Factors such as increased case finding may explain why Michigan had half of the total spinal infections associated with contaminated methylprednisolone acetate in the recent fungal meningitis ...
49 minutes ago | not rated yet | 0
Early use of tracheostomy for mechanically ventilated patients not associated with improved survival
For critically ill patients receiving mechanical ventilation, early tracheostomy (within the first 4 days after admission) was not associated with an improvement in the risk of death within 30 days compared to patients who ...
59 minutes ago | not rated yet | 0
(HealthDay)—Three-quarters of public schools in the metro Atlanta area contain microbes, including bacteria indicating the presence of fecal matter, according to research published in the May 17 issue of ...
39 minutes ago | not rated yet | 0