Sound like something from the Tom Cruise science fiction film "Minority Report"? Well, the future can be glimpsed today thanks to a prototype hand-gesture recognition system called Gestix that was tested during two surgical procedures last fall.

The goal of the test was to try out Gestix in a real operating room situation. It worked better than anyone expected.

Gestix is a video-based hand-gesture capture and recognition system used to manipulate digital images within a diagnostic workstation's GUI. It is designed to recognize specific hand gestures as mouse function commands, specifically pointer movement and button presses. Gestix eliminates the need to use a computer keyboard, mouse, joystick, or touchscreen, enabling medical professionals to remain sterile in an operating suite or intensive care unit environment.

"A hand-gesture vocabulary of commands was selected as being natural in the sense that each gesture is cognitively associated with the command that is meant to represent it," lead researcher Juan Wachs, Ph.D., explained.

Wachs led the development of Gestix during a two-year period while completing his doctoral degree requirements at Ben Gurion University of the Negev in Be'er Sheva, Israel. Gestix was a joint initiative between Ben Gurion University of the Negev and the Institute for Medical Informatics at Washington Hospital Center.

The Gestix system responds to hand motions, eliminating the need for neurosurgeons to scrub again after touching computer peripherals. Video courtesy of Dr. Helman Stern, Ben Gurion University of the Negev.

User personalization

The system requires custom calibration for each user, much like voice recognition software. A video camera is used to capture images of gestures made by a user's hand, which are then used to create a color model that the Gestix computer will recognize.

"After the system recognizes a surgeon's hand gestures, the surgeon must learn and implement eight navigation gestures, rapidly moving the hand away from a 'neutral area' and back again. It's not difficult at all because the system was designed to respond to natural intuitive gestures," explained Dr. Helman Stern, professor emeritus of Ben Gurion's Industrial Engineering and Management department, who was principal investigator for the project.

The gestures are captured by a Canon VC-C4 video camera positioned above a flat-screen monitor displaying the PACS images, using a computer with an Intel Pentium processor and a video-capture device from Matrox Graphics of Dorval, Quebec. Hand movements across the boundary of a "neutral area" in the video frame are tracked.

"When a user wishes to browse the image database, the hand is moved rapidly out of the neutral area toward any of four directions and then back again," Wachs said. "When a movement is detected, the displayed image is moved off the screen and replaced by an adjacent one."

A zoom is achieved when the open palm of the hand is rotated clockwise or counterclockwise within the neutral area. Left/right/up/down gestures evoke actions comparable to turning pages in a book or flipping notepad pages. The rotation gesture is comparable to turning a knob. The system goes into a sleep mode to avoid the tracking of unintentional gestures when a hand is dropped and awakens when a hand waves at it.

Dr. Edward Aulisi, head of neurosurgery at Washington Hospital Center, tested the system during two neurosurgical procedures he performed last fall. Prior to a surgical procedure, he would select diagnostic images from the PACS of the hospital's emergency medical record system (Amalga, Microsoft, Redmond, WA) and load them onto a stereotactic image-guided device (VectorVision2, BrainLab, Westchester, IL). VectorVision2 provides a 3D measurement where a probe or instrument is inside the head using MRI or CT images.

Aulisi uses VectorVision2 with every craniotomy he performs. Sometimes he finds that he wants to use a different sequence or a different series of images. Loading the new images means interacting with the workstation, which means getting scrubbed again afterward, a time-consuming procedure.

"With Gestix, even with an operative glove on my hand, all my commands were understood and the system worked flawlessly," Aulisi said.

The use of Gestix saved Aulisi approximately 35-40 minutes, almost half the time usually required for the total procedure. Even more time savings might be achievable once a surgeon becomes more adept with the system.

"If Gestix were commercially available today, I would be using it. It was easy to catch on to use, performed flawlessly, and would save me a significant amount of time during my surgeries," Aulisi said.

Gestix designers have completed additional refinement of the system, and it has a patent pending. Formal clinical trials and commercialization are expected to begin soon, according to Stern. In an interview with AuntMinnie.com, both Stern and Wachs said that while Gestix was developed for use in sterile environments to prevent the spread of bacteria, it also might be useful in a radiology reading room.

By Cynthia Keen
AuntMinnie.com staff writer
July 31, 2008

Copyright © 2008 AuntMinnie.com