PhD graduate enables headcams to recognize hand gestures

The emergence of digital cameras that can be worn on the head, incorporated into spectacles for instance, offers opportunities to automatically recognize hand movements and gestures. Which is useful for people who are convalescing or for controlling digital applications. Alejandro Betancourt received his PhD with distinction for his research that has made this possible.

The Google Glass may not yet be commercially available but there are many other devices with cameras around that can be worn on the body. In the images made by these devices the wearer’s hands, and all the related gestures, tend to be in the picture. While good methods already exist that enable the hands to be automatically recognized in the images, a good way to allow these devices to see from a first person view what the hands are saying has been lacking.

In seeking a solution, PhD graduate Alejandro Betancourt had to solve a number of problems, such as the highly fluctuating light intensity the camera has to deal with as the wearer moves around. This hinders recognition. Nor is it very easy to identify the right hand from the left or to automatically recognize the posture and movement of the hands. To solve this Betancourt also had to take account of the fact that the computing capacity of a digital device like a Google Glass is limited, although it should be able to recognize gestures straightaway. So one of the methods he employed was Machine Learning.

The Colombian researcher thinks that a wide range of applications is possible using his methods, just like the XBox Kinect is now being used for much more than just playing computer games. He cites as an example a medical application for motor dysfunction whereby his methods should allow automatic monitoring and recognition. To date this has often been a matter of intensive monitoring by a physician. Furthermore, the automatic recognition of hand gestures could allow the camera device or digital applications that run it to be controlled without the need for contact.

For his research Betancourt receives his PhD with distinction from TU/e as well as a doctorate from the University of Genoa in Italy. The PhD is being awarded under the denominator of the Erasmus Mundus Joint Doctorate program on Interactive and Cognitive Environments. His supervisor at TU/e is professor Matthias Rauterberg of the department of Industrial Design. The thesis is entitled:  EgoHands, A Unified Framework for Hand-Based Methods in First Person Vision Videos.