Skip to content

Controlling television with a gesture

Facebook
Twitter
LinkedIn
Research by Sabrina Bilocca

Sabrina Bilocca
Sabrina Bilocca

Advancements in IT have made it possible to integrate computer systems into everyday life. However, cumbersome traditional devices like keyboards, mice, or the TV remote control, are inconvenient and unnatural to use. Hand gestures are a natural means of communication amongst humans. They are used to show feelings, emphasise points, and express thoughts and emotions. The use of hand gestures for humans to interact with computers should come as second nature to us.

Using this approach, Sabrina Bilocca (supervised by Prof. Adrian Muscat) developed a computer vision system which automatically recognises simple human hand gestures to control a TV. Bilocca’s research involved the recognition of isolated gestures as well as multiple gestures in continuous sequences. The greatest challenge was to recognise gestures in a series, since another procedure was needed to spot the key hand gestures while discarding unrelated movements between the key gestures.

The system was developed on a real-life dataset of isolated and a chain of hand gestures. The video samples were pre-processed using two algorithms (computer programmes that do a specific task, like separate the hand from the background) called ‘hand segmentation’ and ‘feature extraction’. The resulting images were input into a machine learning technique for training and testing that allows the computer to learn how to identify the hand gestures in a better way. Bilocca used the Hidden Markov Model as a machine learning. To efficiently spot and recognise key gestures in continuous sequences, Bilocca proposed and implemented an orientation backtracking technique. Using these techniques she achieved a 100% recognition rate for isolated gestures and an 87.8% recognition rate for gestures in a sequence. This compares with existing state-of-the-art techniques.

The project could be further developed as a real-time application to recognise hand gestures as they happen, which would help assist people with disability and the elderly. The skills Bilocca learnt in this project help her in her current position as a network engineer.


This research was performed as part of a Master of Science in Information and Communication Technology (Telecommunications) at the Faculty of Information and Communication Technology, University of Malta. . It was partially funded by STEPS (the Strategic Educational Pathways Scholarship—Malta). This scholarship was part-financed by the European Union—European Social Fund (ESF) under Operational Programme II—Cohesion Policy 2007–2013, ‘Empowering People for More Jobs and a Better Quality of Life’.

Author

More to Explore

Our Post-Truth Reality

Post-truth populism has secured a powerful mandate in the United States of America. This reflects a trend that extends through the world’s liberal democracies and will invite global imitation. In this opinion piece, Jonathan Firbank describes how post-truth populism works, why it works, and why the American election might show us how to fight it.

AGORA: Elections 2024 – Youth Absence and the Far Right Surge

During the run-up to the European Parliament Elections, Prof. Mario Thomas Vassallo grilled two MEP candidates on AGORA, a political talk show broadcast on Campus 103.7. Against the backdrop of numerous elections around the globe, a lack of youth representation, and the rise of the far right, the discussion got us thinking.

Smooth Operator: Improving Surface Finish in Additive Manufacturing

While the advent of 3D metal printing may redefine how designers develop parts for products, the process itself is not without faults. Andre Giordimaina speaks with THINK about the GLAM Project, which aims to improve the process of 3D metal printing by optimising the finish and performance of designed parts.

Comments are closed for this article!