Skip to content

Controlling television with a gesture

Share on facebook
Facebook
Share on twitter
Twitter
Share on linkedin
LinkedIn
Research by Sabrina Bilocca

Sabrina Bilocca
Sabrina Bilocca

Advancements in IT have made it possible to integrate computer systems into everyday life. However, cumbersome traditional devices like keyboards, mice, or the TV remote control, are inconvenient and unnatural to use. Hand gestures are a natural means of communication amongst humans. They are used to show feelings, emphasise points, and express thoughts and emotions. The use of hand gestures for humans to interact with computers should come as second nature to us.

Using this approach, Sabrina Bilocca (supervised by Prof. Adrian Muscat) developed a computer vision system which automatically recognises simple human hand gestures to control a TV. Bilocca’s research involved the recognition of isolated gestures as well as multiple gestures in continuous sequences. The greatest challenge was to recognise gestures in a series, since another procedure was needed to spot the key hand gestures while discarding unrelated movements between the key gestures.

The system was developed on a real-life dataset of isolated and a chain of hand gestures. The video samples were pre-processed using two algorithms (computer programmes that do a specific task, like separate the hand from the background) called ‘hand segmentation’ and ‘feature extraction’. The resulting images were input into a machine learning technique for training and testing that allows the computer to learn how to identify the hand gestures in a better way. Bilocca used the Hidden Markov Model as a machine learning. To efficiently spot and recognise key gestures in continuous sequences, Bilocca proposed and implemented an orientation backtracking technique. Using these techniques she achieved a 100% recognition rate for isolated gestures and an 87.8% recognition rate for gestures in a sequence. This compares with existing state-of-the-art techniques.

The project could be further developed as a real-time application to recognise hand gestures as they happen, which would help assist people with disability and the elderly. The skills Bilocca learnt in this project help her in her current position as a network engineer.


This research was performed as part of a Master of Science in Information and Communication Technology (Telecommunications) at the Faculty of Information and Communication Technology, University of Malta. . It was partially funded by STEPS (the Strategic Educational Pathways Scholarship—Malta). This scholarship was part-financed by the European Union—European Social Fund (ESF) under Operational Programme II—Cohesion Policy 2007–2013, ‘Empowering People for More Jobs and a Better Quality of Life’.

More to Explore

Adrift at Sea: Laws, Morals, and Policies in Malta’s Search and Rescue Region

Since 2016, EU member states have scaled down search and rescue operations that save lives at sea and replaced them with policies intended to reduce the number of migrant arrivals to Europe. These policies of non-assistance and forced returns to Libya render the central Mediterranean one of the world’s deadliest border spaces and force asylum seekers back to a war zone where inhuman and degrading treatment is well-documented. A growing network of civil society organisations continues to challenge these policies in the courts, on the streets, and at sea. This article, the second in a two-part series on migration, is based in part on interviews conducted with Dr Omar Grech, Senior Lecturer in International Law at the University of Malta (UM), Dr Derek Lutterbeck, Deputy Director at the Mediterranean Academy of Diplomatic Studies at UM, and Dr Felicity Attard, expert in International and Maritime Security Law at the Faculty of Laws at UM.

Concentration Camps in Libya

Following the NATO-backed overthrow of Muammar Gaddafi in 2011, Libya descended into a decade of disunity and violence resulting in incalculable suffering and loss of life. Today, much of the country remains a war zone, and migrants in EU-sponsored Libyan detention facilities continue to suffer well-documented, gross human rights violations. This article, the first in a two-part series on migration, is based in part on interviews conducted with Dr Omar Grech, Senior Lecturer in International Law at the University of Malta (UM); Dr Derek Lutterbeck, Deputy Director at the Mediterranean Academy of Diplomatic Studies at UM; and Dr Felicity Attard, expert in International and Maritime Security Law at the Faculty of Laws at UM. 

No comment yet, add your voice below!


Add a Comment