Skip to content

Controlling television with a gesture

Share on facebook
Share on twitter
Share on linkedin
Research by Sabrina Bilocca

Sabrina Bilocca
Sabrina Bilocca

Advancements in IT have made it possible to integrate computer systems into everyday life. However, cumbersome traditional devices like keyboards, mice, or the TV remote control, are inconvenient and unnatural to use. Hand gestures are a natural means of communication amongst humans. They are used to show feelings, emphasise points, and express thoughts and emotions. The use of hand gestures for humans to interact with computers should come as second nature to us.

Using this approach, Sabrina Bilocca (supervised by Prof. Adrian Muscat) developed a computer vision system which automatically recognises simple human hand gestures to control a TV. Bilocca’s research involved the recognition of isolated gestures as well as multiple gestures in continuous sequences. The greatest challenge was to recognise gestures in a series, since another procedure was needed to spot the key hand gestures while discarding unrelated movements between the key gestures.

The system was developed on a real-life dataset of isolated and a chain of hand gestures. The video samples were pre-processed using two algorithms (computer programmes that do a specific task, like separate the hand from the background) called ‘hand segmentation’ and ‘feature extraction’. The resulting images were input into a machine learning technique for training and testing that allows the computer to learn how to identify the hand gestures in a better way. Bilocca used the Hidden Markov Model as a machine learning. To efficiently spot and recognise key gestures in continuous sequences, Bilocca proposed and implemented an orientation backtracking technique. Using these techniques she achieved a 100% recognition rate for isolated gestures and an 87.8% recognition rate for gestures in a sequence. This compares with existing state-of-the-art techniques.

The project could be further developed as a real-time application to recognise hand gestures as they happen, which would help assist people with disability and the elderly. The skills Bilocca learnt in this project help her in her current position as a network engineer.

This research was performed as part of a Master of Science in Information and Communication Technology (Telecommunications) at the Faculty of Information and Communication Technology, University of Malta. . It was partially funded by STEPS (the Strategic Educational Pathways Scholarship—Malta). This scholarship was part-financed by the European Union—European Social Fund (ESF) under Operational Programme II—Cohesion Policy 2007–2013, ‘Empowering People for More Jobs and a Better Quality of Life’.


More to Explore

Can We Change the Contraceptive Pill?

60 years ago, the contraceptive pill triggered a social revolution by allowing women to take charge over their reproductive health. Yet, hefty side effects threaten these same rights. THINK analyses what science has been doing to overcome these side effects.

Food Security in Malta: Is it Possible?

It is unlikely that Malta can ever become self-sufficient when it comes to food. The next best thing for the island is to guarantee sustainable food security. THINK sits with 4 experts to see how this could be possible.

Il-Baqra Tinbieh Kollha: A Journey Towards Abattoir Self-Sufficiency and Sustainability

While plant-based food is gaining popularity, the majority of the world’s population still consumes meat. “Il-baqra tinbieħ kollha” (every part of the cow can be sold), a Maltese saying, emphasises that every part of harvested livestock is used. It is imperative that the meat industry develop sustainable practices to reduce its environmental impact, such as utilising every part of the slaughtered animal.

No comment yet, add your voice below!

Add a Comment