Skip to content

Controlling television with a gesture

Share on facebook
Share on twitter
Share on linkedin
Research by Sabrina Bilocca

Sabrina Bilocca
Sabrina Bilocca

Advancements in IT have made it possible to integrate computer systems into everyday life. However, cumbersome traditional devices like keyboards, mice, or the TV remote control, are inconvenient and unnatural to use. Hand gestures are a natural means of communication amongst humans. They are used to show feelings, emphasise points, and express thoughts and emotions. The use of hand gestures for humans to interact with computers should come as second nature to us.

Using this approach, Sabrina Bilocca (supervised by Prof. Adrian Muscat) developed a computer vision system which automatically recognises simple human hand gestures to control a TV. Bilocca’s research involved the recognition of isolated gestures as well as multiple gestures in continuous sequences. The greatest challenge was to recognise gestures in a series, since another procedure was needed to spot the key hand gestures while discarding unrelated movements between the key gestures.

The system was developed on a real-life dataset of isolated and a chain of hand gestures. The video samples were pre-processed using two algorithms (computer programmes that do a specific task, like separate the hand from the background) called ‘hand segmentation’ and ‘feature extraction’. The resulting images were input into a machine learning technique for training and testing that allows the computer to learn how to identify the hand gestures in a better way. Bilocca used the Hidden Markov Model as a machine learning. To efficiently spot and recognise key gestures in continuous sequences, Bilocca proposed and implemented an orientation backtracking technique. Using these techniques she achieved a 100% recognition rate for isolated gestures and an 87.8% recognition rate for gestures in a sequence. This compares with existing state-of-the-art techniques.

The project could be further developed as a real-time application to recognise hand gestures as they happen, which would help assist people with disability and the elderly. The skills Bilocca learnt in this project help her in her current position as a network engineer.

This research was performed as part of a Master of Science in Information and Communication Technology (Telecommunications) at the Faculty of Information and Communication Technology, University of Malta. . It was partially funded by STEPS (the Strategic Educational Pathways Scholarship—Malta). This scholarship was part-financed by the European Union—European Social Fund (ESF) under Operational Programme II—Cohesion Policy 2007–2013, ‘Empowering People for More Jobs and a Better Quality of Life’.

More to explorer

Maltese Innovation Promises New Era in Accessing Space

A team of researchers and engineers from the University of Malta’s Astrionics Research Group (ASTREA) are developing their own satellite and ground station for Malta to take its place in the night sky above.

Cutting EDGE Research: the Genomes of Maltese Plants

Conservation is a 21st century hot topic. It is a top priority worldwide to ensure the lasting protection of the planet and its natural resources. But how can we even begin to conserve our natural environment if we don’t understand it? Ines Ventura investigates.

Impressions on Branding

Our impression of a brand shapes our experience of the product or service involved. Five students from the Bachelor of Fine Arts (BFA) in Digital Arts looked into this, involving their own interests in the medium.

No comment yet, add your voice below!

Add a Comment

Your email address will not be published.