What would you do if you were stripped of your words? If speech simply didn’t come to you? Sylvan Abela writes about MaltAAC, an Augmentative and Alternative Communication App for the Maltese Language.
Listen to this article as a podcast
In 2016, education minister Hon. Evarist Bartolo faced calls to remove Maltese O-levels as a prerequisite for autistic youths to enter the University of Malta. The Maltese language has a reputation for being quite a tough one to master, and a cross section of parents, politicians, and educators felt that people with language difficulties were being held back in their personal and educational goals. The idea sparked hefty debate throughout the Maltese community, with many expressing their concern that making the Maltese language an optional subject to study would be another nail in its coffin Linguist Dr Sarah Grech (Centre for English Language Proficiency) says Maltese is no different from any other language to learn. What it suffers from is a lack of resources when it comes to mastering it. ‘Television, cinema, games, books, apps—there are so many choices when it comes to absorbing the English language. The same cannot be said for Maltese,’ she says. Thankfully, this situation is changing.
IDENTIFYING TALKING POINTS
Most children naturally convert thoughts into word sequences and couple these to the necessary muscle movements to speak. However, there are others who are unable to translate their thoughts into audible words, creating the need for an alternative means of communication. Enter Augmentative and Alternative Communication (AAC), which is any method of communication that serves to aid or function as an alternative to speech.
Common examples of AAC systems include gestures, facial expressions, and communication boards with visuals that correspond to different words. AAC has proven to be crucial for children who suffer from communication impairments due to cognitive issues such as autism, or motor impairments like cerebral palsy. Over the past 40 years, this field of research has evolved with the noble aim of developing more convenient and accessible AAC solutions for children. But there are challenges.
The main limitation of AAC systems is their communication rate. While most people communicate their thoughts effortlessly, converting ideas into words in a matter of milliseconds, AAC users type letters in a text box or choose pictures from a visual array to construct sentences. In practice, while a child may speak at a rate of 130 to 200 words per minute, AAC users reach rates below 10 words per minute. It is easy to understand why this can be immensely frustrating for AAC users. Thankfully, solutions are on the horizon. Numerous studies have shown that by implementing next word prediction functionality in high-tech AAC systems, the same technology that lets your phone predict what word you will use next, communication rates can be improved significantly. AAC developments in the Maltese language are very limited and no complete high-tech AAC system exists. Our team has risen to the task to tread new ground.
Under the supervision of Dr Ing. Owen Casha, myself (Sylvan Abela, Department of Microelectronics and Nanoelectronics, Faculty of ICT) and Dr May Agius from the Access to Communication and Technology Unit (ACTU, the agency responsible for assessing individuals for AAC solutions and research) joined forces to create a mobile AAC app that would allow children with communication impairments to communicate in Maltese, all while reducing the amount of effort they require to do so and improving their communication rate.
The app owes its inception to SPEECHIE, a project working on a toy that supports language development in preschool children (See THINK issue 24, pg 30). Dr Daniela Gatt, a member of the SPEECHIE consortium, introduced Casha to Agius, who emphasised the urgent need for an AAC app tailored to Maltese children.
Its potential to have a positive impact on people’s lives made it an unmissable opportunity to give back to the community.
At the time, the ACTU team were continuously dedicating resources and effort to customising existing AAC apps originally designed for English. However, this proved problematic due to English being a Germanic language and Maltese being semitic.
In early June 2017, faced with with the selection of a final project for my degree, the development of a mobile AAC app stood out from the list of project proposals. Its potential to have a positive impact on people’s lives made it an unmissable opportunity to give back to the community. I requested a meeting with Casha and Agius without giving it a second thought.
There, we discussed how we could combine the engineering and rehabilitative disciplines to produce a practical and beneficial AAC solution for Maltese children. This initial meeting ultimately paved the way for the birth of MaltAAC, the first AAC app specifically designed for the Maltese language.
DIGGING OUR HEELS IN
Work on the MaltAAC project began with a sit-down at the ACTU, where Agius ran us through the prevailing AAC technologies and identified the essential features that comprise a hightech AAC system. Dedicated speech generating devices (SGDs) were, and still are, considered to be the most appropriate high-tech AAC systems, and for some people, no other AAC solution suffices.
Unfortunately, SGDs come with extremely high price tags, ranging from €3,000 to €15,000. To improve accessibility and affordability, we decided to use a smart device. The next choice we needed to make was the operating system. When deciding between Android and iOS, it came down to market share. In Malta, Android holds the lion’s share with 70% of users, all while being a lot cheaper than Apple devices.
The app features a symbol-based user interface (UI) which looks a lot like the layout used in most high-tech AAC systems intended for children. Primarily, users tap images to input their thoughts. They also have the option to type should they wish to. The image grid is the main component of the UI, which users can resize based on their needs and abilities. This symbol-based system makes the task easier for children, as it helps them visualise the words. We tried to reduce their communication effort even further by using the Fitzgerald Key to colour-code each symbol based on the word’s grammatical role. So nouns are orange, verbs are green, adjectives/adverbs are blue, and social words are pink.
The MaltAAC project is nothing if not a solid step forward in promoting the use of the Maltese language while providing an accessible solution to Maltese children with complex communication needs.
The next step we had to tackle was integration of speech into the app. However, a Maltese text-to-speech engine for the Android platform does not yet exist, so we had to manually record the audio ourselves. Desperate for the project’s success, I turned to my beloved girlfriend, Nicole Fsadni, for support. At that point, she was well-accustomed to hearing about the project and didn’t hesitate to help. Nicole and I scheduled an appointment at the University of Malta’s radio station, Campus FM, and recorded the speech using its state-of-the-art equipment, ensuring that the speech the app generated was the best it could be.
MORE TO IT THAN MEETS THE EYE
But MaltAAC goes beyond translating image-tapping into speech. It enables users to add, remove, and edit words in the app’s dictionary according to preference. Children’s caretakers can protect app features with a security PIN. In addition, they can monitor the child’s progress, back up their AAC profile to the cloud, and lock the device for sole use with the AAC app. To reduce the effort of the child when communicating and improve their communication rate, we implemented a smoothed trigram language model, which generates a list of the most probable next words each time the user selects a new word. The trigram model was most suitable not only because of its ability to provide accurate predictions, but also because of its short training time and low memory consumption, which are essential for swift performance in mobile apps.
After developing the app, we tested it. We identified bugs and measured the language model’s accuracy. It passed both tests, so we moved to the black-box testing stage.
At this point, we ran MaltAAC through its paces. The app was handed over to three occupational therapists, two speech therapists, and two learning support educators working at ACTU, all of whom took it out into the field. These specialists make use of AAC technologies on a daily basis to assist children in their communication needs and therefore could properly evaluate the app. A month later, they came back full of positive experiences and gave us invaluable feedback.
Our next move will be to improve the UI in line with the recommendations made by the AAC specialists. Among other thi
ngs, they suggested switching the position of certain grid elements and changing the interface to a darker colour to improve the visual contrast. We are also working on a Maltese text-tospeech engine, which would replace our home-made voice recordings with synthesised speech. Now MaltAAC has been published on the Google Play Store, and we will eventually start working on an iOS implementation of the app. Undeniably, a laborious journey lies ahead, but the ti me is ripe to capitalise
on the momentum of this project’s success and ride it to the finish line.
WATCH THIS SPACE
The MaltAAC project is nothing if not a solid step forward in promoting the use of the Maltese language while providing an accessible solution to Maltese children with complex communication needs. However, in addition to assisting native speakers, MaltAAC can be taken a step further to serve as linguistic first aid to immigrants arriving in Malta. In this scenario, the app can be used to facilitate their communication and support their integration into Maltese society. The MaltAAC project was sparked by the will to help the vulnerable in our society. From one group, children, we now realise that the app has the potential to help many more groups in our community. We hope to continue giving people the tools they need to help them connect with those around them.
Comments are closed for this article!