A View From Above

Satellite observation helps us understand our environment down on the ground. From tracking ocean currents to studying sinkholes, the applications are endless as long as the technology can keep up. Andrew Firbank speaks with Dr Ing. Reuben Farrugia and Dr Ing. Gianluca Valentino (Faculty of ICT, University of Malta), who are making sure that it does.

Sit outside tonight, and look up at the sky. Our planet is part of a cosmic stage, stunning in its complexity. It’s easy to picture our ancestors, long ago, appreciating the same scene. Only it wasn’t. These days, you’re more likely to spot a satellite flashing overhead than catch a shooting star. 

At some point in our journey, we graduated from looking up to looking down. This new field of science, Earth observation, uses satellites to study our planet’s surface. Satellite remote sensors can detect surface temperatures, identify materials, and even spot different plant species. Every year, more satellites are launched to better track and map the world. But their image resolutions are still lacking. Back on the ground, researchers must perfect techniques to clarify their findings. If satellites are our eyes in the sky, then researchers are the opticians.

Here, Dr Ing. Reuben Farrugia and Dr Ing. Gianluca Valentino, together with their team at the Faculty of ICT, University of Malta, found their niche. It started when Malta announced its first ever National Space Policy in 2017. ‘We immediately realised that we could apply our expertise to solve problems in satellite Earth observation,’ recalls Farrugia during our remote call. They are both younger than you’d expect, given what they’ve achieved. Amicable and sharp, they beam when we explore the countless applications of their work. 

Their first Earth observation project, SAT-FIRE, sprung from a chat with Dr Anthony Galea (Faculty of Science, University of Malta). He was creating Maltese marine current forecasts to aid local divers, search and rescue teams, and coastal monitoring. To do so, he needed frequent sea surface temperature maps from satellite remote sensors, but the images he could source had a resolution too low for Malta’s coastline. This was the first taste of a recurring problem in Maltese Earth observation. Most satellites can’t provide a resolution high enough for a country this small. Luckily, Farrugia and Valentino were uniquely positioned to solve this issue.  

‘Our team’s background is in computer vision, image processing, and machine learning,’ explains Valentino. ‘These are techniques to help exploit and obtain information from images.’ With low-resolution data stunting Earth observation in Malta, they saw how useful their skills would be. ‘I had worked with CCTV cameras to better identify faces,’ Farrugia adds, ‘I focused on de-noising images and increasing their resolution. There are lots of types of satellite sensors: radar, optical, thermal, etc., and we wanted to improve their resolution too.’

Farrugia and Valentino knew that if they could find one satellite capable of taking images precise enough for Malta, then they could use its data to enhance all the other low resolution satellites. For SAT-FIRE, the satellite they needed was Landsat 8. Orbiting our planet every 16 days, it was too infrequent for Galea’s marine current forecasts. But that was overcome. A team of researchers led by Farrugia and Valentino created a neural network that learned to replicate Landsat 8’s quality using daily low-resolution data from another satellite, Sentinel 3. This model was trained using hundreds of Landsat-8 and Sentinel-3 images, and now the neural network can improve resolutions by itself, producing exquisite images from dated data. Dr David Lloyd, a post-doc hired to work on the SAT-FIRE project, developed the deep learning model, while Aaron Abela worked on multi-resolution registration algorithms for his master’s thesis. In the validation of the hydrodynamical model used for the marine current forecasts, Dr Galea was aided by Ms Ioanna Thoma, who also did a master’s thesis on this topic.

Next came SATINET. For this project, our Maltese researchers teamed up with CNES, the French space agency, to map Earth’s surface in 3D. Working with their colleague Prof. Johann Briffa, they designed an algorithm that could combine multiple images taken by a satellite into one 3D picture. ‘You can capture a scene from two cameras at two different angles, where each camera is like an eye, and estimate the depth,’ explains Farrugia. ‘Satellites are the same. They take a picture of the earth, move a bit, take another, move a bit more, take another, and so on. Each picture is like adding another eye.’ They employed AI technology to eliminate any objects in motion, like cars or clouds, for a crisper image. The results are the highest accuracy 3D maps ever produced by a satellite. With this tech, CNES hopes to map 95% of the Earth’s surface in 3D by 2025. The CNES collaboration also allowed master’s student Tharen Abela to spend a 6-month internship at CESBIO, the Centre d’Etudes Spatiales de la Biosphere, to conduct research on forest cover mapping using satellite imagery.  

Farrugia, Valentino, and Briffa are still breaking barriers in Earth observation research today. Their WARM-EO project, like its predecessor SAT-FIRE, is refining low resolution satellite data to map land surface temperatures in Malta. Evaporation cools down fields, so the team, which also includes post-doctoral researcher Dr Mang Chen, hopes to estimate agricultural water usage in Malta to aid conservation efforts. Meanwhile, the Coastal-SAGE project, led by the same team, is underway to monitor and predict landslides on the Maltese coast. Satellite radar sensors can map cliff-side structures and show where landslides are most likely to happen next, helping prevent catastrophes.

Although created with specific goals in mind, the scope for all of these projects is limited only by our creativity. Another possible application is using SAT-FIRE’s temperature-tracking technology to spot illegal sewage dumping into the sea. You might not want to picture it, but sewage is noticeably warmer than seawater. 

Likewise, Coastal-SAGE has been designed to track landslides on Malta’s coast, but the same technology could be used to respond to several natural disasters. Before long, it could help map how volcanoes change as they erupt, or changes in the Earth’s surface following earthquakes. ‘This has been done for 20 or 30 years,’ Valentino clarifies, ‘but the resolutions were much poorer back then. For large events, this was not such an issue, but for small events, you need better resolutions and less noise.’ Their work produces the detail needed for localised events like sinkholes and ground fissures, and it will inevitably aid larger catastrophes too. This would boost emergency response systems. Safe access routes to victims could be sourced with precision from satellites to better guide ground exploration. 

Throughout their journey, our researchers have enhanced satellite data to account for Malta’s tiny size. Larger countries make good use of low resolution satellite mapping, but the Maltese islands require something more refined. Once this resolution barrier is broken, then Malta can also shine a brighter light on the rest of the world. There are plenty of other small nations and regions in need of this technology. Farrugia and Valentino’s team is stepping up to deliver. 

Project #1: SAT-FIRE 

Remote sensor: Temperature

The spark: Dr Anthony Galea (Department of Geosciences) was creating Maltese marine current forecasts. For this, he needed satellite remote sensors to map the surface temperatures of seawater. He had access to data from two satellites:

         Sentinel 3          Landsat 8

Earth orbit frequency (days) 2 – 3   16

Image resolution (per pixel) 1km 100m

For a timely forecast, you need satellites that orbit the Earth at a high frequency. Sentinel 3 seems ideal. But for a coastal region as small as Malta’s, a resolution of 1km per pixel can’t provide the detail needed. Sea would blur with land, distorting any results.

Neural networks: Valentino and Farrugia needed to create a model that used Landsat 8’s high-resolution data to sharpen Sentinel 3’s more frequent images. Here, Farrugia’s background came into play. From his experience in super-resolution, he knew it was possible to train a neural network to match low- and high-resolution images. Every 16 days, the two satellites capture the same scene at the same time, allowing a dataset of low- and high-resolution image pairs to be built. With practice, the neural network learnt to transform the low-quality images into high-resolution ones. SAT-FIRE was the first project in which this technique was attempted on satellite sea surface temperature data, and the results were excellent. 

Status: Complete.

Project #2: SATINET

Remote sensor: Optical

The spark: A trip to France. Valentino and Farrugia won a grant offered by CNES (Centre National d’Études Spatiales, the French space agency) and MCST (Malta Council of Science & Technology). Partly for networking and education, it was also to find a common goal in Earth observation. Arriving in Toulouse, the researchers discovered that CNES was hoping to map 90% of the Earth’s surface in 3D by 2025. They offered to help.  

A new dimension: Satellites can take multiple images of the Earth’s surface in quick succession, like the ‘burst’ mode on a camera. CNES wanted software that could combine these photographs into one 3D image. Luckily, Valentino, Farrugia, and their colleague Prof. Johann Briffa had experience working on this with traditional cameras. By capturing many images and weaving them together, a 3D map can be produced.  Not as simple as it sounds — whilst the satellite travels, the Earth’s surface isn’t static. They employed deep learning and AI technology to create a programme that could eliminate any objects in motion, e.g. vehicles and clouds. The results are the most advanced 3D maps ever produced by a satellite, accurate to within ± 1m 74% of the time. 

Status: Complete.

Project #3: WARM-EO

Remote sensor: Temperature

The spark: The Maltese Energy and Water Agency saw that NASA was using satellite remote temperature sensors to estimate water usage in US agriculture. Evaporation lowers temperature readings, so we can estimate the volume of water used in a given field or on a given crop.  With Malta’s arid climate and lack of fresh water, this knowledge would be invaluable for water conservation. Unfortunately, the resolution used by NASA’s satellites is 500 m. That may work well in the sprawling plains of Nebraska, but here in Malta, you could probably fit 10 farms into an area that size — plus a handful of buildings and a motorway.      

Ultra HD: Valentino, Farrugia, and Briffa found access to a satellite providing 10m resolution, but even that isn’t precise enough for Malta’s fields. They are now applying the same super-resolution technique they used for SAT-FIRE to see if they can achieve an even better image. To do this, they purchased a handful of images from a satellite capable of 3m resolution. If they can train a neural network to replicate that excellent resolution, then they could recreate it with the freely available 10m data. As well as lessening the project’s financial burden, it would be a major step towards accessibility and affordability in Earth observation around the globe.   

Status: On-going. 

Project #4: Coastal-SAGE

Remote sensor: Radar

The spark: Slow-moving coastal landslides: a Maltese phenomenon. The Public Works Department within the Ministry for Transport, Infrastructure and Capital Projects deals with the damage from rockfall and landslides in Malta. These events help define our coastline, but they endanger lives as well as infrastructure. Right now, the best way to tackle them is to respond to disasters once they have occured. Valentino, Farrugia, and Briffa are trying to help the Ministry to predict them, preventing any avoidable damage in the first place. Dr Sebastiano D’Amico and Emanuele Colica, geoscientists at the University of Malta, are helping them gather ground-truth data, such as Digital Elevation Models, using drones.  

Landslides: Satellite remote sensors can shoot down radar waves. When these waves hit an object on the Earth’s surface and bounce back, their characteristics (amplitude and phase) can reveal not only the placement of the object, but even what type of material it is. Successive measurements over time reveal information about changes, such as subsidence and uplifting, that occur to the Earth’s surface. This can show which cliffs are eroding quickest in Malta by tracking their movement downhill. It can reveal which soil patches are weakening, and therefore which boulder lying precariously might start rolling next. 

Status: On-going. 

Acknowledgments

Project SAT-FIRE is financed by the Malta Council for Science and Technology, for and on behalf of the Foundation for Science and Technology, through the Space Research Fund.

Project Coastal-SAGE is financed by the Malta Council for Science and Technology, for and on behalf of the Foundation for Science and Technology, through the Space Research Fund.

Project WARM-EO is financed by the Malta Council for Science and Technology, for and on behalf of the Foundation for Science and Technology, through the Space Research Fund.

Project SATINET is financed by the Malta Council of Science & Technology, for and on behalf of the Foundation for Science and Technology, through the MCST-CNES Space Bilateral Fund.