The way a human tracks motion is both extraordinary and inconspicuous. In both theatre and dance, a lighting graphics engineer designs visuals and lighting to match a performer’s movement. Their motions might delight the audience but are very complex for computers to detect and interpret. Despite continuous breakthroughs, there are still many issues to overcome when tracking the human body across a stage.

Michaela Spiteri (supervised by Alexandra Bonnici) developed a system that allows dancers to control light effects through the dancer’s own movement. Mapping the movement of humans has several problems. Dancers tend to be highly flexible and perform very refined movements. The complex movements sometimes obscure certain body parts, which rapidly appear again, confusing the computer. 

The computer tracked motion through a number of steps. First, it created a mathematical model of the background image. This technique allows the background and dancer to be separated in live video, leaving the dancer’s silhouette.

Secondly, the dancer’s silhouette was then thinned to a skeleton in order to obtain five points: head, hands and feet. A Kalman filter was applied, allowing the computer to continue to track motion even if a point was hidden. The Kalman filter predicts location by assessing past information and predicting where it would be in the future.

The study could stimulate new ways for artists to express their concepts. Additionally, the computer algorithm used can be applied in augmented reality, medicine and surveillance.

This research was performed as part of a Bachelor of Engineering (Honours) at the Faculty of Engineering.