TAMED is a University of Malta project developing ‘affective computing’ in video games, which would enable them to interpret human emotions. Jonathan Firbank goes into detail with Dr Konstantinos Makantasis.Continue reading
If you had a rich malleable canvas that could flip rules on their heads and expose truths we take for granted, wouldn’t you use it? Jasper Schellekens writes about the games delving deep into some of our most challenging philosophical questions.
The famous Chinese philosopher Confucius once said, ‘I hear and I forget. I see and I remember. I do and I understand.’ Confucius would have likely been a miserable mystic in modern mainstream education which demands that students sit and listen to teachers. But it’s not all bad. Technological advancements have brought us something Confucius could never have dreamed of: digital worlds.
A digital world offers interaction within the boundaries of a created environment. It allows you to do things, even if the ‘thing’ amounts to little more than pressing a key. Research at the Institute of Digital Games (IDG) focuses on developing a deeper understanding of how these concepts can be used to teach through doing by looking at people interact with gameworlds, studying how games can impact them (Issue 24), and designing games that do exactly that.
Doing it digital
Two millennia later, John Dewey, one of the most prominent American scholars of the 20th century, proposed an educational reform that focused on learning through doing and reflection instead of the ‘factory model’ that was the norm. Dewey’s idea was embraced, and has become a pedagogical tool in many classrooms, now known as experiential learning.
Let’s not pretend that Confucius was thousands of years ahead of his time—after all, apprenticeships have always been an extremely common form of learning. But what if we were to transplant this method of experimentation, trial and error, into a digital world?
It would allow us to do so much! And we’re talking about more than figuring out how to plug in to Assassin’s Creed’s tesseract or getting the hang of swinging through New York City as Spiderman. While these are valuable skills you don’t want to ignore, what we’re really interested in here are virtual laboratories, space simulations, and interactive thought experiments.
Games make an ideal vehicle for experiential learning precisely because they provide a safe and relatively inexpensive digital world for students to learn from.
Think of the value of a flight simulator to train pilots. The IDG applied the same idea to create a virtual chemistry lab for the Envisage Project. They threw in the pedagogical power tools of fun and competition to create what’s known as serious games.
Serious games are at the heart of many of the IDG’s research projects. eCrisis uses games for social inclusion and teaching empathy. iLearn facilitates the learning process for children with dyslexia and Curio is developing a teaching toolkit to foster curiosity. However, the persuasive power of videogames stretches further than we might think.
In a videogame world, players take intentional actions based on the rules set by the creators. These ‘rules’ are also referred to as ‘game mechanisms’. Through these rules, and experiential learning, players can learn to think in a certain, often conventional, way.
Which brings us to HERE.
Prof. Stefano Gualeni is fond of using games to criticise conventions: in Necessary Evil a player takes on the role of an NPC (Non Player Character) monster, in Something Something Soup Something the definition of soup is questioned, while in HERE Gualeni breaks down what ‘here’ means in a digital world.
HERE sees the player explore the philosophical concept of ‘indexicality’, the idea that meanings depend on the context in which they occur. A fitting example is the extended index finger, which means different things depending on where it is placed and what movement it makes. Point one way or another to indicate direction, place over the lips to request silence, or shake it from side to side to deny or scold.
The game explores the word ‘here’ in the digital world. It sheds light on how much we take for granted, and how a lot of concepts are not as straightforward as we think.
HERE you play as ‘Wessel the Adventurer’, a cat of acute perception that is sent on a quest by a wizard to find magic symbols and open an enchanted cave. Playing on the tropes of role-playing games, the expectations of the adventurer are thus framed in a conventional manner, but not everything is as it seems.
By subverting players’ expectations of role-playing games, they will have the opportunity to discover what they have been (perhaps unwittingly) taught. They will be confronted with a puzzle involving the many versions of ‘here’ that can co-exist in a digital world. Among their prizes is Gualeni himself performing a philosophical rap.
Experiential learning isn’t the only way to learn, but video games, with their interactivity and ability to manipulate the gameworld’s rules with ease, offer a ripe environment for it. The digital realm adds a very malleable layer of possibility for learning through doing and interacting with philosophical concepts. HERE is not alone in this approach.
Words often fall short of the concepts they are trying to convey. How do you explain why people trust each other when there are so many opportunities to betray that trust? Telling people they have cognitive biases is not as effective as showing them acting on those biases.
Explorable Explanations is a collection of games curated by award-winning game developer Nicky Case that dig into these concepts through play. The Evolution of Trust is one of them, breaking down the complex psychological and social phenomena contributing to the seemingly simple concept of trust in society. Adventures in Cognitive Biases is able to show us how we are biased even when we don’t think we are. HERE delves into our understanding of language and the world around us, showing us (instead of telling us) that learning doesn’t have to be boring. Now go learn something and play HERE.
To try the game yourself visit www.here.gua-le-ni.com
Picture an item of furniture. Was it a table, a chair, or a wardrobe? Our ideas of furniture are not oriented around what it does, or even its essential nature, but rather around all the common examples we see around us and the cognitive web of interrelations that builds in our head.
This fictional furniture is just an analogy. Supervised by Prof. Gordon Calleja, I investigated how language affects our preconceptions of what games are and what they should do. Our ideas about games are not related to their potential or their openness but to constructions in our mind.
My research was mostly within the philosophy of language. I explored how our ideas of what a thing is are not grounded in what it does or is, but around a cognitive image, created by what we often see and label it as.
My research was mostly theoretical. It showed how other game researchers might have been misguided or had counter-intuitive results. Their research tried to define what games are and did not realise that this went against how we used the word ‘games’, i.e. as a container for all the things we collectively consider game-like.
That said, I also had the opportunity to test-drive this work in a game I am currently developing with Dr Stefano Gualeni, which analyses what the word soup means.
Apart from answering the long-standing question of what a game actually is, this research also shows how games would benefit from being more inclusive and experimental. By creating these types of games, we can attract people whose cognitive model of a game is different from the norm. Similarly, by designing games that are more open, we can stretch the web of interconnections in people’s minds, potentially showing how games are more akin to life than we realise.
Are you ready to play?
This research was carried out as part of the Masters in Digital Games, carried out by the Institute of Digital Games at the University of Malta.
Joseph Aldape met indie game developer Rami Ismail to chat about indie gaming communities.
Indie games are seemingly unstoppable. As mainstream blockbuster AAA games stutter, new niches are opening up with nearly half of gamers being female and mobile revenue increasing rapidly. In Malta, an important piece in the indie game developer puzzle is the Malta Global Game Jam, which brings coders, designers, artists, writers, and other creatives together to create a game from scratch in just 48 hours.
Run in Malta by the Institute of Digital Games since 2013, the yearly event has grown considerably since its inception, pulling an international crowd from all over Europe. The January event this year included London-based games and pop-culture writer Philippa Warr and Milan-based indie design duo We are Müesli.
After keynotes and workshops to hone participants’ skills, 14 different games were created. The worldwide theme was ‘ritual’. In third place was the create-your-own-god game, Godowbows, and the self-explanatory non-fun game, IKEA supply assistant. In second place the beautifully designed The Passage immersed players into an ancient temple’s rite of passage. Hashtag Master Race won the local event with a game about angels and demons. Internationally over 28,000 people participated.
Apart from a fun weekend, the event is an opportunity for one to practice and learn skills, to build networks and, in a few cases, build promising new IP (Intellectual Property). Participants form a small indie development team every year. Back in the 2013 Malta Global Games Jam, the game And Then We Held Hands saw success and as it was distributed internationally following a $60,000 Kickstarter campaign. The experience can be used to help those already in the industry, or for those wishing to enter the industry, gain confidence to make more indie games or for them to join a big company with proven experience.
Intense events like this play a vital role in the building of a local game development scene that can soon see Malta join its international peers in producing top-notch, international and lucrative games.
Gone are the days when learning a new language meant sitting in a classroom, reading books and practising with classmates. There are language learning apps, programs and games now. But can we really learn a new language just by using this technology? Veronica Stivala finds out. Illustrations by NO MAD.
How do you help school children handle fights, bullying, and other conflict properly? You build a game, of course, and you let children take on different roles in a village. But how does that lead to resolving conflicts? Ashley Davis met researchers Prof. Rilla Khaled and Prof. Georgios N. Yannakakis to find out more
Do you chuckle at the thought of a serious game? The phrase is an oxymoron. How can a game be serious? Games are meant to be fun, frivolous, a way to pass the time. Or else you sometimes hear that games are anything but frivolous. That video game violence in particular is a threat to social order. The idea that games can be used to advance human understanding about the world, and that they can help us to teach, train, or motivate people in some way, is something that still needs to enter our mentality.
Designing games to explore research questions and to solve real world problems is actually a very important aspect of games research, an area of applied research that now has a strong presence at the University of Malta with the establishment of the Institute of Digital Games. Researchers from the Institute work on European-funded projects to create games that tackle serious problems affecting children and adults alike.
Village Voices has been voted the best learning game in Europe at the 2013 Serious Game Awards
Prof. Rilla Khaled and Prof. Georgios N. Yannakakis are two researchers now based at the Institute of Digital Games who work on serious game projects. Khaled’s work focuses on serious game design, while Yannakakis is a specialist in artificial intelligence and computational creativity. Computational creativity tries to build upon the latest technological innovations in human–computer interaction that enable computers to act intelligently to some aspects of human beings. These two areas, game design and game technology, represent a large part of the teaching and research strengths of the Institute.
One game that Khaled and Yannakakis recently helped develop is Village Voices which has been voted the best learning game in Europe at the 2013 Serious Game Awards. It was developed as part of the SIREN project, an FP7-funded interdisciplinary consortium made up of researchers from Malta, Greece, Denmark, Portugal, the UK and the US, along with Serious Games Interactive, a Danish Games Studio.
Let’s take a look at what makes a serious game and think about what made the project a success and what didn’t work so well.
The serious side of Village Voices aims to help school children learn conflict resolution skills. Players take on the role of one of four interdependent villages that are situated in a farm setting and given various quests to complete. Sitting side-by-side at separate computers, they may collaborate, share resources and help each other, or they may spread rumours and steal from each other. Much like any playground setting, children can play nicely, or they can be bullies.
The purpose of the SIREN project was to apply the latest advancements in game technology to the creation of serious games. The brief focused on innovations in procedural content generation, an area of artificial intelligence that automatically builds game elements like game levels or quest structures that would otherwise need to be designed manually. Another part of this innovative technology is detecting the emotions of players. Physiological responses can be measure through various tech like Electroencephalographic (EEG) sensors that can be used to detect a person’s emotional state directly by reading their brain’s electrical signals. Virtual agents were another technology that interested the research team. These agents are believable non-player characters that interact with the player with perceived intelligence.
The idea was to then create a game that would adapt to player behaviour, using emotion recognition tools to create an individual experience for each player. The decision to focus the game on teaching children about conflict resolution came later. Rather than to create a game about bullying behaviour, which is what a lot of people think of when they picture conflict between children, the research team wanted to explore the kinds of everyday conflicts that take place in school-yards. Friendship disputes, differences in opinion, and arguments over the possession of classroom items might seem trivial to adults, but they are important problems for children for whom school is their entire world. The SIREN consortium envisioned a game where players could experience and resolve conflicts in a dynamic setting.
Some people who make serious games say that the serious application of the game should take precedence over fun. They say that serious games should offer players a safe environment to try out new behaviours. Khaled disagreed with this approach to game design. ‘Serious game experiences need to feel real and not trivial. Otherwise why would we then use them to raise a mirror to reality?’
Village Voices allows actions that teachers might find surprising. Players can be destructive in that world. They can steal from each other. The game gives aggressive players a noose with which to hang themselves. Knowing that the person whose labours you just destroyed, or who stole the items you were collecting, is sitting right there next to you intensifies the game’s emotional experience. Exchanges can become heated between players. It is these kinds of heated exchanges that often makes games fun.
Games are usually poor at provoking emotional responses. Village Voices does exactly that. Khaled told me about one session in a British classroom (the game was tested across Europe). A female student had such an upsetting experience that she cried. After reflecting on the incident with her teacher, the researcher, and the other players, the girl later returned to play again. Khaled thought this was a breakthrough learning moment for the student.
So Village Voices is a good learning tool, and it is also fun to play. But how successful was the team in applying game technologies like procedural content generation and emotion detection to its design? Khaled said that the experience of designing a game primarily for the purpose of testing technological innovations was the hardest part of the project. You might think that the role of a game designer is to work out the best solution to a problem given the technologies at hand. However, when the application of technology is the problem, the relationship between design and technology is more complex. Khaled said that the need to include particular game technologies in the design of Village Voices created a situation much like a rock band that needs to accommodate a peripheral member, such as a violin player. ‘While the violin player is not core to the project, the whole project needs to be compromised in some way in order to show off the violin player’s skills. It is not clear that the violinist is going to help the band make a new hit song, but it is clear he has to be there. So the band tries to find the violin player’s most positive qualities because he has got to be there.’
In Village Voices, the violin player’s best qualities are adaptive technologies that make the player experience more personalised. Because support for emotion detection plug-ins was never actually included in the final prototype, the game instead asks players directly how they feel about events in the game and introduces variations to the player experience according to their responses.
So far we have seen that Village Voices was successful according to the popular opinion of game-design peers at the European Serious Games — it won an award. We have also seen anecdotally that it is a provocative, if not fun game, based on the British student’s emotional response. But what does the SIREN team think about the game?
You cannot sit a child down in front of a computer and hope that they will magically learn something
According to Khaled, it can be difficult to implement learning games in classroom settings, and even more difficult to properly evaluate them. Project funding usually runs dry after around three years, and games take most of that time to develop. Gaining access to schools is also difficult. The game is a good fit for classes like social studies that are often held only once or twice a week. Together with the problem of semester breaks and short evaluation periods, as well as the tendency for teachers to have access to only a few computers often equipped with obsolete hardware, researchers would rarely see students engage with Village Voices over a long period of time. All these things place limitations on the design, testing, and evaluation of games for research purposes.
Rigorous evaluation is important as, ultimately, learning games are not black box tools. You cannot sit a child down in front of a computer and hope that they will magically learn something. That vital learning moment comes when players discuss their in-game experiences. As Khaled explained, ‘Playing the game is just half the experience. The other half is the subsequent discussion of the game experience.’
Given that discussion is so essential to the evaluation process, and that it is so difficult to get a sample of those discussions in a research setting, I asked Khaled if it was possible to turn the discussion into a game as well, to include it as part of the package. Khaled mentioned the meta-game, the part of the game where a player is both playing and watching themselves play the game. It is in the meta-game that players achieve the highest level of reflection. It works well as a kind of after-game discussion, a debriefing for players as they leave behind the conflicts of the game world and return to the everyday life of the school-yard; but Khaled added that of course it could be turned into a game. Achieving this level of reflection in the game package itself is just another challenge for the designers of serious games.
The Institute of Digital Games at the University of Malta offers word-class postgraduate education and research in game studies, design, and technology. The inter-disciplinary team includes researchers from literature and media studies, design, computer science and human-computer interaction. Visit game.edu.mt or contact Ashley Davis (firstname.lastname@example.org) for information about the Institute’s Master of Science (taught or by research) and Ph.D. programmes. This article forms part of The Gaming Issue.
Find out more:
Cheong, Y-G., Khaled, R., Yannakakis, G., Campos, J., Paiva, A., Martinho, C., Ingram, G. A Computational Approach Towards Conflict Resolution for Serious Games (full paper). In Proceedings of the 6th International Conference on Foundations of Digital Games, 2011.
Khaled, R. and Ingram, G. Tales from the Front Lines of a Large-Scale Serious Game Project (full paper). In the Proceedings of CHI ’12, 2012.
Vasalou, A. and Khaled, R. Designing from the Sidelines: Design in a Technology-Centered Serious Game Project. In the Proceedings of the CHI Workshop Let’s talk about Failures: Why was the Game for Children not a Success? CHI ’13, 2013.
Some SIREN Gameplay Shots
How can a video game ask questions about life, art, and frustration? Giuliana Barbaro-Sant met up with Dr Pippin Barr to tell us about his game adaptation of Marina Abramović‘s artwork The Artist is Present.
In each creative act, a personal price is paid. When the project you have been working on so hard falls to pieces because of funding, it is hard to accept its demise. The feeling of failure, betrayal, and loneliness is an easy trap to fall into. This is the independent game maker’s industry: a bloodthirsty world rife with competition, sucking pockets dry from the very beginning of the creative process.
Maltese game makers face a harsher reality. Not all game makers are lucky enough to make it to the finish line, publish, and make good money. Rather, most of them rarely do. Yet, if and when they get there, it is often thanks to the passion and dedication they put into their creation — together with the continuous support of others.
Dr Pippin Barr always had a passion for making things, be it playing with blocks or doodling. His time lecturing at the Center for Computer Game Research at the IT University of Copenhagen, together with his recent team-up with the newly opened Institute of Digital Games at the University of Malta, only served to reincarnate another form of this passion: Pippin makes games. At the Museum of Modern Art in New York he exhibited his most well known work: the game rendition of Marina Abramović’s The Artist is Present. He thought of the idea while planning to deliver lectures about how artists invoke emotions through laborious means in their artworks. In The Artist is Present, artist Marina Abramović sits still in front of hundreds of thousands of people and just stares into their eyes for as long as participants desire.
There is more to this performance than meets the eye. Beyond the simplistic façade, Barr saw real depth. Through eye contact, the artist and audience forge a unique connection. All barriers drop, and human emotion flows with a great rawness that games are so ‘awful’ at embodying. Yet, paradoxically, there is a militariness in the preparation behind the performance that games embrace only too well. Not only does the artist have to physically programme herself to withstand over 700 hours’ worth of performing, but the audience also prepares for the experience in their own way, by disciplining themselves as they patiently wait for their turn.
“It’s a pretty lonely road and it can be tough when you’re stuck with yourself”
‘Good research is, after all, creative,’ according to Pippin Barr. By combining his academic background with his creative impulse, he made an art game — a marriage between art and video games. These are games about games, which test their values and limits. Barr relishes the very idea of questioning the way things work. His self-reflexive games serve as a platform for him to call into question life’s so-called certainties, in a way that is powerful enough to strike a chord in both himself and the player. He is looking to create a deep emotional resonance, which gives the player a chance to ‘get’ the game through a unique personal experience. Sometimes, players write about his games and capture what Pippin Barr was thinking about, as he put it, ‘better than I could myself’, or read deeper than his own thoughts.
As far as gameplay goes, The Artist is Present is fairly easy to manoeuvre in. The look is fully pixellated yet captures the ambience at the Museum. The first screen of the game places the player in front of its doors and you are only allowed in if you are playing the game during the actual exhibition’s opening hours in America. Until then, there is no option but to wait till around 4:30 pm our time (GMT+1). The frustration continues increasing since after entering you will still have to wait behind a long queue of strangers to experience the performance work. This reflects real world participants who had to wait to experience The Artist is Present. If they were lucky, they sat in front of the artist and gazed at her for as long as they wanted.
Interestingly, Marina Abramović also played the game. She told Barr about how she was kicked out of the queue when she tried to catch a quick lunch in the real world as she was queuing in the digital one. Very unlucky, but the trick is to keep the game tab open. Other than that, good luck!
Despite that little hiccup, Abramović did not give up on the concept of digitalising the experience of her art. After The Artist is Present, Barr and Abramović set forth on a new quest: the making of the Digital Marina Abramović Institute. Released last October, it has proven to be a great challenge for those who cannot help but switch windows to check up on their Facebook notifications – not only are the instructions in a scrolling marquee, but you have to keep pressing the Shift button on your keyboard to prove you are awake and aware of what is happening in the game. It is the same kind of awareness that is expected out of the physical experience of the real-life Institute.
The quirkiness of Barr’s games reflects their creator. Besides The Artist is Present, in Let’s Play: Ancient Greek Punishment, he adapted a Greek Sisyphus myth to experiment with the frustration of not being rewarded. In Mumble Indie Bungle, he toyed with the cultural background of indie game bundles by creating ‘terrible’ versions with ‘misheard titles’ (and so, ‘misheard’ game concepts) of renowned indie games. One of his 2013 projects involves the creation of an iPhone game, called Snek, an adaptation of the good old Nokia 3310 Snake. In his version, Pippin Barr turned the effect of the smooth ‘naturally’ perfect touch interface of the device upon its head, by using the gyroscope feature. Instead, the interaction with the Apple device becomes thoroughly awkward, as the player has to move around very unnaturally because of the requirements of the game.
This dedicated passion for challenging boundaries ultimately drives creators and artists alike to step out of their comfort zone and make things. These things challenge the way society thinks and its value systems. Game making is no exception, especially for independent developers. An artist yearns for the satisfaction that comes with following a creative impulse and succeeding. In Barr’s case, being ‘part of the movement to expand game boundaries and show players (and ourselves) that the possibilities for what might be “allowed” in games is extremely broad.’
Accomplishing so much, against the culture industry’s odds, is a great triumph for most indie developers. For Pippin Barr, the real moment of success is when the game is finished and is being played. Then he knows that someone sat with the game and actually had an experience — maybe even ‘got it’.