If you were blown away by the futuristic possibilities of Microsoft’s next-generation Project Natal gesture-led gaming technology, then you will be suitably impressed with what a couple of enthusiastic and inventive students from Dublin Institute of Technology are doing with soundbeams and giant invisible keyboards.
Imaginote is the winning team in the Microsoft Imagine Cup 2010 Irish finals and its unique blend of audiovisual therapy with educational applications will be representing the country this year in the world final in Poland.
What the Imaginote duo Nikola Nevin and Marco Castorina have developed could potentially impact on children with physical disabilities, learning difficulties and emotional problems. It is already being trialled with children at the Tyrrelstown Educate Together National School in Fingal.
“Originally it was used only as a therapy tool but after the feedback from children and teachers at Tyrellstown we realised it had all of these extra educational applications,” says Nevin.
Body of music
The novel application is that it is like a giant invisible keyboard where the person plays notes with their body, and by moving around they can move up and down a musical scale.
This is adjustable so you can use it with full-body interaction, or for someone with a physical disability it can be calibrated to work with, say, hand movements only.
“It takes into account a lot more people than a traditional controller would,” explains Nevin.
Imaginote has used soundbeam technology by incorporating it into an audiovisual therapy tool. When someone stands close by a note will sound and animations will appear on-screen. It works by association:
“Sounds are always associated with certain visualisations to help children make connections. So a certain note would represent blue and it will condition you to respond to this automatically.
“This isn’t just going to play a keyboard; you could load guitar sounds into the program and the user is playing guitar simply by moving their body. Plus there is the accompanying visualisations. We have a rhythm game inside Imaginote where musical notes come towards you and you have to move up and down to catch them.
“The equal combination of visual and audio cues means that Imaginote makes it possible for a blind child and a deaf child to play the same rhythm game side by side,” she adds.
Imaginote is also applicable for children with learning difficulties. By substituting notes within the game with letters, words or numbers it can help children with dyslexia or dyscalculia.
Eye on expansion
Going forward, Nevin says Imaginote would like to work with Project Natal technology to expand upon the applications:
“The soundbeam technology we use is 2D-only whereas Project Natal picks up your body in 3D space so you could have so much more expression with it.”
Either way, Imaginote is a winning technology for education:
“Music is a universal language,” adds Nevin.
Pushing diversity
As part of the Imaginote team, Nikola Nevin is one of many young women participating in the Irish entries to the Imagine Cup this year.
Out of the 600 entries from Ireland’s universities, third-level colleges and institutes of technology, one third were female.
“Females who have participated in the Imagine Cup in previous years are getting active in terms of sponsoring or helping other young women coming along in the years after,” notes Clare Dillon, developer and Platform Group lead, Microsoft Ireland.
“Christina Luminea from IT Sligo was involved two years running and she’s been helping out the team there this year. The more women participating this year the more we’re likely to have next year,” says Dillon.
The gender balance is really an integrated issue into overall interest in technology and how cross-disciplinary projects are bringing a lot more colour and diversity into the competition.
“When teams have a range of people with different backgrounds – and not just pure computer science – you start seeing the really innovative approaches to real-world problems,” she adds.
