How one researcher is using VR to find how we ‘hear’ a virtual world

30 Sep 2020

Adrielle Nazar Moraes is a researcher at Athlone IT and Adapt. Image: Adrielle Nazar Moraes

Adrielle Nazar Moraes of Athlone IT and Adapt is exploring the VR world through sound and finding out how it could help in auditory testing.

Adrielle Nazar Moraes holds a BSc in biomedical engineering from the Federal University of Uberlandia (UFU) in Brazil.

In 2018, she was awarded an MSc scholarship at the Athlone Institute of Technology (AIT), funded by Science Foundation Ireland and the Adapt research centre for digital media technology. Her research aims to understand the ability of users to specify or localise the ‘source’ of sound in a virtual environment.

‘My research aims to explore other use cases of VR, going beyond traditional entertainment’

What inspired you to become a researcher?

I started in 2015 as a research assistant in the biomedical laboratory in UFU. During this time, I was in contact with other MSc and PhD students and I believe that this research environment encouraged me to keep on the research path.

My first project looked at the development of a system to capture signals from the muscle to perform a prosthesis control training in virtual reality (VR).

I had the opportunity to meet an amputee and learn his history of how he lost part of his arm. What shocked me was the fact that during one conversation with him, he told me the reason people in this condition abandon more sophisticated prosthesis is because of their complexity.

This event triggered in me a feeling of needing to change the situation and I realised I could help other people doing research. It felt amazing when I saw him using that system to help him and other people with the same condition.

Can you tell us about the research you’re currently working on?

I always dreamt about doing research abroad to change my perspectives and encounter other realities. Then, one day while googling, I came across AIT and my current research theme.

My work started with the development of a VR environment to present the visual information for the user. Since I work with multimedia applications, it is important to consider the user’s quality of experience.

A very common event related to these experiences is named ‘cybersickness’, when what you see doesn’t match with what your body is doing. For this reason, it is important to accurately simulate the virtual-world physics to match what a real experience would be like.

This is extremely challenging for audio because of the way we perceive sounds in space. Each person has a mathematical function to describe how they perceive sound.

During my experiments, I try to understand the interaction of the user with the system in a scenario with two interaction modalities to control the virtual environment. We collect a range of different types of data from the user as they experience VR. This data is used to give insight into the listener’s behaviour, and it is used to understand how people perceive sounds in VR given different listening conditions.

In your opinion, why is your research important?

I believe that there is a constant need to evaluate and create methodologies to measure the impact of technology on people’s lives.

My research aims to explore other use cases of VR, going beyond traditional entertainment. The gold standard test for spatial hearing disorders consists of three listening conditions, and sounds presented through headphones. The sound can be positioned to the left, right or in front of the patient.

However, our brain discriminates sounds within a range of 15 degrees. My experiment consists of 24 possible sound source positions, instead of the classical three from the standard test. As the user progresses through the task, it becomes more challenging, with distractors to confuse the listener.

During the task, physiological data (heart rate, electrodermal activity, temperature) is collected from the user, aiming to understand their level of engagement and immersion with the system. Using the framework I developed, it is possible to increase the complexity of the test and extract information related to performance, correlating it with traditional raw data.

What are some of the biggest challenges you face as a researcher in your field?

As a researcher, I always remember a quote from Edward Osborne Wilson: “We are drowning in information while starving for wisdom.”

For me it has a strong impact especially for research, meaning that in a world with internet access and many sources of information, researchers must commit with their research to disseminate correct and reliable information to people.

Another challenge to consider is the individuality of each person. For example, people perceive sounds according to the shape of their ears or head.

Moreover, we are ‘trained’ to distinguish phonemes we are used to hearing. If I am a native English speaker, it will be extremely hard for me to differentiate phonemes in French if I am not familiar with this language. On the other hand, our brain is extremely efficient in learning, meaning that with the correct training we can overcome these issues.

Are there any common misconceptions about this area of research?

When working with technology, we need to be open-minded about new tools and the learning curve of each person. I believe the biggest challenge for people that are going to try new technology is acceptance. They need to be willing to learn something new, which takes them out from their own comfort zone.

The best way to address this problem is to have detailed information made available to everyone included in the process. My research is in the quality of experience domain, so I aim to mitigate issues related to the learning curve of an application.

During my experiments, I always explain to participants how the application works and what they are taking part in. The objective is to make them feel comfortable with the application and prepare them to perform the required task.

What are some of the areas of research you’d like to see tackled in the years ahead?

I like VR because of the abstraction possibilities. It is easy to transport people from one reality to another. This advantage covers multiple areas, not only the health domain, but also autonomous vehicles, games and entertainment.

For being so complex, I believe we can explore much more from VR than we can imagine. That is one of the reasons I believe in this research area.

Are you a researcher with an interesting project to share? Let us know by emailing with the subject line ‘Science Uncovered’.