southampton

Published:: 11 December 2018

Music is ubiquitous in all cultures and plays an important role in people’s wellbeing and quality of life. At Southampton, researchers are helping those with severe to profound hearing loss to perceive music through their cochlear implant using an online tool.

Music is all around us; it’s a fundamental part of the world,” says Dr Ben Oliver , Associate Professor in Composition, at the University. “Whether you’re at a wedding, funeral, listening to birdsong, or even in a lift, you hear music,” he adds.

Cochlear implant limitations

While cochlear implants (CI) can help those with severe to profound hearing loss to perceive speech, many CI users are unable to enjoy music through their implant. “This because the devices don’t convey pitch information very well,” says Dr Rachel van Besouw, former Lecturer and now Visiting Research Fellow in the Institute of Sound and Vibration Research at Southampton.

Rachel explains that a cochlear implant has an array of small electrodes that are inserted into different regions of the cochlea (part of the inner ear) to give pitch information. “Up to 22 electrodes are doing the job of thousands of hair cells that would normally be in the cochlea, so frequency resolution is poor. Pitch and the ability to recognise the timbre of sounds is compromised,” says Rachel. “Being able to recognise voices and instruments is a big limitation of the device.”

The case is also different for CI users who were able to hear in the past. “They may remember what pieces of music used to sound like. With a CI, the same pieces of music will likely sound different and perhaps even quite horrible, which can lead to a sense of loss,” says Ben.

Joining forces

The interdisciplinary team from across the University joined forces to create ‘Compositions for Cochlear Implantees’, an initiative to help CI users to appreciate music again. The team includes hearing scientists and audiologists, musicians, a composer and music therapist from the University’s Institute of Sound and Vibration Research, Auditory Implant Service and Music department. They work with external bodies such as the National Cochlear Implant User Association and the Ear Foundation. Major music artists such as Cliff Richard, Philip Selway from Radiohead and 10cc have donated songs to the initiative.

Rachel, Ben and the team consulted with CI users about the development of a music rehabilitation resource to find out what people would find useful.

Through a series of workshops we gained a better understanding of the music styles and structures that can be appreciated by people with a CI.

Interactive experience

The workshops informed the composition of seven original musical works by Ben for CI users, which were performed by an ensemble of professional and outstanding student musicians at a special concert in 2012. “The seven pieces of music were bespoke for CI users; instruments such as the harp were used due to the crisp beginning of notes and a lot of the sounds were short, rather than sustained notes,” explains Ben. “The compositions worked in tandem with visual cues such as video and live audience participation, in order to help users stay engaged and keep time with the music.”

The new insights from the workshops and concert were also used to guide the development of a computer-based music rehabilitation programme called the Interactive Music Awareness Programme (IMAP). The aim of this digital toolkit of interactive software is to help CI users to distinguish, recognise and appreciate different musical sounds. “The programme includes exercises where a CI user could change the levels of different instruments in a piece of music to learn the individual sounds of instruments again,” says Ben.

Over the life of the project we have found that users, with limited perception of music at the beginning, can now identify specific instruments and start to appreciate the music again. For a CI user, practise is key for improving music perception.

Online toolkit

The researchers then developed the IMAP into a free online platform, which all CI users can benefit from. Richard Polfreman , Associate Professor of Music and expert on the interface between music and technology, redesigned the IMAP software, enhancing some of the features so that it could be a downloadable system. “Users can now go to www.morefrommusic.org , download software and carry out exercises on a regular basis so that they can retrain themselves to understand and start enjoying the music,” says Richard.

“We now have over 250 CI users and nearly 200 professionals who have registered to use the IMAP resource, and the exercises are also used in workshops in the Auditory Implant Service,” says Ben.

In 2018, as part of the CHOICE-cochlear implant home care project, funded by the Health Foundation, collaborators from the Auditory Implant Service, Music and Electronics and Computer Science at the University developed the mixer app from the IMAP software into an easily accessible web app.

‘More From Music for CHOICE’ now also includes a way for people to save their favourite music mixes and the team is hoping to add more tracks in the future. As technology has evolved, it has become possible to change IMAP into a web-based application, which was not feasible even just a few years ago.

“Fundamentally this project is about the importance of music, the perception of music by CI users, and how quality of life can be increased by learning to enjoy music again,” says Richard. “We are using music to benefit others in a direct way – helping people who have lost something from their lives to get that back – it’s very rewarding work,” he adds.

Watch the 'Compositions for Cochlear Implantees' concert