Better living through vibrations

Sound design for applications in society

HKU Music and Technology is about designing music and sound for diverse purposes: movies, theatre performances, commercials, games, and all sorts of devices – from vacuum cleaners to communication tools for people who have trouble hearing or seeing. Lecturing researcher Gerard van Wolferen has been working on sound applications for people with a disability for years. Together with students and healthcare institutions, he creates solutions for the auditory or visual impaired by mixing art and technology.



foto: Mats van Soolingen
A friend of Gerard’s son was once forced by a physical condition to follow another schooling method than his classmates, now twenty years ago. Van Wolferen decided to scan the textbook to make it accessible to him anyway, so he could still join the rest of his class. This may not have much to do with the musical aspect of Music and Technology – the school Van Wolferen has been teaching and lecturing at for about 25 years. But it does display how he combines his artistic approach with technological knowledge. In his own words: ‘It is about being able to empathise with another while using your skills and imagination to enhance their quality of life, without thinking of a revenue model. This is what separates HKU’s designers from the research & development departments of commercial companies. We are not looking to sell our technology, but want to create something of actual benefit for an individual user.’

Heartbeat

In the many examples he gives, the name Bartiméus pops up frequently. This institution for the blind and partially sighted has become a solid partner throughout the years. It all started with the design of the audio game ‘Sneller’ (‘Faster’) by several HKU interns. In this shooting game, set in space, the blind can win from those who can see. The partnership grew from here. When Van Wolferen went to Bartiméus to show his communication concept for the deafblind, the timing appeared to be perfect. ‘They were just struggling with a deafblind client who injured himself to gain the attention of caretakers. With various cohorts of students, we therefore developed a portable device called ‘Heartbeat’. It uses vibrations to indicate distance between you and other people, and you can draw their attention by simply clicking a button. It worked: the client stopped self-harming. Eventually, Heartbeat was produced in a limited range. Van Wolferen thinks it would have been a great start-up for students: a Heartbeat production company. 'We proved that we were not just a bunch of creatives with pointless experiments, and won the trust of Bartiméus. It still created great opportunities for our students as well.’

Co-design with users

A more recent example, on a much larger and international scale, is ASSAM (Assistants for Safe Mobility): a European project of ten partners from Germany, Spain and the Netherlands. Together they work on modular navigation tools that allow the elderly, despite their declining abilities, to remain mobile for as long as possible. The project offers adjusted rollators, electric wheelchairs and three-wheel bikes with a special navigation system that safely guides the user from A to B, avoids obstacles and directly contacts a helpdesk assistant when necessary. ASSAM’s design method is iterative and user-centred; a perfect fit with the Dutch participants from HKU and Bartiméus. They contributed by designing and carrying out the user tests. ‘With the help of our software developer, we created a unique test system that takes every variable into account. The elderly users go out on a walk with their own care assistants, while they use our tablets to film what happens and register the individual user experiences. This differentiated data is then fed back, through a HKU server, to the producers of the walking aids and navigation systems in Bremen and Barcelona.’

Thinking in reverse

These are just a few of the many applications and partnerships. Other projects are the design of a responsive room for young people with autism, together with the University of Portsmouth and Pompeu Fabra in Barcelona. Or a vibration vest that allows deaf people to enjoy the dance floor, with TNO (the Dutch organization for applied physics). Also remarkable is the diagnostic music instrument, developed with UMC medical centre, in response to a question from a music therapist and child psychiatrist. They wondered if the small acoustic drum that they use for stimulus processing with autistic children, could also serve as a diagnostic instrument. The assignment was included in the hardware classes for second-year HKU students. This led to the development of a digital version; the Bela Drum, that could not only produce vibrations, but register and analyse them as well. ‘Yet such a direct request for a specific application is exceptional’, Van Wolferen adds. ‘In most long-term partnerships, we mainly signal needs that we must solve through discussion. It’s what we teach our students as well: to respond to the social needs around you. We are not scientists or engineers; our work is much more intuitive. We can think out the whole process in reverse, or “out of the box”. And that can lead to valuable results.’


Cooperating partners

HKU, Bartiméus, UMCU, TNO,
Mediate: University of Portsmouth, Universitat Pompeu Fabra;
Vibration vest: TNO, ByWire;
ASSAM: Deutsches Forschungszentrum für Künstliche Intelligenz, Universitat Politécnica de Catalunya, Centre de vida independent, Johanniter-Unfall-Hilfe e.V.

Lecturing researchers and HKU artists

Gerard van Wolferen, Hans Timmermans, Hugo Verweij, Marc Groenewegen, Maurits Lamers, Marina Toeters

HKU support

David Crombie, Jan IJzermans