Tool for Deaf and Hearing People
Rhine-Waal University of Applied Sciences
2017
A system on HoloLens that bridge two way communication between Deaf and hearing people by translating from spoken to sign language and provide sign options to reply back. Moreover, it also helps the user learning new vocabularies in both spoken and sign language.
Communication Mode
The system translates spoken to sign language and provide a preset of appropriate replies according to the input and speak it out via the speaker of the HoloLens.
Learning Mode
The system detects an object by image recognition and provides the vocabulary in both sign and spoken language. Also, to make sure the user is learning the right word, the system provides similar images from Google Search.
The Process
Empathizing User
Understanding the context, culture, needs and pains of Deaf/hard of hearing people
It was a great challenge in understanding an extreme type of users like Deaf people because of lack in accessibility and a sign language translator was always required. Majority rejected to collaborate but fortunately, a team member's personal network is familiar with Deaf culture and fluent in ASL. So, we could interview 4 Deaf people from Canada via Skype. Then we came up with a persona, Benny.
Define & Design
It was validated that there is a need in communication for Deaf people as the way sign and speaking/written languages work differently. The main findings were the need in translating, as well as learning tool for both Deaf and hearing people.
​
So, we came up with a tool that can be used by Deaf and hearing people to learn both sign and speaking/written language together.
Prototyping
We later brainstorm and generated sketches of how the AR interface for this tool could look like. Since our challenge was to develop a HoloLens application, we kept the investment in the early design very low by doing a paper prototype.
Design Evaluation
Due to the difficulties we face in access to users, we tested the low-fidelity prototype with hearing people and asked them to imagine they cannot hear. The test scenario was for them to explore the tool, and learn the word "tree" in both written and sign language.
