A raised eyebrow, a cautious smile, a daring glance: facial expressions reveal a lot about someone’s feelings. But what if you cannot see them? For the visually impaired this is a huge loss of quality in communication. The Emotion Whisperer’ is a subtle tool that can prompt the visually impaired with the body language they’re otherwise missing out on. It comes with a pair of camera glasses that sends images of conversation partners to an emotion recognition app. These are analyzed and translated into a sensory signal: emotions like fear, surprise or happiness are felt by a specific vibration of the small device in the palm of your hand. You can actually feel someone smile. The Emotion-whisperer is a concept with which blind designer Simon Dogger Graduated the Design Academy Eindhoven. He won several awards including the IF talent award. Now this concept is being realized into a product. With financing and co-operation with the University of Amsterdam, the Technical University of Eindhoven, the OMCZ (Oogheelkundig Medisch Centrum Zaandam) and the technical partner Vention we are now into the validation research and the development of a V2.
As a user, user expert and designer Dogger is able to indicate his loss, research and analyse the impact and use his designer tools to turn this into an innovative and human centered solution. This is about the feeling of loss within the visually impaired world, giving them a lower quality of communication. Effects leading to reduced participation, increased isolation and lower self-esteem. The Emotion-whisperer is a tool providing blind and low sighted users to reach equal levels in communication and connection. The Emotion-whisperer consists out of three parts. A pair of glasses with integrated camera sending images of conversation partners through bluetooth to the smart-phone. An application with emotion recognition software is able to recognise the intensity levels of the six base emotions. A wearable receives this data through bluetooth. Each emotion is represented by an individual vibration part on the wearable. The recognized intensity levels are translated into according intensity levels of vibration.
The University of Amsterdam has developed a validation research program. Research-question: Can stakeholders learn and recognize substitute emotions through vibration on the skin? For this; purpose a research setup simulates optimal working emotion recognition software. Over the course of an hour the six base emotions are taught and finally tested upon recognition. To know if stakeholders are able to use the device is crucial for the course ahead.
The V1 is built in an Android smart phone with open source emotion recognition software. Using the internal camera and executing the haptic feedback through the vibration parts of the phone. Due to low quality of the software the app is able to recognize limited emotions and emotion intensities. Presently we are developing new types of wearables. A bracelet and a phone sleeve are both able to execute the vibrations. Tests will lead to the best choice concerning implementation in social context. The glasses with integrated camera will be included in later versions.
With the results of the validation research and a new design for the wearable the V2 will be developed starting in 2020. Goal is to build a product that can be tested; independently by stakeholders for a longer period of time. Because a second validation research consists out of six emotions with each three different intensity levels this cannot be done in a one hour research on location. For this purpose, a learning and research module is made that can be done independently over the course of five days. The V2 integrates this module into the application. Together with a wearable, stakeholders can do the validation research at home. To reach the true goal of the app we also implement high quality emotion recognition software to indicate real time functioning. The addition of this feature can measure the effect the app plays in social context and deliver user experience knowledge.
Oogheelkundig Medisch Centrum, Health Tech Yard, Elitac, Visio, Creative Industries Fund NL
University of Amsterdam, Technical University of Eindhoven, Vention Technologiys BV, Elitac BV, Koninklijke Visio Nederland
The following ZIP-file includes Hi-Res images and a description of the project.
Please contact Simon about any publications.