Project: Using artificial intelligence to facilitate communication between deaf and hearing people

2020-11-01 – 2024-06-30

Abstract

According to the WHO, over 5% of the world population suffers from hearing loss. This is expected to double by 2050. Hearing loss is estimated to cost society over 750 billion US dollars globally, per year. The Flemish government subsidizes up to 36 hours of interpreter aid per deaf person per year, which also leads to high costs. These interpreters facilitate communication between people with hearing loss and those without. After all, the latter often do not know sign language. Imagine an artificial interpreter, which reduces the need for human interpreters in many situations. Such a tool would be of great use to deaf people and their relatives, and allow re-allocation of government spending to increase their quality of life, as fewer interpreters would be required in everyday life. This project will lead to the creation of such an artificial interpreter through a structured research approach with attention to linguistic properties of sign languages. We will have an AI agent learn how humans move and communicate through gestures, using the wealth of unlabeled sign language video data at our disposal. As a side effect, we will gain knowledge that is also useful in the fields of social robotics and gesture recognition. Using our connections with linguistics researchers and the Deaf community, we will be able to bring this project to fruition. This project will provide potential for several applications, most importantly a real-time sign language interpreter tool.