Proposed tech measures echo created by hand movements; could help boost communication between deaf and hearing populations
BUFFALO, N.Y. — A University at Buffalo-led research team has modified noise-cancelling headphones, enabling the common electronic device to “see” and translate American Sign Language (ASL) when paired with a smartphone.
Reported in the journal “Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies,” the headphone-based system uses Doppler technology to sense tiny fluctuations, or echoes, in acoustic soundwaves that are created by the hands of someone signing.
Dubbed SonicASL, the system proved 93.8% effective in tests performed indoors and outdoors involving 42 words. Word examples include “love,” “space,” and “camera.” Under the same conditions involving 30 simple sentences – for example, “Nice to meet you.” – SonicASL was 90.6% effective.
“SonicASL is an exciting proof-of-concept that could eventually help greatly improve communication between deaf and hearing populations,” says corresponding author Zhanpeng Jin, PhD, associate professor in the Department of Computer Science and Engineering at UB.
Before such technology is commercially available, much work must be done, he stressed. For example, SonicASL’s vocabulary must be greatly expanded. Also, the system must be able to read facial expressions, a major component of ASL.
The study will be presented at the ACM Conference on Pervasive and Ubiquitous Computing (UbiComp), taking place Sept. 21-26.
For the deaf, communication barriers persist
Worldwide, according to the World Federation of the Deaf, there are about 72 million deaf people using more than 300 different sign languages.
Although the United Nations recognizes that sign languages are equal in importance to the spoken word, that view is not yet a reality in many nations. People who are deaf or hard of hearing still experience multiple communications barriers.
Traditionally, communications between deaf American Sign Language (ASL) users and hearing people who do not know the language take place either in the presence of an ASL interpreter, or through a camera set-up.
A frequent concern over the use of cameras, according to Jin, includes whether those video recordings could be misused. And while the use of ASL interpreters is becoming more common, there is no guarantee that one will be available when needed.
SonicASL aims to address these issues, especially in casual circumstances without pre-arranged planning and setup, Jin says.
Modify headphones with speaker, add app
Most noise-cancelling headphones rely on an outward-facing microphone that picks up environmental noise. The headphones then produce an anti-sound – a soundwave with the same amplitude but with an inverted phase of the …….
Source: http://www.buffalo.edu/news/releases/2021/09/005.html