Research News
The headphone-based system uses Doppler technology to sense tiny fluctuations, or echoes, in acoustic soundwaves that are created by the hands of someone signing. Photo: University at Buffalo.
By MELVIN BANKHEAD III
Published September 9, 2021
A UB-led research team has modified noise-cancelling headphones to enable the common electronic device to “see” and translate American Sign Language (ASL) when paired with a smartphone.
Reported in the journal “Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies,” the headphone-based system uses Doppler technology to sense tiny fluctuations, or echoes, in acoustic soundwaves that are created by the hands of someone signing.
Dubbed SonicASL, the system proved 93.8% effective in tests performed indoors and outdoors involving 42 words. Word examples include “love,” “space” and “camera.” Under the same conditions involving 30 simple sentences — for example, “Nice to meet you.” — SonicASL was 90.6% effective.
“SonicASL is an exciting proof of concept that could eventually help greatly improve communication between deaf and hearing populations,” says corresponding author Zhanpeng Jin, associate professor in the Department of Computer Science and Engineering at UB.
But before such technology is commercially available, much work must be done, he stresses. For example, SonicASL’s vocabulary must be greatly expanded. Also, the system must be able to read facial expressions, a major component of ASL.
The study will be presented at the ACM Conference on Pervasive and Ubiquitous Computing (UbiComp), taking place Sept. 21-26.
Communication barriers persist
These are the acoustic soundwaves created by signing the phrase “I need help.”
Worldwide, according to the World Federation of the Deaf, there are about 72 million deaf people using more than 300 different sign languages.
Although the United Nations recognizes that sign languages are equal in importance to the spoken word, that view is not yet a reality in many nations. People who are deaf or hard of hearing still experience multiple communication barriers.
Traditionally, communication between deaf American Sign Language (ASL) users and hearing people who do not know the language takes place either in the presence of an ASL interpreter, or through a camera setup.
A frequent concern regarding the use of cameras, according to Jin, includes whether those video recordings could be misused. And while …….
Source: http://www.buffalo.edu/ubnow/stories/2021/09/sonic-asl.html