Hey Siri, Read My Lips: Silent speech recognition enables covert communication
HAL reads lips.
From IEEE Spectrum:
“Hey, Siri, add proctologist appointment to my calendar for 12:50pm tomorrow.”
“Hey, Siri, play Wind Beneath My Wings by Bette Midler on repeat.”
“Hey, Siri, what’s 52 + 25?”
“Hey, Siri, Google me.”
There are lots of digital assistant commands one should never say out loud in front of other people. But
in a few years, proximity to eavesdroppers might not be a problem
anymore, because we’ll be able to silently mouth all that embarrassing
dictation.
Called “silent speech recognition,” the technology
essentially reads lips. Users mouth the words of a command or message,
and the device deciphers the words, based on the movements of the face
and neck.
The feat can be accomplished using surface electromyography, or
sEMG. Electrodes are placed on the skin at key places around the mouth,
along the jaw, under the chin or on the neck. Muscle movements in those areas generate neuromuscular signals—essentially an electrical code. Algorithms trained on silent speech then translate the electrical signals, decoding what the user is saying.
A flexible patch offers a simple, ergonomic way to attach the device to the face quickly—something many previous designs lack. “I’ll be curious when that comes out to see what they’ve done,” says Geoff Meltzner, vice president of research and technology at VocalID, a synthetic voice company, who was not involved in the Soochow project. “I think this could solve an ergonomics or ease-of-use problem,” he says.
But a flexible material addresses just one piece of the challenge. A handful of other groups have, since NASA’s pioneering work 15 years ago, been quietly building the nuts and bolts of a silent speech recognition device. One of them is Delsys,
a wearable sensor maker, which has been working on the technology since
about 2006, says Meltzner, who was the primary investigator in the
early years of the project.
Delsys’ current prototype has eight custom, rigid electrodes that
are placed on the face and neck. They are connected to a computer where
algorithms perform the decoding. The system can recognize about 2200 mouthed words—about 5 percent of the 42,000
words in the average English speaker’s vocabulary. (And yes, this
writer told Siri to do that math while no one was within earshot.)
In a study
published in June 2018, Delsys’ device proved to be about 91 percent
accurate in translating silent speech. One can imagine the
potential applications: any phone user who doesn’t want to be
heard, people working in loud environments, and military personnel who
need hands-free covert communication.
The tool not only saves us from divulging private information, but
could also make us sound smart. Imagine chatting with a group of people,
and being able to secretly Google something, or do complicated math, or
search for the name of an acquaintance you should totally already know,
just in time to introduce her. Smooth.
But no one is going to be fooled by someone mouthing the words “Hey Siri, what is Sylvia’s last name?” while wearing a headset that wraps from his ear to his chin.
To address that problem, researchers at the MIT Media Lab are developing a covert communication device they call AlterEgo.
The current prototype is bulky, but it can decipher words
iterated silently with the mouth closed. “We call it internal
articulation,” says Arnav Kapur, a graduate student at MIT who is heading up the project.
To communicate with the device, the user makes slight internal
movements with the mouth closed, kind of like the way one might form
words while silently reading, says Kapur....MORE
Jeff Bezos: “Alexa, send nudes to my secret admirer.”
Alexa: “Got it. Sending nudes to the National Enquirer.”
