Severely paralyzed man communicates using brain signals sent to his vocal tract
It's the first time someone with speech loss has communicated simply by attempting speech.
A severely paralyzed man has been able to communicate using a new type of technology that translates signals from his brain to his vocal tract directly into words that appear on a screen. Developed by researchers at UC San Francisco, the technique is a more natural way for people with speech loss to communicate than other methods we've seen to date.
So far, neuroprosthetic technology has only allowed paralyzed users to type out just one letter at a time, a process that can be slow and laborious. It also tapped parts of the brain that control the arm or hand, a system that's not necessarily intuitive for the subject.
The USCF system, however, uses an implant that's placed directly on the part of the brain dedicated to speech. That way, the subject can mentally activate the brain patterns they would normally use to say a word, and the system can translate the entire word, rather than single letters, to the screen.
To make it work, patients with normal speech volunteered to have their brain recordings analyzed for speech related activities. Researchers were then able to analyze those patterns and develop new methods to decode them in real time, using statistical language models to improve accuracy.
However, the team still wasn't sure if brain signals controlling the vocal tract would still be intact in patients paralyzed for many years. To that end, they enlisted an anonymous participant (known as Bravo1) who worked with researchers to create a 50-word vocabulary that the team could decipher using advanced computer algorithms. That included words like "water," "family" and "good," enough to allow the patient to create hundreds of sentences applicable to their daily life. The team also used an "auto-correct" function similar to those found on consumer speech recognition apps.
To test the system, the team asked patient Bravo1 to reply to questions like "How are you today?" and "Would you like some water?" The patient's attempted speech then appeared on the screen as "I am very good," and "No, I am not thirsty."
The system was able to decode their speech at up to 18 words per minute with 93 percent accuracy, with a 75 percent median accuracy. That might not sound great compared to the 200 words per minute possible with normal speech, but it's much better than the speeds seen on previous neuroprosthetic systems.
“To our knowledge, this is the first successful demonstration of direct decoding of full words from the brain activity of someone who is paralyzed and cannot speak,” said Edward Chang, MD, Chair of Neurological Surgery at UCSF and senior author on the study. “It shows strong promise to restore communication by tapping into the brain's natural speech machinery.”
The team said the trial represents a proof of principal for this new type of "speech neuroprosthesis." Next up, they plan to expand the trial to include more participants, while also working to increase the number of words in the vocabulary and improve the rate of speech.
