Researchers have demonstrated a system that can translate brain activity into synthetic speech.
Earlier projects have shown that it’s possible – although it took months to perform a task that most people do instantly.
Now a complementary new study has seen researchers decode spoken words and phrases from the brain signals that control speech, in realtime.
Often patients who experience facial paralysis due to brainstem stroke, spinal cord injury, neurodegenerative disease, or other conditions may lose their ability to speak. However, the brain regions that normally control speech are often intact and remain active in these patients.
Researchers set out to see if they could develop the technology that allowed them to express the information in their brains as fluent speech.
The new study, by researchers in University of California in San Francisco (UCSF) neuro-surgeon Eddie Chang’s lab and led by postdoctoral researcher David Moses, PhD was published yesterday (30 June) in Nature Communications.
Volunteers had a tiny patch of electrodes placed on the surface of their brains to record brain data while they listened to a set of nine simple questions.
The team developed a set of machine learning algorithms equipped with refined phonological speech models, which were capable of learning to decode specific speech sounds from participants’ brain activity.
The volunteers initially responded out loud to the questions, training the machine learning algorithms to detect when they were hearing a new question or beginning to respond, and to identify which of the two dozen standard responses the volunteer was giving with up to 61% accuracy as soon as they had finished speaking.
A key finding was that context is important, and helped to improve the algorithm’s speed and accuracy, so identifying the question as well as the answer raised the accuracy levels.
A new study is now underway at UCSF to determine if the neural interface implant can be used to improve communication and movement in patients suffering from stroke, neurodegenerative disease or brain injury.