Symphonic Labs, an audio tech startup, has recently launched an online tool that showcases their AI technology for lip reading. The tool, called readtheirlips.com, allows users to upload videos of speakers, and the AI will identify inaudible speech by analyzing the movement of the speaker’s lips. This technology has applications in noisy environments or when speaking quietly to voice assistants in public settings. Users can upload short video clips to the site, and the AI will return text of what it calculates is being said, as long as the speaker’s face and lips are clearly visible.
During tests of Symphonic Lab’s lip reading AI, accuracy was generally high, with minor errors in some portions of the speech. For example, “to try to comfort them” was transcribed as “to try and comfort them,” and “will recall every day in gun violence” was transcribed as “or what we call everyday gun violence.” Overall, the software showed promise in accurately transcribing speech as long as the speaker’s face was clearly visible. The company has also tested its AI on footage of notable figures like VP Kamala Harris and Taylor Swift, showcasing its versatility across a range of speakers and environments.
In addition to contemporary figures, Symphonic Labs tested its lip reading AI on silent film era clips to see how it fared with older, grainy footage. While the accuracy of the transcriptions cannot be confirmed, it was interesting to see the AI’s interpretation of what iconic movie stars like Gloria Swanson might have been saying in silent films. The technology showcased on readtheirlips.com is a part of Symphonic Lab’s wider project, MAMO, a Mac OS software application that integrates this technology with personal computers, enabling users to issue voice commands silently.
Chris Samra, an engineer with Symphonic Labs, explained that the startup aims to build interfaces that feel telepathic without the need for bulky hardware or implants. The lip reading AI serves two main purposes: enabling fast and silent communication and analyzing speech in loud or distant environments. Although readtheirlips.com is more of a tech demo, the long-term goals include making conversational interfaces more accessible for mass consumers and improving accessibility for people with conditions like Dysphonia and hearing impairment. The latest update to the software allows for personalization, allowing users to train the AI with their own voice and vocabulary for better accuracy.
This technology may be particularly useful for individuals who feel uncomfortable using voice assistants in public settings. A survey by PwC found that most U.S. consumers prefer to interact with voice assistants at home due to concerns about privacy and the perceived social awkwardness of using voice commands in public. With Symphonic Lab’s lip reading AI, users can dictate commands in public spaces without vocalization, additional hardware, or wearable mics, allowing for a more discreet and comfortable user experience. As the technology continues to evolve, it holds the potential to revolutionize the way we interact with voice-controlled devices and interfaces.
