Artificial intelligence company LiRA, a startup with student roots at UNC-Chapel Hill, is making a rapid push on the next phase of its technology development. The venture is crowdsourcing selfie-style videos to train its lip-reading technology, which is designed to restore a higher quality of life to patients who lose their voices due to critical airway surgery or other physical trauma.
When a helicopter airlifting a patient touched down at UNC Hospital in the spring of 2019, first-year ENT resident Andrew Prince had no idea how the disturbing situation that was about to unfold would set him on the path to soon co-found a tech company. The patient was a woman from western North Carolina who could barely breathe and required emergency intervention to avoid suffocation. After running tests, doctors found that the patient had a huge tumor in her throat. They performed an emergent tracheostomy—surgically creating an opening in the neck into the windpipe that allows air to fill the lungs. The procedure saved the patient’s life but cost her something dear: her voice. Because the breathing tube passing through the opening bypassed her vocal cords, the woman—like many tracheostomy patients—couldn’t speak. Approximately 100,000 tracheostomies are performed annually in the United States.
For Prince, a current otolaryngology and head and neck surgery resident at the UNC School of Medicine, the aftermath of the procedure etched an unforgettable memory. “The patient woke up in pain, realizing that she was in a hospital, couldn’t speak and had a throat tumor. This created a whirlwind of grief that resulted in a complete breakdown,” said Prince. “She became extremely distraught and began pulling at the tubes and lines connected to her. It was a safety risk, so she had to quickly be sedated.” Prince walked away unsettled. “I left thinking that there has to be a better way, because that situation is not why I got into medicine—to have to silence people,” he said. “And that’s when it clicked: what if we could develop a really robust lip-reading solution?
Fast forward to 2023: Prince has co-founded and is now CEO of LiRA, a startup that is developing a lip-reading technology for voiceless patients. Prince, who continues to work as an otolaryngology head and neck surgery resident, co-founded LiRA (which stands for “Lip Reading Assistant”) in the fall of 2019 with four other UNC-Chapel Hill graduate students: Andres Tello (Kenan-Flagler Business School), Nga Nguyen (Gillings School of Global Public Health), Dina Yamaleyeva (Department of Biomedical Engineering), and Alison Schaefer (Department of Biomedical Engineering).
Yamaleyeva, co-founder and product design/marketing officer, says that “LiRA’s technology is an app that translates lip movements into speech” and is “like speech-to-text, but for lip movements.” The technology, based on artificial intelligence, allows voiceless patients—including those who experience vocal cord trauma, tracheostomy, or complete surgical removal of the voice box—to use an app on their phone to record video of themselves forming words with their lips. The technology analyzes the video, translating it into audio for others to hear. The current patient alternative—writing thoughts on paper—is a frustrating, exhausting mode of communication.
“When these patients lose their faculty of speech, they withdraw socially. They stop caring because they get so fatigued and caught in this darkness because they can’t connect with anybody as humans,” said Prince. “We want to restore a part of their humanity that, unfortunately, is taken away from them, certainly with the best intentions possible: to save their lives and to give them future quality of life. But, right now, they lose too much life quality. It’s not an even exchange.”
Tello, LiRA co-founder and chief commercial officer, says voicelessness also has negative business ramifications for the wider healthcare system. It’s expensive for hospitals and clinics to help patients cope with the mental and social effects of voicelessness, and those costs quickly add up for a U.S. healthcare system that’s already hit hard by billions of dollars attributed to waste.
Because the company’s product is founded on artificial intelligence, Tello says that a particular type of data—selfie videos of people reading sentences—is essential for training the technology to read lips. “We need data,” Tello said. “That’s our life source.”
To accelerate its data collection, LiRA partnered with a software firm in Virginia to launch its LipTrain campaign. The campaign makes it easy for anyone to contribute selfie videos. People who want to help simply visit the LipTrain website, where an easy-to-use tool allows them to record themselves on their phones or laptops as they read sentences that appear on the screen. The videos feed into LiRA’s learning algorithm, teaching it to more accurately translate specific words.
“The LipTrain campaign builds a data pipeline through micro volunteerism,” said Tello. “Anyone in any setting—on a couch, at a desk, on the beach—can use their mobile device to read out sentences. That data feeds into our pipeline, which trains our algorithm to become smarter and more accurate.”
Yamaleyeva said the company needs LipTrain contributors now more than ever. LiRA is using the videos to finalize development of its minimal viable product (MVP), an early version of its technology that LiRA will pilot with real patients to measure the technology’s impact. More than 1,600 people have submitted videos. Yamaleyeva says that if 500 more people contribute two minutes of video each, the technology’s lip-reading accuracy can soar.
“LipTrain is the most crucial aspect for us to capture engagement. The more data, the better the accuracy,” she said. “It’s also important to get diverse contributors so that when it is time to translate a patient’s lip movements, the lip-reading algorithm will be familiar with a variety of facial features, genders, and skin tones, thus improving accessibility for all people.”
As of July 2023, LiRA has operated on completely non-diluted funding received through grants, awards and prizes. The company secured its earliest funding from innovation-focused programs at UNC-Chapel Hill. “We started off with our first checks coming from competitions at UNC,” said Tello. “And we still remember how amazing it felt to get our first $1,500 check and how we thought we were going to change the world. That feeling doesn’t go away.” LiRA received awards and grants from numerous University programs: the Eshelman School of Pharmacy’s E(I) Lab, Launch Chapel Hill, the Kenan-Biddle Partnership Grant, and Innovate Carolina’s 1789 Student Venture Fund. LiRA was also a finalist in a 2020 innovation pitch competition hosted by the Gillings School of Global Public Health.
As its technology development needs expanded and the onset of the COVID-19 pandemic forced them to spend less time on other pursuits, LiRA buckled down and focused on other off-campus sources. “The beginning of the pandemic is when we truly dug in, and we started to relentlessly apply to everything that had a remote possibility of giving us a step forward,” said Prince. “In the first 18 months, we probably applied to between 100 to 200 accelerators, small grants, awards, competitions—anything and everything we could get our hands on just for further resources.”
LiRA’s success with private foundations includes the Clinton Foundation and an NC IDEA MICRO grant, which is a $10,000 project-based award from NC IDEA that helps young companies validate and advance ideas. Competitions have also become a lucrative proving ground. LiRA won first place and a $10,000 prize at the Launch Place’s 2021 Big Launch Challenge Pitch Competition, a first place $20,000 grant at VentureWell’s 2021 Verizon Forward for Good Challenge, co-winner of the 2021 RIoT Your Reality Challenge, and second place for a $30,000 prize at the 2021 Tulane Business Model Competition. The company also placed first at the 2021 Tech Tank pitch competition hosted by Covintus, the firm it partnered with to launch its LipTrain campaign platform. In 2023, LiRA was a finalist in the 2023 TriVent Startup Competition, earning a $25,000 prize as the top AI venture.
Despite its success with foundations and competitions, the startup’s most pivotal funding is the $276,000 federal Small Business Innovation Research (SBIR) grant it received from the National Science Foundation, says Tello, because “that one shifted the priorities back to the technological development” and focused the team around “solely working on those types of initiatives.” The company also received a corresponding $75,000 grant from the One North Carolina Small Business Program, a state program that awards matching funds to North Carolina businesses that receive an SBIR or STTR award.
Balancing grant applications and competitions with technology development time is something Prince says young companies should monitor. “For every moment that you spend developing a pitch deck or writing a response to a question, you’re not driving development of the technology. And it’s that balance that I’ve always found interesting: How do you do 100 percent of everything at the same time?”
Prince and Tello feel that the venture is poised to pursue larger funding, including a potential seven-figure SBIR Phase 2 grant and a Series A venture capital round. Even with significant opportunities on the horizon, the greatest lesson for Tello involves humility and efficiency. “No competition is too small, and you have to humble yourself—even if that means going back to the smaller competitions,” he said. “For first-time entrepreneurs, understanding capital efficiency is so important. If you can build that metric into the fabric of your company and demonstrate that you were thoughtful about spending every dollar, it’s going to pay dividends when you’re getting ready for larger investor rounds.”
Launched during the COVID-19 pandemic, LiRA continues to operate as a virtual company. It received physical workspace as a member of the Launch Chapel Hill accelerator, but pandemic-era restrictions limited the team’s ability to use the physical space.
Although the team thrives virtually, it may eventually investigate a physical location. “In terms of our tech arms, they’ve been in California, Massachusetts and North Carolina,” said Tello. “Being virtual has worked for the business so far, but I think more face-to-face time would be incredible.” To that end, Tello says that LiRA is currently evaluating a physical location as an incentive and part of the prize package for a new accelerator opportunity.
Although the original five founders graduated and started other full-time professional positions, they all still hold important roles with LiRA. “We’re all still involved, at minimum, in advisory roles,” said Tello. “But the positions we’ve prioritized are CEO, CTO and a fusion of operations and finance.”
In its earliest days, LiRA turned to Carolina student interns—a valuable practice that has evolved with new technology development and funding demands. “We hired undergrad and grad students, which was a great management experience for us because we got to see them flourish on their projects,” said Tello. “Right now, we’re focused on tech-driven initiatives that require a very specific set of skills, and as we moved into other types of funding opportunities, it became clear that we needed to have the talent and capabilities to contend as an AI company. Those needs shifted, and our recruitment focused more on people with PhD-level computer science, deep learning and machine learning skills. That eventually led us to hire our current CTO, who has the experience to go deep on the tech side.”
LiRA’s entrepreneurial learning path began when the grad students—all strangers to one another—met by happenstance in the fall of 2019, nearly six months after Prince’s formative encounter with the tracheostomy patient. The students attended E(I) Lab—a program where student teams form to develop solutions to unmet healthcare challenges—hosted by the Eshelman School of Pharmacy.
“The first E(I) Lab session put us in a room, and it was like a science fair, where you walk from poster to poster, and you naturally gravitate toward an idea or problem. It was based on unmet needs that the healthcare community had submitted to the E(I) Lab,” said Tello, who said the initial problem presented to the group was that the electrolarynx—a battery operated voice machine that produces sound after removal on the entire voice box—doesn’t provide a human mode of communication. “Five of us from different disciplines—business, medicine, public health, and biomedical engineering—were walking around these poster boards, and there was a moment when I clearly remember standing there, and we all looked at each other like: ‘Maybe we should do this. We could put together a solution.’”
The team’s experience at E(I) Lab included award funding, faculty mentorship, and education on entrepreneurial practices like customer discovery and the business model canvas. After working at E(I) Lab, the group used other University learning resources, including Launch Chapel Hill, the CUBE incubator at the UNC Campus Y, and Kenan-Flagler Business School’s Entrepreneurship Center. “Each of those resources had its own seminar series, instructors and mentors, and that’s when we were able to ask questions and learn from experts,” said Prince. “We pressure tested business ideas about our business model, customer persona, customer discovery and go-to-market strategy.”
Prince’s learning experience leading LiRA makes him optimistic about market-driven medical advancements that extend beyond his daily realities as a surgeon. “As a change-maker surgeon in the making, I think there are many ways a corporation can drive innovation and improve care for our patients,“ he said. “The beautiful aspect of a for-profit corporation is that it requires other forces beyond science to drive it forward. Industry and commercialization are avenues for getting more resources to improve technology and to get it into the hands of people who wouldn’t otherwise benefit from it.”
In a few minutes, you can help someone find their voice again. LiRA needs you to record yourself saying common sentences that voiceless individuals need to communicate. Every video makes a difference in teaching LiRA’s learning algorithm how to more accurately translate specific words. Participate in LipTrain.
| UNC-CH