Music video app uses facial recognition to improve resident emotional well-being

Music video app uses facial recognition to improve resident emotional well-being
July 26th, 2021

A Milwaukee senior living and care provider is using a new music video app to try to engage residents and improve their emotional well-being through personalized music.

Ovation Communities says it is the first long-term care provider in the United States to use the SOLO app, developed by a technology startup in Israel. The app uses facial recognition technology and artificial intelligence to monitor a resident’s emotional state. Using that information, the app chooses music videos that are familiar and meaningful to the resident and, ideally, will amplify positive motions, promote relaxation, calm agitation and provide meaningful interaction, according to an Ovation spokesperson.

Research suggests that music offers emotional and behavioral benefits for older adults, particularly those living with dementia and cognitive issues. Songs and melodies are thought to evoke responses of serenity, energy and movement, as well as nostalgia, memory and a renewed experience of past emotions.

“Music allows people to become more engaged in their reality,” said Kevin Farinelli, Ovation Jewish Home music therapist. “It’s really wonderful to see someone’s face light up when they’re listening to the languages, sounds and sons they love.”

The SOLO app allows the Ovation activities team to help residents set up tailored profiles with tags for language, music genres and other preferences. When a resident uses the app, a facial scan of 42 muscles recognizes seven emotions and plays music video clips adapted to their current emotional state. Calming music, for example, might be used to relieve anxiety, whereas upbeat songs could be used to ease sadness.

The new technology allowed caregivers to engage residents one-on-one in an interactive, contactless way when the pandemic put a hold on group activities, according to Ovation. The effort helped residents feel part of a broader group and community through the commonality of song. A dashboard allows staff to monitor resident changes, trends and overall emotional well-being.

According to SOLO, after the first 30 days of use of the app, senior living and care communities reported a 92% resident engagement rate and a 76% improvement in resident mood.