Background

Even if they cannot see, blind people can actually ‘watch’ and enjoy various forms of videos like us such as movies and videos on Youtube, as they experience these videos through music, dialogue and sound effects, which in turns help paint the picture in their heads. But there’s one thing that blind people can never experience before : music video, since most music videos are mainly music and visuals that tell a story, so for people who cannot see, watching a music video give them no other enjoyment than listening to the music track.

Describe the creative idea

Visible Track is the first online service that lets blind people ‘see’ millions of music videos that are available online. By simply entering accessibility-featured website, copy and paste any music video link from Youtube or Facebook Watch. The service utilises as many as 6 Ais—e.g. scene splitting, scene classification, action classification and object classification—that work together on different tasks to help music videos become visible & universally accessible to everyone. All of which help intelligently add sound effects to each scene automatically to help blind people paint the ‘scene’ in their head. Moreover, the service also uses image-captioning AI and text-to-speech AI to generate audio descriptions to describe what’s going on in the scene. Alternatively, blind people can also use Visible Track service via messenger bot on THAILAND ASSOCIATION OF THE BLIND Facebook page.

Describe the strategy

Prior to this project, the art of music video has never been accessible to blind people. But today’s technology enables us to reduce inequality and improve quality of life for the blind. Visible Track is the first online service that lets blind people ‘see’ millions of music videos that are available online with the help of AI technology. The idea is to go beyond visual limitations of music videos and use AI-enabled sound depiction of scenes, allowing music video content to be accessible to the visual-impaired.

Describe the execution

The service utilises as many as 6 Ais—e.g. scene splitting, scene classification, action classification and object classification—that work together on different tasks to help music videos become visible & universally accessible to everyone. All of which help intelligently add sound effects to each scene automatically to help blind people paint the ‘scene’ in their head. Moreover, the service also uses image-captioning AI and text-to-speech AI to generate audio descriptions to describe what’s going on in the scene.

Describe the outcome

More than 5,000 hours of music videos played • Over 75% of “Thailand Association of the Blind” members have used the service • 80% of surveyed users report experiencing new enjoyment of music videos • Data shows that Visible-Track.com have been used from many countries beyond Thailand.