Title | VIDEO LYRICS |
Brand | EAT KANAZAWA |
Product / Service | PRODUCT |
Category | A01. Innovative Technology |
Entrant | DENTSU INC. Tokyo, JAPAN |
Idea Creation | DENTSU INC. Tokyo, JAPAN |
Idea Creation 2 | QOSMO Tokyo, JAPAN |
Production | DENTSU CREATIVE X Tokyo, JAPAN |
Production 2 | QOSMO Tokyo, JAPAN |
Name | Company | Position |
---|---|---|
Kaoru Sugano | Dentsu Lab Tokyo | Creative Director |
Nao Tokui | Qosmo, Inc. | Executive |
Kouki Yamada | Freelance | Natural language processing programmer |
Kazuyoshi Ochi | Dentsu Lab Tokyo | Film Planner/ Copy Writer |
Ryosuke Sone | Dentsu Lab Tokyo | Film Planner/Director |
Jun Kato | Dentsu Lab Tokyo | Creative Producer |
Masafumi Fujioka | Dentsu Creative X. inc, | Producer |
Hiromi Nakamura | Dentsu Creative X. inc, | Production Manager |
Kiyonori Higuchi | Office Higuchi | Music Producer |
Ken Katsuno | Startline. inc, | Cameraman |
Mayumi Matsuzawa | Freelance | Hair & Make Up |
Tetsuro Isshi | Freelance | Animator |
Karaoke videos were originally created as loosely-organized stories that would serve as a common base upon which to create wide variety of interpretations. By “showing” these videos to a computer and creating new lyrics that call to mind completely new stories, we aimed to give karaoke an entertaining new twist. And by perfectly syncing the lyrics and the visuals, we wanted to create a more immersive experience that would maximize people’s enjoyment of karaoke.
A convolutional neural network learns from numerous combinations of photos and their descriptions. We then conduct a video scene analysis. A programmed AI creates captions for an unknown karaoke video frame. It repeats this process to quickly write more captions for the video. The system extracts distinctive words from the captions. It searches for related words and synonyms, then selects words for the lyrics. The system calculates the number of characters in the original lyrics, then replaces them with entirely new lyrics it has generated.
On January 29, 2016, we conducted a demonstration at the art event eAT KANAZAWA 2016. There, we presented new lyrics for existing Japanese songs in real time. As our official release, we presented a demo of original songs and video on the project website. At present, implementation as a new service and promotional activities are under consideration within the karaoke industry.
Karaoke is a well-known form of entertainment from Japan which is loved all over the world. But unfortunately, even though the music and lyrics on the screen continue to change, the video playing in the background is often times not the song’s official music video but rather an automatically-generated compilation of outdated stock clips, preventing people from really enjoying themselves at karaoke and actually becoming a cause of stress. Karaoke-goers have long been puzzled by the illogical, nonsensical videos that accompany karaoke tracks.