Title | VIDEO LYRICS |
Brand | EAT KANAZAWA |
Product / Service | PRODUCT |
Category | D03. Data Visualisation |
Entrant | DENTSU INC. Tokyo, JAPAN |
Idea Creation | DENTSU INC. Tokyo, JAPAN |
Idea Creation 2 | QOSMO Tokyo, JAPAN |
Production | DENTSU CREATIVE X Tokyo, JAPAN |
Production 2 | QOSMO Tokyo, JAPAN |
Name | Company | Position |
---|---|---|
Kaoru Sugano | Dentsu Lab Tokyo | Creative Director |
Nao Tokui | Qosmo, Inc. | Executive |
Kouki Yamada | Freelance | Natural language processing programmer |
Kazuyoshi Ochi | Dentsu Lab Tokyo | Film Planner/ Copy Writer |
Ryosuke Sone | Dentsu Lab Tokyo | Film Planner/Director |
Jun Kato | Dentsu Lab Tokyo | Creative Producer |
Masafumi Fujioka | Dentsu Creative X. inc, | Producer |
Hiromi Nakamura | Dentsu Creative X. inc, | Production Manager |
Kiyonori Higuchi | Office Higuchi | Music Producer |
Ken Katsuno | Startline. inc, | Cameraman |
Mayumi Matsuzawa | Freelance | Hair & Make Up |
Tetsuro Isshi | Freelance | Animator |
We created an automatic lyric generation system based on the original karaoke video by employing image analysis via a convolutional neural network, enabling the perfect synchronization of the video and the lyrics.
A convolutional neural network learns from numerous combinations of photos and their descriptions. We then conduct a video scene analysis. A programmed AI creates captions for an unknown karaoke video frame. It repeats this process to quickly write more captions for the video. The system extracts distinctive words from the captions. It searches for related words and synonyms, then selects words for the lyrics. The system calculates the number of characters in the original lyrics, then replaces them with entirely new lyrics it has generated.