CategoryA02. Applied Innovation
Idea Creation R/GA TOKYO, JAPAN
Idea Creation 2 SHISEIDO CO., LTD. Tokyo, JAPAN
Production R/GA TOKYO, JAPAN
Production 2 NOMURA Tokyo, JAPAN
Production 3 TYO CAMP KAZ Tokyo, JAPAN


Name Company Position
Niklas Lilja R/GA Tokyo Executive Creative Director
Anthony Baker R/GA Tokyo Executive Technology Director
Seamus Higgins R/GA Sydney VP Chief Creative Officer APAC
Toru Nagahama R/GA Tokyo Design Director
Jedy Chen R/GA Tokyo Senior Creative Technologist
Naru Kudo R/GA Tokyo Senior Producer
Aleksandra Nowacka R/GA Tokyo Producer
Marvin Varela R/GA Tokyo Senior Software Engineer
Ayumu Nagamatsu R/GA Tokyo Senior Software Engineer
Kumi Tominaga R/GA Tokyo Creative Technology Director
Tyler Bowers R/GA Tokyo Associate Creative Director
Mark Lloyd Ruck R/GA Tokyo Associate Creative Director
Reynan Shimada R/GA Tokyo Experience Designer
Scott Rothman R/GA Tokyo Senior Software Engineer
Masa Tanaka R/GA Tokyo Accociate Creative Director
Sayaka Ota R/GA Tokyo Creative Director
Jooyoung Lee R/GA Tokyo Junior Visual Designer
Sookie Lee R/GA London Designer
Wing Luo R/GA New York Creative Director & Senior Technical Director
Bill Feng R/GA Tokyo Junior Software Engineer
Kiran Lokhande Hansen R/GA Los Angeles Experience Design Director
Keisuke Nakajima R/GA Tokyo Senior Strategist
Mariko Futamura R/GA Tokyo Senior Art Director
Makiko Miyake R/GA Tokyo Copywriter
Naoto Hieda R/GA Tokyo Senior Software Engineer
Tomoya Matsuura R/GA Tokyo Software Engineer
George Sugitomo R/GA Tokyo Executive Creative Director
Masayuki Tanaka R/GA Tokyo Senior Copywriter
Ikuo Toyama R/GA Tokyo Creative Director
Debbie Pan R/GA Tokyo Art Director
Abe Chuang R/GA Tokyo Copywriter
Scott Kundert R/GA Tokyo Senior Producer
Mariko Oka R/GA Tokyo Accout Director
Yosuke Suzuki R/GA Tokyo Managing Director
Joanna Saw R/GA Singapore Tech Intern
Tetsuro Kanegae Shiseido Executive Creative Director
Takashi Maruyama Shiseido Research Scientist
Masato Kosukegawa Shiseido Chief Creative Director
Mika Ishii Shiseido Producer
Asuka Hashiguchi Shiseido Copywriter
Maki Kobayashi Shiseido Designer
Keisuke Yoshida NOMLAB Designer
Hiromasa Imaizumi NOMLAB Account Director
Takeharu Hanokizawa TYO CampKAZ Producer
Hiro Kano NICE RAINBOW Director
Kenta Takasaki TYO SPARK Director
Takaya Suzuki STEP Producer
Amariei Catalina ACTIVE CINE CLUB Editor
Azusa Kikuchi TTR Editor
Eri Ueda TYO CampKAZ Production Manager
Eri Ueda TYO CampKAZ Production Manager

Why is this work relevant for Innovation?

In contrast to the superficial and inaccurate two-dimensional effects produced by Snapchat or FaceApp for lighthearted selfie-gratification, the Beyond Time experience generates individually accurate aging simulations that are built upon actual data from a century of research into the science of aging. Built in partnership with Japan’s leading experts in senescence and simulation technology, our project represents a world’s-first technology that creates an immersive 3D age simulation experience in real-time.


Beyond Time was influenced by generative adversarial networks and other recent advances in machine learning, as well as applications of the technology to produce creative work like in The Next Rembrandt. Though this project, we sought to apply machine learning to the creation of an emotional experience that transcends time in the real world. Since the processes and algorithms of popular mobile apps are proprietary, we wrote and developed original software for the express purpose of creating our scientifically-accurate age simulation experience, taking care to work closely with Shiseido Research and legal teams to ensure that non-anonymous data was not retained in our systems for the protection of our participants’ data rights.

Describe the idea

The mind affects the body as much as the other way around. As a leader in the beauty category, Shiseido recognizes the importance of mental wellness and positivity as essential keys to physical beauty. Beyond Time is an innovative interactive installation designed to make it possible for individuals to speak with digitally aged or de-aged versions of friends, family, and loved ones. In doing so, we sparked emotional conversations that transcended time, led people to discover beautiful things about growing old, and helped them think differently about the effects of physical aging.

What were the key dates in the development process?

2018 Jan—Mar: Technical research / Definition Apr—Aug: Concepting / Aging system development / Draft copy and UX development June—Aug: Installation design August: First user tests Sep—Nov: Installation design Finalization Sep—Dec: UX, Copy, Design and Aging system update / Data processing system development / MVP development Nov—Dec: Data collection December: 2nd user tests 2019 January: Data collection Feb—Mar: Software development finalization Feb. -Mar: Final installation construction April: Quality assurance testing April 13 2019: Grand Opening at Shiseido Global Innovation Center

Describe the innovation/technology

Beyond Time is three high-spec computers, multiple Kinect sensors, IoT connected buttons, multiple 4k cameras, plus synchronized LEDs, directional microphones, variable-opacity walls for privacy, and more than fifteen thousand lines of code across six different programming languages. We combined these components into a bespoke technology solution based on cutting edge face capture and aging research. We built a fast, easy-to-setup and reliable three dimensional system to generate one of the largest face modeling databases in the world. We then created the first real-time, fully three-dimensional age and age-reversing simulation engine that detects motion, recognizes individual faces, generates their 3D face models, aging textures and even a dynamic wrinkles layer, and then synchronizes the experience to take participants on a journey through.

Describe the expectations/outcome

In the beginning, we were tasked with “creating something that will attract visitors to Shiseido’s new Global Innovation Center.” By the end, we engaged over 3,100 people in an emotional experience that continues to be fully-booked today. We developed and implemented groundbreaking 3D facial data capture and skin color recognition solutions. We generated one of the largest 3D facial databases in the world. We created a new platform for open conversations for changing perceptions on aging. And most importantly, we helped the iconic Shiseido brand make a cultural statement that beauty goes beyond young and perfect skin, and that growing old is a beautiful process.


Video URL