Describe the creative idea

UNLIMITED SPACE is an interactive installation which integrates 2D typography of large-scale online data and 3D capture of dynamic body motion. On entering the space, in an immediate response to movements, keywords collected from our service websites are projected onto the walls, arranged to take the form of each individual’s body shape. Under the concept of “Immerse themselves in the cloud of collective wishes and dreams. ”, UNLIMITED SPACE connects the real world of customers with the intangible world of data.  In UNLIMITED SPACE, the world of our data, which shapes our daily lives, is connected with the shapes of our bodies. The installation is a space that can expand the viewer’s sense of the world. All the viewer needs to do is walk into the space and enjoy the interplay between movement and words, possibly finding a similar connection to the words typed by someone else, somewhere unknown.

Describe the execution

We, Rakuten Group Inc., provide over 70 different internet services, including a marketplace which is widely used on a regular basis by millions of customers in Japan. When customers search for products, the site response is near instantaneous, yet the amount of data processed in parallel is enormous. This happens in order to fulfill our demands, but we, as customers, rarely feel its existence in our daily life. We developed our art installation to allow viewers to experience the dynamics of online data and providing a valuable means through which to connect to the intent of others, when they search for products for themselves or gifts for others.  Our system consists of three parts.  First, "Big Data Processing" extracts trending words from queries to our services. Each word is typeset with polygonal form in our original font in a word plane, and many such word planes are mapped to a typeset plane. Each typeset plane contains approximately 15,000 word planes, where words can appear multiple times to fill the area.  Second, "3D Human Shape Capture" captures the shapes of visitors in real-time. To capture the 3m × 6m installation space, we built a multi-camera capturing system consisting of three calibrated depth cameras connected to a PC, and a UHD projector. We employ Azure Kinect for depth capturing and estimating body shape using its body tracking feature. The body shape extraction runs in parallel on three GPUs, one for each depth sensor. As a result, we achieve video rates for shape acquisition, using NVidia GeForce RTX 2070 SUPER GPUs.  Finally, "Typographic Visualization" allows the typeset words to be mapped onto surfaces of visitors’ shapes. This integrates 2D typography and dynamic 3D body shape visualization, what we call interactive kinetic typography.