PROJECT HUEY

Innovation Spike

Case Film

Presentation Image

TitlePROJECT HUEY
BrandGOOGLE
Product / ServiceGOOGLE CLOUD
CategoryA04. Early-stage Technology
EntrantGOOGLE BRAND STUDIO APAC Sydney, AUSTRALIA
Idea Creation GOOGLE BRAND STUDIO APAC Sydney, AUSTRALIA
Media Placement STAB MAGAZINE Byron Bay, AUSTRALIA
Production R/GA Sydney, AUSTRALIA
Production 2 GOOGLE BRAND STUDIO APAC Sydney, AUSTRALIA
Post Production GOOGLE BRAND STUDIO APAC Sydney, AUSTRALIA

Credits

Name Company Position
Tara McKenty Google Creative Head
Kazuha Okuda Google Head of Strategy
Blake Kus Google Creative Technologist
Shuhei Iitsuka Google Creative Technologist
David Arcus Google Creative Technologist
Simone Takasaki Google Head of Production
Claudia Cristovao Google CCO
Joel Cruz Google Creative
Erin Tuschiya Google Producer
Jihye Lee Google Designer
Fredrik Fridborn Signality Creative
David Habrman Signality Creative
Tom Bird Stab Creative
Seamus Higgins R/GA Creative
Celia Karl R/GA Creative
Ben Miles R/GA Creative
Shaun O’Connor R/GA Creative
Darragh O'Connell R/GA Creative
Henry Cook R/GA Creative
Ciaran Park R/GA Creative
Mikael Rousson Signality Creative
Dimitrios Scoutas R/GA Creative
Tom Twiby R/GA Creative

Why is this work relevant for Innovation?

Surfing isn’t just a notoriously difficult sport to learn, but also to judge. Waves are highly variable, the flow or style of a surfer’s maneuver is subjective, and bias from judges towards particular surfers can occur too. Artificial intelligence can solve this, but is often perceived as a threat – something that will replace humans. Instead of replacing judges, Huey supplements their work by providing accurate and fast metrics on things that are hard to see for humans: a surfer’s speed, height, rotation, etc. Only a person can be a judge, but Huey can see what they might have missed.

Background

Machine Learning can produce incredible insights, but often the stories around its value are prosaic and unrelatable. Huey is about a unique human story – ML working in tandem with people to remove biases and improve a sport. Huey is composed of existing ML tools, like human pose estimation and pattern recognition, as well as traditional computer vision tasks, like optical flow for speed, and depth perception for distance. The initial working prototype – built with Tensorflow – could predict surfers’ maneuvers and speed from archival footage of past surf competitions. To that limited dataset, hours of surfing footage from a wave pool were added, to create robust quality for the ML model. As COVID19 prevented Huey’s first live judging event from happening, that has become a live demo instead. The dataset keeps being added to over time, which makes Huey progressively even more robust.

Describe the idea

Huey aims to remove unconscious bias from judges in surfing. Surfers of a particular age, gender, status looks or even nationality might be judged beyond their performance. This tool creates metrics for evaluating the skill of a surfer on aspects that are factual. Judges can compare exact values on how high a surfer rose from a wave, how vertical they were and how much they rotated – and close the gaps of uncertainty our unconscious biases can hide in. The tool itself is the first of its kind, and has been built on the Tensorflow platform with training data gathered specifically for this project. Huey is the result of a small group of passionate creatives, engineers and surfers with a limited budget and resources. It ultimately seeks to erase bias from sports, one wave at a time.

What were the key dates in the development process?

19th July 2019 - Proof of concept complete 21st Sept 2019 - Money secured from stakeholders to carry out Feasibility study 14th of November 2019 - Feasibility and testing 21st of March 2020 - Wave Pool Event August - December 2020 - UI design and GFX exploration December 2021- March 2022 - Develop and test judges’ interface and app 22nd of March 2022 - Stab High Surf Competition wave pool launch (Our first Surfing event to demonstrate Huey was March 2020, due to Covid19 we were unable to fly international surfers to Australia to compete in this event, we pivoted the event and hosted a demonstrational event, and used the footage from that day to contribute to training data.)

Describe the innovation / technology

Our ML model has been trained from hundreds of hours of surfing footage. The training footage combined cameras from three fixed positions, allowing us to gather data and predict a surfer’s position in 3D. The footage was annotated with keypoints across a surfer’s body and surfboard as part of the training process. This allows us to predict the pose and position of both the surfer and their surfboard. Our tool can take a camera feed from a live event, and in real-time detect a surfer’s height from the wave, their verticalness, rotation, projection and their completion and type of maneuver. Project Huey is built on top of TensorFlow with training and processing run on Google Cloud. The initial proof of concept, training data library, the feasibility study, and prototype has been completed. We are currently working on the MVP and exploring the tool’s UI and visualisations for the judges and viewers for the Stab High Surfing event in March 2022.

Describe the expectations / outcome

While Project Huey was developed for surf competition judging, the scalability into other, and mostly less complex sports, is the logical next step. Besides the implied long-term outcome of freeing sports judging from biases, Huey aims to start a bigger conversation. A conversation about what humans are good at, what machines are good at - and how we can maximise these strengths. We found ourselves in an advertising landscape where robots dance, machines paint and algorithms write poems. Project Huey looks to innovate on where we choose to innovate and how we use technology for what. We hope that Project Huey will become a role model on ethical, productive and truly beneficial use of technology that does what humans can’t - instead of replacing their subjectivity.