ProjectMade to MeasureClientLaokoon

A personalized data and AI-driven storytelling experience


The Challenge

One of germany’s most forwardthinking filmmakers approached us with a bet – and a lot of data.


The Result

An interactive, data- and AI-driven storytelling experience reaching over 50.000 views in the first month.

Our Expertise

  • Ideation
  • UX & UI Design
  • Full-Stack Development
  • Data Analysis & Visualization
  • AI & Machine Learning


The Brief

One mission, many perspectives

The project was initiated by the artist and filmmaker group Laokoon. In 2020 the group called on people from all over Europe through social media spots to hand over their personal google data and anonymously provide them with this data for an artistic experiment.

With "Made To Measure" they wanted to illustrate how online companies, advertisers, and other data traders use the information they collect, by telling a story – based submitted data of real persons.

A transmedial experience

The website experience closely intertwines movie and interactive elements by inviting the visitors to answer questions and discover underlying data through explorative visualizations. To achieve this, we designed and developed a custom video player that seamlessly switches between video playback and interactive elements allowing coherent integration. Our custom-made solution includes an individual approach to host and stream media content. It enhances the experience, guarantees a reliable media delivery in different viewing situations, and minimizes the users' data footprint—a core aspect for a project which focuses on shining a light on conscious usage of personal data.

For every platform

We optimized all videos, visualizations, and interactions for desktop and mobile use cases. Our custom editorial workflow allows us to control and edit the whole experience, regardless of targeted device type or medium, and creates a new model of interactive storytelling on every platform.

Data to understand

As part of the data exploration, we built an internal tool to perform research tasks like finding occurrences and patterns to the Google search queries in the raw dataset, which consisted of 50.000+ searches and visits. This prototype built the foundation for the final data visualization, allowing visitors to explore a fragment of the collected google searches.

Data to tell a story

The whole interactive narrative reveals itself through data fragments that uncover details about the experience and identity of the character – an Austrian woman in her 30s. In a real-time animation overlays the experience also shows her searches which – more often than not – implicate personal feelings, fears, and desires.

From data to awareness

Without knowing, each visitor acts as a data source while interacting with websites. The application uses device profiling and tracks and analyses the behavior of each visitor: How actively do they participate? Which questions or scenes do they skip? How much is the mouse moved? What kind of device do they use? All of this data is collected and used to calculate the psychological profile of the visitor. Without any warning, this visitor is presented with this profile in the climatic "breakdown" of the narrative. The website reveals itself as a judgemental speaking digital clone of the protagonist.

A digital clone trained by machines

The data-influenced climax, spoken by the synthetic voice of the protagonist, required machine learning in two languages. In collaboration with Jack Rusher, we cloned the voice using state-of-the-art artificial intelligence tools to generate any text to speech to achieve this.

Layers

Background

In cooperation with researchers from the Stanford Media and Personality Lab, Laokoon built a comprehensive personality profile from the data, just like Facebook, Google & Co. do through algorithms of billions of people. This profile became the basis for the whole interactive movie. Five years of this person's life were reconstructed in detail on a large theatre stage and filmed. A few months later, the original and her datafied double met.

Our interactive cross-media then uses this data and weaves it into a fascinating story, combinging realtime interactive data visualizations, video streaming and a synthesized voice in an unprecedented way.

It attracted more than 50.000 visitors in the first month and was part of the Ars Electronica Theme Exhibition.

Trailer

Credits

Related Projects
View more data visualizations and information interfaces
© 2011 - 2022
Studio NAND GmbH