A personalized data and AI-driven storytelling experience

The project was initiated by the artist and filmmaker group Laokoon. In 2020 the group called on people from all over Europe through social media spots to hand over their personal google data and anonymously provide them with this data for an artistic experiment.

With "Made To Measure" they wanted to illustrate how online companies, advertisers, and other data traders use the information they collect – by telling an interactive story, based on the submitted data of a real person. We helped them realize that vision.

A transmedial experience

The website experience closely intertwines movie and interactive elements by inviting the visitors to answer questions and discover underlying data through explorative visualizations. To achieve this, we designed and developed a custom video player that seamlessly switches between video playback and interactive elements allowing coherent integration. Our custom-made solution includes an individual approach to host and stream media content. It enhances the experience, guarantees a reliable media delivery in different viewing situations, and minimizes the users' data footprint—a core aspect for a project which focuses on shining a light on conscious usage of personal data.

For every platform

We optimized all videos, visualizations, and interactions for desktop and mobile use cases. Our custom editorial workflow allows us to control and edit the whole experience, regardless of targeted device type or medium, and creates a new model of interactive storytelling on every platform.

Data to understand

As part of the data exploration, we built an internal tool to perform research tasks like finding occurrences and patterns to the Google search queries in the raw dataset, which consisted of 50.000+ searches and visits. This prototype built the foundation for the final data visualization, allowing visitors to explore a fragment of the collected google searches.

Data to tell a story

The whole interactive narrative reveals itself through data fragments that uncover details about the experience and identity of the character – an Austrian woman in her 30s. In a real-time animation overlays the experience also shows her searches which – more often than not – implicate personal feelings, fears, and desires.

From data to awareness

Without knowing, each visitor acts as a data source while interacting with websites. The application uses device profiling and tracks and analyses the behavior of each visitor: How actively do they participate? Which questions or scenes do they skip? How much is the mouse moved? What kind of device do they use? All of this data is collected and used to calculate the psychological profile of the visitor. Without any warning, this visitor is presented with this profile in the climactic moment of the narrative. The website reveals itself as a judgemental speaking digital clone of the protagonist.

A digital clone trained by machines

The data-influenced climax, spoken by the synthetic voice of the protagonist, required machine learning in two languages. In collaboration with Jack Rusher , we cloned the voice using state-of-the-art artificial intelligence tools to generate any text to speech to achieve this.