Democratising incrementality measurement at the7stars

Click through rates (CTRs) have dominated the digital effectiveness conversation for 30 years due to their relative ease of measurement. Their consistency in the ever-evolving world of digital marketing has been an attractive attribute for anyone looking to optimise campaign performance - and the speed, length and clarity of the performance feedback loop is compelling in a digital tracking landscape that is in constant flux. But with CTRs outside of SEM routinely under 0.1%, it should now be well established that CTRs are a poor leading indicator of campaign effectiveness.

Yet, agencies like the7stars still regularly receive briefs listing CTRs as a campaign KPI. Often it’s still identified as a unifying metric to compare creative effectiveness, or to provide fast feedback for optimisation. However, CTRs tell us nothing useful or meaningful about the business outcomes marketers are ultimately tasked with driving; nor is it aligned to any specific channel role.

So why does it persist on briefs? It’s because, at the other end of the spectrum, measuring the incremental business outcomes driven by a campaign has always been harder. Traditionally, marketing analysts have focused on using multi-factor analysis methods (usually Market Mix Modelling) to tackle this. However, MMM outputs - although powerful - can represent abstract insights to navigate, and arrive with a significant reporting lag (often 10 weeks+).

Experiments on the other hand, are much simpler to interpret, and generate faster feedback loops. The concept of ‘going dark’ to measure the incremental effect of a campaign is intuitive and appealing, and the corresponding outputs paint a compelling picture of marketing’s business contribution.

Robust media experiments are notoriously difficult to design though, and require several interdependent factors to be managed. Opportunity costs like foregone sales must be carefully positioned to the business; overlapping planning territories must be targeted cleanly across multiple departments and media owners; alternative category response rates, target audience penetration levels and physical availability must all be optimised to construct a robust experimental campaign that doesn’t break the bank for advertisers.

Taking everything together - ‘speed to design’ is typically too slow for the traditional client/agency response to brief process, and incrementality experiments have therefore become an underutilised tool among the marketing community.  

At the7stars, we have developed an internal tool to increase that speed to design – Gravity Experiment. Developing this platform has significantly reduced the friction between setting a hypothesis with clients and designing a media plan that robustly tests that hypothesis. Our design algorithm is powerful enough to balance all interdependent factors and leverage the regional granularity available in digital channels to design campaigns with the lowest possible measurement risk and at the lowest possible media cost. All in a matter of seconds.

Over the course of the last year we have developed a web-based platform that is used agency wide, and has truly democratised incrementality experiments. It is flexible enough to be used across all channels and can even stack multiple channels and multiple hypotheses into a single experiment. The user-friendly interface means it can be used across all teams, short-cutting analyst time at the design stage.

Gravity Experiment has step changed our approach to test and learn as an agency. We can learn more, and with greater speed and accuracy than before. We can more elegantly communicate media’s contribution to business growth for our clients, and the continuous loop of hypothesise, plan, execute, learn, iterate - has levelled up our clients media strategies in a year of unprecedented disruption.

The platform has enabled us to execute experiments as diverse as an eight figure ‘scale up’ media plan for a DTC client, testing higher BVOD weights to mitigate inflation for a retailer, and to test the relative incrementality of Facebook and Google for a streaming campaign.

The tool has helped us to successfully evolve the conversation on digital marketing effectiveness in the agency. Gravity Experiment allows us to create a rich stock of effectiveness learnings at the rapid pace digital marketers have become accustomed to; all while delivering attribution measurement and insight at a higher level of integrity and reliability than ever before.