Metrics: Making Meaning out of Data
Last week I wrote about how the way we've gone about using data and technology for audience targeting has led us astray in our quest for more reach and, in turn, greater share for the advertiser.1
What I wrote in conclusion was that an advertiser doesn't really need more or better data (while both can, though not always, be useful) but better metrics planning.
What does that mean?
Let's define metrics. A lot of people use metrics and data interchangeably. They are not. Data are what metrics read. Metrics are a method of measuring something. So, you’ve got to answer: what’s being measured? But before that, you’ve got to answer: why measure at all? The easy answer — so I know what’s going on — doesn’t offer much on its own.
A valuable starting place is a framework I borrow from my early years in this business as a media auditor. In those days, the agency I worked for -- Hawk Media in San Francisco -- was responsible for posting all Burger King's Spot TV. That means, we'd look at the media plans for every DMA purchased for Burger King's "spot fill" effort (spot fill was a way to lift rating points against a buy target in a market that had "weak" delivery against that audience when reached by national television; "Seinfeld" was popular, but there were less people watching it in Wilkes-Barre than there were in New York, e.g.). The process to breakdown the analysis was always the same: what was planned? What was purchased? What was delivered? The metrics used to determine the "success" of the buy then was how closely the costs and the ratings held from planning through to delivery, and how those compared to the market overall that was under examination.
Based on what I learned then, the framework I use now for metrics planning is this:
Verification: prove you bought what you planned and got what you paid for
Validation: that your plan was the right one, a good one, or one that went towards success of delivering the objective
Determination/recommendation: provides indication and direction for what to do next; changes, improvements, nothing at all
A plan will vary based on a few dependencies relating to the objective and availabilities, but the framework structuring it based on the above is pretty stable. The big myth with metrics is that they are meant to reveal truth. But this is not a philosophic exercise to articulate a Kantian "ding-an-sich" (thing-in-itself). What we want is a dependable process built around finding what works.
When the numerati took the lead in the advertising industry by way of digital, the business saw measurement as the way to find out what's realy going on with how ads work. But as we automated things, we started measuring only what was countable. It wasn't long before measurement got confused with what can be measured, and data became the most important thing, instead of what data could tell us. Instead of the increasing volume and granularity bringing us closer to what we wanted to know, it took us further away. As the science historian, Theodore Porter, wrote, "Quantification is a technology of distance" and that the objectivity we think it brings us derives its force from cultural contexts not from what's being quantified.
A good metrics plan keeps the distinction between measurement and data clear, and focuses on defining what works. It's a process that can help set up for a repetition of success and a way to find out what went wrong when success is not the result.
Meskauskas, James. “The Myth of Targeting.” Media Darwin, Media Darwin, 19 Jan. 2022, https://www.mediadarwin.com/writings/the-myth-of-targeting. ↩︎
