AN OUTDOOR RATINGS MANIFESTO

Traditional Sample-Based Research Won’t Solve Outdoor’s Ratings Problem.

By Erwin Ephron

Try counting grains of sand with your fingers. That’s what the ratings try to do today, and with about as much success.

Media are fragmenting and measuring the tiny pieces drives consumer-survey sampling crazy. But the industry seems not to have noticed.

Let’s start with local television, which gets the most press. The 2-sigma relative error on a rating of 0.5 from a sample of 1,000 is almost as large as the rating itself -- plus or minus 90%. Which means you can be pretty sure the 0.5 rating is bigger than zero and probably not bigger than 1.0. That’s really vague.[1] And as the ratings shrink, worse is yet to come.

But to get sampling error down to a manageable size, say +/- 20%, requires a sample of 16,000 and that’s clearly unaffordable. So we as an industry have decided we don’t need precision to buy media. We’ll buy averages and we’ll buy it in bulk.

But why buy in bulk when individual unit data can be reported at a reasonable cost? For TV we can incorporate cable set-top box data into the ratings and use sample-based research to model viewer demos. This would give us tighter ratings for all inventories and allow us to target, package and buy better. It would be a much improved television ratings service. The technique is called data integration, and you’ll be hearing a lot about it in the next few years. It is the future of ratings research.

Which brings us to Outdoor.

Hi Tech vs. High Sample-counts

Because technology is expensive, there is a perpetual tug-of-war in research between high tech and high sample counts. Today, a well-respected researcher, Nielsen, is pushing Outdoor to rely on technology in the form of small meter-based (GPS) consumer panels for its first ratings service. [2]

The satellite technology is impressive, but there are serious problems. The cost is high, so the Outdoor ratings will run out of sample. It’s not very technical. It’s Pareto’s Principle, the old rule of volumetric concentration.

For traffic on roads (as well as cereal in bowls) the highest volume 20% of a population will account for a lopsided share of the total volume. For Outdoor, the most heavily trafficked 20% of units will absorb most of the GPS panel’s response (it samples traffic), and leave very little sample to measure the balance.

This will make most Outdoor inventory unreportable, or reportable with extreme uncertainty, simply because there’s not enough sample.

Here are some rough numbers.

If we use a GPS sample of 800 respondent-weeks and the average respondent passes 1,500 out-of-home locations in a week (the Arbitron GPS number), that will generate a sample of 1,200,000 Outdoor exposures. Sounds like a lot. But there are 13,000 out-of-home units to be measured in a market like Chicago. And if 2,600 units (20%) absorb 720,000 exposures (60%), there is very little sample left to measure the other 10,400, especially if we want demos and not just gross persons ratings.

Buying And Selling Badly In Bulk

My estimate is the majority of the out-of-home in Chicago would not meet minimum standards for reporting. That would limit the individual unit ratings to the largest Outdoor locations and report the bulk of Outdoor as average unit or packages.

Packages as the basic media unit don’t work for buying and selling Outdoor because we don’t know enough about the medium to package it intelligently.[3]

Without individual unit demo data, sellers have no good way to construct the packages for inventory management and buyers have no good way to construct packages for targeting. It becomes media management by trial and error. A game of Blind Man’s Bluff.

Kiss VAI Goodbye

Another problem is package reporting can’t use VAI adjustments. The much needed Visual Accessibility Indices, which are used in Europe to correct Outdoor exposure data for “probability of seeing”, are unit specific. They cannot be applied to packages of audience, because they require the exposure contribution of each unit in the package. Packages force users to revert back to the old traffic-count data.

So using average Outdoor measurements will keep us from using better Outdoor measurements.

Data Integration is the future

Consumer survey data alone, be it GPS meter or travel log, is not sufficient to provide the detail an Out-of-home ratings service needs. The samples must be far too small for so widely dispersed a medium.

The key to reporting Outdoor ratings for most of the inventory in a market is to start with the traffic data associated with each unit location. Government and improved Traffic Audit Bureau data can generate counts of potential exposure for each location and GPS survey data can be introduced into the database to model reach, frequency and demographics. This is the approach Arbitron is taking.

Outdoor’s measurement solution is similar to the Internet, where large-sample server data needs to be combined with small-sample survey data to be able to report the smaller websites. And it forecasts television research, where, large-sample set-top box data will have to supplement small peoplemeter samples, even when they become passive.

The process is called data integration and it, as well as new technology, is the future of Outdoor and all audience research.

[1] If you think 2-sigma is an exaggeration, so is a 0.5 rating. Today most are smaller.

[2] Please note: the writer is a consultant for Arbitron, a competitor of Nielsen.

[3] The demo characteristics of the area where the unit is located won’t do for targeting or packaging, because people from other areas travel past that location. To learn who they are we do consumer surveys.

- September 3, 2003 -

^ top of page ^      < back <