I’ve been on the front lines of PR measurement for over 10 years in my role at General Motors. In that time, I’ve sat through more vendor pitches than I care to remember and watched many squirm uncomfortably as I poked at the black box that is their particular proprietary methodology.
To make matters worse, everyone has their own definitions for what you would think would be common terms so you truly have to poke and prod to make certain that you know what each vendor’s system is actually measuring . As an example of the lack of clarity in definition, I was in a meeting the other day with some very smart folks and there was discussion of the definition for impressions. You had some equating it to circulation, others were trying to add on factors like where it was in the publication, others were calling it opportunity to see (which I know from experience also has several definitions of its own), and it went on from there.
As technology continues to evolve, the measurement world keeps changing. New companies come in and others go out. Mergers and acquisitions happen constantly. New communication formats like Twitter, Facebook, YouTube, etc. are introduced.
With all that in mind, here’s a very basic reason why having measurement standards in place is important. Companies work with different research firms and PR agencies depending upon their needs. If we have standard definitions across the PR world, you can then compare measures from an event with Agency A to an event with Agency B to understand what tactics worked, what didn’t and how you might change/improve those events the next time. If the measurement systems don’t match, it is much harder, and often nearly impossible, to assess how the events compare.
I understand that there are almost as many different measurement systems as there are stars in the sky. In the PR measurement world, you have firms that specialize in measurement, research departments at PR agencies, corporate types who have developed their own DIY method of measurement, and the college professors who are trying to teach the next generation of PR practitioners the importance of research within the context of strategy. A lot of time, money and effort have been spent on developing the proprietary tools and measurement processes currently in use so there is reluctance to change. However, even within the proprietary systems, I believe there is room to standardize the basic measures. The value will then come from how you utilize the measures to move strategy forward rather than debating the pros/cons and definitions of your particular measures. The more we can remove debate about measurement process and instead focus on strategy, the faster we all win.
Jackie Matthews leads Communications Research and Measurement for General Motors at its headquarters in Detroit, Michigan. She is an elected member of the IPR Commission on PR Measurement and Evaluation and will be inducted into PR News’ Measurement Hall of Fame at its measurement conference in April, 2012.
Jackie, very valid point. I am India head of CARMA International, a global media analysis company. Lot of our global mandates do arise from this customer need of having consistent and globally benchmarked measurement systems across markets they operate. Taking it beyond, very rightly PR measurement industry does need to come together to create some standards that can be a guide for the customer to understand / evaluate / use various measurement solutions.
Imagine a world in which the goal of medical research was to prove to the patient that the doctor has done a swell job. Not how to treat a condition in most cases; not how well an intervention worked on a particular case. No, the goal is just to make the doctor look good.
Unfortunately, this is the world – much of the time – of public relations research.
Jackie Matthews’ post provides a compelling insight into why the PR research sector is stalled in interminable debates. We do not need proprietary methodologies for PR research (do you really want to fly into an airport that has its own proprietary method of air traffic control? Go to a dentist who has her own proprietary method of root canal?).
The “science beneath the art of public relations” demands that we mature and start to apply rigorously and consistently scientific principles. We need to have meaningful industry standards, and focus, as Jackie Matthews writes, on strategy – when it’s working and when it’s not.
The encouraging part of this discussion is that we have Jackie Matthews and General Motors pushing for such a maturation of the PR research sector. As current IPR efforts relating to research standards goes forward this year, it is reassuring to know that companies such as GM “get it” and will be in support. GM and other big buyers of research can drive the market for more authentic, useful, and ethical communications research.
Sounds like you are holding the starting gun for a race to the bottom. If anyone even mentions “impressions” in a conversation about PR measurement, you should stand up and throw them out of your office. Impressions is an advertising term and anyone who tries to apply it to the measurement of public relations outcomes is a charlatan.
But there are worse things out there. For example, Facebook’s chief marketing officer, David Fischer, recently told the Wall Street Journal,”The average Facebook user has 130 friends, which equates to four degrees of separation to one million people.”
Four degrees of separation? Is that the methodolgy they are using to “monetize” Facebook and justify its $100 billion market-cap valuation? I hope I never see that methodology come near the PR business.
Bill Huey
Strategic Communications
scbhuey.com
Jackie – thanks for posting the clarion call for standards. Your last two sentences resonate most strongly for me — “The value will then come from how you utilize the measures to move strategy forward rather than debating the pros/cons and definitions of your particular measures. The more we can remove debate about measurement process and instead focus on strategy, the faster we all win.”
To @Mike’s point, something like “impressions” may seem easy to operationalize in the advertising context, but is “reach” the same thing? How about “opportunities to see,” or “ratings points?” Do we rely on audited circulation, internet dayparts or divining rod?
I don’t think anyone wants to use obtuse definitions unrelated to other media understandings, but without a clear view of what we actually are talking about when we use terminology, we’re in risky territory.
That said, Jackie’s main point should be a no-brainer — if everyone is keeping their methods in a black box, we can’t compare fairly. Standards will help us do so, while respecting the secret sauce that drives innovation and profit.
Cheers!
Sean Williams, member, Institute for PR Measurement Commission
@Commammo
Jackie,
Good points. But the industry is making exposure/audience measurement more difficult than it needs to be.
One of the things that drives me nuts when hearing wacky operationalizations for terms like “impressions” is that nearly all sizing-related measurement terms already have standard definitions from their use in advertising. Just because a term is being applied in a PR measurement context doesn’t mean that its definition needs to be unrelated to the rest of the media world’s understanding of the term. For most of these terms, we’re talking about exposure measurement, and that doesn’t differ one bit, conceptually, whether it’s exposure to an ad or a story.