The Barcelona Declaration of Research Principles
Barcelona Declaration of Measurement Principles slides (PDF)
A new declaration of standards and practices to guide measurement and evaluation of public relations was discussed in Barcelona this week and adopted by delegates attending at the 2nd European Summit on Measurement, organized by AMEC and the Institute for Public Relations.
The language may not yet be perfect – and on the surface, some of the principles may seem obvious – but this is a credible attempt by some 200 people from more than a dozen countries to address the need for clear standards and common approaches to measuring and evaluating public relations results.
“What we’re doing here is setting a baseline to build on,” said David Rockland, Partner/CEO of Ketchum Pleon Change and Global Research, in moderating discussion. “This is the basic philosophy behind a set of standard practices.” The standards will be refined based on feedback received in Spain and additional input gathered here [please comment below] and by AMEC over the coming weeks.
For additional information and perspective, Institute for Public Relations research papers are linked wherever appropriate.
Here’s a summary of the principles, also posted at AMEC.
1. The Importance of Goal Setting and Measurement
Fundamentally important, goals should be as quantitative as possible and address who, what, when and how much impact is expected from a public relations campaign. Traditional and social media should be measured as well as changes in stakeholder awareness, comprehension, attitude and behavior. Paul Holmes was present and rightfully took exception with the measurement term “target audiences.” Instead, we need to think in terms of communities of “stakeholders” as the power of communications shifts from companies and institutions to communities of individuals.
For more on goal setting, see the Institute’s recent paper on “Guidelines for Setting Measureable Public Relations Objectives.”
2. Media Measurement Requires Quantity and Quality
This principle acknowledges that overall clip counts and impressions are usually meaningless. Instead media measurement should account for impressions among stakeholder audiences and quality (eg. tone, credibility of the source and media outlet), message delivery, inclusion of 3rd party spokespersons, prominence and visual dimension. Importantly, this principle also suggests that quality can be defined as negative, positive or neutral.
3. AVEs are not the Value of Public Relations
There was near-total agreement on this principle in Barcelona (92%) but the group was split on what other validated metrics to use in place of AVEs. (Weighted Media Cost was one suggestion.). The legitimate intent here is not to debate the validity of AVEs (which simply measure the cost of media space) but to move beyond this measure once and for all. Also, this principle acknowledges that multipliers are “silly” and should never be applied unless proven to exist in a specific case.
4. Social Media Can and Should be Measured
- Organizations need clearly defined goals and outcomes for social media
- Evaluating quality and quantity is critical just as with traditional media
- Media content analysis should be supplemented by web and search analytics, sales and CRM data, survey data and other methods.
- Given the scale and volume of social media, technology-assisted analysis may be necessary.
- Measurement must focus on conversations and communities, not “coverage”
- Understanding reach and influence is important, but existing sources are not acceptable, transparent or consistent enough to be reliable. Experimentation and testing are key to success.
A recent paper published by the Institute’s Commission on Public Relations Measurement and Evaluation (download here) outlined practical steps for public relations practitioners who want to adopt web analytics as part of their media measurement strategy.
5. Measuring Outcomes is Preferred to Measuring Media Results
- This principle suggests that: Outcomes include shifts in awareness, comprehension, attitude and behavior related to purchase, donations, brand equity, corporate reputation, employee engagement, public policy investment decisions and other shifts in audiences regarding a company, NGO, government or entity as well as the audience’s own beliefs and behaviors.
- The proposal that “benchmark and tracking survey research are the preferred practices for quantitative measurement” almost certainly will be expanded to include and acknowledge the value of qualitative research methodology. (Some researchers suggest that in addition to being descriptive, PR research is dominated by a short-term quantitative tradition. Some contend “that no everything that counts can be counted and not everything that can be counted, counts.”)
- Standard best practices in survey research — including sample design, question wording and order and statistical analysis – should be applied in total transparency.
6. Business (read: Organizational) Results Can and should be Measured Where Possible
- Models that determine the effects of the quantity and quality of PR outputs on sales or other business metrics, while accounting for other variables that drive sales, are a preferred choice for measuring consumer or brand marketing. Related points are:
- Clients are creating demand for market mix models to evaluate the impact of consumer marketing
- The PR industry needs to understand the value and implications of market mix models for accurate evaluation of consumer marketing PR in contrast to other measurement approaches
- The PR industry needs to develop measures that can provide reliable input into market mix models
- Survey research can also be used to isolate the change in purchasing preference or attitude shift resulting from exposure to PR initiatives.
An important point for consideration was made by Ketchum’s John Paluszek, also chair of the Global Alliance: “The results of other organizations that we serve are critically important. Our field is growing in its service to NGOs, charitable organizations, governments, the military; organizations that fall outside the business perimeter. We should be talking about ‘organizational results’ instead of only ‘business results.'”
7. Transparency and Relicability are Paramount to Sound Measurement
- PR measurement should be done in a manner that is transparent and replicable.
What’s next?
With additional comment from the global PR measurement community, the Barcelona Principles will be further refined and distributed in mid-July for further discussion. Also, we look forward to exploring additional principles at the Institute for Public Relations 8th North American Summit on Measurement October 6-8 in Portsmouth, New Hampshire.
So, what do you think?
Robert W. Grupp
President and CEO
Institute for Public Relations
I have read most but not all posts. My perspective is from the boardroom of a Fortune 100 company. I would ask us all: where is the connection to creating sustainable, competitive advantage? I believe the seminal work done around the Barcelona Principles delivers the PR/Comms industry to a whole new level of performance, credibility, and practice. Now, it is time to step up another rung where strategy informs the work, and the PR/Comms people become the key link to informing the two most important constituents: employees and customers.
My comment comes from years of viewing marketing that focused on the finer points of execution while failing to deliver little more than incremental value. This dearth of strategic-driven marketing led other disciplines to fill the void left by marketers. M&A heated up to infuse the P&L with growth and profit, and the cost accountants came in to trim the fat through strategic sourcing and a plethora of turnip squeezing measures intended to unleash more bottom line profit. Meanwhile the marketers got lost in coupons, trade incentives, and predatory discount schemes tied to contracts. This was marginalization at work as marketers lost sight of their chief role: create, capture, and sustain value.
What most enlightened CEOs want is sustainable growth. The best way to deliver that is by creating competitive advantage. The path begins with well-informed strategy based on a clear view of segmentation, sharply defined target markets, and stunning positioning built on a well-defined value proposition that informs the PR/Comms people on what the firm is attempting to do.
Marketing is full of mumbo-jumbo that can be off-putting and, at times, bewildering. Marketers are under great pressure to deliver the goods. Often they take shortcuts. ANd, it seems when efforts fall short, they have ample data to support explain it away, or buy new life.
As for my colleagues who have done the pioneering work in measurement that has been burning bright for five years, Bravo! The work has been groundbreaking. And now, it is time to move to the next level, where you don’t get to sit at the big table they demand you sit at the big table.
Hats off for starting the revolution. I will be a vocal supporter and provocateur!
Tim McMahon PhD
Creighton University
I certainly do hope that these principles crystallize in some hard procedures to measure PR effectiveness. Until then, they’re just more industry blather. The only thing they reinforce is the need for professional research ahead of a program that supports business objectives.
Mark, let me build on your final and very important sentence.The first step in the process is to understand business goals and objectives, and then the communications goals and objectives supporting the business goals. The second step is to build in, as part of the program development process, measurable objectives. The paper below discusses this key process. Research and measurement should be built in, not bolted on.
https://instituteforpr.org/topics/setting-measurable-objectives/
You can hear me talk about my approach at the following presentation at the 2012 International Public Relations Association conference: http://bit.ly/R2DZNw
As you note, the Barcelona Principles are just that, principles. The Coalition for Public Relations Research Standards is working on developing standards and best practices for use by the profession. As the International Organization for Standardization (ISO) notes, the adoption of standards is market-driven. Standards will be adopted to the extent that clients (corporations and other) insist that in-house departments, agencies, and research firms adopt and use standards.
#7 should say “replicability” not “relicability”… proofing for typos enhances credibility and gets better results 🙂
Thanks for pointing that out! We have fixed the typo.
Thank you, I’ve recently been looking for information about this topic for ages and yours is the greatest I have found out till now. But, what in regards to the bottom line? Are you certain about the supply?|What i do not understood is in reality how you’re not actually much more well-liked than you may be right now. You’re so intelligent.
I’m sorry. Replicability is a word. I’m not so worried. I see you just forgot the “p”. And I misused “your” — bad day. Sorry!
We’re in communications and your using a term that doesn’t exist on the Institute for PR website? I’m confused and worried. What the heck is relicability? And since you’re talking about replication, maybe you meant replicability–which I will also point out, IS NOT A WORD.
This is an excellent essay. It throws light on the research aspects. But I wish u to tell us the kinds of research. In this way, v will be able to know the value/priority of such kinds of research. Thanks. God bless u for guiding the research schlars.
This is a great move in the right direction to ensure communication practitioners remain relevant in years to come. There is an urgent need for PR activities to be measured against key business objectives and principles that guide business. AVE has historically assumed that communication is the end result – i.e. the fact that exposure was achieved means the communication goals were met. Now, communication is a means to an end – the resultant effects of exposure should be what communicators seek out.
For years, the South African PR landscape has used AVE as a unit of measure – now, people are taking note that while AVE serves a purpose (how much space did I get), the business outcomes are what will determine success of a media campaign. Did you increase sales? Change perceptions? Change behaviours?
We cannot drop AVE, but we can add to.
Daniel Munslow
Chief Communications Officer
Talk2Us
Johannesburg, South Africa
This declaration is a small step to achieve the perfect, objective and complex standard of PR measurement and evaluation. This declaration is useful, but not enough, because these principles are well known amongst PR professionals, so it can’t be a real compass for them.
I hope the real, accepted, relevant standard will be prepared by e.g. IPR asap.
Gergely Bognar
Comprad Communications
http://www.comprad.hu
Hungary
What is most useful about the Barcelona Declaration is that we now have an “authoritative” document to help clients understand that we are not just making up measurement voodoo. Warren Delos Reyes says dropping AVE totally could confuse clients. I know many who haven’t even made it that far along the measurement curve. I am a big believer that we need to be measuring outcomes, but it’s not always easy, especially with clients who see communicators as the people who write brochures or organise cocktail receptions.
Kristen Sukalac
Prospero Communications
Paris, France
And while we’re at it, could we come up with a better term than “impressions”?
It’s borrowed from advertising and means the total number of exposures or opportunities to see the advertisement delivered by a media schedule, i.e.,
reach X frequency = gross impressions.
As such, it isn’t helpful or useful as a measure of PR message delivery. Call them media cubits or doolallys or something else, but let’s get away from using impressions.
Bill Huey
Strategic Communications
Atlanta
While it can be said that AVE is not really the value of PR (literally and figuratively) I could say that it should not be totally be taken out since AVE reflects the audience or readership of that certain media. Moreso, in the SEA region wherein most if not all PR practitioners still use AVE as one attribute for measurement eliminating it completely may just confuse them. What should be done is proper education and dissemination of its proper use when it comes to its terminology and proper use. It could also be suggested that Media Measurement companies be given some free hand to still use AVE but should employ or add some other measurement factors like tone, messages, share of exposure, spokesperson, etc. to make it more well rounded and provide more insights and analysis. I do agree that having AVE alone and not adding other measurement attributes is not proper method of measuring PR value.
Congrats for the hard work, it is really useful to people like me, already in the business but lacking conceptual basics, and this is a very good source of study.
But as Julian mentioned, communication is reaching a risk of being too term-oriented. Simplicity is the key
While the implementation of measurable PR objectives bring many benefits on corporate reputation, the absence of them may easily cause loss of efficiency and considerable number of crisis. By avoiding to be “measure” focussed, I think PR industry today needs new and updated transparent measurement criteria especially for the social media. I agree with Tina about the conceptual basics, and I’ll be studying on this matter scholarly. Congratulations for the progress that has been made. Looking forward to see the additional Barcelona principles… Dr.Idil Sayimer,Turkey.
That’s one small step for PRers; one giant leap for the industry.
First off, congratulations on a first step to create a stronger basement for evaluation & measurement in PR.
I appreciate all the hard work that many of us put into this declaration.
While I’m generally enthusiastic, I feel that the industry is still very much focused on talking about names (web analytics, CRM, media resonance) than concepts or methodologies that might be suitable. Sometimes it seems as if we fight over the supremacy of terms – not over the best way to measure the impact of communication.
Also, I see the danger of getting too “measure” focused. Measurement is important but we need to measure what we think will have an impact not what we know to have impressive but potentially useless numbers as a result. While I acknowledge that we hopefully got past the AVE discussion I think we might still be tempted to go with whatever seems most impressive (although we get into trouble when clients ask us what it actually means…).
Finally, under point six it is mentioned that survey research can be used to isolate change. This is only partly true. There are experimental designs involving surveys which might provide us with conclusions in how far PR is responsible for changes. However, simply asking people wether it was “PR which did it” provides us with nothing but a very biased view of that individual. And we cannot even possibly start to think of a wording for that question which will not prime the interviewee in a certain way – leave alone influence their mind to actually tell them the truth of what actually happened in this little black box while our message was going through. While PR has indeed underestimated the power of survey methodologies for quite some time it’s not the holy grail in research. A survey on its own can never determine causality. That’s something we really need to keep in mind.
Despite those points I am happy about the progress that has been made and hope that some of it will trickle down into everyday work.
Well done,
Julian