When anyone asks why standards are important, I cite a recent conversation with a client. It started with what seemed a simple question: “Are we able to see how many people saw the stories, and how do you calculate reach?”
A dozen emails later, we finally sorted out that what the client called reach, we call opportunities to see. What she was calling placement, we call items. And what we referred to as “placement” didn’t matter to the client at all.
Which is why, in early 2011, the members of the IPR Measurement Commission, AMEC, PRSA and the Council of PR Firms began work on a set of standards for both traditional and social media measurement.
How does one establish a standard?
Any standard- setting process takes an unbelievable number of late night and early morning intercontinental phone calls on top of some solid thoughtful time to sort out just exactly what is going to be a standard or a guideline or a best practice.
For us on the IPR Social Media Measurement Standards Committee, it was a question of starting where the Barcelona principles left off. Principle #6 said that “social media can and should be measured.” In 2011, attendees at the Lisbon Summit made defining those social media measures its number one priority.
The first step was to sort out what groups were working on the same issue and how much progress had they made. Which is why we held a meeting in New Hampshire last October that brought together the Word of Mouth Marketing Association, the IABC, the Digital Analytics Association, the Advertising Research Foundation and others ( see below).
Dubbed “the Conclave” we quickly realized that we were all wandering around in a standard-less miasma of shared confusion. We decided that there were five broad categories of standards that needed to be addressed. They were:
- Content
- Reach & Impressions
- Engagement
- Influence & Relevance
- Impact & Value
For each, we agreed to four-step process. One of us would draft a set of standards and solicit feedback from members of The Conclave. After incorporating the comments, we would then announce them to the industry as a proposed interim standard and accept further comments. Once the comment period was closed, we would release them to a customer panel for adoption. After that happens, the standards would be released to the marketplace as interim standards, ready for implementation.
For the first item on the list, Content, as in what constitutes content and where it comes from: We quickly realized that, given the scope of possibilities for content definition, no single group of people was going to be able to control all the various methodologies for collecting content. However, we felt that vendors owe it to clients to fully disclose how data and content was being collected. We concluded that the best approach would be to call for a transparency table, essentially the social media equivalent of that food nutrition label on the side of a cereal box. Here is that label, and as you can see it doesn’t recommend one specific way of doing things, but rather provides for the client, a reference so he or she knows what is comparable between vendors.
Filled out, the form might look like this:
Specifically, the table captures critical information about social media content sources and methods to provide full transparency and easy comparison across analyses. Including: What content and channels are included? How is the data collected? How deep is the analysis? Are multiple languages captured? Via native-language queries? How are key metrics calculated for reach, engagement, influence and opinion/advocacy? How is sentiment coded? How is irrelevant content (bots, spam blogs, etc.) filtered? What proprietary methods were used in the analysis? What search strings were used? We strongly recommended that a full, detailed list of search terms should be attached to the form.
Where are we now?
Now that the Transparency Table has been published as a proposed interim standard, we will work with major clients, agencies, research providers and software vendors to get their commitments to use the table, include it in their reports and capture their feedback on what works and what needs refinement. That feedback will then fuel modifications as needed for review by Conclave members to deliver a formal “approved industry standard” later this year. While the table itself may seem straightforward, we believe it will play a critical foundational role for clients who need clarity and transparency from their providers — and enable better education of the industry about critical content sourcing and methodology issues that are fundamental to sound social media analysis.
We are currently tackling the issues of Reach & Impressions as well as Engagement with a goal of having standards available by the next Conclave meeting October 3rd. The topics of Influence & Relevance will be tackled later in the fall and we’ll explore Impact & Value early in 2013. We are now also working closely with the Media Research Council that has been appointed to create standard definitions of social media reach by the Interactive Advertising Bureau and WOMMA.
We look forward to your input and feedback. What do you think about our priorities? How will you use the Transparency Table? Is our preliminary guidance on target? What are we missing?
For more information, follow the posts at www.smmstandards.org .
Katie Delahaye Paine is the founder and CEO of KDPaine & Partners LLC. She is the author of the recently released Measure What Matters, Online Tools for Understanding Customers, Social Media, Engagement, and Key Relationships and is the publisher of “kdpaine.blogs.com” and “The Measurement Standard”.
While I agree with Harry’s comments, I can also see how using this sort of table would be very helpful from a client’s point of view, for social media evaluation, and also, to a lesser extent, for traditional evaluation. I have had similar drawn out conversations with clients as Katie has, as we try to reach a common ground on the terms we use, and what we each mean by them. I imagine it will be a useful aid to give clarity until the industry comes to some agreement on measuring things like reach. I look forward to seeing how this develops over the coming months.
I commend the initiative and the progress towards standards is important. But I question the narrowness of some terms/labels. For instance, ‘sentiment’ is a widely used term, particularly in the US, where it seems to be stand as a measure of positivity/negativity and the overall message rating of an item. However, sentiment, as defined in most dictionaries, refers to a judgement prompted by feeling or emotion. At best, it is opinion. However, the qualitative dimension of media content is often based on facts that are selected and presented and particular messages that may have nothing to do with emotion of feelings. Others refer to ‘tone’, ‘favourability’ or ‘positivity/negativity’. Should these widely used (and arguably better) terms be used or at least included in the table?
I find this concept of standard rather curious. In the article it is stated: “Here is that label, and as you can see it doesn’t recommend one specific way of doing things”. That seems be just the opposite of what a standard is designed to accomplish. A standard is not a suggestion. It is common approach that everyone adheres to.