This post summarizes a roundtable discussion by the IPR Measurement Commission.
Katie Paine (Paine Publishing): All the things that we have traditionally measured don’t seem as valuable today. What are we as organizations valuing, and what should we be measuring? And especially for folks that are in bigger organizations than mine, do you sense that that’s changing? Are you measuring and valuing different things?
Long ago, the “Grammy of Communications” went to the person who got the most impressions and reported the most hits or clicks or whatever. That’s not going to happen today. I don’t know a lot of organizations that measure on that. Generally, they’re measuring on some kind of change, whether it’s increased awareness, or increased traffic, or saving lives. So, are we valuing things that are different and if so, what are they?
Elizabeth Rector (Cisco): I think it’s an “and.” I think some of the standard metrics aren’t going away, it’s just multiplied, and then we are asking how they come together. That’s blowing up in my world. And finding comms people…finding people who can have that mindset and that blend, because most of my kind of comms insights people have been very much traditional comms, and some of the outcome, like the reputation and brand and all of that. But as we’re getting into thought leadership, and driving to the website, and views, and paid media that’s all now under comms’ umbrella.
Working with marketing, because marketing is the lower end of the funnel, but now we have to hook into marketing with the paid, the thought leadership. Finding people who can think across all those is really hard. And what does that mean when you bring them together? Even educating the stakeholders who still say, “How much PR did you get? How many articles?”
So we’re starting to educate our executives about the modern communications organization, of the paid, earned, and owned. I guess we talk about shared, but we don’t talk about that as much. So, it’s really hard from an insights and analytics perspective to find people who can go across the board.
Katie Paine (Paine Publishing): Got it. That’s interesting, I’m working on a measurement framework for a client, and it’s the first time that I’ve seen upper-funnel metrics, mid-funnel metrics, and lower-funnel metrics – they’re using the marketing terminology and inserting the role of media at each stage.
But to your point, Elizabeth, it’s a combination of trust barometer, web conversions, share of influence, share of voice, and things like that. It’s a blend. Anybody else seeing the same thing?
Joseph Czabovsky (University of North Carolina): I don’t think the fundamentals have changed. As you were saying Katie, it’s still the different elements of funnel, or whether we’re doing awareness, attitude, or behavior, essentially. I think that’s still the core of what we do and what we measure. I entered this space 10 years ago when we were still trying to come up with how to measure social media, and now it feels like the argument’s the same – the challenge of what are we measuring or who are we measuring.
But especially the question of the who or the whom, that seems to be the biggest challenge for the next decade. Talking about non-PR measurements for example, I track the COVID data every day. And if you watch a national discussion, Arkansas could be this this week, Michigan could be next week, and then the next week, Arkansas is suddenly going well or badly, and Michigan is now going well or badly, and they’re in the opposite direction.
So, looking at those macro questions versus who are we talking about?
We’re only going to get more fragmented as an algorithm audience. And I think that’s something as a field that we’re going to have a big challenge with over the next 10 years, especially as we lose more and more of our traditional media forms that we build our measurements around.
Katie Paine (Paine Publishing): Do you see that that as a shift looking at measuring the audience as opposed to measuring the media that gets to the audience?
Joseph Czabovsky (University of North Carolina): Yeah, I’ve been arguing for an audience-focused and then media approach, and I feel like a lot of times we approach it media, down to audience. The example that I always think about is from the old movie “The Truman Show,” when he’s stuck in this world where everybody’s marketing to him, including his wife. And so she markets the slicer and dicer and all these things, and she’s talking to the external audience, but in the moment, she’s talking to him, and after her marketing pitch, he just looks around and goes, “Who are you talking to?”
And I always think about that example in terms of measurement, because if we don’t start with audience first, and then go back out to some of our questions or some of our media that we’re analyzing, I think we sometimes assume we’re talking about the right audience, but we’re really just relying on platform-based metrics.
Rob Jekielek (The Harris Poll): And that’s very much the reality in most marketing spheres as well. If you’re looking at social and programmatic and all of that, there’s very few people that are able to take an audience-first perspective. All your core KPIs are channel-based. That’s problematic, right? I think it’s a big struggle for a lot of organizations, both communications and marketing, getting to an audience-based perspective.
And it’s a really hard problem to solve, it’s not an obvious tweak. Elizabeth, you probably have some perspectives here. Like, you’re seeing all this stuff flowing all the time in everything you’re looking at. But it’s a very challenging problem, isn’t it?
Elizabeth Rector (Cisco): Absolutely.
Rob Jekielek (The Harris Poll): Real audience profiling is very hard, right? There’s no perfect model for it.
Elizabeth Rector (Cisco): Well, and from an outcome-based perspective, we look at CIOs, for example. We look at CIOs, IT decision-makers from our brand, and reputation outcome. Are we shifting those perceptions? Analysts, employees, and media, who are our key audiences, so we come about it from both angles.
We did intent-based modeling and tried to map what CIOs are searching for based on what we know, but you can’t get that persona-based data. We do, from our web analytics, have a lot of persona-based data, so you absolutely can get it from that, but third party search, ad, you can’t get that.
Rob Jekielek (The Harris Poll): And you guys are probably ahead of the curve in being in B2B tech. B2B tech is much closer to integrating all the parts of the funnel, right, because it has clear sets of calls to action, clear sets of, “How do I sell? Who am I selling to?” It’s super targeted, much more so than most other spaces.
Katie Paine (Paine Publishing): I don’t know. Mary, jump in here, but I would say that life sciences is pretty targeted.
Mary Miller (Miller Comms): Well, I totally agree. And I’m listening to this conversation and I’m thinking, “One thing that really hasn’t changed is…” I agree with what’s been said before, that audiences are really who were targeted… it’s the people and their opinions and their behavior, etc. Katie, yeah, life sciences is very very targeted. Specifically health care, it’s even going right down to the changing volume of prescriptions and things like that – that’s something that’s measured.
But one of the things that I’ve been wondering lately, it seems I don’t hear anybody talking about real surveys being done with stakeholders anymore. That people just don’t take time to take surveys. Is that true? Are we all just buying metrics that you can buy or tap into? Is anybody still doing true surveys?
Rob Jekielek (The Harris Poll): Yes. In-depth, 100%.
Mary Miller (Miller Comms): Oh, I’m glad to hear that.
Katie Paine (Paine Publishing): And Mary, I’m launching one next week, so…
Mary Miller (Miller Comms): Oh, excellent.
Rob Jekielek (The Harris Poll): Especially in health care.
We can match physician numbers and prescription rates and all those kinds of things. But it’s still very difficult to tie those things to campaigns, although you have the person, you know their exact prescription rate, you know how often they’re visited by reps, etc…
Katie Paine (Paine Publishing): One of the things that is true is the fact that it is harder to get people to respond to surveys without an incentive.
But if you say, “This is gonna make your life easier because we’re gonna be better at serving your needs,” and then, “Oh, by the way, to what extent are you likely to recommend us?” I mean, you throw the reputational questions in amongst the, “How can we better serve your needs?” questions, and they’ll respond. You do need a larger sample base to begin with, but people are still responding to surveys.
Mary Miller (Miller Comms): That’s good to hear. Patricia, what do you see as far as surveys go?
Patricia Bayerlein (Gagen MacDonald): There is still a lot of surveying inside the organization, although there are questions about how much and how many and how even people will respond. But we’re seeing surveys around the standard things like organizational health, but I also think there are new topics that are coming up based on greater understanding of employees and talent and the current situation that I think we’re seeing across the pandemic about all the different choices in hybrid work.
And really how to reach people in these different states, how to work with people…leaders, to reach and engage people, and how to measure that engagement in this rapidly changing world of work. And then, we’re also seeing a lot of interest in measures around well-being, measuring not just diversity, but inclusion and equity. So there are additional topics that are coming our way that are creating, I think, new ways of thinking about how to measure and what to measure, and is it only survey-based?
I think they’re trying to be better listeners inside the organization.
So by default, they’re using the pulse survey methodology, because I think it’s very easy for people to understand when thinking about different ways to really understand perspectives and sentiment, and combine different methods to really get a true picture.
Katie Paine (Paine Publishing): Is it just me, or is anybody else seeing, as a measure of PR success, metrics like recruitment, retention and reputation as an employer? Are you seeing that? Joseph’s nodding his head.
Mary Miller (Miller Comms): Yeah. Definitely.
Joseph Czabovsky (University of North Carolina): Yeah. Especially this summer, and even during COVID. I’ve been looking at a lot of secondary data about the flight of everybody wanting to leave their work environments, especially when you get to questions of, “Do people want to come back to offices?” that relates to all of those other kind of attitudinal questions, Katie, that employers have in that space.
Katie Paine (Paine Publishing): Several clients that I’m now working with are measuring cultural perception, both internally and externally.
Mary Miller (Miller Comms): I think the pandemic had a lot to do with that and the work from home situation, people started looking at their employees in an entirely different way, and how engaged they were. And all of the D&I issues that have come forward, and women in the workplace, etc.
Katie Paine (Paine Publishing): Chelsea, welcome. What are you seeing in terms of measurement?
Chelsea Mirkin (Cision Insights Americas): Thank you. Good to be back. So what am I seeing in terms of measurement?
Katie Paine (Paine Publishing): What’s different now that wasn’t around in 2019 or 2016?
Chelsea Mirkin (Cision Insights Americas): I’m hearing from every single one of our customers now is, specifically thinking about where our work applies and where people want to go is a desire to connect our own media coverage and social media coverage to business outcomes, right? It’s like the Holy Grail. We’ve been talking about this forever, but what I’ve seen is that agency teams seem to be going off and building their own tech stacks to try to answer this question.
There seems to be a lot more of consolidation of different third-party data streams to try to get at answering this question. So it’s not just about siloed approaches to doing media measurement, and a siloed approach to doing reputation measurement, or survey research. We really are looking for ways to bring all of that together to figure out what’s driving it. Is the media coverage driving a change in awareness and behavior? Is it some other part of the marketing mix?
And then I’d say the other trend is marketing mix. Bringing it together with other earned, owned, paid, all in one view, one picture, seems to be a demand that we’re hearing quite frequently. So it’s not enough to just have earned media off in a silo measurement, kinda these point solutions off telling stories, we need to be telling a cohesive story with all the different third-party data sources.
Katie Paine (Paine Publishing): Lusine, what are your thoughts on what’s different now that you maybe weren’t measuring or weren’t valuing as much a few years ago?
Lusine Kodagolian (Stratint Research): I would agree with the earlier statement that audiences really are a much bigger focus. Although segmenting people in social media into different groups is still not a straightforward process. We have to go around exploring different options for different audiences. Some are easier than others. If you’re looking for policymakers, then obviously it’s easier to build that just based on bio terms. If you’re looking for sustainable investors, that’s much harder, because you are looking at a group that doesn’t easily identify themselves.
And the second thing that we are seeing is ESG. It’s sort of becoming part of the reputation, much more so than it did before. But at the same time, there is also some ambiguity as far as how does it combine with reputation? Is it part of reputation? Is it separate? So different clients look at it differently. So those are the things that we are seeing a lot more than we did even a year ago.
Katie Paine (Paine Publishing): Is ESG seen as being tied to stock price or being tied to culture and ability to recruit and keep talent?
Lusine Kodagolian (Stratint Research): I think it varies. In life sciences, for example, it’s probably culture more so than the stock price. In other companies, like real estate, for example, or fintech, they tie it a lot more closely to stock price. I think it’s the fact that there is no clear definition as to what kind of an impact does this ESG component have, so everyone interprets it in their own way.
Katie Paine (Paine Publishing): What I’ve done in the past is done a whole bunch of analysis of ESGs for us in stock price and found that, in certain industries there is a correlation. There was absolutely no doubt that the higher the ESG score, the higher the stock price over time. And that’s not accounting for crashes and other things, but within the time frame I was looking at, there was definitely a connection. But I’m surprised that there isn’t more of that out there. Rob, and folks from the research world, do you see any of that?
Rob Jekielek (The Harris Poll): A lot of people say they want to financially model different things, but I think when you actually get into the details of how to do that, people get a little bit overwhelmed. So we do some pretty extensive work on financial modeling, but it’s not for the faint of heart. But you need to have people who know how to do it.
You need people to understand how financials work, how a business works, and an active mechanism for looking at facts, not just on stock price, but on margin, on growth potential, on IPs, etc.
Katie Paine (Paine Publishing): Is there now a standard definition of ESG scores? Because I know when I was first writing about them, they were all over the map.
Rob Jekielek (The Harris Poll): There is a lot of standards that are being looked at for how to even connect those things. I think one of the better general perspectives is what’s being done by the World Economic Forum run by Brian Moynihan from Bank of America with the big four accounting firms, where they are getting much stricter parameters in how you connect them, but there is still no de facto standard.
Katie Paine (Paine Publishing): Well, at least we’re getting closer. You know, there didn’t used to be standards for PR measurement either, so anything is possible. Jim, what are you seeing that you haven’t seen in the last few years?
Jim Simon (Simon & Associates): Having been to the corporate side most of my career, and now with corporate finance, I see a real distinction I’m sure some of you are seeing too between the U.S. and European and other regions when it comes to ESG. They really were the pioneers more in Europe than we were here. CSR was the language of the day here, and now it’s more ESG. I agree with all the previous comments. I think there is a weighting issue, depending on the industry group you’re in.
So if you’re in oil or energy, I would say “E” is a big deal, “G” should be a big deal for everybody, and “S” is sort of in between. ESG filters vary by industry group, the region you’re in, and your awareness of it generally. And I think overall, ESG is finding its way more and more into corporate suite discussion and I’m certainly seeing it with clients. And it’s the old story – where you have a few big players who draw attention to it, people like Jamie Dimon (CEO of JP Morgan Chase), then the industry group begins to think about it more.
From there, you reach a flashpoint where it becomes more embedded in the general consciousness. That’s what I’m seeing and hearing.
Katie Paine (Paine Publishing): It is showing up as a metric on communications dashboards. I have been told that increasing your ESG score is part of your overall annual measures of success.
Mary Miller (Miller Comms): It’s just as important to the employees as it is to external stakeholders. And I feel so strongly that companies can measure the effectiveness of their communications on ESG with employees to get a good idea of where they stand with their external stakeholders.
Katie Paine (Paine Publishing): So you are suggesting you measure employee stuff to figure out where the external…
Mary Miller (Miller Comms): Well, to find out where your messages may be deficient. Not that you wouldn’t measure it with your external audiences. I feel that employees are like a barometer for what you’re going to find with your external stakeholders, and if you know you’ve got something that’s not resonating with employees, then I’d change it to your external audiences without asking the external.
Joseph Czabovsky (University of North Carolina): I’m also finding more and more that the internal and external are trying to sync as much as possible. Organizations have a certain brand so the ESG for Ben & Jerry’s is going to look very different than Coinbase. Coinbase got a lot of attention last year because they banned political conversations in internal Slack chats. They lost a significant part of their employee base, but then also recruited a lot of other employees who wanted that environment. And that’s a very different model than a Patagonia or a Ben & Jerry’s version of ESG.
Since a few of you were talking about investor relations, I always pause to wonder, is this like a black swan case where it just doesn’t fit the model, and that’s okay? That’s where I always play in the sandbox, so I’m always in that space. But retail investors in the investor space, for example, I’m not sure segmentation works, in the sense that segmentation would traditionally say they’re retail investors so they’re going to be a different kind of audience.
But it’s a different kind of segmentation than, “Are you gonna go to this movie or not, or vote for this candidate or not?” Because in many ways, retail investors’ purposes are so different from regular institutional investors, and they’re able to control so much of the market in a very small part of the market. But when your audiences are so different from each other, it’s not like saying these are industrialists versus aviation investors.
These are investors that are more than apples and oranges. They are like apples versus beef or something, you know. I don’t know if our traditional segmentation applies. So, a question to the group is, does that just mean we’re not measuring it properly? Or do we have to think of those as sometimes we have to change the whole model for different kinds of segments or audiences, even within the same industry?
Katie Paine (Paine Publishing): That’s a very interesting question. And I would assume that if you didn’t have research, you could probably find some commonality between different segments of that audience, either by the types of stocks that they care about or the types of things that they care about, but it’s interesting that now institutional investors are a lot more monolithic than retail investors. Michael, what are you seeing about ESG and culture and stuff like that in Australia?
Michael Ziviani (Precise Value): Let’s first say there has been this wonderful case study here of a big financial organization who promoted someone whilst harassment allegations were circulating. And as a result, some staff started to get very upset at that and leave the organization. They lost vast amounts of market capitalization as big investors and some pension funds started to look at that and question what was going on.
And finally they came to their senses, and things settled down and they’ve reversed that promotion, but they have seriously lost ground. And I think that’s a bit of a wake-up call to our market that’s been slow in the areas of ESG, largely, because of the measurement questions, and you now have more active shareholder and stakeholder activism here. So the market’s changing, but also to circle back to the earlier question of marketing.
To me, it’s this balance beam. It’s the balance beam between reputation and brand. And you can’t build a good brand without reputation, right? The two are fundamental and have to be measured, and often operate in lockstep, and that feeds the funnel. So the funnel is not just pure brand, the funnel to me is reputation measures as well.
Katie Paine (Paine Publishing): Did anybody see the Charlene Li piece, I think it was on LinkedIn, on “Culture Eats Strategy for Breakfast?” Her point was, it’s a little bit of both, you can’t have one without the other. But this whole notion of ESG, culture, who you are internally is going to affect you externally might be the takeaway from this conversation.
Which is probably the biggest difference now; the old idea that you had external and internal as separate departments is no longer. Your employees are your spokespeople whether you like it or not. They’re going to influence your audiences, because they are speaking to your customers whether you like it or not.
What I loved about Charlene Li’s point, that if your culture is deep and strong in your organization, you are much better able to react to major changes in the marketplace. So your strategy is baked in cement, but if that goes out the window, your culture knows how to respond if you have a strong culture.
