This blog is provided by the IPR Measurement Commission based on the original blog published by PR Academy
As 2022 drew to a close, the time felt right to reflect on a year that was meant to bring a level of calm after the pandemic disruptions. Instead, we got a war in Ukraine, an evolving global economic crisis, more bad news on the global climate, and seemingly more political polarisation. Amid all this, professional communicators needed to ‘stay sane inside insanity’ and provide evidence-led solutions to often intractable problems:
- Should my organisation / spokesperson take a stand on this politically contested issue? And what should that stand be?
- How do we address a highly diverse audience (not to mention stakeholder environment) with consistent coherent messaging?
- How can our organisation contribute to making things less uncertain, and how can communication help with that?
Leadership around the world in any kind of organisation appears to be in crisis, and concepts like VUCA (for something that is volatile, uncertain, complex and ambiguous) and wicked problems (for which no known solution exists) are being used to describe the phenomenon (as in this article, reflecting on COVID responses; for a more comprehensive discussion, see this peer-reviewed paper that Darren Lilleker and I published in 2021).
From the perspective of PR, if this is where we are, then how can we get to a better place, and what might be the role of measurement and evaluation on the journey?
In a recently published study, University of Missouri scholar Luke Capizzo explores this rather complex question through interviews with PR practitioners. His insights on the measurement and evaluation of wicked VUCA problems in PR are worth considering as “the growing frequency of intractable problems and contentious issues … requires a broadening of our perspectives” (Capizzo, 2022, p. 4).
His analysis of current models and frameworks concludes that they “measure progress toward organizational goals that dictate consensus outcomes as preferable, rather than incorporating alternative conceptions of success” (Capizzo, 2022, p. 2). He offers an alternative, “holistic measurement model for intractable problems” that widens the perspective from organisation-centric in a traditional sense (the ‘output – outtakes – outcomes’ model), toward societal impact in a triple bottom line sense (considering business interest, but also social and environmental interest):
Jim Macnamara, for many years a thought and practice leader when it comes to the evolution and improvement of the measurement and evaluation of public communication (advocating organisational listening as the fundamental skill), provides a similar perspective in his recent work, which among others is based on large-scale project commissioned by the World Health Organisation post pandemic (Macnamara, 2023). He applies an ‘architecture of listening’ that sees organisational communication as a two-way process: from the organisation to stakeholders, communities, society at large, through a logical progression from inputs to activities, to outputs, outcomes and ultimately, impact. And then the reverse, from the environment to the organisation, through the same logical steps:
In: Macnamara, J., 2022. Organizational Listening in Public Communication: Emerging theory and practice. Sydney: University of Technology Sydney, p. 65.
Both Capizzo and Macnamara recognise the work of AMEC, especially the Integrated Evaluation Framework, as foundational as they “fit neatly within the quantitative, metrics-driven management frameworks favored by organizational leaders and professional communication associations” (Capizzo, 2022, p. 1). But when organisations and communicators are facing wicked VUCA challenges, previously neat fits might no longer suffice. As Sam Knowles would put it (for example in his new book, which I’ve reviewed for PR Academy), the time has come to ask smarter questions to get to better answers.
For the last two years, I have been leading the AMEC International Certificate in Measurement and Evaluation at PR Academy, and the centrepiece of this course are AMEC’s freely available planning tools, from the Measurement Maturity Mapper questionnaire to assess an organisations ‘M&E maturity’, to the Planning Sheet and ultimately, the Integrated Evaluation Framework to plan, execute and evaluate communication activities.
These are excellent, practically useful tools, grounded in reflective practice as well as academic theory and research evidence (enshrined in the Barcelona Principles). The more practitioners are familiar with them and apply them in their day-to-day work, the more our field will develop and improve. In particular in light of the aforementioned wicked VUCA challenges, I encourage students as well as clients to use those tools and make them work for their specific contexts and challenges. Modified and advanced frameworks with additional metrics, grounded in curious exploration through asking smarter questions and getting to more relevant insights – all this will help us navigate our ‘interesting times’.
Further reading:
Capizzo, L. (2022). What counts amid contention? Measuring perceived intractable problems in public relations. Public Relations Review, 48(2), 1-11. https://doi.org/https://doi.org/10.1016/j.pubrev.2022.102179
Macnamara, J. (2023). Measurement, evaluation + learning (MEL): New approaches for insights, outcomes, and impact. In D. Pompper, K. R. Place, & C. K. Weaver (Eds.), The Routledge Companion to Public Relations (pp. 225-236). Routledge.
Thomas Stoeckle is Partner of Analytics and Insights at Dot I/O Health, Part-time Lecturer at Bournemouth University, Course Leader at PR Academy, and Visiting Lecturer at Quadriga University. He is a member of the IPR Measurement Commission.
