This blog post is presented by the IPR Commission on Measurement and Evaluation.
Evaluation has been on the lips of industry leaders and academics in public relations for more than four decades, as reported in a historical analysis by Fraser Likely and Tom Watson (2013). But, despite this, a number of surveys and studies have concluded that evaluation of public relations (PR) remains in a state of “stasis” (Gregory & Watson, 2008) and “deadlock” (Macnamara, 2015).
In particular, research shows a focus on measuring outputs rather than identification of outcomes and impact; a lack of alignment of PR achievements with organizational goals and objectives (Tench et al. 2017; Zerfass et al., 2017); and often a lack of methodological rigour, as evidenced in continuing use of advertising value equivalents (AVEs) (USC & The Holmes Report, 2017).
In searching for answers to explain industry lag in evaluation and trying to find a way forward, studies have revealed a number of stated reasons including lack of budget, lack of standards, and lack of demand. However:
- Almost two decades ago Walt Lindenmann (2001) pointed out that “research doesn’t have to put you in the poorhouse”, identifying a range of low-cost methods for evaluation;
- David Michaelson and Don Stacks (2011) identified a need for standards and a number of initiatives have been launched over the past seven years including the Coalition for Public Relations Research Standards (Marklein & Paine, 2012), the Barcelona Principles (AMEC, 2015), and a recent Institute for Public Relations (IPR) Task Force on standards;
- A European study found evidence that some employers do not want rigorous PR evaluation (Baskin et al., 2010), but given that accountability is a hallmark of professional practice (Vriens et al., 2016) and performance management is widely established, this claim needs to be challenged.
In 2018, a special panel session at the International Communication Association (ICA) conference in Prague called for “new voices” and new perspective on the age-old issue of evaluation of PR. Some interesting new perspectives – some provocative – were presented. It is worth reflecting on a few of them.
Performance measurement immaturity
One of the interesting concepts put forward was that management in organizations sometimes does not demand or appreciate rigorous evaluation of PR because the organization itself lacks maturity in its overall performance measurement systems (PMS). In such environments, proposals for PR evaluation encounter what the authors, Stefania Romenti and Grazia Murtarelli (2018), call “conflicting logics” – that is, calls for PR evaluation conflict with the overall logic of the organization in relation to evaluation. They proposed that a maturity model of organizational performance measurement would provide a good indicator of the extent to which an organization was likely to support and appreciate PR evaluation.
This is an interesting and under-discussed barrier to PR evaluation. It is gratifying that the International Association for Measurement and Evaluation of Communication (AMEC) has recently announced the launch of a maturity model called the Measurement Maturity Mapper (AMEC, 2018).
“Functional stupidity”
A provocative and easily misunderstood concept presented at the ICA conference is the theory of functional stupidity. Not even the senior academics in the room had heard of this one. Functional stupidity refers to illogical outcomes at a function level – not individual stupidity. So don’t take offence at this notion. This theory was developed by management academics Mats Alvesson and André Spicer (2016) as an explanation of why some things that seem obviously necessary are not done in organizations and why intentional avoidance occurs.
In applying this to PR evaluation, Jesper Falkheimer and Howard Nothhaft (2018) said that a key to understanding this paradox is that many practitioners know that they over-promise. This is not necessarily lying or intentional misleading. In the competitive environment of agency ‘pitches’ to clients and the need to impress management in organizations, PR and communication practitioners are pressured into optimizing the projected outcomes and impact of activities. In some cases, management sets targets that are unrealistic and unachievable. In such situations, rigorous evaluation is not in the interest of practitioners in PR and communication management. It is actually in their interest to avoid rigorous evaluation or to use simplistic methods that show activities and outputs rather than outcomes or impact.
The theory of functional stupidity leaves one question unanswered: How do practitioners get away with avoiding evaluation or using invalid metrics such as AVEs or vanity metrics? The immaturity of overall performance measurement knowledge and systems in some organizations identified by Romenti and Murtarelli (2018) might explain this.
These are two innovative ways of looking at the evaluation challenge. The next question then is: “What strategies can practitioners adopt to address these?’ Two suggestions are as follows:
- If possible, work for organizations with mature performance measurement systems, as they are more likely to appreciate and support PR evaluation.
- Try to improve the performance measurement of the organization overall if it is immature by acting as a strategic advisor to senior management and taking leadership in an organization. Evaluation can be an opportunity for PR practitioners to extend their influence and input to management.
Jim Macnamara, Ph.D., is Distinguished Professor of Public Communication at the University of Technology Sydney (UTS). He is also a Visiting Professor at The London School of Economics and Political Science, Media and Communications Department. Recipient of The Pathfinder Award and The Don Bartholomew Award in 2017, Jim is a member of the IPR Measurement Commission as well as Chair of the Academic Advisory Group of AMEC.
References
Alvesson, M. & A. Spicer (2016), The Stupidity Paradox: The Power and the Pitfalls of Functional Stupidity at Work, Profile Books, London.
AMEC (Association for Measurement and Evaluation of Communication). (2015), “Barcelona principles 2.0”, London, available at: http://amecorg.com/barcelona-principles-2-0
AMEC (Association for Measurement and Evaluation of Communication ). (2018), “Measurement mapper – M3 debuts Nov”, media release, available at: https://amecorg.com/2018/06/measurement-mapper-m3-debuts-nov
Falkheimer, J. and Nothhaft, H. (2018), “Evaluation – an example of functional stupidity?”, paper presented to International Communication Association panel session, New Voices in PR Evaluation: Innovative Approaches and New Research Avenues for a Field in Stasis, 26 May, Prague, Czech Republic.
Gregory, A. & Watson, T. (2008), “Defining the gap between research and practice in public relations programme evaluation – towards a new research agenda”, Journal of Marketing Communications, Vol. 24 No. 5, pp. 337–350.
Likely, F. & Watson, T. (2013), “Measuring the edifice: Public relations measurement and evaluation practice over the course of 40 years”, in Sriramesh, K. Zerfass, A. and Kim, J. (Eds), Public Relations and Communication Management: Current Trends and Emerging Topics, Routledge, New York, NY, pp. 143–162.
Macnamara, J. (2015), “Overcoming the measurement and evaluation deadlock: A new approach and model”, Journal of Communication Management, Vol. 19 No. 4, pp. 371–387.
Marklein, T. & Paine, K. (2012). “The march to standards”, presentation to the 4th European Summit on Measurement, Dublin, Ireland, June, available at: http://amecorg.com/downloads/dublin2012/The-March-to-Social-Standards-Tim-Marklein-and-Katie-Paine.pdf
Michaelson, D. & Stacks, D. (2011), “Standardization in public relations measurement and evaluation”, Public Relations Journal, Vol. 5 No. 2, pp. 1–22, available at: http://apps.prsa.org/Intelligence/PRJournal/past-editions/Vol5 No2
Romenti, S. and Murtarelli, G. (2018), “Conflicting logics between communication departments and management”, paper presented to International Communication Association panel session, New Voices in PR Evaluation: Innovative Approaches and New Research Avenues for a Field in Stasis, 26 May, Prague, Czech Republic.
Tench, R., Verčič, D., Zerfass, A., Moreno, A. & Verhoeven, P. (2017), Communication Excellence. How to Develop, Manage and Lead Exceptional Communications, Palgrave Macmillan, London, UK.
USC (University of Southern California) Annenberg Center for Public Relations and The Holmes Report. (2016), Global Communications Report 2016, available at: https://annenberg.usc.edu/research/center-public-relations/reports/global-communications-report-2016 .
Vriens, D., Vosselman, E. & Gross, C. (2016), “Public professional accountability: A conditional approach”, Journal of Business Ethics, October, pp. 1–18.
Zerfass, A., Moreno, A., Tench, R., Verčič, D. & Verhoeven, P. (2017), “European Communication Monitor 2017. How strategic communication deals with the challenges of visualisation, social bots and hypermodernity. Results of a survey in 50 Countries”, EACD, EUPRERA and QuadrigaMedia Berlin, Germany.
