This post is written by the 2017 IPR Don Bartholomew Award For Excellence in Research winner, sponsored by 
Download the full study here.
What is the current state of nonprofits’ public relations measurement and evaluation? How can we better measure and evaluate communication as well as better use findings? This paper provides an accessible, yet comprehensive introduction to the current state of nonprofit communication evaluation for professionals and scholars.
Introduction
Since the 1970s, the measurement and evaluation of public relations have been one of the top research priorities (Watson, 2008; Volk, 2016). However, many organizations still have not systematically evaluated their communication activities. For example, researchers today refer to the status of public relations measurement as “stasis” (Macnamara & Zerfass, 2017, p. 319) or “deadlock” (Macnamara, 2015, p. 371). Unfortunately, scholars have been less interested in professional topics, such as practical measurement methods (Volk, 2016). The difficulty in measurement and evaluation also applies to nonprofits.
Nevertheless, the field of nonprofit communication evaluation is well underway. Some researchers and professionals have attempted to employ new methods to measure the impacts of nonprofit communication and use measurement findings, such as econometric model or marketing mix modeling. Still, despite practical and theoretical importance, such new communication measurement methods remain little discussed. Thus, this paper examines:
1.) The current state of nonprofits’ public relations measurement
First, this paper examines the current state of nonprofit communication measurement by synthesizing existing findings from five recent surveys and a content analysis of 15 years of nonprofits’ reports.
Key Findings
- Although nonprofit organizations value measurement and evaluation, fewer nonprofits measure their communication.
- Interestingly, 71% of nonprofits communication professionals reported that they conduct measurement in a survey, while a content analysis found that only 13% of international and nonprofits measured communication.
- Communication professionals mostly measured metrics focusing on output, or how many mentions they achieved, rather than attitude change, financial and strategic impacts.
- In particular, only half or less nonprofits communication professional say that they track key specific metrics (2017 AMEC).
- Nonprofits conduct more quantitative research than qualitative research, while the bigger the organization is, the more qualitative and quantitative research organization conducts.
- Nonprofit organizations primarily use M&E results for internal and external uses; to report to board of directors (94%) and funders (93%), as well as to plan/revise initiatives (91%) and strategies (86%).
- The primary audience for evaluation were executive staff (CEO or ED), Board of Directors, and Funders.
- A majority of organizations spend less than the recommended amount on evaluation.
- The top challenges were lack of staff and time, insufficient budget, and limited expertise in evaluation.
- One opportunity was the leadership’s expectation and support.
- Lastly, communication professionals in nonprofits display rather moderate capabilities in measurement.
- Nevertheless, 71% wanted to learn more about measuring communication effectively.
2.) How nonprofits can measure impacts of public relations and use measurement data to improve their communication practice.
Researchers and professionals have continued their endeavors to develop methods to prove the value of public relations and to improve public relations. By reviewing academic literature and best practices provided Ketchum Analytics, the paper offers ways to measure nonprofits’ public relations impacts and use measurement findings.
Key Findings
- Academic efforts to measure public relations’ impacts can be summarized into three categories: compensating variation approach to benefit cost ratio (BCR), media coverage and financial performance, and econometric model. These attempts focused on proving the value of public relations. Findings also reflect Volk’s review (2016) that scholars have been less interested in practical measurement methods.
- Unlike academic efforts that mostly focus on seeing and proving the value of public relations, public relations professionals went a step ahead – Predictive analytics. Public relations professionals attempt to use measurement data to optimize the mix of communication activities and message elements. Such use of measurement data can maximize the communication outcomes, such as donations or patient visits. There are two approaches: survey approach and marketing mix modeling approach.
3.) Caveats, debates, Future directions
Lastly, caveats, debates, and future directions surrounding public relations measurement will be discussed. Some are about definitions and terms, such as whether we should use the term Return on Investment (ROI) and how to calculate it. Some are methodological, such as whether we can use predictive modeling to examine the impacts of programs, policies, and practices.
Key Findings
- Researchers and professionals have shared the concerns for using and measuring PR Return on Investment (ROI). Still, the progress to define, use, and measure PR ROI has not been made.
- Many attempts to measure public relations’ impacts reviewed here did not have counterfactuals and used methods that cannot test cause-and-effect causality (e.g., concomitant time series and Granger testing); Such methods test if a particular variable precedes another.
- Predictive modelling cannot show cause-and-effect relationships and cannot reveal the impact of programs, policies, and practices; Still, predictive modelling can identify better indicators for the outcomes and make use of them.
Conclusion
Despite nonprofits’ limited measurement of public relations, public relations scholars and professionals have developed the cutting-edge methods to measure its impacts and use measurement findings to improve messages and channels. However, such methods need to be used with care, particularly because what they show may be the temporal precedence and correlation, not the causal relationship. Moreover, public relations professionals and scholars focus on measuring economic impacts, yet understudied examining its societal impacts.
Yet, there is hope. Fortunately, researchers in other fields have extensively developed rigorous causal inference design and analysis strategies to obtain unbiased estimates of the causal effects of programs, policies, and practices. Working closely with external measurement and analytics experts can be more realistic, effective, and cost-saving than developing in-house capabilities.
The potential of measuring public relations’ impacts has not been fully realized. It is future nonprofit public relations professionals and scholars’ roles to work together with measurement specialists to develop rigorous measurement and innovate use.
Better and more communication measurement will help advance evidence-based communication for nonprofits to make a difference. I hope this paper helps you better understand nonprofits’ public relations measurement and evaluation.
Jungkyu Rhys Lim is a Ph.D. Candidate at the University of Maryland, studying public relations and strategic communication, focusing on risks, crises, and disasters. Jungkyu Rhys Lim studies behavioral and communication interventions to help individuals, communities, and organizations better prevent, prepare for, respond to, and recover from the public health risks and natural disasters.
