Guidelines for Measuring Trust in Organizations (Updated)

Download (PDF): Guidelines for Measuring Trust in Organizations (updated 2013)

Introduction
In 2003, a coalition of organizations representing 50,000 public relations and communications professionals gathered in New Jersey to discuss ways to restore trust in American business. Buffeted by scandal and crisis since the Enron debacle, the average citizen’s belief in the integrity and honesty of corporate American had reached an all-time low. The coalition agreed on three basic actions that they could recommend to each and every CEO in America.

1. The CEO should articulate a set of ethical principles closely connected to their core business processes and supported with deep management commitment, enterprise-wide discipline and training.
2. The CEO should create a process for transparency that is appropriate for current and future operations. It should include an oversight committee, culture audit and consistent messaging. CEOs should ensure that they have professional, competent counsel to serve as a strategic integrator, champion, bridge builder, catalyst, facilitator and record keeper for appropriate transparency.
3. The CEO should establish a formal system of measurement. CEOs should make trust a corporate governance issue and a board priority tied to compensation

As a result of that statement the Measurement Commission of the Institute for Public Relations created a white paper that offered standards and guidelines for trust measurement.

In the ensuing decade we have seen the rise of online commerce and the decline of brick-and-mortar stores, necessitating whole new levels of trust. Social media has taught us that a single tweet can build or destroy an organization’s trust with its stakeholders. Media itself has changed, raising enormous issues of trust in the veracity of reporting and content.

Social media has ushered in a transparency revolution. It is no longer possible to hide behind a “corporate message,” because anyone from Wikileaks to your competition can expose any inconsistencies in seconds. As a result your stakeholders are putting greater emphasis on character value.

This document offers an update of these standard guidelines to take into account a number of new factors that impact the measurement of trust, namely social media, research in transparency and authenticity, and new research in public relations and trust measurement.

Preface
In 2007, when Facebook had just expanded beyond its initial college marketplace, its communications team – including its agency, public affairs chief, product PR managers, and VP of communications – met in Palo Alto to set goals for 2008. At one point during the meeting, founder Mark Zuckerberg walked in to address the group. After his remarks he took questions. I was privileged to be part of the gathering, and I asked him how he would define success for his communications team a year hence. His response was stunning: “I would want people to trust us more.”

That wasn’t the answer I expected, but it made realize just how critical trust is in a world shaped by friends and stakeholders that you have never met. Today, improving trust in your brand is probably the most common, if unarticulated, measure of success.

At the time, I provided Mr. Zuckerberg and the team with the original version of this paper. But, in the intervening years, communications, public relations, and most of the world we live in has changed.

What hasn’t changed is the need to measure trust. It’s not just a “nice to have” piece of research. Trust has a direct impact on the financial health of an organization. Following the Enron scandal, its accounting firm, Arthur Anderson, was essentially forced into bankruptcy because its clients lost confidence in its results. In 2011, highly trusted brands like Target, FedEx, Penn State, Netflix, and Lowe’s all underwent crises that were either started or exacerbated by social media. Every single one had financial consequences, whether it was a decline in stock price, or the loss of key recruits to the competition.

I therefore decided it was time up update these guidelines, in the hope that organizations will agree with us that setting up a system to measure trust is a critical and achievable component of corporate governance.

The Value of Trust and the Cost of its Absence – A True Story
Here’s a true story about the vital importance of measuring trust between people and organizations. Once upon a time not very long ago, there was a state university in a small town in New England. Both the university and the town needed new soccer fields. One of the university’s alumni, a successful local entrepreneur, stepped forward and offered to donate $6 million so the university could build them. A site was selected, town officials were notified, and the university assumed it would soon be hosting soccer tournaments.

Now, the university was a venerable institution, and it took care to maintain its reputation of quality and prestige by communicating about itself to the world. But it failed to understand that some very important changes had been occurring in the town. For much of the university’s history, most of the people who lived in the town year round either worked for the university or had family or friends involved there. Over recent decades, however, rising real estate prices and property taxes had forced many of the faculty and staff out-of-town. Their houses were bought up by retirees and commuters who had no particular connection with the university. So the town’s permanent residents, who had once formed a sympathetic constituency, gradually changed into an inactive, disengaged public. As far as they were concerned, the university was just something to put up with. Whether they trusted the university was unknown and as yet untested.

Several years before the soccer fields were planned, the university had embarked on another large construction project, a 6,000-seat sports and entertainment arena. The university, a tax-exempt and local zoning-exempt state entity, chose to simply notify the townspeople of its plans. It completely failed to anticipate that the town’s permanent residents might object to potential parking problems or the absence of any local tax benefits. As a result, many townspeople felt railroaded by the university and town officials, and, after a contentious political campaign, the town leadership was replaced.

Let’s skip ahead a couple of years to the soccer fields proposal. The university, despite its strong public relations department, had been making no effort to understand the concerns of its constituencies. It was unaware that a good part of the town’s permanent residents had become well-organized and were potentially quite hostile. When the university announced its new construction project, the reaction was swift, noisy, and disastrous. A citizens group that had started as a simple email list during the previous election campaign now was now 2,000 members strong, including several community leaders who had close ties to the state capital. The group therefore had no problem attracting the attention of the statewide media. They also used e-mail and their listserv to ensure that every university trustee and every politician heard their complaints.

Less than two months after being announced, the proposed soccer fields were cancelled, the $6-million gift was rejected, and the university president stepped down. The school and the community paid a stiff price for the university’s failure to measure the town’s trust. Worse still, it would take a decade before suspicions were put to rest and trust rebuilt.

What Is Trust?
Trust has been a widely studied concept both by itself and, most importantly, as a component of the quality of relationships. Research by James and Laurie Grunig and Linda Hon and others have shown that trust is one of six independently measurable components of relationships. James Grunig has identified three dimensions of trust that are measurable by the Grunig Relationship Instrument:

• Competence: The belief that an organization has the ability to do what it says it will do, including the extent to which an organization is seen as being effective, and that it can compete and survive in the marketplace;
• Integrity: The belief that an organization is fair and just;
• Dependability/reliability: The belief that an organization will do what it says it will do, that it acts consistently and dependably.

Although the experts are not in complete agreement, trust between an organization and its publics are generally described as having the following independently quantifiable characteristics:

•  Multilevel: Trust results from interactions that span coworker, team, organizational, and interorganizational alliances. Which is why you need to cast a wide net when you survey your publics on trust.
• Culturally-rooted: Trust is closely tied to the norms, values, and beliefs of the organizational culture. Therefore it is critical to understand the self-image and self-definitions of your publics if you are going to accurately measure trust.
• Communication-based: Trust is the outcome of communications behaviors, such as providing accurate information, giving explanations for decisions, and demonstrating sincere and appropriate openness. Which is why communications metrics are critical in trust measurement.
• Dynamic: Trust is constantly changing as it cycles through phases of building, destabilization, and dissolving. So it is important to measure trust on a continuum over time.
• Multidimensional: Trust consists of multiple factors at the cognitive, emotional, and behavioral levels, all of which affect an individual’s perceptions of trust.

What Is Trust Measurement?
The science of measuring trust incorporates any and all research designed to determine and quantify how people perceive your brand or organization. It is the summation of all their interactions that coalesce in a perception of their relationship with your brand. More specifically, trust measurement is a way of giving a result a precise dimension, generally by comparison to some standard or baseline and usually is done in a quantifiable or numerical manner. It seeks to answer questions such as:

• Have the behaviors, programs and activities we implemented changed what people know, think, and feel about the organization, and how they actually act (as exhibited by protests, votes, and purchases)?
• Have the actions or behaviors of my organization had an impact on the trust that our constituencies feel towards our organization?
• Have those public relations and communications efforts that we initiated to build trust had an impact – that is, “moved the needle” in the right direction? If so, how can we support and document that with research?

Traditionally, all valid trust measurement has been conducted via some form of survey research. However, with the advent of social media, a number of other techniques have been suggested, including content analysis of social networks (i.e. Twitter, Facebook, etc.) and web analytics. Let’s look at the pros and cons of these alternative methods.

Using Content Analysis to Measure Trust
The proliferation of content, and specifically personal opinion as voiced in blogs, Facebook, and Twitter, has led many to try to mine this data for personal opinions and sentiment. Typically this mining has been done using some form of Natural Language Processing. And, while it appears to have some validity in measuring sentiment in entertainment (for instance, books, movies, and television programs), substantial further research is necessary before we can judge with certainty whether what someone posts to a Facebook page in fact really represents trust.

Manual content analysis, however, is a more realistic possibility. In an experiment conducted with three top research universities, my firm, KDPaine & Partners, found that it was possible to create a set of coding instructions based on the Grunig Relationship Instrument that a human coder could use to detect and record elements of trust in social media conversations. However, the experiment also showed that there were several caveats for using content analysis to determine trust levels.

Some level of emotion must be present to analyze it. If you are analyzing a topic that does not evoke sentiment, it will be impossible to discover sentiment in it. In the university study, for consistency purposes we removed discussion of sports from the analysis. Once we did that, most of the emotion and passion went with it. Of 2,000 total items analyzed, only 265 (13%) actually contained any of the relationship concepts we were trying to study. So, if you are a B-to-B company or one for which most of the conversation about you is about science or academic research, then there may not be enough sentiment expressed to be measurable.

It is all a “general public.” In social media, it is extremely difficult to accurately associate an individual poster with a specific stakeholder group. A person’s Twitter handle or even Twitter identity may or may not reflect the real person behind the handle. There is seldom enough informational detail to identify whether that person is your employee, your customer, or your competitor.

If you are going to analyze sentiment, you need to start with Twitter and Facebook. Consumer opinion is far more frequently expressed on Twitter and Facebook than anywhere else. Therefore, if you are measuring trust, or for that matter any other sentiment, you need access to as complete a collection of Twitter and Facebook content as possible. Most services only get about 15% of the total Twitter fire hose, and even services that have a full fire hose license only get about 85%. Facebook poses an even greater challenge since most content providers only provide what is publically available, which again is a small percentage of the universe. Personal blogs are another good source of sentiment, but you will need to make the effort to exclude content farms and fake sites that are robotically created.

Using Web Analytics to Measure Trust
A number of social media pundits have suggested that consumer actions, as tracked by quantitative web metrics (for instance, the number of fans, followers or forwards), are accurate expressions of consumer trust. In theory the concept sounds good: if more people follow you on Twitter, or like your content on Facebook, one might assume that they trust you. However, the reality of the social web gets in the way of any validity to this form of trust measurement. Here is why:

You want to humans –not “bots” – to trust you. There has been a proliferation of applications that enable you to automatically follow anyone who tweets about a specific topic. You really have no idea whether those followers are actual people or just spam bots that are set to automatically follow any Tweet on a particular topic. You can frequently end up with bots following bots.

Just because someone follows or likes you, does not mean they trust you. The majority of followers never engage with the people or brands they follow. In addition, many people follow or like a brand or a person because they disagree with them and want to keep tabs on what they are saying. This is especially true on Facebook, where consumers increasingly go to post complaints. “Liking” a page enables them to listen for other consumers who have similar issues.

Love at first site doesn’t happen very often. Relationships and trust are based on actions, not clicks. If we were talking about romantic relationships as opposed to social ones, liking or following would be the equivalent of winking on Match.com. When you click on that picture, you are not saying, “I want to have a relationship with this person.” You are simply expressing interest in that person’s picture. Until you have some level of real interaction – like having coffee together or an email exchange – chances are that a wink will be just be a wink.

Six Simple Steps to Set Up a Trust Measurement Program
There are six basic steps that are required to measure trust.

Step 1: Define the publics and stakeholders you want to study or with which you want to build trust.
Before you can create a system to measure the degree to which some group trusts or mistrusts your organization, you must carefully define the persons you are interested in and understand what is important to them. If you are not sure who your stakeholders are or what priorities they have, refer to Brad Rawlin’s paper “Prioritizing Stakeholders for Public Relations.”

Step 2: Set SMART goals and objectives.
There is almost always a direct and measurable benefit to an organization when it measures and understands its relationships with its constituencies. For instance:

• A high level of trust helps cultivate relationships with donors, consumers, shareholders, and legislators who are needed to support organizational goals.
• Higher employee trust means lower turnover, which has a direct impact on the bottom line.
• Higher employee trust also increases the likelihood that they will be satisfied with the organization and their jobs. As a result they are more productive, more likely to support the organization, and less likely to interfere with its mission.
• Trust from the financial community is critical to an organization’s access to capital and therefore its ability to grow.
• Trust from the media can often avert or reduce a crisis.

Begin your program by establishing SMART objectives, that is, objectives that are Specific, Measurable, Achievable, Realistic, and have a Time Factor. Your SMART objectives will define success of your program, and therefore will determine your metrics. Typical objectives might be to:

• Save time
• Reduce barriers
• Gain efficiency
• Reduce legal costs
• Eliminate barriers to expansion
• Reduce lost revenue
• Support the stock price
• Neutralize opposition
• Shorten sales cycle
• Better communicate our messages

One important element of setting objectives is to be very clear about the type of trust you need to build, since that may influence your metrics. Andrew Tucker, in a paper delivered at the 2006 International Public Relations Research Conference, postulated three varieties of trust:

• Short-term trust, which is based on financial performance and product quality metrics which might be measured by something simple like ranking on Fortune’s Most Admired list, or other public lists.
• Medium-term trust (which he calls “reflexive mistrust”), which translates roughly to, “I can’t totally trust you because I don’t know you, but I need to trust you somewhat to do my job or fill my need.” This type of trust would be tested and measured with either a survey or a behavioral experiment.
• Long-term trust, which is manifested by loyalty, for which you can use measures of customer and employee loyalty and level of forgiveness in a crisis.

In addition, there is transferred trust. Transferred trust is the trust gained when an organization has so frequently resorted to obfuscation or spin that their opposition takes on greater trustworthiness by comparison. Transferred trust is exemplified by BP, Bank of America, and Murdoch’s News Corporation. This type of trust is of particular importance for corporate communicators to consider in the new, social media-dominated world. We’ve all heard the usual interviews with both sides in traditional media. But in social media, the obvious “spin” and frequent obfuscations are exposed and thus trust in whatever the other side says grows, regardless if it is true or not.

Step 3: Establish a benchmark: What do you want to compare results to?
Measurement is essentially a comparative tool, and you always need something to compare results to. Whether it is your own trust over time or a comparison of your results to a peer company, you must provide a benchmark and a context for the results.

So, for example, suppose you are investing in a corporate social responsibility (CSR) program to build or rebuild trust in your organization. Before beginning the program, you would conduct a trust measurement survey to establish a benchmark. Then, after the program has run for a year, you would repeat the survey. By comparing the two results you can determine the impact of the program.

Step 4: Decide upon your metrics
Metrics typically fall into three categories:
Activity or output metrics: For example, number of times your brand is mentioned as trustworthy by the media.

Attitude or perception metrics, also known as outtake metrics: For example, change in perception of your organization as a trustworthy organization.

Outcome metrics: For example, percent change in the number of people that reinvest in your company and/or that become repeat purchasers.

Metrics are typically articulated with percentages, as in:

• Percent increase in trust score (or decrease in mistrust)
• Percent reduction in sales cycle
• Percent reduction in legal fees
• Percent decrease in lawsuits or complaints filed
• Percent increase in repeat customers
• Percent reduction in employee turnover rate
• Percent increase in employee satisfaction and trust score
• Percent communication of key messages

Step 5: Select a measurement methodology, instrument, and/or tool.
Before you even begin to define the appropriate tool(s) to collect new data, it is always smart to dig around within your organization to find out what data already exists. Are there customer surveys, employee morale studies, or other research data that can be analyzed for trust components? If you are starting from scratch, bear in mind that there is no one, simple, all-encompassing research tool, technique, or methodology that can be relied on to measure and evaluate trust in all organizations. The choice of a tool depends on your objectives and the type of metrics you choose.

A good place to start is with the trust dimension statements of the Grunig Relationship Instrument, typically administered in a pre/post longitudinal study. The following are agree/disagree statements. Respondents are asked to use a 1 to 7 scale to indicate the extent to which they agree or disagree with each statement. The statements are followed by which dimension of trust they measure, for instance, Integrity. The statements marked “(Reversed)” present the relationship in a negative, rather than positive, light.

1. This organization treats people like me fairly and justly. (Integrity)
2. Whenever this organization makes an important decision, I know it will be concerned about people like me. (Integrity)
3. This organization can be relied on to keep its promises. (Dependability)
4. I believe that this organization takes the opinions of people like me into account when making decisions. (Dependability)
5. I feel very confident about this organization’s skills. (Competence)
6. This organization has the ability to accomplish what it says it will do. (Competence)
7. Sound principles seem to guide this organization’s behavior. (Integrity)
8. This organization does not mislead people like me. (Integrity)
9. I am very willing to let this organization make decisions for people like me. (Dependability)
10. I think it is important to watch this organization closely so that it does not take advantage of people like me. (Dependability) (Reversed)
11. This organization is known to be successful at the things it tries to do. (Competence)

Step 6: Analyze results, make recommendations and measure again.
The most important part of the measurement process is to analyze the data and learn from it. What are the actionable points, how can you change and improve? What should you react to? What should you ignore? What needs to be done today?

It is important to measure failure first. Identify any opposition to management goals and decisions before it develops into issues or crises. You also need to help management understand that certain decisions might have adverse consequences on a public, so that management might make different decisions and behave in a different ways than it would have otherwise. Frequently, good communications can help your stakeholders accept a decision that management wanted to make before communication took place. For example, the public would be more likely to forgive a social media faux pas, find a change to a product more acceptable, or support a downsizing that takes employee interests into account.

How to Improve Trust
A key to good trust measurement is that it be actionable. Now that you have this data, what can you do with it? The short answer is to use it to become more authentic and transparent. In his seminal research on the link between transparency, authenticity, and trust, Brad Rawlins showed that the more you are transparent and authentic, the more likely people will be to trust you.

But it’s not just about communicating transparently. To build trust you must be transparent. Recent research by Brad Rawlins implies that greater organizational transparency tends to build trust, while less transparency tends to diminish it. “You can’t just say what you want to say, or even what you are legally required to say. You have to be willing to say the things that no one wants to hear,” said Rawlins. His final point, which returns to the concept of reflexive trust, is that if everything we say is tainted by falsehoods, no one will believe anything we try to communicate.

Long before you worry about communications, if you measure your trust level and find it lacking, you need to make sure that the things that you do, you espouse, and you believe in are the same. Rawlins suggests mistrust originates not with your stakeholders, but with management. The bottom line today is that the integrity of an organization’s leadership is essential to trust. There can be no disconnecting between what you believe and what you do. More importantly, if you work for an organization that doesn’t have integrity, there is no point in worrying about or measuring trust because it simply can’t exist. Put another way, if you want to increase the organization’s trust, you need to do trustworthy things.

There are many times when a change in trust does not lead to immediate changes in behavior or attitude, so it is important to set realistic expectations. Trust and good relationships keep publics from engaging in negative behaviors such as litigation, strikes, protests, or negative publicity. As a result, you may have difficulty measuring a behavior that did not occur because of a good relationship. This is one area when listening to the social media conversation can be helpful. Social media makes it very easy to vent. If the amount of venting remains low – less than, say, a dozen tweets in a 24-hour period – it may well blow over. But if you see a sharp increase in tweets, it might be time to retreat to the war room and figure out how you might be able to restore trust.

One way around this is to take periodic (once a quarter or three times a year) pulse checks of your stakeholders to test their perceptions. Another way to approach it is to look at your organization’s share of negatives as compared to that of peer companies or competitors. If your share of negatives is declining while the others’ are holding steady or going up, then you know your strategy is working.

Be aware that there may be a long lead time between the development of a good relationship and a positive outcome. For example, if you make a habit of inviting members of a community to social events and other relationship-building occasions, they may begin to see you as friendlier or nicer or easier to deal with. As a result you are more likely to find that differences or misunderstandings between that audience and your organization can be resolved by conversations rather than litigation. However, it may take months or, more likely, years, before your efforts have a measurable effect on legal fees.

Resources for Further Study
There are many books available that discuss and describe both qualitative and quantitative research techniques. Here are three that specifically discuss such techniques from a public relations perspective:

• Using Research In Public Relations, by Glen M. Broom and David M. Dozier (Englewood Cliffs, NJ: Prentice Hall, 1990)
• Primer of Public Relations Research, by Don W. Stacks (New York: The Guilford Press, 2002)
• Public Relations Research For Planning and Evaluation, by Walter K. Lindenmann (available from the IPR Commission on Trust Measurement and Evaluation)

Two useful resources for qualitative and quantitative research techniques are from the Advertising Research Foundation’s: “Guidelines for the Public Use of Market and Opinion Research,” and “The ARF Guidelines Handbook: A Compendium of Guidelines to Good Advertising, Marketing and Media Research Practice.” Both are available from the Advertising Research Foundation, 641 Lexington Avenue, New York, NY 10022.

A Trust Measurement Checklist
If you’re thinking of commissioning trust research, you should make sure you go through the following checklist and have the requisite data on hand.

• Write down the specific goals and/or objectives for your program.
• Make a list of the stakeholder groups that impact your success.
• List the various channels and media outlets that are important.
• List the various campaigns that may impact your trust or awareness levels.
• Make a list of opposition groups and competitors.
• Find out what other research exists, including customer satisfaction or employee attitude studies.
• List all the important media outlets and channels that matter to your stakeholders.
• If part of the project involves an assessment of media coverage, determine who will be responsible for collecting the mentions, clips, or copies of broadcast materials that will have been generated. What are the ground rules and/or parameters for clip and/or broadcast material assessment?
• Define the timeline for the trust measurement effort. What are the budgetary parameters and/or limitations for the assignment? Do priorities have to be set?
• Clarify how the collected information will be used, and by whom.

Questions for the Research Suppliers, Agencies, and Consulting Firms that Conduct Trust Measurement and Evaluation Studies
Before a trust measurement and evaluation research project is launched, there are important questions that should be asked of those who will carry it out:

• What will be the actual research design or plan for the project? Will there be a full description in non-technical language of what is to be measured, how the data will be collected, tabulated, analyzed and reported?
• Will the research design be consistent with the stated purpose of the study? Will there be a precise statement of the universe or population to be studied? Will the sampling source or frame fairly represent the total universe or population under study?
• Who will actually be supervising and/or carrying out the project? What are their backgrounds and experience levels? Have they ever done research like this before? Can they give references
• Who will be doing the field work? If the assignment includes media content analysis, then who will be reading the clips, or viewing and/or listening to the broadcast video/audio tapes? If the assignments involve focus groups, then who will be moderating the sessions? If the study involves conducting interviews, then who will be doing those and how will they be trained, briefed, and monitored?
• What quality control mechanisms have been built into the study to assure that all readers, moderators, and interviewers adhere to the research design and study parameters?
• Who will be preparing any of the data collection instruments, including tally sheets or forms for media content analysis studies, topic guides for focus group projects, and/or questionnaires for telephone, face-to-face, or mail survey research projects? What role will the organization commissioning the study be asked, or be permitted, to play in the final review and approval of these data collection instruments?
• Will there be a written set of instructions and guidelines for the readers, the moderators and the interviewers?
• Will the coding rules and procedures be available for review?
• If the data are weighted, will the range of the weights be reported? Will the basis for the weights be described and evaluated? Will the effect of the weights on the reliability of the final estimates be reported?
• Will the sample that is eventually drawn be large enough to provide stable findings? Will sampling error limits be shown, if they can be computed? Will the sample’s reliability be discussed in language that can clearly be understood without a technical knowledge of statistics?
• How projectable will the research findings be to the total universe or population under study? Will it be clear which respondents or which media vehicles are underrepresented or not represented at all, as part of the research undertaking?
• How will the data processing be handled? Who will be responsible for preparing a tab plan for the project? Which analytical and demographic variables will be included as part of the analysis and interpretation?
• How will the research findings and implications be reported? If there are findings based on the data that were collected, but the implications and/or recommendations stemming from the study go far beyond the actual data that were collected, will there be some effort made to separate the conclusions and observations that are specifically based on the data and those that are not?
• Will there be a statement on the limitations of the research and possible misinterpretations of the findings?
• How will the project be budgeted? Can budget parameters be laid out prior to the actual launch of the assignment? What contingencies can be built into the budget to prevent any unexpected surprises or changes once the project is in the field or is approaching the completion stage?

Katie Paine is a member of IPR’s Measurement Commission, founder and senior measurement consultant at Paine Publishing.