When Data Closes Doors: Lessons for Sharing Unpopular Findings

22 February 2021, 0600 EST

Photo courtesy of the Negative Psychologist.

When sharing unpopular findings, what obligations (if any) do scholars have when policymakers do not care to hear the message?

This is a guest post by Tricia Olsen, associate professor of business ethics and legal studies at the Daniels College of Business and Josef Korbel School of International Studies at the University of Denver. It is part of an occasional series discussing the ethical dilemmas that arise when academics engage with policymakers and the broader public. This series is part of the Rigor, Relevance, and Responsibility project of the Sié Chéou-Kang Center for International Security & Diplomacy, which seeks to make ethical considerations an integral part of policy-relevant research and engagement. The program develops knowledge around, and informs the practice of, responsible engagement so that future generations of academics can engage in the policy world with confidence and clarity. This program is supported by the Carnegie Corporation of New York.

It was the ideal scenario for a policy-engaged scholar: high-level policymakers had reached out with a series of questions that could be answered empirically and with a need for new data. I sought and received funding to collect the data, which was in line with my expertise, existing scholarship, and research trajectory. I hired a team of amazing students to work with me to create the database. I encountered interesting, unforeseen findings and shared them with high-level policymakers. Seemingly ideal. Instead, I subsequently waded through a quagmire of challenges and mismatched expectations. Data collection efforts are important; informing policymakers and ensuring policies are effective and appropriate. You may be thinking of course it’s important and informs policymakers! But what if our data collection yields unexpected results? How do we criticize the very institutions (and its stakeholders) we are working with when their good will is necessary to bring about better policy outcomes? When sharing unpopular or “inconvenient” findings, what obligations (if any) do scholars have when policymakers do not care to hear the message? 

I’d like to share my lessons learned so that, perhaps, your story will end better than mine.

Lesson 1: Clarifying Your Role as a Scholar

When my conversations began with a contact at the UN Working Group on Business and Human Rights, there was a lack of clarity as to the role of research in this particular context. The UN Working Group member assumed that our research would help achieve their policy goals, rather than potentially question the validity of those goals. This is sometimes the case, but the challenge emerges when it isn’t. A quick clarification was all I assumed was needed to assuage any misconception of my role. In retrospect however, this type of disconnect requires a more detailed conversation; a clear definition of roles, possible results, and future outcomes—while also revisiting the topic periodically. This first recommendation is crucial as it sets the stage for your partnership. What is your role as a scholar and what is your role within your partnership? Can these two roles coexist?

Lesson 2: Recognizing Misaligned Incentives

While we had numerous conversations about filling the data gap and the powerful role that data could play in data-driven policymaking, the findings I had to share did not fit what the UNWG wanted or were willing to hear. In psychological research “motivated inference” describes a scenario in which people have strong motivations or incentives, and thus they are very selective in the sort of evidence they absorb or internalize. We’ve all been there: once we’ve made a decision, we only want to hear information that supports that decision.

Kunda explains it this way: “The motivation to be accurate enhances use of those beliefs and strategies that are considered most appropriate, whereas the motivation to arrive at particular conclusions enhances use of those that are considered most likely to yield the desired conclusion” (p. 481). I wanted to be accurate. The UNWG wanted to continue on their pathway, not assess whether it was the correct one. I was motivated to play a part in helping them better understand the nuances of the challenges of business and human rights through data. The UNWG was motivated to improve adoption of the UN Guiding Principles on Business and Human Rights. The fact that scholars and policymakers contend with juxtaposed incentives isn’t inherently wrong, however failing to acknowledge our mismatched incentives will lend all parties to dissatisfaction. There is an elephant in the room, and it is better to point it out than to potentially get trampled.

Lesson 3: Sharing Unpopular Research and Fostering Conversation in the Midst of Contention

Conversations leading up to these meetings focused on the dearth of systematically collected data on business and human rights and the UNWG’s commitment to understanding how evidence could inform their policy work. What I did not fully understand or appreciate is that while this is what was said and felt, certain types of data and findings would simply not facilitate further action. After years of collecting data, I was very excited to present findings at a variety of UN-sponsored events. I shared that, in contrast to narratives about the “governance gap,” there is evidence that states are not necessarily, or always, weak. I shared that while not ubiquitous, victims of corporate human rights abuse often work to access remedial mechanisms at a greater rate than many would have assumed. And, perhaps most controversial, the data I collected shows that, in the context of corporate human rights abuses, the state was either complicit in, or assisted in, the abuse of approximately 30 percent of all allegations. 

The findings were (are!) intriguing and I was so deep in the data, I assumed others would think so, too. When it came time to share, I was so wrapped up in these findings, I expected they would spur greater inquiry and interest.  At a minimum, I expected some type of engagement: perhaps thoughtful glances or some furious note taking in the audience. As I presented these findings, instead of pleasantly surprised and engaged faces, the room fell silent.

That was the end of that. I did not have the opportunity to share additional findings with the UN Working Group on Business and Human Rights (though some of my work on access to remedy can be found on the Office of the High Commissioner on Human Rights website). And, what is worse, the findings and their possible influence were muted. While misaligned incentives likely explain this in some part, the UNWG also became quickly consumed with the debate over whether there should be a treaty on business and human rights.

If I had a redo button, I would frame unpopular findings as the beginning of a conversation. It would entail saying something as simple as “I know what I’m telling you is unexpected and even unwanted, but how can we use this information as we move forward?” There are countless conversations to be had in light of (and despite of) unpopular findings. Sparking conversation is a good outcome, much preferred to stopping discussion all together. On a more practical level, this is a case of missing the “meeting before the meeting.” While I had the opportunity to engage with my contact at the UNWG, I had not met nor did I have a working relationship with the other UNWG members. Were there an opportunity to socialize the findings with other UNWG members and learn more about the constraints they faced prior to my presentation, I imagine I would have been able to engage more deeply with the UNWG. Had I been more aware of this, perhaps I could have framed the findings in a way that empowered them to act on, rather than ignore, problematic findings.

To be clear, this account is not to denigrate or criticize the work of the UNWG or any others advocating for improved corporate behavior or victims’ access to remedy. It is simply an observation that without clarifying the role of scholarship, recognizing misaligned incentives, and thinking through how to share unpopular findings, efforts will be spent defending one camp rather than understanding how to better coordinate and collaborate on existing efforts. John Dewey noted that today’s institutions are the residue of yesterday’s problems. If we academics cannot position our work to, at a minimum be heard if not embraced, especially in complex policy areas like business and human rights, we might find that policymakers are simply adding to the residue.