Confronting Biases in Policy-Engaged Research: The Case of NATO and Russia

15 March 2021, 0800 EDT

This is a guest post by Rachel Epstein, professor of international studies and associate dean of faculty & research at the Josef Korbel School of International Studies at the University of Denver. This post is the first in an occasional series discussing the ethical dilemmas engendered when academics engage with policymakers and the broader public. This series is part of the Rigor, Relevance, and Responsibility project of the Sié Chéou-Kang Center for International Security & Diplomacy, which seeks to make ethical considerations an integral part of policy-relevant research and engagement. The program develops knowledge around, and informs the practice of, responsible engagement so that future generations of academics can engage in the policy world with confidence and clarity. This program is supported by the Carnegie Corporation of New York.

As early as middle school, we are teaching young minds to think critically and notice bias when it inevitably arises in news and media. Yet as academics, there is an illusion that we are free from bias and conflicts of interests that permeate all other parts of the world. To perpetuate this illusion only hinders policy engagement and deepens the divide between academics and practitioners. PhD training addresses part of this problem pretty effectively—by teaching us to consider alternative explanations in depth, to articulate the limits of any given study, and to avoid making sweeping statements about future developments that are intrinsically unknowable. 

However, in light of a recent review of literature on NATO enlargement, I ponder whether there is a critical strategy to be added to the discourse.  Antithetical to what we are taught at the advanced level—to strive for objectivity— academics should openly acknowledge political commitments where they exist, because of course they will exist. These political commitments can lead us to become “stealth issue advocates,” in the words of Roger Pielke, where social scientists claim to be arguing from expertise but are in fact arguing from a political position. And we may only be dimly aware of doing this; the first victim of the deception may be the researchers themselves, in terms of not recognizing their own biases.

This is not to suggest that researchers fail in general to approach their topics with an open mind or that our political commitments cannot change as a result of our research—ideally, they would. But in the area of NATO enlargement’s hypothesized effects on the Russian regime’s conduct in recent decades, there is evidence that in striving for objectivity, scholars actually just hide their biases rather than incorporating them explicitly into the debate, which has in turn undermined the quality of discourse. NATO enlargement is a particularly good case through which to examine this issue because although the biases are relatively subtle, they have had undeniable influence on the conclusions scholars draw and the stridency of their claims.

The early debate about NATO enlargement’s likely effects on Russia generally had various contours. Critics of enlargement to East Central Europe in the 1990s, such as Michael Mandelbaum and George Kennan, argued that it would incite Russian nationalism, elevate that country’s sense of humiliation and defeat, and lead to the dramatic worsening of Russia-Western relations. Other skeptics downplayed Russia’s objections or potential capacities, finding fault with the policy instead on the basis of its limited utility and high cost. Meanwhile, supporters of enlargement, including yours truly, speculated that Russian domestic politics would take their own course regardless of NATO actions or concluded that NATO enlargement would be a productive hedge if Russian revanchism resurfaced and/or a useful tool for stability, even if it didn’t. But early on, whether NATO enlargement would cause Russian aggression, revanchism and heightened hostility with the West was essentially unknowable.

So have we learned anything in the intervening decades about who was correct? While the short answer is “no,” this has not stopped committed observers, myself included, from marshaling confirmatory evidence for their side. On the surface, the debate over NATO’s effect on Russia takes the following form and reflects an update to the version outline above. Critics of NATO enlargement, as of 2021, could plausibly argue that NATO enlargement since 1999 had encroached on Russia’s sphere of influence, pushed Western forces up to Russian borders, and provoked Russia into launching conflicts in Georgia and Ukraine. Meanwhile, having fed Russia’s sense of insecurity and encirclement, critics argue, the leadership embarked on a series of ferocious domestic repression measures, including poisoning, jailing or killing journalists and opposition members, curtailing protest and severely limiting the free flow of information. None of this was necessarily inevitable, this reasoning suggests. 

The other side of the argument insists, however, that the Russian regime was riddled with corruption and was intent on covering it up. Governments from Yeltsin to Putin very likely engaged in subterfuge that resulted in the massacre of their own citizens to build support for authoritarianism and relentlessly pursued imperialist ambitions, regardless of the will or sovereignty of neighboring states. Whether NATO enlarged or not, this argument goes, democratic endurance in Russia was never likely. And given Russian and Central-East European history, it was better to secure the small, vulnerable states to its west rather than risk further curbs on their autonomy—or worse—infringements by Russia.

But beneath the surface, another debate was playing out—and this is where consequential bias is revealed. While some scholars prioritized relations between “major powers” (particularly Russia and the US in this case), others were more concerned about guaranteeing the rights, prosperity, and security of the countries in “Zwischeneuropa”—those states sandwiched between Russia and Germany that had long been beholden to large power rivalry. As two scholars, Goldgeier and Shifrinson, concede themselves, while they “agree on how to go about evaluating the costs and benefits of enlargement,” they nevertheless “disagree on the merits of the policy because [they] place different weights and assign different probabilities to those different factors.” And both the weights and probabilities depend largely on the personal beliefs of the author.

It turns out that the value judgments Goldgeier and Shifrinson point to permeate the literature without scholars being in direct dialogue about how central those differences are to the conclusions they draw. On the side of privileging “great powers,” Stephen Cohen, for example, argued that after the break-up of the Soviet Union in 1991, “the foremost goal of U.S. policymaking should have been a Russia. . .that was prospering, politically stable, at peace and fully cooperating with the U.S. on the most threatening international problems”. John Lewis Gaddis agreed, stating that among the most important rules of foreign strategy was to “treat former enemies magnanimously” and that NATO enlargement violated not only this basic principle of diplomacy, but every other, as well. Mandelbaum’s assertion that the policy of enlargement was “largely irrelevant to the problems confronting countries situated between Germany and Russia” mirrored another sentiment in the literature, which was that East Central Europe was not in the Western strategic interest to protect. As Dani Reiter matter-of-factly pointed out, “The West did after all accept Soviet annexation or domination of all of these states during the Cold War without taking military action”.

For many specialists working primarily on East Central Europe, however, the statements above are politically problematic, even if there are few empirical disagreements. For example, many ECE specialists would agree on the goal of constructive Western relations with Russia—but would not concede that such relations should come at the expense of Eastern Europe’s ongoing exposure to possible Russian violence, intervention and revisionism. Second, there is no empirical disagreement over Western and US abandonment of Eastern Europe following World War II—though many scholars of the region do contend that it was a catastrophic mistake, not to be repeated. Third and finally, proponents of NATO enlargement are continually asking their critics whether the sentiments of those “situated between Germany and Russia” should matter in this controversy. Defenders of the enlargement policy argue that from a democratic perspective, we should listen to those populations and leaders who had rarely, if ever, willingly succumbed to Soviet domination. And here, there is an empirical correction—the drive for NATO enlargement originated within postcommunist countries—so, by those countries’ own assessment, it was not “irrelevant” to the problems they confront.

It is perhaps to be expected, though certainly not universal, that researchers, depending on the empirics they absorb, the history they read, and the field research they do, end up internalizing to a certain extent the values and interests of the countries and populations they study. This kind of socialization should not necessarily be discouraged, even if it could be. On occasion, it leads to sharp insight, as when Valerie Bunce, long more immersed in the politics, economics and languages of ECE than her Sovietologist counterparts, identified fissures in the Soviet empire that presaged its collapse, even before Gorbachev came to power. 

On the one hand, scholars acknowledging their political positions might clarify for policymakers where empirical differences end and where value judgments begin. On the other hand, even that formulation may prove to be naïve in the sense that the empirics we gather and analyze are probably never really separate from our political priors, a point that goes against much social science training and the striving for objectivity. Nevertheless, for constructive policy engagement, researchers could acknowledge more clearly when they are arguing over the probabilities that certain events will take place based on empirics as opposed to when they are arguing that something is more probable because of an underlying but inexplicit political commitment.