What Is Policy Relevance, and (How) Should We Reward International Relations Scholars for It?

19 September 2013, 1209 EDT

The subfield of international relations seems to suffer from an inferiority complex. While most subfields of political science do their research and trust that the results are relevant to policy for a good reason, many an international relations scholar complains that the subfield is not relevant enough. The latest example is Campbell and Desch, who worry that rankings of departments are biased against policy relevance in international relations scholarship. Not surprisingly, Stephen Walt chose the bandwagoning strategy in this case (gated content, sorry).

While I don’t worry about rankings too much (I am perfectly capable of judging the quality of academic research myself, thank you very much), I do agree with some of their claims. It is odd to exclude books from consideration in a ranking, given that many of the major ideas in international relations scholarship are presented in books even today (for examples of epic books that have changed our field for so much better, see here and here). Exclusion of interdisciplinary journals from consideration is also highly problematic, since solving the world’s most pressing problems require a combination of social and natural sciences.

Where I strongly disagree with Campbell and Desch is on the inclusion of non-peer reviewed work in the evaluation of departments and scholars. Publishing in Foreign Affairs, they believe we should reward publication in outlets that do not use peer review, such as… *drum rolls*… Foreign Affairs:

“One cut at trying to rank programs by the presence of their faculty in non-academic publications involved factoring in Foreign Affairs and Foreign Policy articles.

This seems wrong to me. Academics work at universities as scientists. Our research differs from that being produced by think thanks in that academic research must survive peer review by other scholars, and this peer review is based on scientific criteria such as theoretical innovation, the quality of the research design, and contribution to knowledge. Academics have only one advantage, and that is the scientific method. If it were not for the scientific method, I would not pay any attention to academic commentary. I have many good friends who work for think thanks, and they are much better informed about current events than I am. That is their advantage. But my research is more systematic than theirs, and that is my advantage. Both types of researchers are needed.

What about policy relevance? Are my standards a recipe for irrelevance and obscurity, as some claim? No, they are not. Good scientific research on important topics is almost by definition relevant to policy. The world is full of non-governmental organizations, social movements, and governments that desperately need rigorous research to improve their programs and policies. Academic policy relevance should be defined as the ability to use the scientific method to contribute to policy formulation. Insightful commentary based on a gut feeling or authority is not academic policy relevance. It results from a fundamental misunderstanding of what the role of academic institutions is in the global society. International relations scholars who feel the need to comment on current events based on their personal views or experiences can do so, but their policy relevance must be evaluated based on their ability to use the scientific method to add value.

I would not mind publishing in Foreign Affairs myself. I certainly would not mind if my academic salary were increased for all of my reports, consulting, and field projects that are changing the world out there even as I write this. I spend a lot of time doing policy relevant research, and I can point out to real change without using misleading proxies such as my name in Foreign Affairs or the number of times my corny jokes have been tweeted. But as an academic, I am more than happy to subject my work to peer review. If my arguments are logically flawed or my identification strategy weak, I should not be rewarded just because some policymaker out there wants to justify a policy by referring to an Ivy League academic who is of the same opinion. International scholars should work harder than ever before to do the kind of research that survives the difficult process of peer review. While peer review is capricious and the outcome noisy, I believe truly excellent and policy relevant research will find its way to a good journal.