Tag: policy relevance

The Importance of Being (Pragmatically) Earnest

Photo courtesy of the Guardian UK.

When engaging with policy audiences and organizations, how can one be truthful when telling the whole truth may be counterproductive?

This post is part of an occasional series discussing the ethical dilemmas that arise when academics engage with policymakers and the broader public. This series is part of the Rigor, Relevance, and Responsibility project of the Sié Chéou-Kang Center for International Security & Diplomacy, which seeks to make ethical considerations an integral part of policy-relevant research and engagement. The program develops knowledge around, and informs the practice of, responsible engagement so that future generations of academics can engage in the policy world with confidence and clarity. This program is supported by the Carnegie Corporation of New York.

Continue reading

Do Policy Recommendations = Policy Relevance?

Whether scholars embed policy recommendations in their work is a flawed measure of whether work is policy-relevant.

Across a series of articles and book chaptersMichael Desch and Paul Avey have argued international relations scholarship is declining in policy relevance, with IR scholars falling into what Stephen Van Evera has called a “cult of the irrelevant”: a hermetically-sealed professional community that values technique and internal dialogue over broader societal and political relevance. As evidence, they cite data demonstrating a marked decline in the frequency with which articles in top IR journals provide policy prescriptions.

Continue reading

Thoughts on making the most of APSA for the alt-ac attendee

Graduation Cap and Diploma on White with Soft Shadow.

C/o Bluestocking, 2008 Uyen Le

APSA is nearly upon us again, and I thought I should write something profession-related as I got back into blogging. My first thought was to make fun of annoying questions, but I already did that (six years ago…but still relevant). And there is a lot of advice floating around for grad students or others on the market. Instead, I thought I’d focus on an area where my experience is more unique: navigating academic conferences while working outside academia (or alt-ac*) and–in my case–trying to get back in.

For just a little context, I am currently in a tenure-track job but had always been on the policy-academia border. I worked in the defense industry in DC before grad school, and continued working part-time after I started (as I attended school in DC). I then switched to the think tank world (working part-time with the Pew Research Center). After graduating, I went on the academic job market but ended up getting policy jobs–first with the University of Maryland’s National Consortium for the Study of Responses to Terrorism (START) and then full-time with Pew. After a few years out, I decided to try the academic job market again, and got my current job.

Continue reading

National Security Generalists and Learning the Lessons From Lost Wars

A friend posted this piece on facebook: “Why Nerds Should Not Be In Charge of War.”  It draws from the new PBS Vietnam War documentary by Ken Burns and Lynn Novick to argue that it happened because of the prominent role played by “generalists.”  Yes, Robert McNamara and his gang of Whiz Kids are mighty arrogant, and they have much blame to share for the war.  Indeed, McNamara, unlike certain other arrogant former SecDefs, has spent the time since trying to grapple with what he had wrought.  There is something to the idea that we need folks involved who are regional experts.  Indeed, there has been much debate about whether we political scientists did area studies wrong by insisting on generalizable theory and advanced methods.

Continue reading


Against Policy Relevance: A Polemic


Over on my Facebook feed, there’s a good discussion going on about Adam Elkus’ “The Problem of Bridging the Gap.” Elkus’ post amounts to, quite deliberately, a medium-length polemic against “policy relevance.” That is, he aims to provoke.

For example, Elkus argues that:

It judges the value of academic inquiry from the perspective of whether or not it concords with the values, aims, preferences, and policy concerns and goals of a few powerful elites. Why, if anything, do we judge “policy relevance” by whether or not it helps governmentpolicy elites? Surely governmental elites, politicians, think-tankers, etc aren’t the only people who care about policy! The “policy relevance” model is simply a normatively unjustified statement that political scientists and social scientists in general ought to cater to the desires and whims of elite governmental policymakers.


It demands that academic inquiry ought to be formulated around the whims and desires of the people being studied. One does not see this demand outside of the political science policy relevance wars. No one asks psychologists whether experiments are “relevant” to lab rats because it would be absurd to base research around what the experimental subject wants. Psychologists also do not care whether or not the college students that are paid to populate their experiments find their research “relevant” or understandable. Nor do neuroscientists inquire about the preferences of neuronal populations or biologists the opinions of ant colonies. Yet political scientists ought to cater to a narrow set of policy elites that they (partly) study?

You should go read the whole thing. Continue reading


Blogging the Gap

Yesterday, I had the chance to participate in the Bridging the Gap workshop led by Bruce Jentleson.  It is an effort every summer to help younger scholars figure out how to engage the policy world in a variety of ways, including figuring out how to write and publish op-eds, how to get into government for short periods of time (like the Council on Foreign Relations International Affairs Fellowship that changed my career/life), how to engage think tanks and more.

Continue reading


Ready for the Weekend Quasi-Linkage: Are Political Scientists Useless?

Steve and I had a good Twitter exchange with Tom Ricks about whether or not political science is useless to policymakers, particularly quantitative work and modeling. I thought this exchange was funny because today I saw that Colin Kahl, friend and more importantly, a damn good political scientist, was just appointed as Vice President Biden’s National Security Advisor.

Ricks’ broadside has provoked a few choice blog posts from Steve, Paul Staniland, Tom Pepinsky, and Henry Farrell. (Dan Drezner also had a good one on the topic from 2012). I also thought Ezra Klein’s column from a week or so ago on why he finds academic political science so useful for understanding American politics to be in the same vein. In different forms, they all chide Ricks for ignoring a host of new books and articles that speak to important real world issues.

Ricks references a good forthcoming ISQ article by Mike Desch and Paul Avey that includes elite survey data where policymakers express rather dismissive attitudes towards political scientists, particularly of the quantitative and game theoretic variety.

Having attended the Minerva Initiative conference last week where all of the newly funded projects were given a few minutes to talk about our research, I know that the Pentagon is interested in sophisticated modeling efforts, including game theoretic work, geospatial mapping, MRI analysis, large N work, and other methods. So, what gives?

A number of people in the academy are worried about the disjuncture between academia and political science. Having participated in several Bridging the Gap conferences, I know how hard political scientists are working to try to be relevant and thoughtful. This is also something that Carnegie and other foundations are supporting to get right. They just announced a $1 million award to former colleagues Frank Gavin and Jim Steinberg on this effort to improve the relevance of academic work. In the same announcement, Carnegie also awarded a big award to the University of Denver on non-violent protest movements as well as other grantees.

Some people conflate qualitative and historical analysis with policy relevance and large N work and modeling as theoretical and un-related to real world problems. While I’m largely qualitative, I recognize that there is quite a lot of good quantitative work to be done, and that even if the Big Data revolution is overblown, it has enormous implications for both scholarship and practice. We need to move beyond just the simple critique of the irrelevance of political science based on methods. Instead, we should look at whether the questions people ask are important ones, and if political scientists are able to communicate their findings in a way that is accessible and relevant to policymakers. With efforts like The Monkey Cage, Political Violence at a Glance, The Duck, and others, you can’t say we aren’t trying!


Human Rights Research and Researchers in IR: Are We REALLY that Odd?

Before APSA last week, I had the privilege of attending a small conference put on the Teaching, Research, and International Policy (TRIP) Project at William and Mary.  The conference was a chance for researchers in different research areas to write about the policy-relevance of their issue area and compare research and researchers in their area to the larger IR community.  It relates to the discussion going on the last couple of weeks on ISQ’s blog.  All of the participants had the opportunity to use the TRIP project data on journal articles in top-IR journals and survey data from IR researchers around the world.  I learned lot about how interactions with the policy/practitioner community differ across issue areas.

Continue reading


NYT and Poli Sci Public Engagement

The ISA mess is the gift that keeps on giving.  Now Nicholas Kristof has written a piece in his NYT column that “addresses” the controversy.  The problem is that the column is out of date.  Not just in focusing on the ISA proposal that has been beaten back by the forces of reason (that would be me and other bloggers?), but that other canards get lumped in.  While some noted bloggers have been denied tenure, it is highly unlikely that their blogging did them in.  Indeed, there is more pressure by lots of folks (presidents, provosts, deans, grant agencies) to do more outreach.

Continue reading


The Numbers Do Not Lie: People Don’t Like Numbers

Perhaps the first Monkey Cage post at the Washington Post presents some numbers that show that policy-makers tend not to like the higher tech kind of poli sci (or theory)  We knew this from previous TRIP reports and other studies, but still it is important to consider such stuff, especially given that quantitative work (in IR, anyway) is now about as prevalent as non-quant work.

One might be tempted to argue that we should stop or reduce quant work given that a key audience may not like it so much.  My first reaction was to think about baseball.  The rise of statistics to evaluate players–as depicted semi-accurately in the Moneyball book, a bit less accurately in the movie–was resisted by those in the game.  That did not mean that the numbers did not capture key dynamics.  Indeed,  knowing the results proved to be quite helpful to those who were willing to learn or hire people who understood them.

As someone who is far more comfortable with qualitiative work but has published some quant, I tend not to be as fearful of the rise of the (quant) machines as others but also see the point that the quant work has its limits.  In all things, I am a big fan of portolios and of diversity.  Just as professional baseball still relies on scouts to complement the numbers, the professionals in politics need both numbers and stories, quant and qual analyses.  After all, these politicians who do not like to read numbers sure as hell rely on them as they run for office via polling and market analyses.  Seems to me that they should keep on relying on numbers when they govern.

So, again, the answer is not to run against the latest in political science but find ways to make it digestible to both policy folks and general publics.  That this post appeared in the Monkey Cage as it starts its new life as part of the Washington Post is then especially appropriate.  The MC’s aim is to do precisely that–take poli sci and present it in ways that publics and policy folks can get easily without mastering the methods behind the analyses.  I do think that policy folks also will have increasingly stats-literate folks working for them, just as baseball and basketball teams hired the whiz kids who never played professionally but provided much insights with their scientific study of the games.

We can continue to think of ways to improve our dissemination of the knowledge we create.  Sorry, the grant I am writing this month requires a knowledge mobilization plan so this jargon is inescapable right now.  But I don’t mind thinking about such stuff–if I want public money (Canadian money in this case), I should and do accept the responsibility of trying to figure out how I will share my findings beyond the academy.  This responsibility does not shape the methods I choose to study the stuff, but it does mean I will take seriously how I plan to communicate what I learn.


Professing and the Policy World

There is so much criticism of the academic enterprise these days, asserting that professors are too focused on research and not enough on teaching and not enough on relevance to the policy world. These critiques are hardly new, but bear more weight in a time of austerity. It is easy to point to some work that seems hardly relevant and some professors who seem least interested in engaging the “real world,” but I am constantly reminded of the opposite—professors who become deeply engaged in policy-making one way or another.

Continue reading


Quote of the Day


In short, there’s no reason at all to consider microeconomics the “real” economics and macroeconomics some kind of flaky impostor. Yes, micro is a lot more rigorous — but if it’s rigorously wrong, who cares?


On Paradigms, Policy Revelance and Other IR Myths

I had every intention this evening of writing a cynical commentary on all the hoopla surrounding Open Government, Open Data and the Great Transparency Revolution. But truth be told, I am brain-dead at the moment. Why? Because I spent the last two days down in Williambsurg, VA arbitrating codes for a Teaching, Research and International Politics (TRIP) project (co-led by myself and Jason Sharman) which analyzes what the field of IR looks like from the perspective of books. It is all meant as a complement to the innovative and hard work of Michael Tierney, Sue Peterson and the TRIP founders down at William & Mary, who have sought to map the field of IR by systematically coding all published articles in the top 12 peer-reviewed disciplinary journals for characteristics such as paradigm, methodology, epistemology and policy relevance. In addition, the TRIP team has conducted numerous surveys of IR scholars in the field, the latest round capturing nearly 3000 scholars in ten countries. The project, while not immune from nit-picky criticism about its methodological choices and conclusions, has yielded several surprisingly results that have both reified and dismantled several myths about the field of IR.

So, in the spirit of recent diatribes on the field offered by Steve and Brian, I summarize a few of the initial findings of our work to serve as fodder for our navel-gazing discussion:

Myth #1: IR is now dominated by quantitative work

Truth: Depends on where you look. This is somewhat true if you confine yourself to the idea that we can know the field only by peering into the pages of IO, ISQ, APSR and the like. Between 2000-2008, according to a TRIP study by Jordan et al (2009), 38.8% of journal articles employed quantitative methods,while 30.4% used qualitative methods. [In IPE, however, the trend is definitely clearer: in 2006, 90% of articles used quantitative methods — see Maliniak and Tierney 2009, 20)]. But the myth of quantitative dominance is dispelled when we look beyond journals. In the 2008 survey of IR scholars, 72% of scholars reported that they use qualitative methods as their primary methodology. In our initial study of books between 2000-2010, Jason and I found that 58% of books use qualitative methods and only 9.3% use quantitative (the rest using mainly descriptive methods, policy analysis and the rare formal model).

Myth #2: In IR, it’s all about PARADIGMS.

Truth: Well, not really. As much as we kvetch about how everyone has to pay homage to realism, liberalism, constructivism (and rarely, Marxism) in order to get published, the truth is that a minority of published IR work takes one or more of these paradigms as the chosen framework for analysis. Surveys reveal that IR scholars still think of Realism as the dominant paradigm, yet realism shows up as the paradigm of choice in less than 10% of both books and article. Liberalism is slightly more prevalent – it is the paradigm of choice in around 26% of journal articles and 20% of books. Constructivism has actually overtaken realism, but still amounts to only 11% of journal articles and 17% of books in the past decade. Instead, according to the TRIP coding scheme, most of the IR work is “non-paradigmatic” (meaning it takes theory seriously, but doesn’t use one of the usual paradigmatic suspects) or is “atheoretic”. [Stats alert: 45% of journal articles are non-paradigmatic and 9.5% atheoretic, whereas books are 31% non-paradigmatic and 23% are atheoretical).

So, Brian: does IR still “really like” the isms?

Myth #3: Positivism rules.

Truth: Yep, that one is pretty much on the mark. 86% of journal articles AND 85% of books between 2000-2010 employed a positivist methodology. Oddly, however, only 55% of IR scholars surveyed report to see themselves as positivists. I’m going to add that one to the list of “things that make me go hmmmmm…..”

Myth #4: IR scholarship is not oriented towards policy.

Truth: Sadly, true. Only 12% of journal articles offer policy recommendations. [Ok, a poor proxy, but all I had to go on from the TRIP coding system]. Books are slightly more likely to dabble in policy, with 22% offering some sort of policy prescriptions – often quite limited and lame in my humble coding experience. Still, curiously, scholars nonetheless perceive themselves differently. 29% of scholars says they are doing policy-oriented research. This could be entirely true if they are doing this outside the normal venues of published research in the discipline and we’re simply not capturing it in our study (blogs, anyone?). All of which begs several questions: are IR scholars really engaging in policy debates? If so, how? Where? If not, why not? (Hint: fill out the next TRIP survey in the fall 2011 and we’ll find out!!)

(Note to readers: I was unable to provide a link to the draft study that Jason and I conducted on books, as it is not yet ready for prime time on the web. But if you have any questions about our project, feel free to email me).


Key Constraint on Policy Relevance

Dan Drezner has a great post today about how the foreign policy smart set (his phrase) gets so frustrated by domestic politics that they tend to recommend domestic political changes that are never going to happen.

I would go one step further and suggest that one of the key problems for scholars who want to be relevant for policy debates is that we tend to make recommendations that are “incentive incompatible.”  I love that phrase.  What is best for policy may not be what is best for politics, and so we may think we have a good idea about what to recommend but get frustrated when our ideas do not get that far.

Lots of folks talking about early warning about genocide, intervention into civil wars and the like blame “political will.”  That countries lack, for whatever reason, the compulsion to act.  Well, that is another way of saying that domestic politics matters, but we don’t want to think about it.

Dan’s piece contains an implication which is often false–that IR folks have little grasp of domestic politics.  Many IR folks do tend to ignore or simplify the domestic side too much, but there is plenty of scholarship on the domestic determinants of foreign policy/grand strategy/war/trade/etc.  Plenty of folks look at how domestic institutions and dynamics can cause countries to engage in sub-optimal foreign policies (hence the tradeoff implied in my second book–For Kin or Country).

The challenge, then, is to figure out what would be a cool policy and how that cool policy could resonate with those who are relevant domestically.  That is not easy, but it is what is necessary.  To be policy relevant requires both parts–articulating a policy alternative that would improve things and some thought about how the alternative could be politically appealing.

Otherwise, we can just dream about the right policy and gnash our teeth when it never happens.


© 2021 Duck of Minerva

Theme by Anders NorenUp ↑