Earlier today, Brian Rathbun criticized “rationalists” for making assumptions.* I have some thoughts on the matter, and I expressed many of them on Twitter. And in the language of Twitter — snark. That was not the best response, and I regret the tone of my reaction, if not the substance thereof. I’d like to try now for a more constructive dialogue.
Let’s start where we agree. Brian writes
It seems for rationalists that assumptions are statements that one makes to make the building of theoretical models easier. It does not matter if they are true, only if they are useful. Assumptions in rationalism are just things you don’t touch. It is a synonym for elements of an argument that are not subjected to empirical analysis or testing. I guess this is a necessary evil to make formal models in particular work. Otherwise one can’t find equilibria and generate expectations of outcomes. (emphasis added)
That’s a reasonably accurate statement of the position I and other “rationalists” would take. At least, the bolded portion is.
However, Brian’s statement implies that formal theorists are unique in their reliance upon assumptions. That’s simply not true. No one is capable of avoiding assumptions, in their research or their daily lives. Assumptions are not just statements that one makes in order to make the building of theoretical models easier. They are simplifications that make life possible.
Each and every day, each and every one of us makes a large number of fundamentally unfalsifiable assumptions. (I’ll get to false ones further down.) As Harrison Wagner writes in War and State,
[P]eople often literally bet their lives that [human behavior is predictable] by driving a car at seventy miles an hour down a highway while separated from cars traveling at the same speed in the opposite direction only by a painted yellow line. And in buying the car they drive they will have bet a lot of money that wherever they go there will be people willing to supply them with oil and gasoline to keep it running and fix it when it breaks down. Human behavior is, in fact, very predictable, and if it were not, social organization would be impossible.
What makes human behavior so predictable? Why are we all comfortable making these and similar assumptions? You might be thinking that the answer is empirical observation. If so, you might want to read a little bit about the problem of induction. Obviously empirical observation conditions the assumptions we make, but we’re still forced to make assumptions. One might even go so far as to argue that it is the (perhaps implicit) assumption that other people have well-ordered preferences, which include among them a desire for self-preservation, that makes us comfortable betting our lives that other drivers will not cross those precarious yellow lines.
Though there are many disagreements among political scientists about how best to study political phenomena, I’m not aware of anyone whose work does not assume, at least implicitly, that the past provides some evidence of what the future may hold. Yet if you believe that it does, as I do, you have made a non-falsifiable assumption. If you doubt that, kindly describe for me the empirical test you would conduct that would allow you to falsify the claim that the world was created this morning by a mischievous omnipotent being whose sole desire was to amuse itself by watching us search in vain for patterns that do not exist. If such a being created us all with memories of a past that never occurred, and populated the world with physical evidence of such a past, how would we know it?
Now, you may think that I’m just knocking over a strawman. After all, Brian already acknowledged that objectivity is impossible. I assume (heh) that he would be perfectly comfortable agreeing with me that we cannot know for a certainty that the past happened. (Though I’d be interested to see where the conversation goes if I’m mistaken in that respect.) The real issue is not whether it is possible to avoid making assumptions altogether, but whether the types of assumptions typically made by game theorists are so implausible as to render the models upon which they are built worthless. If I’ve understood him correctly (and I will grant that I may not have), Brian contends that they generally are. My sense is that Brian sees value to game-theoretic models rooted in psychology, as are often found in behavioral economics, but no others. If that is indeed his position, it will surprise no one to learn that I disagree. But it may surprise you to know that the reason I disagree is not that I’m willing to defend the truth-value of these assumptions. In some cases, I might actually be willing to argue that they’re less distorting than is commonly thought, but I’m happy to admit that most mainstream game-theoretic models, including those I analyze in my own work, contain a variety of assumptions that are patently false. The reason I disagree with Brian is that I reject the claim that any theoretical model is only as good as its assumptions. As should you. Because that is not a defensible position.
Those are strong words. I’m trying to avoid being as obnoxious as I was earlier today on Twitter, but I stand by them.
Models are like maps.** They are objects that we use to help us get from where are (a state of relative ignorance) to where we want to be (a state of slightly improved understanding). To ask whether a map is “true” is as nonsensical as it is to ask whether a model, or the assumptions that comprise it, is “true”. No one has ever thrown their 
Garmin out the window while screaming, “Damn you! Where’s the elevation? The world is not flat!!” Depending on what you wish to know about the world, you might very well require a map with topographical information, however. When people like me say that we shouldn’t ask whether assumptions are true, only whether they are useful, we’re not simply insisting on a new name for the same thing. Whether an assumption is useful or not depends on the question you are asking. The simple bargaining models analyzed by Fearon 1995 are quite useful for helping us address the question of “why war and not negotiation?” They are not very useful for answering the question “how does the likelihood of war vary with the regime types of the belligerents?” To answer that question, you would need to make different assumptions. Just as the map you want when you’re driving through Boston is not the map you want when you’re exploring Boston on foot. If you’re on foot, you may well want a subway map. And note that I’m not just talking about the fact that different maps make different decisions about what to leave out. Many maps distort the relative size and shape of things. The information they do provide is often deliberately false. eWhen’re not just talking about accidental errors.
Take, for example, Fearon 1995.*** Brian notes, correctly, that if states were assumed to be risk-averse, the claim that there are always agreements both sides prefer to war would no longer hold. There is no disputing this. What is very much open to dispute, however, is what we should infer from this. As Brian would have it, “if [the assumption of risk-neutrality] turns out to be false, the model completely falls apart.”
That’s true only if you take “completely falls apart” to mean “contains other outcomes in equilibrium in addition to those that already exist.” And, yeah, the incentive to misrepresent private information, or rapid shifts in power, would still matter if we allowed for the possibility of risk-acceptance. (If anyone is interested in seeing a formal proof of this claim, please say so in the comments and I’ll be glad to write a follow-up post.) But Brian is absolutely right when he says “You don’t need any of that stuff about incentives to misrepresent one’s reservation price to explain war. Some just prefer it to peace, costly as it is.”
I’m just not sure that means what he thinks it means.
What Brian has argued (correctly) is that Fearon did not identify all possible explanations for war, only those that satisfy certain properties. So what? Does that mean that his work “is like a house with a rotten foundation. It will come crashing down”? Well, again, it depends on what one means by “crashing down”. Models very similar to Fearon’s have been used to generate non-obvious observable implications about war onset, duration, and outcome, and those implications have been empirically corroborated (see this paper, which tests a key prediction of this paper, or perhaps this paper, among a variety of others). That’s not my idea of “crashing down”.
If we stipulate that some leaders really enjoy taking risks, what does that get us? Well, if we can’t point to any observable information that would allow us to distinguish ex ante between those leaders that hold such preferences and those that do not, we’ve basically got a tautology. Why not just say that war is caused by warmongerism? I will grant you that such explanations are logically consistent — which is a property I value a great deal. But I’m not sure it’s particularly useful. I can point to observable factors that tell me when wars are more or less likely to happen if I focus on information and commitment problems. I’m not sure I can do the same by focusing on risk-acceptance.
Until now, we’ve also assumed that the only purpose of a model is to generate testable hypotheses about the world. Theoretical models can serve other purposes. If a dozen or so publications appear claiming that X causes Y because of A, B, and C, then a theoretical model that shows that premises A, B, and C do not logically imply a causal relationship between X and Y tells us something worth knowing. But I’ll admit that the nature of this type of contribution is more modest. It is more akin to identifying a plank of rotting wood in the bridge we’re trying to build than it is extending said bridge. I think that’s valuable, but I won’t try to sell it as something it’s not.
To sum up, there is no way to avoid making assumptions. If we were to literally assume nothing, our lives would be impossible. Scholarly inquiry of every kind would have to be abandoned. If instead we read Brian a bit more loosely and simply seek to avoid assumptions whose truth-value is highly suspect, things wouldn’t be quite so bad. But we’d still be giving up a lot more than Brian appreciates. I’m not, by any means, arguing that everyone should formalize the arguments — only asking that those who tout the virtue of pluralism practice what they preach.
*I use the term with scare quotes because I don’t know what it means. I have the sense that game-theorists are necessarily members of that set, but game-theorists frequently assume the very things I’ve been told over and over again that “rationalism” leaves no room for. It also seems that people who primarily conduct empirical analysis can be deemed “rationalists” if the use the words “cost” and “benefit” often enough, while those write lengthy diatribes against “rational choice” might go on to conduct research on the relationship between the costs of abstention and voter turnout.
**I do not take credit for this metaphor. I adopted it after reading the work of Clarke and Primo (see here and here), whose work I cannot possibly recommend highly enough to anyone interested in such debates.
***This is one of my favorite articles, and I’ve blogged about it many times before (see here especially).
All I’d add is that models can do more than be harnessed to hypothesis-testing or used as ways to check internal consistency. Understood as ideal-types, models of all sorts can also be sued as baselines against which to compare actual occurrences so as to adjudicate between different kinds of causality, particularly between adequate and accidental causality. A lot depends on whether the explanation in question is supposed to terminate in a single case or in an empirical generalization; I prefer the former, usually. But on the broader issues, completely agreed. (Of course, on the specific issue of rational-choice models, I have some ethical objections to the whole perspective, but that’s an entirely different kind of debate that doesn’t touch on these basic epistemic status issues.)
Excellent point about models as baselines, PTJ. I should have mentioned that.
I’d like to hear your ethical objections to rational-choice at some point, by the way.
I wonder though, if usefulness is the criteria, where that leaves Pol Sci, not to mention economics. Pol Sci in the US, as far as I can tell is in a dreadful state, cuts to funding and with many arguing that it has no relevance Surely relevance is linked to ‘useful’. For a snarky, but humorous take on it check this out: https://www.youtube.com/watch?v=idHQoCUfPZ4
Fair question, Colin. Personally, I think there is value to understanding the world better even if that understanding does not have immediate, short-term policy relevance. But I understand that many do not agree, and we do receive a lot of taxpayer assistance…
Thanks Phil, but I’m not sure what sense I can make of the idea of ‘understanding the world better’ in the context on an instrumentalist treatment of assumptions. Also, my Tomtom is pretty accurate to be honest (most of the time). And by that I mean it’s not a literal representation (what representation could be?), but the relationships it describes are realistic, not just useful; or maybe better, it is useful because they are realistic. So for example, when it tells me to turn left in 1K, into a given road, it mostly gets it right, and when it’s when it doesn’t that we throw it out of the window. Likewise the London underground doesn’t really look anything like the map, but it’s useful because the relationships it describes do hold in the real world; if they didn’t it wouldn’t be useful. But my ‘abstractive and entity assumptions’ gets at these issues anyway (I think!)
Excellent points. The degree to which theoretical models in political science and economics describe relationships that are realistic is certainly not anywhere near the same level as your GPS, but these models can be said to be useful in precisely that sense. There are patterns anticipated by these models that no one would have thought to look for otherwise that do appear to be operating in reality. That’s the sense in which I would describe them as useful.
Just an FYI: the “models are like maps” idea is also in JD Singer’s “Levels of Analysis” article.
I’m ashamed to say that I forgot that. It’s apparently been far too long since I’ve read that piece. Thanks!
The assumption of rationality can underpin methodologies that don’t explicitly treat models as ideal-typical instruments of prediction or even, necessarily, of the sort of model-actual disjunction analysis that PTJ discusses. It also lies at the heart of some very venerable hermeneutic traditions.
For example, if you define an action as rational when it accords with some rule or norm, and you assume that actors are rational, you end up with a guide to interpretive study of the particular social contexts in which action takes place. You can ask, ‘what scheme of norms would need to be in place in order for this action to be rational?’ This allows all sorts of interesting analyses of discourse, of ritual, of performance, and so on – all hinging upon the assumption of a certain kind of rationality.
I worry about taking this direction. Cultures have many norms and many rules, not all of which are operative at any one time (or necessarily held in common by all individuals within that society). I’ve always thought that rational choice theory should stick to its guns re individual maximization; to open up that box, to include a hereafter term (to take a random instance) in a model of suicide bombing, is to strip these kinds of models of both parsimony and falsifiability. Jon Elster is good on this. … I guess I always wonder: all actions are rational, in so far as they are the products of a cognitive process (and here I fear PTJ will come in and — one hopes affectionately — slap me down, as I know nothing of this “model-actual disjunction analysis”). So why are we so obsessed about reframing everything through the lens of individual rationality?
The sort of methodology I was alluding to was what I understand Peter Winch to have suggested in _The Idea of a Social Science_ and you are correct to point out that it only gives us a fleeting glimpse of some norms embodied in the action of some person and some time.
But it definitely isn’t individualistic. Far from it. Norms only make sense, in this methodology, if viewed as fundamentally social things. By understand norms as rules which constitute and guide a community, and by ‘decoding’ the actions of individuals according to how they follow the rules, we can make inferences about what norms are in place.
PTJ’s comment connects to this. Imagine that we have a theory – as an ideal-typical model – which states that suicide bombers are highly radlcalised ideologues whose actions are entirely reasonable (read: rational) according to the norms of sacrifice and commitment that are present in their ideology. But then imagine that in observing the behaviour of bombers, we find it very difficult to interpret what they’re doing as committed martyrdom. This suggests to us that more or different norms are in operation than what we previously assumed; we see no easy chain of reasoning that leads from posited norms to observed behaviour, and since we assume actors are reasonable, we must re-evaluate our assumptions (or, perhaps, our observations).
As for the suggestion that all actions are rational, this might be true by definitional fiat. That is, many philosophies/methodologies differentiate human action from other human behaviours on the grounds that action is some self-conscious attempt to connect means and ends, or to consider some nexus of beliefs and desires in order to make optimal choices. Other perspectives would have rationality assessed, for example, according to the extent to which a choice is a sound calculation, or confers a survival benefit, where action can be irrational when it derives from bad critical thinking.
In PTJ’s terms, if I understand them rightly, what you are describing here is a method, not a methodology. (He may well be fighting a losing battle on this point, however.)
I would hope that any example I try to provide of a methodology in action will appear to have a method to it! In this case, the methodology is to define rationality as ‘action according to a rule’, and to define rules as inter-subjective and ontologically primitive social material. The structuring of enquiry according to the question ‘in virtue of what rule is [action] rational?’ is also a methodological principle. The various ways in which that question is answered – the processes by which observations are collected and analysed in order to produce a sound account of the rules – would be methods. This is, at least, how I understand the difference between methodology and method, and I wouldn’t at all be shocked (though a bit dismayed) if I’ve got it wrong.
I should have refreshed the blog before commenting on Brian’s post. Ninja’ed + KO’ed by Phil. Again.
The overarching defense of assumptions here is extremely strong, but I am not sure it answers the question of why *these* assumptions and not others. There’s no problem here so long as we are careful to stipulate the epistemological value of what we do. There is a sociology-of-science problem, however, insofar as rational-choice assumptions become treated as self-evidently correct while others require a different burden of proof. I think we have reached a point where this is the case, with specific kinds of rational-choice models (formal or informal) (1) given explanatory privilege in the field and (2) conflated with really real descriptions of how the world operates. In reacting to this state of affairs, Brian is on solid ground.
I’ve noted this repeatedly in my concerns about the over-reliance on “off-the-shelf” mechanisms — often derived from Fearon’s interventions in the field — as adjutants to correlational analysis.
My impression from reading this exchange on an iPhone while walking to do other things is that Phil is defending formal theory (a minimal position) here and Brian is unsure if he is attacking formal theory or rationalism. But then I am probably not the right person to ask, since I’m not just watching this fight but sitting in the bleachers drinking overpriced domestic beer and shouting while waving a foam finger saying “WE ARE NUMBER ONE.”
In fairness, I’m not sure if Brian conflated the two or I did. The two are certainly not synonymous, but attacks on “rationalism” that are not also attacks on formal theory are relatively rare (though I have seen them).
You’re absolutely right, Dan, that I haven’t said much here about “why these assumptions?” Perhaps another post is in order…though I’d like to hear more from PTJ before I write such a post.
What I will say, for now, is that I’m sympathetic to the concern that different assumptions face different burdens of proof. That’s troubling, and I think you are probably right. On the other hand, I think a lot of people really overestimate how easy game-theorists have it. Maybe big name game theorists do…but I suspect there’s a reason Fearon hasn’t published some of his best papers, and that his most influential piece appeared in a (very good) subfield journal rather than a general journal. I constantly get rejections where the reviewers fail to engage the argument I’ve made in any way. I have a lot of friends who complain about the same thing. I can’t imagine it’s easier for people who explicitly acknowledge making the types of assumptions that aren’t as mainstream in poli sci, so I’m not disagreeing with you about that point. I’m just trying to say that I think the field as a whole is a lot more hostile to “rationalism” than many people realize, in *addition* to being unjustifiably hostile to other approaches.
We touched on this in our podcast discussion, IIRC. I wonder if there’s something else going on: if work that is much more explicit about teasing out the logic of those purported “mechanisms” faces a higher hurdle than those that invoke those we ‘know’ to be true?
Of course, the question is empirical… of the kind that it would be nice to have better data about in ways that require peer-review processes and outcomes to be more transparent. And clearly there should be disparate burdens of proof. (One need only postulate “magical mental powers” as an assumption to see the problem!) So, uh, interesting point.
We did indeed, briefly.
My gut is that you’ve hit the nail on the head, though you’re right that we’d need better data to really sort this out. I also agree with what you said above about empirical work that makes some hand-wavey references to Fearon’s work. There is an aura of truthiness around some “rationalist” work — at least, that which has already been published by big name scholars — that allows people to borrow its prestige even when it’s far from clear that the argument their making actually comports with the work their citing.
It sure would be nice to run some field experiments to sort these things out though. :)
I’m not sure the unfortunate fact that atheoretical empirical work is more likely to get published if the author couches the interpretation of their results in certain language is a knock against “rationalist” work though. Whether it is valuable to make certain theoretical assumptions is independent of whether people inappropriately grant prestige to anyone who cites work that makes those assumptions. Why not focus on *that* instead of criticizing the assumptions?
Jeez, Dan, you act as if it’s unlikely that dictators can calculate the consequences of their actions as easily as Garry Kasparov or Deep Blue.
A few questions/comments from someone far less knowledgeable in these debates (though clearly the field is still in need of some clear-headed debate, as confusion continues to pervade!)
1. Where do assumptions come from? I suppose there can be many sources, but one idea is that they may stem from formerly descriptive/historical work (by author X, on subject Z) which, if widely read and accepted (its accuracy aside,) over time becomes so embedded in a scholarly community that its claims turn into the conventional wisdom or shorthand for “how actors think/behave,” broadly speaking.
If this is plausible, and in agreement with Dan and Brian, I suggest that what must be encouraged is more careful historical (primary source) work to see whether in fact rationalist assumptions realistically hold or whether they indeed roughly correspond to how historical actors reason and act. Similarly, we’d all be well advised not to “transpose” mechanisms or assumptions across issues, or time-space, but instead aim for more bounded goals.
What frustrates me is that this type of work seems to currently have little purchase in a field like IR (or Political Science in the US) that is so enthused by large-N generalizations. That obsession in my view sort of forces scholars to make a lot assumptions as they go about gathering and analyzing huge amounts data for a great many cases.
Does this make any sense?
That does make sense, indeed, GM.
In answer to your question, assumptions come from a lot of places. Sometimes, they are made purely for convenience. Sometimes, they are made for the sake of argument, as in the example I gave above of wishing to demonstrate that the very premises others have pointed to by way of explaining their correlational results do not actually imply the story they’ve told. Sometimes, they are made to establish a simple baseline. Sometimes, they are informed by historical evidence of a variety of forms.
I think the field would benefit a great deal from being more open to careful primary source work. But it’s important to keep in mind that those of us who make these assumptions do so in the hope that it helps us appreciate the logic of an argument better, or to give us a foundation upon which to build, or to identify observable implications. When the goal is one of the former, testing the assumptions offers nothing. Regardless of what is found, the contribution of the argument remains the same. The goal is the latter, the best way to evaluate whether it was successful is to see if the observable implications are consistent with empirical observation. If they are, it’s not at all clear why we should care whether the assumptions that went into the model that led us to look for the pattern we never previously looked for were true or not. In other words, I agree with you that we stand to gain a lot as a field by recognizing that searching for broad generalizations are but one way of gaining insight, but there *often* isn’t much to be gained by testing assumptions.
This is not to say that assumptions are inviolate, as Brian claimed rationalists argue. I’ve been pretty clear in the past (on my personal blog), as have others, that it can be entirely valid to question assumptions, under certain conditions. If a scholar assumes something to be true always and everywhere that is usually true in most cases, with the exceptions serving to weaken the pattern anticipated by the model but never reverse it, then there’s no harm done by making that assumption. If a scholar assumes something to be true that is patently false, but is unnecessary to the results they derive, and making more realistic assumptions would only require a much longer proof of the same substantive result, there is again no harm done by the assumption. A great many of the assumptions “rationalists” make fall into these two categories. When people criticize those assumptions, it’s unproductive.
BUT, that said, there are situations where the contribution offered by a theoretical model critically depends on a particular assumption being true. If there is cause to doubt *those* assumptions, that’s worth pointing out. Game theorists are well aware of this, though. Most of my own work focuses on demonstrating how important results that have been established by previous game theoretic work is sensitive to implausible assumptions. It is not true that we treat our assumptions as sacrosanct and refuse to critically evaluate them or allow others to do so. We’re just trying to convince people that reflexively questioning every assumption, even when our conclusions would remain the same if we relaxed them, is unproductive.
Well said, Phil.
Google ‘The Game of Torture’ and the immense normative problems underlying formal theory’s blindness to the real world becomes obvious, alarming, and something to be condemned.
Great debate. But I wonder if Brian’s proper target was formal theorists (or if he intended it to be). Like methodologists who work on measurement (who are always happy to engage in endless conversations about how scarily fraught it all is), game theorists tend to take assumptions seriously. Indeed, game theory is mostly *about* playing around with assumptions and seeing how slight changes affect model results (i.e., the opposite of the “untouchability” that Brian asserts). Reading empirical work as much as I do, however, Dan’s point re “which assumptions” is really apt. It’s not that rational choice is hegemonic — or that formal theory is necessarily even that influential — it’s that we’ve had the advent in the 1990s and the 2000s of the rational-choice-framework-plus-empirical-tale-tacked-on. Some of this work is great, but some of it uses very simple, very primitive off-the-shelf rationalist models to “shed light” on an empirical case. Of course, it doesn’t shed light on an empirical case, often radically and fundamentally misunderstands that case, and occasionally gets cited at double the rate of careful and actually illuminating empirical work. Of course, if Brian’s aim had been bad empirical work that doesn’t address the applicability of its rationalist assumptions to the case at hand, then there’d be little to debate here.
I just wanted to throw out a few things that I’m surprised haven’t come up. (1) agent-based modeling: What I find interesting about ABM is that it is able to model a multitude of different actor decision rules in a single interactional setting (which formal theory generally cannot do). So you can have rational maximizers, altruists, and imitators, and simply shifting the proportion of these in a multi-actor setting results in very different outcomes or patterns of diffusion. Where does one get information on how to set proportions of actor types? Some say public opinion, some say you extrapolate from other observed patterns. Once again, there’s a lot of sophisticated discussion about this stuff, and I echo someone’s comment (Phil’s?) that the late 2000s are not the early 1990s. (2) The deeper problem is that most of our models of behavior and of interaction are not truly relational. Rationalists accept this critique with one side of their mouths, but we don’t do much about it, because it’s simply easier to do methodological individualism. That choice has consequences, however, for the accuracy of our models (and the efficacy of our policy interventions).
Well said, Adrienne.
I think this is sufficiently well said that it deserves its own post.
by “sufficiently” I mean AWESOMELY.
Phil believes that “No one is capable of avoiding assumptions, in their research or their daily lives” – but this is merely a belief. In reality it is perfectly well possible to live without any assumption, provided one does not cling (as is very common) to the propositional mental mode, drifting from one predicative element to the next, for instead ‘listening’ to the subjacent qualities in experienced reality without judging prematurely and for seeing where this leads. The point is making mental life itself into an experience instead of all that presumptuous calculus about ‘external’ aspects of reality, based on assumptions – ‘talking’ into the subject matter before giving it a chance to unravel more and more fully to awareness. – You might enjoy extensive publications on this possibility; see for example https://edoc.unibas.ch/1421, or for an application to socio-economics http://www.ijtr.org in the archives in Vol. 3 and 4.