I’ve had it. Recently I was asked to review a book that will not be named for a journal that will not be mentioned. It was by an author with a pretty good reputation with an excellent press on a subject that I am well informed on. (I won’t mention names. Dan can take him or her to the woodshed later.) I thought I would be doing the field a service and forcing myself to read what I thought would be an entertaining book that I might not otherwise have the time to read. The problem: it is f&ck*ng mess.
Actually that really isn’t the real issue. The real problem is that I am absolutely positive that this book will receive great reviews and probably win a prize. It has glowing blurbs on the back by luminaries in the field that are entirely unjustified and indicate either 1) they have not read the book, or 2) they have read the book but are friends with the author, or 3) they have not read the book, are not friends with the author but have all suffered major brain injuries within the last year. But it is the kind of thing that passes for good qualitative research in international relations right now. And that sucks.
I will be more specific, presenting what I see as the faults of this book, but which really characterize many if not most books in this vein. I will offer them in a positive light, as admonitions for young scholars to do better work, with enough profanity to capture my indignant rage and serious intent.
1) Do not be a bad historian. If you are going to do macro-historical work that relies on comparative case studies, be ready to read at least one goddamn primary document. I am really, really tired of seeing book after book that relies on secondary sources. This is academic hearsay. It is not admissible. And do not, under any circumstances, quote some historian’s conclusion as evidence for your argument. Get off your ass. Do the work. Historians’ work has all the problems that ours does. They are not the Pope.
If you write a book in international relations on a subject where the country’s official secrecy act is no longer in effect and you do not use primary sources, you have no excuse. And even if there is such a law, that probably means it is a relatively contemporary subject. People do have mouths. They can be interviewed. So unless your subject is how it feels to be part of a mass genocide or the politics of public policy towards the deaf and mute, this rule applies to you.
2) Do not be a statistician, much less a bad one. Show causality. The whole point of doing qualitative work, as opposed to statistical, is usually to trace a process. So get out your pencil and trace it. Don’t simply engage in some kind of half-assed correlative argument that this factor is present when this factor is present so you are right. We want to see not just the smoking gun, but the casings, the bullet, the body, and the hand on the trigger. This will probably require some reference to primary documents. See #1. If you ‘t do that you are just a statistician with a small N and no math skills.
3) Do not fall in love with your own ideas. This way your theory and evidence will match. Almost every book or article starts with an idea that is interesting to its originator, and the problem is that idea is a hard one to break up with. Almost any initial hunch is wrong in some way, even if there is also probably something to it. My first book looked for a common partisan alignment on foreign policy across countries. Didn’t exist. My second book looked for the role that identity played in U.S. multilateralism. None.
But it is very clear when you read a lot of academic work that that love never dies and authors will do anything to maintain that relationship. They will twist the truth, ignore obvious inconsistencies, or make excuses for their argument. Don’t do that. Marry. Get divorced. Marry new trophy spouse. Let the initial idea take you into unchartered waters and see where it takes you because that is inevitably somewhere new, but also more interesting.
4) Do a proper literature review. Make sure you have exhausted all the different ways that someone might go about explaining your explanandum and deal with them. Decisively. Do not pretend they are not there. It is rude and also lacks academic integrity.
5) Avoid the two-by-two table. It is a common joke at academic talks that all the great arguments involve two-by-two tables. I am instantly skeptical of every piece I ever read with a 2×2. 90% of the time they are terrible. I think that qualitatively-oriented academics are sensitive about the criticisms they get for lacking parsimony and generalizability and seek to armor themselves by creating simplistic typologies instead of learning math. That is stupid. Embrace context or go to stats camp.
I do both quantitative and qualitative work, but my best work is the latter. We can complain all we want, and I have, by the dominance in the field of certain ‘isms’ and methodologies, but we have to bear part of the blame. They have a point about our fuzzy conclusions and lack of rigor. We do a lot of really bad work, and we have to get better.
This has to be a personal code. The fact that I am reviewing a book with one of the best presses in the business that makes all of these mistakes indicates that there is no professional incentive to do any of this. Only Dan checks people’s footnotes. It must come from your own sense of personal integrity. But I will be watching…..
As someone who used almost exclusively–and cited exclusively–secondary source materials, I don’t share your obsessions with primary documents. The point is to know the key debates and issues and not simply choose what works best for your arguments.
I still owe two book reviews. I’m not sure why we do them anymore, given that the typical review comes out years after the book does.
We do book reviews so people have something to put in their tenure and promotion files indicating that at least someone has read their book. Whether we should or not is an entirely separate question, but that’s the only actual practical use of book reviews I am familiar with.
When I get, e.g., Int’l Studies Review or Perspectives on Politics, sometimes the first thing I turn to is the book review section. Obviously I look at it very selectively and often cursorily but it is possible to run across interesting things. If reviews’ only purpose is to be tenure-and-promotion file fodder, then I doubt, frankly, that anyone
would bother reading or writing them. I don’t know much of anything about the tenure process, but don’t people have to write letters about the work of the person in question? If so, why do they need the book review? Also, I take exception to the implication that book reviews are just pointless scut work. Writing a conscientious book review, even (sometimes especially) a short one, takes effort.
On primary/secondary sources: It depends on the time frame and the nature of the project, doesn’t it? Some of the undisputed classics of historical sociology (e.g. Moore, Skocpol, Wallerstein [I’m not giving titles, I think you know the books I’m referring to]) rely entirely on secondary sources. Primary sources can strengthen an argument and there are contexts in which not using them would be extremely questionable, but the notion that secondary sources are ‘inadmissible hearsay’ is quite overstated. (For one thing, if that were the case Lustick wouldn’t have bothered to write that APSR article some years ago about how to select them. Not that he was necessarily right.)
Brian’s point on primary v. secondary sources in historical
sociology is not one of scope, but of methodological consistency. Historical
data is not produced ex nihilo, waiting to be plucked by the
historically-informed social scientist, but embedded within deep, divisive methods,
theories, and discourses. Relying solely on secondary data (as Dan suggests) or
being selective about primary sources (as Lustick contends) can lead to
dangerous distortions in the collection of data.
Recent work by economic historians (Greif, for instance)
betray the tyranny of false choice between social science rigor and the study
of the past. Yet, political scientists continue to betray the study of the past
because they can keep publishing historical sociology without ever getting ink
on their hands, as appears to be the case with Brian’s phantom reviewed book.
There’s a whole discussion to be had about zeroing in on supposed “smoking gun” documents as if they decide the argument. We do this a lot in the context of “here’s a memo where the reasoning accords with my theory; therefore my theory is correct.” I don’t see how that “method” can survive a day of being involved in the actual production of such documents, let alone reading them from the perspective of a working in the government.
I agree with LFC here. I find PoP reviews to be helpful in two ways. First, it there is always the possibility that there is a book in there that I have not heard of and would want to read. Book reviews may come out two years later, but serious critiques of theories in books can take even longer. Second, it gives me a better feel for other takes on a book that I have already read besides my own. This is especially helpful since I tend to read different books than my fellow grad students.
I also wanted to mention that I do wonder about the use of primary source documents, or at least the debate surrounding them. I was a History undergrad so I had it pounded into my head that primary sources were a must (as well as the idea that all scholars have biases). However, for the purposes of political science I can see where Dr. Nexon is coming from. Especially since most primary sources are also other people’s interpretations of events. I guess it depends on the type of study you are doing. All of which is to say this is an area that I feel we would be better off having a debate about and setting out the points of each side if possible.
I’d say that the value of a 2×2 or any other typological table is in the hands of the user. I’ve seen them used well, and atrociously.
“I am really, really tired of seeing book after book that relies on
secondary sources. This is academic hearsay. It is not admissible.”
You may want to moderate that statement. What makes our own interpretations of primary sources inherently superior to those of other authors? Nobody is an expert on everything, and sometimes it makes sense to defer to others.
I don’t understand the argument here beyond “I read a book that sucks in several ways.” Are the book, author, and journal not being named because the review hasn’t been published yet, or because it’s not going to be written? If the latter, why isn’t it being written? Per Nexon’s question in comments, one of the purposes of reviews *should* be to let the author know whether or not his/her work is any good. The Damoclean sword that hovers above every author’s head needs to drop if the work is bad.
If there’s some sort of self-censorship going on because the author is successful/prominent/well-connected, that doesn’t reflect terribly well on the field generally. I hope I’m misreading the post, but the way it reads, that’s one possible interpretation.
But, Brian, I love a good 2×2. More importantly, I worry that the desire for an all-inclusive lit review may be pushing students into more unreadable work. I have ranted elsewhere about lit reviews: https://saideman.blogspot.com/2011/04/let-me-review-literature.html. Folks should definitely read all the relevant stuff, but do not need to tell the reader everything that they have ever read, only that which is germane. Of course, folks might disagree about the germane-ness of different arguments, but the risk of reviewing too much needs to be taken seriously.
You know, if you didn’t like the zombie book… you… you… could have just… said —WAAAAAAAAAHHHHHHHH!!!! I want my mommy!!!!!!
One other thing: I agree with point #3 in the post. I can understand, say, the reluctance of a grad student not to give up his/her argument halfway through a dissertation, but in general people should be more willing to discard their pet notions and go where the evidence seems to be leading.
correction: remove the “not”
should read: “the reluctance of a grad student to give up [etc.]”
Wow. I was going to suggest you as a reviewer on my book. I am never going to suggest you as a reviewer on a book. In fact, after reading this, I think I’m going to cry and eat some ice cream and re-think opening a cupcake shop in Islington. :-(
I noticed these exact problems, down to the 2×2 tables, as an undergrad taking an “advanced” course in a sub-discipline of comparative politics. The main models and concepts were generally interesting, and we read plenty of sweeping claims, full of clever insights. The problem is that if someone is smart enough to actually understand and get something out of the sophisticated theories, they are smart enough to realize that the “arguments” for these theories, at least as stated, are based not on sophisticated empirical research and analysis, but on cherry-picked anecdotes and statistical insinuation i.e. bullshit.
We read some decent empirical research of course, and some of the models seemed in isolation to be useful conventional wisdom, a decent first approximation of a complex reality. But from a broader or simply more distant perspective, the syllabus, the course, the professor, and the discipline all seemed divorced from reality, pathologically insular, single-minded, and averse to disagreement. I hope that scholars who still engage with reality take your advice and start thinking and working harder than they have been, both for their own sake and the sake of those who they teach and advise.