I thought I would offer my take on the series of posts below on how to reform the peer-review system. Part of the problem is bad reviews. We can remove the anonymity of them, but that raises the obvious problems (bad blood, retaliation, pulling punches, speaking truth to power, etc). A better solution would be to rely on the expertise of editors. A really stupid review is obvious to even non-experts. And if one reviewer says X and the other says ‘opposite of X’, then you should know that you have to get a third opinion, not just reject the piece for not garnering sufficient support from the beginning (this is Dan’s point).
Why can’t editors do this? The biggest problem seems to be workload. They just don’t have the time to carefully sort through reviews and evaluate them as critically as the paper under consideration. But how do we lessen their workload? Write less. Read more. This goes for graduate students and professors. Graduate students are “socialized” earlier these days and this generally means they are pushed to publish. The tenure standards are higher everywhere now. None of this is good for the field. All of this means too many papers that don’t tell us very much. And they make editors’ jobs impossible.
Political science and IR today reminds me of elementary education. Most everyone in educational psychology seems to agree that kids really shouldn’t be pushed early on, that the social part of early grade school is more important than the academic, that they aren’t cognitively ready to do proper book learning. Yet we expect more academically of our little ones than ever before. They screen them in my town for their competence in the ABC’s upon entering kindergarten, then tell us what to work on over the summer. Go f*&k yourself.
Most everyone also agrees that some kids, particularly boys, develop more slowly intellectually, but of course eventually catch up. And also that it is better to think creatively rather than to do rote learning. Yet homework starts in first grade and the first thing they focus on is penmanship. Ugh.
Without saying that grad students are just kids, we have to recognize that there is a growth process in graduate school, and frankly even in the assistant professor stage. By pushing younger academics to write too early and too much, we deny them to chance to read widely and think more creatively. We also encourage people to be carbon copies of their advisors because if people don’t have a chance to develop their unique voice they will merely parrot their mentors. The metric of success becomes a simple process of counting journal articles and where they are placed, not WHAT THEY SAY THAT IS NEW AND INTERESTING. This gets to some of my earlier complaints in Stuff Political Scientists Like, that we fetishize new data to the detriment of new ideas.
This is partly because the people who are doing the judging (tenured faculty members, journal reviewers, search committees) are themselves writing too much and don’t have time to properly read and critically evaluate other people’s work. We have created an academic environment where everyone is in his or her own bubble. No wonder we all get (and write) lousy reviews. I personally don’t keep up with the journals as I would like to and this makes me sad.
Maybe I am idealizing the (fairly recent) past, and maybe Berkeley was just different, but when I was in grad school, not so long ago, grad students didn’t judge academics by how many APSR articles they had published, but by their ideas. Oh, Schweller, he is the ‘balance of interest’ guy. Oh, Moravcsik, he is the ‘liberal intergovernmentalist.’ I frankly had no idea about how much people had published, only what they had published.
Of course, this probably cannot be changed because it is an arms race. I can’t tell my students to slow down, to stop and think, because they will get cut off at the knees by the more ‘socialized’ grad students in other programs on the job market. Or they might not get tenure at a university that simply counts beans. I find this not only sad but also deleterious to the discipline. I can’t do much about it, but my kids are going to spend the summer in the yard, not with workbooks.