There have been a lot of posts recently, both here and elsewhere, on the perils of peer-review -- particularly on irresponsible, unaccountable reviewers, as well as on whether contemporary philosophy (and so, reviewers) have fetishized rigor to an unhealthy (and comical) extent.
In light of these issues, I'd like to ask ye, my fellow Cocooners, the following two questions:
- What do you look for as a journal reviewer? (Please answer only if you've been a journal reviewer!)
- What do you think journal reviewers should look for, and why?
I suppose I'll kick things off by giving my short-answer to both questions.
- What I look for as a reviewer, and why: It depends on context. If I am reviewing a primarily critical paper or "negative argument" within a paper (one that takes another piece of philosophy to task), I look for rigor. I want the author to really make a convincing case that the piece they're criticizing is in error. On the other hand, if I am considering a piece of positive philosophy, I aim to balance rigor with interest, and sometimes prioritize the latter over the former. Because I recognize that interesting new contributions to philosophy can be, and usually are, chock full of holes and failed arguments (see e.g. Kant, Rawls, Etc.), what I really want to know is: has the author motivated an interesting new way of looking at the problem(s) at issue? Or, more bluntly, I ask myself: is this a paper people are likely to talk (and write) about? (Notice: a paper can be one of this sort even if its argument has a lot of questionable components).
- What I think reviewers should look for, and why: I've made no secret of my belief that many reviewers focus far too much on rigor and not enough on interest. I've read a lot of rigorous papers in top journals lately that I just don't think are very interesting at all (no, I won't name them!). My experience has been that Mumford is right. Journal reviewers are far, far too conservative. When we prioritize rigor over "interestingness" -- when we insist that a person defends each and every premise of their "to the hilt" -- we end up publishing papers that are so conservative that they're just not that interesting. There's a reason why Groundwork for a Metaphysics of Morals, A Theory of Justice, Quine's "Two Dogmas", etc., are so widely discussed...and it is not rigor. As far as I am concerned, RM Hare's hilariously aggressive review of A Theory of Justice is more or less right in its details. Rawls' book is full of problems, left-and-right. But so much the worse for Hare. Rawls' book was interesting, and generated an incredible amount of conversation. Rigor is important. Interesting is more important. Or so say I.