It is my impression that many referees think of peer review as essentially trying to come up with objections against the main thesis of any paper they are reviewing. That is, it appears that many referees subscribe to the following principle of peer review:
(PR) If there is a serious objection (or serious objections) against the main thesis of the paper under review, then recommend rejection.
Now, (PR) implies that no philosophy paper is worthy of acceptance. Why? Because Philosophy’s track record suggests that, at some point, someone will come up with a serious objection to a philosophical thesis that seemed to be well-defended at the time. In other words, for any given philosophical thesis, there are probably unconceived objections, i.e., objections that no one has thought of yet. If that is the case, then those referees who adhere to (PR) should never recommend acceptance, since there are probably unconceived objections against the main thesis of any given paper they are reviewing.
So, if you are a referee who adheres to (PR), then you are committed to the claim that no philosophy paper is worthy of publication in any journal. If this consequence strikes you as absurd, then you should change your refereeing ways.
I think it depends on what is meant by 'serious objection.' Here is an attempt (not necessarily successful) at what 'serious objection' means when the objection sufficiently warrants rejection.
Say that a "serious objection" is one that does not have a relatively simple fix, and aims to quell a relatively obvious objection.
Say that an objection has a "relatively simple fix" iff: the reviewer can anticipate a defensible reply on the author's behalf, or has good reason to believe that some such reply exists. (Charity is important here.)
Say that a "relatively obvious" objection is one that most experts on the topic (admittedly a fuzzy set) would identify as an objection.
Still, the big problem is that there can be good and bad assessments about what has a relatively simple fix, and about what most experts would identify as an objection. (Of course, human fallibility is bound to affect the process to some extent.)
A helpful rule of thumb, I think, is to ask whether raising the objection to the paper would, itself, make for a worthwhile paper. (This point isn't original to me, but I forget where I heard it.) If a worthwhile paper would plausibly result, then normally the fix isn't relatively simple and the objection is relatively non-obvious.
I am unsure whether the foregoing is altogether adequate. But hopefully it helps, if only to assist others in formulating their own thoughts on the matter.
Posted by: T. Parent | 01/02/2014 at 03:28 PM
Hi Moti: to follow up on T. Parent's point, I don't think many referees would say they accept or obey (PR). I think they would say they adopt something like the following:
(PR)* If there is a serious objection (or serious objections) against the main thesis of the paper under review THAT THE PAPER (1) REALLY SHOULD HAVE DEALT WITH AND (2) COULD HAVE DEALT WITH ADEQUATELY, then recommend rejection.
This principle, obviously, contains two contentious phrases (concerning what the paper should *and* could accomplish), but -- or so I say -- these are precisely the kinds of contextualized judgments that reviewers and editors should make when evaluating an article.
If there is some obvious objection that the paper should have dealt with, but did not, then there are two options: (1) recommend reject, or (2) recommend R&R. I take it that what (PR)* also recommends in this case is a judgment on the reviewer's behalf about whether *this* article could actually deal with the objection sufficiently well (in which case one should recommend an R&R), or whether some other, more remedial article is necessary to head off that objection *before* this article (the one under review) can really get off the ground.
And that, I take it, is a matter of judgment. I've had reviewers reject papers when I felt like I *could* deal with their objection in the paper -- and in those cases I did feel like the reviewer made the wrong recommendation (they should have given me an R&R). In other cases, though, I've been able to see that I really need to write another paper to deal with that issue, and in those cases I've accepted the rejection as justified.
Thoughts?
Posted by: Marcus Arvan | 01/02/2014 at 04:23 PM
One rough heuristic I've heard people use to help further qualify PR is to ask whether the possible serious objection would be interesting and valuable as its own paper in and of itself. If the answer is yes, that's a sign that the original work makes an important contribution and a mark towards publishing. If the answer is no, that's a mark toward rejection, that seems to prevent your puzzle about publishing above.
Posted by: Wesley Buckwalter | 01/03/2014 at 11:25 AM
Thanks for the comments, everyone.
I certainly hope all of you are right that most referees do not subscribe to PR. Judging by my experience, FWIW, I have seen too many referee reports that go like this:
Author argues that p. Here’s a problem for p. Therefore, reject!
I like Ted’s rule of thumb and Wesley’s heuristic: roughly, raise an objection only if addressing it would make for a worthwhile standalone paper. As Marcus points out, however, judgments about what would make for a worthwhile paper will vary significantly among professional philosophers.
Posted by: Moti Mizrahi | 01/03/2014 at 12:48 PM
Moti: interesting paper in Argumentation, by the way. I just read it, and pretty sympathetic to its overall thrust. I've always found Alston-style arguments to be quite underwhelming for more or less the reasons you give. And don't even get me started in the "partners in crime"-style arguments that are in so much vogue in recent defenses of moral realism (viz. "alternatives to moral realism face similar objections, so moral realism is no worse off than them" -- as though that is actually a good reason to take *any* of the views that face those problems seriously). Anyway, cool paper - thanks for bestowing it upon us! ;)
Posted by: Marcus Arvan | 01/04/2014 at 04:01 PM
Thanks very much for reading my paper, Marcus. I’m glad to hear you agree with its main thesis. I agree with you about what you call “partners-in-crime” arguments. (I like the label.) They are quite lame. I suspect it might have something to do with failing to distinguish between motivating a view and supporting a view. But I’m not sure.
Posted by: Moti Mizrahi | 01/04/2014 at 10:30 PM