Should one negative review be sufficient for a journal rejection?

In the comments section of my recent post on resubmitting rejected manuscripts to new journals, Recent Grad writes:

I have a question that's somewhat related to the topic of resubmitting rejected manuscripts without making changes. A little background: I'm a recent grad. I've had around 5 reviews for a couple of different papers (including one R&R and rejections for the other submissions). Every single time I've only had one referee report. You can imagine that only having one report complicates the question of whether to do revisions before resubmitting. Referees can be so divergent I find myself very hesitant to make changes based on only one report--unless the criticisms happen to strike me as significant and obviously correct.

In response, Amanda wrote (my emphasis):

I think it could be any of those things. The possibilities, in order of my estimated likelihood:

1.The report you got was the first of two reviewers to turn it in. Because one rejection is enough for total rejection, there was no need to wait on the second report...

I've gotten a few revise-and-resubmit verdicts based on one positive report and one negative report--cases where a journal editor seemed to side with the positive reviewer over the negative reviewer. However, I've heard a lot people (including I think some journal editors) say what Amanda says: that, all too often, editors may reject a manuscript based on only one report. While I sort of understand why this can occur (good journals have high standards for acceptance, and receive far more manuscripts than they can publish), I also find it a bit puzzling - and for a few reasons.

First, as we all know--both from the peer-review process and philosophical discussion more generally--different readers can judge the same piece very differently. Indeed, philosophers are often quite split on the value of a given piece of work. I reviewed a paper a while back and found out after the fact that one reviewer recommended rejection, another revise-and-resubmit, and a third outright acceptance with no revisions! Given how differently different people can judge the same piece, isn't it a bit premature to judge a submission on the basis of a single review?

Second, there appear to be a lot of false negatives in our current peer-review process: very influential papers that were rejected by multiple venues before being published. While I understand the desire of journals to avoid false positives (i.e. inadvertently publishing bad pieces), isn't there also a danger in tilting the process too far in that direction? If excellent papers that go onto enjoy wide discussion and esteem are routinely rejected by multiple journals, doesn't that suggest that editorial processes might be weighted too far in the direction of avoiding false positives?

Finally, on the same note, one might think (as I'm inclined to think) that with creative works in general--in art, science, and philosophy--polarizing works are actually more likely to generate substantial discussion and influence than less polarizing works. For example, the film Blade Runner received mixed reviews when it first appeared--yet is now widely regarded a masterpiece. Initial reaction to Darwin's Origin of Species was also notoriously hostile. Led Zeppelin's first album was met with stinging criticism - yet it too is now regarded as a masterpiece. The same kind of thing occurs in philosophy too. Frege is said to have thought so little of Wittgenstein's Tractatus that, in all of their correspondence, there is no evidence that Frege read past the first page!

Given that most philosophy articles are never cited, what could be better than publishing articles likely to divide readers? Those who love a given piece are likely to respond to it, but so are those who are critical of it: a good recipe, I think, for generating more active and lively debate!

Comments

[Recent Grad]

~I wrote the post Amanda responded to.

The policy of rejecting based on just one bad report does make a certain amount of practical sense (as long as you actually have more than one reviewer!). If, as I'm used to hearing, journals are overwhelmed with submissions to the point that they simply don't have enough space for every good article they get, then one solution is to error on the side of rejection--i.e., that way, you reduce the pool of good articles to a publishable size without compromising quality (increasing the rate of false positives). Of course, this method does have downsides. As Marcus points out, it will be harder for divisive or controversial papers to get through. And I also suspect that the current refereeing practices encourage people to write safer and more specialized papers (as Marcus and others have also noted). But I imagine that the best way to mitigate these issues would be to have some journals that prioritize publishing papers that try to make "bigger", more controversial points. I doubt most editors would change their practices unless the rate of submissions were to go down significantly.

[Amanda]

I agree with you Marcus! And I wish there was a way to change entrenched practices. Because it is a zero-sum game though, there needs to be some policy of publishing controversial work over "safe" work that gets two "accepts". If the editors were to just start accepting more "one accept" papers without changing anything else they would clearly have more papers than they have space for.

Another option which I support but always gets rejected is to have more papers per issue. We have such an abundance of quality over space there is no need to worry about a few extra papers per journal bringing down the quality. The answer I always get is it is impossible because of reasons about space and copy-editing. This doesn't make sense to me, since switching to an online format makes way more sense these days, and I just don't see the value in caring so much about copy-editing. Would anyone's philosophical life really be changed if journal papers had a few typos?

[Pekka Väyrynen]

The going practice varies from journal to journal. For many journals where submissions are handled by specialist section/associate editors, the default is that when you've sent a paper out to referees in the first place, you wait for all the reports to come in and make a judgment based on not just the referee's recommendation but also the referee's reasons for the recommendation and your own reading of the paper. (This is the default procedure in the two journals I'm involved with as a section/associate editor.) It isn't uncommon for papers to get through where the referees are split. If a generalist journal doesn't have a team of specialist section editors, you would expect the editor to be guided much more heavily by the referees' recommendations; it's a real shame that sometimes reports aren't screened for quality well enough. So space isn't the only practical concern; editors' level of expertise with the paper's subject matters a lot too, and this can vary depending on how the journal is set up.

Also: I wouldn't read too much into a rejection based on just one report, without knowing what journal is in question. Some (even fairly top) journals use just one referee as a default, and only sometimes call in another referee.