I sent out a bunch of papers for review this winter break, and the experience got me thinking about several things:
- Long review times
- Lack of reviewer comments
Although I think the peer-review process in philosophy has arguably improved as of late--and although I have had many positive experiences with great reviewers and editors--I nevertheless think our discipline could use a whole lot more discussion/debate about these practices, in part because (in my experience) other fields are very different (I'll say more about this shortly). Anyway, here are some thoughts:
1. I think we should have a discussion about best-practices for desk-rejections.
Recently, at the Eastern APA, I was having a conversation with several people who work at a Leiter top-5 program. We got to talking about our experiences with a particular journal--a journal that has a habit of desk-rejecting papers in a day or two, sometimes just a few hours. Each of us noted that we had sent papers to this journal many times and been desk-rejected in a day or two every time. All of us thought this was pretty curious. First, given that the journal is very highly ranked, and given that it appears as though something like 90% of papers are desk-rejected, the practice seemed to us to disproportionately put decision-making in the hands of one (or few) individuals. Given research on implicit bias, etc., we worried about this. Second, and perhaps more importantly, it raised the general question of what desk-rejections should be for. On the one hand, it seemed fine to many of us to desk-reject a paper that clearly falls below some minimum standard--a standard clearly not worthy to send out to reviewers. But to desk reject on other grounds--for instance, that a paper isn't novel enough? My sense was that this seemed dubious to all of us: that the very point of peer-review should be for reviewers to make those kinds of judgments. While I recognize that some may say that if one journal's desk-rejection practices seem too strict, or biased, or whatever, there are always other journals to submit to, I'm unconvinced. There are very few highly-ranked journals in many subfields, and if even a few of them have problematic desk-rejection practices, the effects can be significant. [I should also note, perhaps, that on more than one occasion where I've had a paper desk-rejected, I've had one or more visitors to my personal website from the city where a journal's editor-in-chief resides, raising questions in my mind at least about respect for respect for anonymized review at the desk-rejection level].
In any case, given that desk-rejections play a significant role in publication processes--and given that different journals function very differently in this regard--I want to suggest that this is something that our discipline should really be discussing, and perhaps set clear "best practice" standards for (via APA policy?).
2. I think our discipline should have a more serious discussion about long review times.
We've discussed this here before, and it is something that many others have discussed, but I think it is worth revisiting again. Although I could be wrong, my impression is that journal review times in philosophy are quite unlike--that is, far longer, on average--than journal review times in other fields. I have submitted to journals in psychology and political science, for instance, and review times have never been longer than 4-6 weeks (and nearly always with comments justifying the editorial decision--more on this shortly). A brief look at Andy Cullison's philosophy journal wiki shows that inter- and intra-journal review times vary widely. Some journals' review-times are between 1-2 months, whereas others average 6 months, 9 months, and in some cases, well over a year. Moreover, even those that have shorter review times on average typically have severe outliers (journals that average 2 months in some cases appear to have papers that take 11 months, or in one case 24(!) months).
Although again this issue has been discussed, I still think it could use more discussion. Long review times--particularly at top-ranked journals--can be expected to disproportionately affect early-career scholars in temporary academic positions. If, for instance, you are an adjunct, post-doc, or VAP who needs to add publications to your CV to get a job, you may simply not have time to submit to a journal that averages 6-12 months. Long review times at top-ranked journals may lead people in temporary positions to send excellent papers to lower-ranked journals with faster turnaround times, thus biasing the entire process of publishing in top-ranked journals against them (and in favor of people in more permanent positions). Indeed, I think long review-times arguably invite a "self-fulfilling prophesy effect" where individuals in stable, permanent academic positions have better opportunities to publish in top-ranked journals compared to people in temporary positions. I, for one, have consistently felt pressure to "set my sights lower" due to the tenuousness of my employment situation.
Some might say that many philosophy journals have improved substantially in recent years. However, while I think the Cullison wiki broadly bears out this claim (some notoriously slow journals have reduced their average review times by several months), my impression is that philosophy journals still lag far beyond journals in many other fields. I'm very close with some people in psychology, for instance, and they've told me that my experience sending papers to psych journals is typical: no more than 4-6 weeks turnaround (far shorter than just about every journal in philosophy).
Now, of course, some might say that reviewing philosophy papers is more onerous than reviewing papers in other disciplines--but, from everything I've ever heard from anyone in our field that I've spoken to, this seems false. When you actually sit down and review a philosophy paper, you can typically do it in a few hours. The problem, it seems, is that reviewers in our field can (and do) get away with procrastination (indeed, I should know: I've been a guilty party before, sad to say). Anyway, this seems problematic to me--particularly given how rare reviewer comments are in our field (again, more on this shortly). Just as with desk-rejections, I want to suggest that our discipline should have a clear and open discussion about turn-around times and norms/practices for making them what they should be: the 4-6 week average typically of many other fields.
How could such a norm be enforced? I have a few ideas. Here's one: if a reviewer is (A) asked to review for a journal and refuses, or (B) accepts but takes longer than 4-6 weeks, they could be barred from submitting to that journal for, say, a period of one year. Given that reviewers at top-ranked journals tend to be people who submit papers for review at those very journals, a discipline-wide policy like this could, I think, be reasonably expected to lead to 4-6 week turnaround times. And here's another idea: exclusivity forfeiture. Currently, philosophy journals have a common practice of not allowing one to submit a paper to other journals at the same time. One way to reduce journal review times might be to have a disciplinary-wide norm that after some amount of time under review (2 months), the journal can no longer require exclusivity in this regard: the submitter should be free to submit the paper elsewhere. Given that journal editors and reviewers don't want to waste their time reviewing on a paper that might get published a rival journal, this too might put pressure on journals to ensure that their reviewers get their work done on time.
3. I think our discipline could use a good discussion of (lack of) reviewer comments.
I've submitted papers to psychology and political science journals. Not only have turnaround times been no longer than 4-6 weeks in every case. I have never once not received at least a brief, but detailed explanation of grounds for rejection. Philosophy journals, however, are very different. It is not uncommon to send a paper to a philosophy journal, have it spend 2-6 months under review, and receive no reviewer comments. This too seems to me problematic. Given that the very point of anonymized peer-review is to reach publication decisions that mitigate bias, etc.--and given the many ways in which reviewer recommendations without comments could be motivated by bias (they don't like the paper's conclusion, they don't like the topic, they know who the author is. etc.)--it seems to me that our discipline should follow others and require comments justifying editorial decisions to be provided to authors. No one, it seems to me, should have to wait 2-6 months to hear from a journal and be provided no explanation for the decision arrived at. At any rate, greater discussion of this issue seems to me well warranted.
Do you agree? Disagree? I'm curious to hear everyone's thoughts!