The Philosophers' Cocoon

Boilerplate journal rejection language?

In our newest "how can we help you?" thread, a reader asks:

Recently got desk-rejected by Ethics after waiting 4 months. The comment " I am afraid, however, that it is inappropriate for the journal." is boilerplate right?

Interesting question. I'm not sure this is boilerplate. I took a look at a couple of desk-rejects that I received from the same journal, and didn't see this kind of language. Curious to hear what others think!

Asking editors for referee clarifications?

In our newest "how can we help you?" thread, a reader asks:

I'm working on revising an R&R, but for one comment by a reviewer, I'm finding it difficult to understand exactly what the reviewer's concern is. Part of the issue is that the comment is full of typos and incomplete sentences. But that aside, my question is more general: Is it ever alright to ask the journal's managing editor to reach out to the reviewer on your behalf in order to ask the reviewer a question of clarification?

This is an interesting question, and I'm not sure what the answer is. I've never approached editors with a query like this, and wonder whether any readers have (and what editors and referees might think if approached in this way). Anyone have any helpful experiences/tips to share?

Presenting papers currently under review?

In our November "how can we help you?" thread, an anonymous postdoc writes:

I've seen a few discussion here about presenting papers before they are under review. And the consensus seems to be that this is fine (as I think it should be).

However, I'm not sure you've touched on presenting papers that are currently under review.

Specifically, I am giving a talk next week on a paper that is already under review, and at a venue where I expect some of the most obvious reviewers for the paper will be attending (although it is not a particularly niche paper, so the probability the reviewers are actually there is reasonably low).

The following reasons make me think this should not be a problem: 1. I'm pretty sure I've seen people do this before. 2. If the reviewers are in the audience and haven't started reading the paper yet then this is really just an instance of pre-review presentation.

But the final reason goes rather the other way: 3. If the reviewers have started reading the paper and my presentation changes their views of the paper (either positively or negatively) then they might need to excuse themselves (or feel that they should) from review—and that would be annoying for everyone (me, the reviewers, the editors) due to the wasted time and effort all around.

So, have I made a mistake? And assuming that I will present the paper anyway (it would be difficult to change now), is it better to flag that the paper is already under review or not to mention it?

My sense is that it is fine to present papers that are under review at journals. I suppose it could "pollute the reviewer pool", as it were, particularly if the paper is on a niche topic where there aren't many reviewers for the journal to choose from. But this is a problem even before a paper is under review, as presenting at conferences or colloquia can disclose your identity to potential reviewers too. I also don't think it's a good idea to flag it to the presentation organizers or participants. But, I'm not sure. I have to confess that I've never thought about the OP's questions before myself!

What do you all think?

Addressing peer-review bottlenecks in job apps?

In our newest "how can we help you?" thread, a reader asks:

I am a postdoc on the job market and I only have minor publications on my CV. I currently have three publications under review; one of this paper has been reviewed for a year at a highly-regarded journal in my main AOS and another is stuck in a third R&R round since May. The difficulty for finding reviewers may be explained by the fact that I use an unusual empirically-informed methodology in philosophy.

Would it be a good idea to address the impact of the reviewing bottleneck on my research in my job documents? And if so, how? Instead of directly addressing this situation in my cover letter, a way to hint at this situation could be to add on my CV and/or research statement the dates since my papers have been under review or through R&R. Or, would it be better to have my letter writers addressing this situation in their letters?

These are good questions, but although I could be wrong, I unfortunately suspect that addressing these issues in job-market materials (either in a cover letter or letters of recommendation) is unlikely to help very much. From a hiring perspective, search committees have incentives to hire people who publish enough--and publish quickly enough--to get tenure. They also typically have hundreds of candidates, many of whom have robust publishing records, and who they may favor for that reason alone.

Fortunately, I think, all is not lost. Search committees, at least in my experience, also care a great deal about the nature and quality of a person's work and will sometimes hire someone with few publications because their research is particularly unique and exciting. My department hired someone like that a few years back--someone whose research was so different than everyone else in our pile that we just had to interview her, and she knocked our socks off and we ended up hiring her.

Petitioning an editor to reconsider a verdict?

A reader wrote in recently with the following kind of question, which I paraphrase (with their consent) to preserve their anonymity:

If you submit a paper to a journal, the referees provide mixed verdicts (one positive, one negative) and your paper is rejected, does it ever make sense to petition an editor to reconsider their decision, particularly if you think the negative report is flimsy, unfair, etc.? Have any readers tried this? If so, did your petition work?

We actually discussed this issue a few years ago, and the few comments there were illuminating. But I think it might be helpful to hear from more readers, as well as perhaps from some editors!

 

Why three reviewers?

In our newest "how can we help you?" thread, a reader asks:

I have submitted a paper to a special issue of Synthese, and it is now under review of the third reviewer. Is it normal to get a paper reviewed by more than two reviewers (in my case, three) in a special issue? Does this mean that there are conflicting suggestions from the first two reviewers?

Another reader, "ref", responded:

Yes, there probably was a split verdict, but one was promising enough to warrant a 3rd opinion

This seems probable to me, but I wonder whether journals sometimes commission three reviewers even without a split verdict. Any editors care to weigh in?

What article statuses at journals mean?

In our August "how can we help you?" thread, a non-tt faculty member asks:

Does anyone have a good understanding of what article statuses, e.g. initial check or awaiting associate editor recommendation mean? I think some are fairly self-evident in retrospect, but when I first started submitting to journals I was quite confused and anxious.

I think, of course, the best thing to do is to not check article status every hour, but some general answers may be helpful to people like my former self.

My experience is that different journals often have somewhat different article status processes, but here are a few that I can recall offhand along with what I think they mean:

• Initial checking --> article is with the managing editor, who makes sure the article is fully anonymized, within the journal's word limit, etc.
• Reviewers contacted --> the editors are trying to find reviewers
• Under review --> some journals seem to use this for every stage of peer review.
• Awaiting reviewer scores --> self-explanatory
• Reviews completed --> self-explanatory
• With editor --> this may occur early in the review process, indicating the article is under desk review by relevant editor(s), who are deciding whether to send the paper to reviewers; but it may also occur after reviewer scores are received, indicating the editor is in the process of reading their reports and making a decision.
• Awaiting associate editor recommendation --> referee reports are completed, and the journal has some associate editor(s) decide on the basis of the reports (and their own reading of the paper?) whether to recommend acceptance, rejection, etc., to the Editor in Chief/Editorial Board.

It's also worth noting that sometimes papers can go from "reviews completed" to "with editor" (or some such) back to "Under review." This may happen, I think, if the reviewers were split and the editor is seeking out a third reviewer to break the tie. But really, I'm only speculating there, and like the OP I generally prefer not to check my article statuses very often (I don't find it very productive, though if a paper has been under review for several months, I may check in to see where things are at).

Do any readers (particularly journal editors) have any inside insights to share on what article statuses mean (and what's going on behind the scenes)?

Tips for handling multiply-(desk)-rejected papers?

In our newest "how can we help you?" thread, a reader asks:

I have written two papers that get regularly desk rejected. I've just created an excel spreadsheet to track that. These are papers that are the result of the realisation of a research project I've got some external funding. Several people who are more experienced and more accomplished than I am and who read the papers tell me that they are good: top-15 on Leiter's list (I've managed to publish something of that stature in the past). These two papers were collectively rejected 12 times in the past 6 months and were rarely sent to external reviewers... but when they were sent for external review, the reports were ok (some mild acceptance, some R&R recommendations).

It seems that when one actually reads the papers, then the impressions are mostly positive; the problem is that the editors keep throwing my works in the bin without sending them for an external review... I'm very frustrated because I'm running out of journals that my papers are suitable for... And I start to wonder if my institutional affiliation and nationality are not playing at least some role in this chain of rejections (I come from a country that is not philosophically cool, like UK or Germany, and my university is internationally relatively unknown)...

Do you have any ideas what the problem is? And what might be a solution?

These are great questions, and I'm curious to hear what readers think. Has this sort of thing happened to you? If so, do you have any tips for the OP?

Checking in on an R&R?

In our newest "how can we help you?" thread, ABD asks:

I understand that you are supposed to wait three months after submitting an article before reaching out to the journal to inquire about the submission, but I'm wondering if I should also wait three months before inquiring about a submitted revision for an R&R?

My sense is that it's probably best to think of R&R's like any other submission. Also, for what it's worth, I've never had much luck with checking in on papers. Every time in the past that I've done so, I got a rejection soon after, so these days I just prefer to wait. I also try to send papers to journals that have a reputation for having good turnaround times, so that I don't run into this kind of quandary.

But these are just my thoughts. What are yours?

Is it ethical to post preprints?

Liam Kofi Bright, Remco Heesen, and I have argued for transitioning toward a crowdsourced approach to peer review. Yet, one common concern raised to our alternative approach is that it would effectively undermine anonymized review--the main mechanism in our current system for mitigating bias in favor and against authors on the basis of knowing their identity. I honestly appreciate this concern, though I still think the overall benefits of moving toward a hybrid peer-review model like that used in math and physics (which combines a crowdsourced approach with traditional journal review) are likely to outweigh the costs. 

But regardless, I recognize that not everyone agrees that peer-review should change, and this has gotten me thinking a bit about the ethics of particular actions surrounding these issues. For example, one thing that advocates for change might do is to post preprints on PhilArchive or PhilSci Archive, normalizing the practice. But, is it ethical to do this?  Last week, I asked whether journals or referees care whether authors post preprints--and I'm glad I did. For, as Jake Nebel chimed in, at least one philosophy journal, Ethics, does care about this. Their official editorial policy for Submission Eligibility reads:

We ask that authors not post or publish online manuscripts that they have submitted or will be submitting to Ethics, as doing so may compromise the anonymous review process or may disqualify the submission on grounds of prior publication.

So, I'm curious about a couple of things. First, irrespective of whether other journals have a policy like this, is it ethical for authors to post preprints--either on an online archive or on their person website--prior to journal submission (given the manner in which it might compromise anonymized peer review)? If it's not ethical, what alternative steps for promoting reform to peer review (for those of us who advocate for reform) are ethical? I'm really curious to hear what everyone thinks!

Tips for reviewing book proposals?

In our newest "how can we help you?" thread, ECR asks:

I have recently received my first invitation to review a book proposal, and was wondering what are the main things to consider as a referee. Should I go about in a similar way I would review a journal article, or should I be stricter with regard to originality, potential impact, or other aspects?

This is an excellent question, and I'm really curious to hear what readers think. My (admittedly anecdotal) sense is that the reviewing standards for book proposals are quite different than for journal articles. Because it's normally taken for granted that an author of a book manuscript will have ample time to revise the manuscript prior after receiving a contract (prior to a final "clearance review" of the final manuscript prior to publication), the job of a book proposal reviewer isn't to decide whether the book is publishable as-is. It is to give the editors a recommendation as to whether the book seems promising enough to either (A) place a full draft of the manuscript under review, or (B) offer a contract...

Writing papers that overlap significantly

In our July "how can we help you?" thread, T asks:

I have two papers that deal with the same issue but from slightly different perspectives. Paper 1 has been accepted for publication. Paper 2 is a work in progress. Paper 2 utilizes and relies upon a criticism of a view that I develop more fully in paper 1. However, how do I incorporate this criticism in paper 2 without plagiarizing myself? I figure I should first cite myself, of course. But then can I basically copy and paste? Or should I rephrase the criticism, or expand it? How to go about this? Thanks!

These are good questions. I've faced this issue many times, and I normally try to rewrite things significantly for a few different reasons.

Do journals/referees care about preprints?

There's been quite a bit of talk in the discipline in recent years about how overwhelmed journals are with submissions these days, and what to do about it. Liam Kofi Bright, Remco Heesen, and I argue in our paper, "Jury Theorems for Peer-Review", for transitioning to a crowdsourced approach where (much as in math and physics) authors would standardly upload preprints to an online repository (e.g. PhilArchive) and have papers openly reviewed there. Although Bright and Heesen argue elsewhere for abolishing peer-review at journals altogether, I demur. In math and physics, authors standardly upload preprints to the ArXiv prior to sending papers to journals. ArXiv papers are often discussed publicly on blogs and the like, and often a fairly clear consensus emerges on the merits of the paper. Papers can also be revised in light of feedback, and new versions uploaded. But none of this takes place instead of journal review. Authors still send their papers to journals, and there's still a formal peer-review process in addition to the more informal public peer-review for preprints. 

I like this hybrid model for several reasons. First, it seems to have worked quite well in math and physics. Second, it seems like a kind of "best of both worlds" approach. In my experience, some people are (understandably) skeptical about moving to an open, crowdsourced-only approach. Keeping formal peer-review at journals may assuage those who are skeptical of that alternative, ensuring that journals still serve a purpose: namely, ensuring that there is a stage of peer-review where hand-selected experts and editors with demonstrable expertise put a formal "stamp of approval" on a paper. On the flip side, a crowdsourced approach may serve as a helpful kind of "calibration check" on how journals are doing in this regard, and vice versa. If open, online reviews of particular papers diverged substantially from journal decisions, then it could be illuminating to examine why. For example, as I've noted before, a pretty wide variety of influential and Nobel Prize-winning economics papers were rejected from very good journals--and as Bright, Heesen, and I note in our paper, there's a pretty substantial literature that indicates a conservative bias in peer-review, which we contend may be partially explainable in terms of journal incentives (e.g. greater incentives to reject than accept), etc. So, having both types of peer-review work side-by-side could be very illuminating.

Do editors pay attention to past rejections?

In our newest "how can we help you?" thread, a reader writes:

Maybe this calls for speculation, but I wonder if other readers have their own sense of whether journal reviews are truly blind. In particular, I am curious if others have the sense that, after they have been rejected at a journal a particular number of times, over a particular span of years, their new submissions receive less consideration than others.

I certainly hope this isn't the case! And, being a quasi-Bayesian, I'd understand if it were. I just find myself thinking, with respect to certain journals, that my rejection pile is so substantive that (maybe?) they are inclined to reject an article of mine based on past evidence.

This is an interesting query, and I'm curious to hear from authors and editors! Here are couple of quick thoughts...

Dealing with harsh journal rejections?

In our newest "how can we help you?" thread, a reader asks:

How do you deal with very critical rejections at journals, the sort that claims that there is almost nothing good in your paper?

I have submitted a draft to 3 places so far, the first two are top ones, and I got mixed reviews recommending acceptance and rejection (which also contains encouraging comments). Then I submitted to a third one, not so top, as the possibly last try. To my surprise, the two referee reports are very negative and criticize everything I said that they selected as "superficial" "mistaken" "not minimally novel" etc. While I try to take in everything the referees say, it is painful to do that. Do you have such experience and how do you deal with it? Also, do you know why less top journals can give much more harsh reports? That seems counterintuitive. (In the early rejections, even for the negative part the referees only say something like the arguments are not as decisive as the author seem to think, not utterly dismissive.) Thanks a lot.

This is a great query, and I'm very curious to hear from readers. Here are my thoughts and experiences...

Why do stages of peer review take so long?

In our July "how can we help you?" thread, S asks:

I submitted an article to a journal 1.5 months ago, and it has not been sent out to reviewers yet. Is this standard?

I'm an early-career person, so I really need publications at the moment and am becoming nervous.

Another reader, 'R&R', reported being in a similar position:

I came to post about the same thing as S. I submitted a *revision* to a journal two months ago, and it hasn't been sent back out for review. It's just gone from "Editor Assigned" to "Editor Invited" and then back again. Is there any reason at all that it should take that long to send a revision back to its original reviewers? And at what point is it appropriate to say something to the editors about it?

Unfortunately, my sense is that this kind of stuff is pretty standard, and I've experienced it myself. I'll also second S's point that this is a real issue for early-career people who need publications to get a job and/or tenure and promotion. I was often totally stressed earlier in my career about how long peer-review took, and it influenced my publication strategy (such as which journals to send things to) a great deal.

Norms for writing, research, and citations

In our newest "how can we help you?" thread, a reader writes:

It is a probably basic question about academic norms. When you write a paper, concerning any train of thought (claims, arguments, etc.), should you simply think about it on your own and write it out as your own plus a reference to similar work if you do find such work, *or* search the literature and try to find if someone else makes similar claims, arguments, etc., and then simply explain this person's idea in your paper, not claiming it to be your own even if you come up with it yourself? I intend this to be a very general question---the choice in question can occur any number of times when writing a paper. Thanks in advance.

This is a great query, and it echoes a set of questions raised by NK in this thread a few months ago:

I wonder if it might be worth starting a more general discussion here about the *point* of citation. I, for one, would be interested to hear others' views on when, and why, one *ought* to cite a given work.

One suggestion made above is that you ought to (try to) cite everything relevant. Another (to which I take Helen to have objected, though the following is an attempt to describe it a bit more charitably) would be that you ought to cite relevant work that you think is good/worth reading and engaging with. These obvious differ in at least one important way: on the latter proposal, citations would be prestige-markers; on the former, they wouldn't.

Are there other options? And what is there to be said for and against each option?

I'm also curious what others think of the following sociological hypothesis: when you cite "low-prestige" work (whatever exactly that means), you thereby lower the perceived prestige of your own (i.e., the citing) work, in the sense that others are going to be less likely to read your work and/or take it seriously (if/when they take note of the character of your citations).

N.K.'s last point (that citing "low-prestige work" can lower the perceived prestige of your own) is one of the main reasons that I think authors are under an obligation to do some basic research--via doing some simple searches for key words in PhilPapers and Google Scholar--before writing, and to cite and/or discuss explicitly up front any relevant precursors (including similar arguments) to the one they intend to write about.

Bias against innovation in peer review?

In our newest "how can we help you?" thread, a reader asks:

I don't know if this is the place to ask, but I wonder if there is some bias against innovative research in our profession, including publication. Do you have the impression that oftentimes the more interesting research has a harder time of being published than a very polished but a little boring paper? I have gone through a few rounds of grad student admissions, and I found that many colleagues are more sympathetic towards all-round okay candidates than those who appear to be very innovative but not so okay in some respects. I judge the latter to have a higher intellectual ability but the former seems to be standardly preferred. Sorry about the ranting---I am not sure if it makes sense to anybody besides me.

This is an interesting question, and I'm curious what readers think. Thankfully, there's actually quite a bit of research on this question on peer review in general, which Liam Kofi Bright, Remco Heesen, and I briefly touch upon on pp. 21-22 in our forthcoming BSPS paper, "Jury Theorems for Peer Review." In brief, to the extent that this has been studied, peer review does appear to have a conservative/status quo bias (see also here). But, in a way, this seems entirely natural to me. As the saying goes, "Extraordinary claims require extraordinary evidence." Insofar as innovative work takes chances and pushes the envelope (so to speak), it's quite natural for readers to be skeptical about it (this also just seems plausibly on Bayesian grounds). Fortunately, if the history of philosophy and science have shown anything, it's that innovative work can pass peer review regardless--and, of course, most of us serve as reviewers. So, if the OP is frustrated by what they take to be a bias against innovation, there are at least two things they can try to do to change things: try to publish innovative work and be open to it as a reviewer!

But these are just my thoughts. What are yours?

Dealing with unresponsive journals (again)?

In our newest "how can we help you?", a reader asks:

Here's a(nother) question about dealing with unresponsive journals:

I submitted a paper last summer (2021). It is with a good journal in my subfield and is one of my better papers. It initially received a 'revise and resubmit, no commitment to publishing' in November '21, and I resubmitted in December '21. The paper has been untouched since then (i.e., for about seven months). The editors have not responded to two emails asking whether the paper is still actively under review.

When do I pull out? At the nine-month mark? The one-year mark? I would of course like to publish with them, but continuing to wait could lead to problems for my career. Thanks in advance for any advice.

Good questions. We've discussed how to handle slow and unresponsive journals before, but the question seems to come up fairly often, so I think it's important to discuss. Personally, I think I'd send the editors one more email and pull the paper within a week or two if they don't respond. There are always other good journals out there, and if you got a revise-and-resubmit at this journal, chances are you'll get one (or perhaps even an acceptance) at another good journal. But these are just my thoughts. What are yours?

Thanking previous referees in submissions?

In our newest "how can we help you?" thread, a reader asks:

I have a paper that was rejected from a journal, but with an unbelievably helpful referee report that is instrumental in rewriting the paper. How do I acknowledge this anonymous reviewer when I resubmit the paper to a new journal? Is it odd to acknowledge a reviewer from a previous journal's review process in a new submission?

Good question. Another reader submitted the following reply:

[U]sually submissions should not include any acknowledgements, so then there's no problem. Once the paper is accepted and you add acknowledgements you can acknowledge a reviewer from a previous journal.

I respectfully disagree. When I submit papers to journals, I often include acknowledgments to anonymous referees--usually in footnotes at places in the paper that have been substantially revised due to referees' suggestions. I don't mention which journal(s) the referees were from, as I think that could introduce biases into the review process. But I tend to thank referees because I think they deserve the acknowledgment, and because I worry that if I incorporated a ton of their feedback into the paper without acknowledging them, it might irritate them (or worse) if they are asked to review the paper again for the new journal. I've also never been asked to remove these kinds of acknowledgments in an initial submission precisely because they are anonymized (though if I ever were asked to remove them, of course I would). My understanding at any rate, has always been that the reason that journals ask you to "remove acknowledgments" is to preserve anonymized review and prevent bias (which might occur, obviously, if you write things like, "I thank Derek Parfit and audiences at NYU and Princeton for their feedback").

Anyway, this is just what I do and my experience with my approach. What do you all think?

Deciding whether to review a paper?

In our newest "how can we help you?" thread, an early grad student writes:

I'm wondering how to decide whether to referee a paper. I just got my first request, and from what I can gather from the abstract, this doesn't seem like an editorial call I'm particularly qualified to make. Would appreciate any thoughts.

This is a great query. Another reader (an editor) submitted the following reply:

Do not agree to referee a paper you believe you are not qualified to referee. Referees are to give their expert opinion - if you are not an expert, then your opinion is irrelevant.

I think this seems right as far as it goes (if a bit blunt!), but I'm curious whether we might be able to give the OP more helpful positive advice. How do you decide whether to review a paper? 

Advice for a 'journal horror story'?

In our March, "how can we help you?" thread, a reader writes:

I have an ongoing journal horror story and I could use some advice.

I submitted a paper to a journal in March 2021, almost 14 months ago. Around 5 months in I started contacting the journal about the paper's status once a month. The managing editor's replies were very uninformative each time, but after a while they said that the journal was in the process of looking for a new publisher and that everything was slowed down as a result.

Once a year had passed I decided to contact the chief editor of the journal. The journal has a triple blind review review process, so I sent an anonymized email. A week later I got an R&R verdict on the paper, with quite positive reviews. I also got an apologetic letter from the managing editor, which revealed some details about the process. Apparently, everything in the journal has to go through the managing editor. At one point, the subject editor handling the manuscript had a query for the chief editor regarding the paper. The query had to go through the managing editor, who decided not to pass it along for now. It sat there for three months, because the managing editor thought that the chief editor should only be bothered with stuff regarding the new publisher deal. It was passed along to the chief editor only once I reached out.

I submitted a revision a month ago, and the paper is still stuck at the "undergoing initial checking" stage, which usually takes a day or two. I contacted the managing editor again, who said, as they did many times before, that they're busy with this new publishing deal and my paper will move to the next stage very soon.

I'm afraid it will be months before the paper is even passed along to a subject editor and then to reviewers, and it might get new reviewers and who knows what will happen then. I'm not sure what to do. Should I contact the chief editor again? I don't want to bother them again but I don't know what else to do.

I have to confess that I'm really not sure what to advise here. Given that the revision has only been there a month, and R&R reviews can routinely take several months, I'm not sure that I would check in with the chief editor yet. Then again, given what happened during the first round review process, maybe checking in makes sense. I guess if it were me, I might wait another month or so, but again, I'm not sure. What do you all think?

Journal formatting requirement info?

A reader writes in:

Are you aware of any resource that has grouped journals according to their basic formatting requirements (these journals are straight Chicago; these journals are straight APA; these journals are Chicago with modifications X, Y, & Z)?  I’m just thinking that might be a massive time-saver for picking the next journal after a rejection.

This is a good question. My sense is that traditionally, most journals have permitted initial submissions to be in any professional format, only requiring authors to put their manuscripts in the journal's style upon acceptance. However, my recent experience is that an increasing number of journals seem to be requiring a very specific format (e.g. Chicago) to even consider a submission (presumably to decrease submissions?). Obviously, this can be a real inconvenience to authors, so it would be great to hear in the comments section whether there are any resources available online where this kind of information is compiled. Alternatively (or in addition), it might be good to compile in the comments section which journals require a specific style at first submission. 

Increasing journal prestige by increasing rejections?

In our February "how can we help you?" thread, a reader writes:

I'm curious about others thoughts on this, especially in light of the ongoing conversations about a scarcity of reviewers.

I recently learned that a journal (that will remain nameless) has adopted an interesting policy in an attempt to become more prestigious: reject a significantly larger percentage of submissions. This strategy is relevant to reviewer scarcity because many (if not most) of these rejections are desk rejections.

Does this seem like an artificial way for a journal to become more prestigious? I'm not sure, maybe it's just brilliant! After all, people value things that are harder to obtain/achieve, and one way to make a journal article seem more valuable (on a CV, for instance), is to make it scarce Maybe prestige itself is somewhat artificial, and this is just an excellent illustration of that.

One reader submitted the following reply:

I worry about this type of tale. How on earth would you come to learn this? Or is it some sort of urban myth floating around. I assume any journal that aimed to do this would want to hide the fact.

I'm not sure about this. Why would a journal want to hide that they are becoming more selective? William Peden submitted the following more nuanced response:

If the desk rejections are arbitrary, it's an artificial but also quite risky strategy. To develop prestige, a journal needs a reputation as both as place where bad submissions tend to be rejected AND where good submissions tend to be accepted. Risking the latter has consequences, e.g. if authors send work they think is great to a journal and it gets desk rejected, then the authors might reason that the journal isn't a good fit for their future research, even if the author thinks that research is excellent. On the other hand, if the desk rejections are not arbitrary, then this can be a good way to justifiably obtain prestige. Again, there is a risk: firstly, if the journal has a problem of too many submissions, this policy can make it worse! That's because authors would rather have a quick desk rejection that waiting months and months for editors to find reviewers. Secondly, if the journal doesn't have an oversubmission problem but lacks prestige, a high rate of desk rejections can make the attractiveness of submitting there even lower. How can all of these incentives interact? I am collaborating with some economists to model exactly that sort of question. One thing we have proven beyond all reasonable doubt: smart editors should always desk accept our papers.

This seems right. Increasing selectivity (and openly signaling this) could increase a journal's perceived prestige, but it's a risky strategy--and so what I expect more often happens is that good journals slowly become more selective as their prominence and submission numbers increase, so as to mitigate these risks. But I'm not a journal editor, so I'm just speculating here. What do you all think? It might be good to hear from journal editors in particular!

Adding (side) material to satisfy referees?

In our February "how can we help you?" thread, a reader inquires:

I'm looking for input & shared experiences on the following R&R situation.

Got an R&R a couple months back which I've revised and am ready to resubmit. It's for a journal with no word limit. Still, the revised version is now 3,000 words longer than the original. But, 2,000 of those words belong to a section developed exclusively to address the complaint of the more negative review which was a BIG complaint. Here's an analogy that explains what I mean by BIG.

Imagine you write a paper on a topic in moral psychology, say blame. Then the negative reviewer says, "You can't talk about blame without talking about how it relates to X issue in moral metaphysics (e.g. realism about moral properties)." That's what I am dealing with here.

As the writer, a natural thought is: "This isn't a paper about moral metaphysics; it's a paper about moral psychology. Would that I could just do the response work in a footnote that notes connections but is direct about this paper's frame. But obviously this person thinks you can't do one without the other, so I better please them, and not by just writing a footnote."

I suppose one response to my situation is that my response doesn't have to be 2,000 words. Sure. And I'm trying to cut it. But given the nature of their issue, it seems they were asking that quite a lot get done. So yeah, looking for advice.

I'm curious what readers think about this. I've heard that some people may try to push back against referees on stuff like this, writing in their response to referees why they think the requested revisions aren't warranted, or at least, why they think they can just address the issue quickly in a footnote. This isn't my approach, however. My experience as an author has been that if one referee cares about X, chances are pretty high that other referees will care about X too--and in turn, readers of the article when it comes out in the journal probably will too. So, for better or worse, my general policy is to do what referees ask, and then engage in clever editing so that I can keep the paper under the journal's word limit. The only downside, I think, is that this can result in bloated 'Frankenstein'-like papers (see picture below). But whatever: such is life in academic publishing. Or so say I. What do you all think? What would you advise the OP?

Publishing short papers?

In our newest "how can we help you?" thread, a reader asks:

I have a question on short papers. I recently had a paper rejected from Analysis. I still think the paper has merit, so I would like to submit it to another journal. But I am worried that another journal will see that the paper is ~4000 words, i.e. much shorter than the usual full-length journal paper, and desk reject it. So the question is: is it a problem if a paper is significantly *under* a journal's word limit? Or are editors quite happy to consider shorter papers?

Good questions. Another reader submitted the following reply:

I think short papers are not a problem. Do not just lengthen the paper to make it the word limit. That would be silly. I have published short pieces. But you should probably have some one look at it before you send it in. Ask a friend who will be honest with you. You do not want to contribute to the referee crisis.

I agree. I think it all depends on how interesting the short paper is, and whether there's any reason that it needs to be longer. You can always send the paper out as-is to a journal or two and see what happens. Maybe it'll get accepted, or maybe you'll get feedback from referees saying it needs to be further developed and lengthened. Alternatively or in addition, I'd suggest sending the paper to a few outside readers to see what they think. Are there things they think the paper needs to do that the 4K version doesn't? If so, consider lengthening it! But these are just my thoughts. What are yours?

Trouble contacting journals?

In our February "how can we help you?" thread, a reader asks:

I have a paper that's been under review at a journal with typically decent turnaround times for over 6 months. I sent an email via the journal's online system after 5 months asking if there were any updates. I received no reply, but I was now able to see who the managing editor is for the paper. I sent another email via the journal's system at 6 months again asking for an update and also received no reply. My question is whether it is appropriate to reach out to the managing editor via their email address at their college/university.

One reader submitted the following reply:

[A]bsolutely do not use the managing editor's university e-mail. Use the journal system - that is what it is for.

Okay, but as the OP notes, they've sent multiple emails to the journal via the online system and have received no reply. So, what are they to do? Any thoughts?

Data from editors on 'the referee crisis'?

A reader writes in:

I have closely followed the discussion of Helen’s recent essay on the peer review system.  It was disheartening to hear about Helen’s experience as an associate editor.  If anything, it makes me appreciate the work editors go through even more.  Thank you for that.

On the other hand, Ben Bradley’s post seems to paint a very different picture... I think the community would benefit from more data.  So, I am floating the following idea: a new post soliciting relevant information (such as # of accepted requests, # of declined requests) from journal editors.

I think this is a great idea. If any readers are journal editors and have this kind of information available, please feel free chime in down below in the comments section: how many referee requests do you need to send out, on average, per referee who accepts an invitation?

Using PhilPeople features to find referees?

In response to Helen's recent post on the troubles that journal editors have finding referees and comments by a number of readers there that they are rarely asked to review, Mike Titelbaum (University of Wisconsin-Madison) helpfully writes in that David Bourget and David Chalmers suggest the following feature at PhilPeople:

"Our 'find a philosopher' feature of philpeople is a good start. You have to pick the relevant topic quite narrowly and order by number of publications in the topic, past five years. That gives you as top hits people that are active in the area and have solid publication records. If you don't do 'past five years' you get more senior people, who aren't going to be new to you or likely to be available. I bet many editors would benefit from these simple instructions."

I hope this helps in some small way with the ongoing 'publication emergency', as it might help authors get more timely verdicts while better distributing the burdens on editors and referees.

Seems like the peer review system has given up the ghost

I am an associate editor for Ergo, and I was idly wondering last night what would happen once I reach Reviewer Z.

The situation is this: A minimum of two referee reports are needed for each paper that is sent out for review. Each referee is assigned a letter, and I've already burned through Referee invites A through F in a couple of days.

Each referee says the same thing "too busy". One wrote to me the following "Sorry, but this is the 13th refereeing request that I've received so far in 2022 [this was 15 February], and I've accepted most of the others. I try to limit myself to accepting about one per week on average."

Similarly, for the Journal of Analytic Theology, I along with the other executive editor and managing editor made a happy dance when finally, reviewer 8 for a very specialized paper said yes to a referee request. 

Just a couple of days ago, one of the journals I am on the editorial board for wrote to us

"Since the COVID-19 pandemic began it has been increasingly difficult for us to find referees to review papers that are submitted to our journal. At the same time, we receive more papers every year. It is common these days that we have to approach six or seven people before we are able to find someone to review a paper. In light of this situation, I would like to ask you for help and request you to review papers more frequently than before."

It has come to the point that the most suitable referee(s) for a paper are almost never available. It takes us so long to find reviewers, sometimes a month, six weeks or more (this situation is also exacerbated by the fact that many people don't respond to referee requests at all). This lengthens the span of time even more for the total review process. All sorts of small fixes don't work anymore, e.g., shortening the time that people get to review, asking for alternatives (this is still very much appreciated, but unfortunately alternative referees are just as unavailable). 

Of course, it could be that my editorial experience is atypical--I'm just one scholar.

But it is my strong suspicion that the peer review system is finally broken beyond reasonable repair. We've seen a slow worsening of the situation and the pandemic has finally broken the system. People are burnt out and overburdened, job candidates increasingly desperate. 

In light of this, we need to do something. We owe it to authors to change the system, I think, rather than tinkering at the edges. Any suggestions (ranging from proposals to overhaul the entire system and how to do this practically, such as by our own Marcus Arvan in this paper co-authored with Liam Bright and Remco Heesen) welcome. *

Refereeing papers that are critical of you?

In our newest "how can we help you?" thread, a reader writes:

I was recently asked to review a paper that engages with some of my work. Since the journal asking for the review made the manuscript available to me, I was able to look it over before agreeing. I quickly discovered that the paper is very critical of one of my papers, and that it also includes a number of serious mischaracterizations and misunderstandings of my arguments (which I found very irritating). On the one hand, I feel the urge to accept the review request so that I can correct these mischaracterizations. On the other, I know that I am not a neutral party in this matter, and so I am also somewhat inclined to decline the review on account of my obvious bias. What do other people do when they encounter this kind of situation? Is it better to decline on account of my bias, or to accept on the grounds that I am an authority when it comes to what I say in my own papers?

These are very good questions. From what I've heard elsewhere on social media, my impression is that editors ask targets of critiques to referee papers fairly often. Further, in one way, this sort of seems like a good practice--for the journal has an interest in publishing good papers, and if anyone is well-placed to prevent mischaracterizations, then it is presumably the author being critiqued. On the other hand, it does (as the OP notes) introduce a kind of conflict of interest into the review process. After all, if anyone has an interest in not being embarrassed or their arguments disproven, it is the author whose work is being critiqued. So, I wonder whether editors have a way of spotting this. When editors receive a review from an author whose work is being critiqued, do they subject it to especially careful scrutiny, reading the paper for themselves to determine whether the referee's review is fair? 

I'm really curious to hear what readers think of this practice, as well as from editors--and on whether the OP should accept the referee assignment. What do you all think? 

How to get asked to referee?

In our newest "how can we help you?" thread, S writes:

How can I get papers to referee? Where to start? I am a junior researcher with a few publications, but not from EU or USA, and without much of a network. Any advices?

Several readers submitted replies. Michelle wrote:

[I]f the journals you are interested in refereeing for use Manuscript central or similar then create a profile and add you keywords. When the editor looks for reviewers you'll show up in the search. You can also email the journal and ask to be added to the reviewer list I imagine.

And Ref wrote:

You are less likely to be asked to referee for journal that you have not published in. So it is crucial that you publish in good places, if you want to referee paper for journals. I have refereed over 170 papers, many for leading journals in my sub-field - I am not junior. But that is largely because I have published in these journals (I also have three books with a leading press). Further, once you start refereeing, it helps if you are effective at getting useful reports in to the editor in a timely fashion. No kidding, most of the time I get my report in within 5 days of agreeing to referee the paper. Further, I write my reports for the editors - I given them a professional assessment of the paper, and provide justifications for my evaluation.

These both seem like good suggestions, though for what it is worth, I was asked to referee for a number of good journals early in my career before publishing all that much. What do you all think?

Contacting EICs in peer-review delays?

In our most recent "how can we help you?" thread, a reader asks:

A question about journal etiquette. I have a paper that has been under review at a journal for nine months. For the past two months, I have not been able to get any response from the managing editor about the status of my paper. My question is: is it ok to write to one of the chief editors of the journal about this? Also, the journal is triple-blind, but as far as I understand that applies to the area editor who is handling the submission.

Good question! Another reader submitted the following reply:

I got a paper off the backlog at [redacted], where it had been stuck for nine months with no response from the ME, by emailing someone from the editorial board. I tried the chief editor first, but they had not responsive either. But the editorial board member was very apologetic and got me a decision in a week. Extreme times call for extreme measures, I think. My only caveat: be extra polite if the CE works in your area or it's a journal you need to publish in at some point (I purposely chose a board member as far removed from my area as possible).

I'm curious what other readers think. Do any of you have any helpful tips or experiences to share?

How to know if you're a good referee?

In our most recent "how can we help you?" thread, a reader writes:

I referee more and more for journals, and there is this question at the back of my head everytime I send a report: How do I know if I do a good job as a referee? Or: What are good signs that you do a good/bad job as a referee?

A bit of context: When I send my reviews to editors, I usually also send them a message saying "if you need more information, or if my report is unclear, please let me know". I never hear back from the editors. When authors get an R&R, they are not dismissive of my comments, but this is hardly good evidence (as an author, being nice is part of the game!). We've all had experiences with bad referees. How am I supposed to know I'm not one of them? Maybe getting new invitations to referee from the same editors is a good sign. But perhaps they don't even keep track of this.

Good questions! In my experience, one of the best indications of one's performance as referee are how one's reports compare to other referees' reports. First, some journals have the (very good) practice of sharing all referee reports when an editorial decision is rendered not only with the author, but with other reviewers. As a reviewer, I've often found this to be very eye opening. On some occasions, I've seen that my worries were shared by other referees--which gave me some independent evidence that I had raised the right concerns about the paper. On other occasions, my reports seemed better than the other reports, giving more detailed feedback. However, on still other (thankfully rare) occasions, other reports led me to wonder whether I did a good job with my report (as they either understood something in the paper better than I did, etc.). For these reasons, I think the practice of journals sharing referee reports with all of a paper's referees is a great practice, and I only wish more journals did it (as in my experience, very few do). Finally, of course, sometimes journals share other referees' comments (or the author's reply to them) when authors submit revised papers for revise-and-resubmit verdicts. I find this to be really helpful for the same reasons. Other than this, though, my sense is that all one really has to go by is one's own internal sense of whether one is doing a good job.

What do you all think? What indications do you think a referee can have that they are doing a good job? And, are there things that journals should do in order to give one a clearer picture of how good of a job one is doing?

How are R&Rs handled?

(Update: comments now open!)

In our most recent "how can we help you?" thread, a reader writes:

When I re-submit a revised manuscript to a journal after an R-and-R decision, will they go to the editor first, or will they be sent out directly to the reviewers? I ask because I think I should clear something I've done in the revisions with the editor (a slight compromise of anonymity) before the reviewers see it.

Good question. My sense is that all submissions (both initial submissions and R&Rs) go to editors first, and then the editor decides whether to send it out to review, and if so, to whom. Sometimes (e.g. for an R&R with 'minor revisions'), editors may not send out an R&R to reviewers at all, deciding for themselves that you have adequately addressed the reviewers' concerns. Other times, the editor may send out the R&R to some of the reviewers but not others (my sense is that this can happen if one referee initially submitted an incompetent or unfair review: an editor might not be interested in hearing that referee's opinion again). Finally, sometimes editors may send out the R&R to all of the previous reviewers, or even (sigh) a new reviewer, if one of the original reviewers is unavailable. I guess my only concern about the OP is their note that they are interested in "clearing something" with the editor that might involve a "slight compromise of anonymity." If the journal is triple-anonymized (so that editors do not know the author's identity), this seems to me to be problematic. Although I can understand situations where this might be tempting (such as a reviewer misinterpreting some of the author's previously published work cited in the paper!), authors should still respect anonymous review in submissions and resubmissions, I think. But perhaps the OP can chime in and provide further context in the comments section.

Anyway, these are my thoughts. What are yours? 

How often should one referee for journals?

In our most recent "how can we help you?" thread, a reader writes:

I'm at the point in my career when I'm just starting to receive fairly frequent requests to referee journal articles, and I'm finding myself really unsure of how many of these I should be doing. Would cocoon readers be willing to share their own approaches to this? How much refereeing do people do? And do people have good rules of thumb for deciding when to say yes and no to referee requests? (E.g., a certain number of reports per month, or a certain number of reports per reports one has received oneself, etc?) This is one of those mysterious parts of the profession where I just have no idea if I'm pulling my weight, doing way more than I should be doing, or roughly getting it right. Thank you, everyone!

Excellent questions! One reader submitted the following response:

At one point, I was refereeing 20 papers a year, and working as an editor for PLOSOne, handling about 6 papers a year (which meant getting referees and making a final decision on papers). Now I am a co-editor on a journal, and I no longer do PLOSOne. So I referee only about 12 papers a year, and only in a select set of journals (the best in my sub-field or very highly regarded general journals). I also review a few book proposals and manuscripts a years for various presses. Without knowing your circumstances, I think if you are doing 12 a year, then that is fine.

Reviewing a bad paper for a journal?

In our newest "how can we help you?" thread, a reader writes:

For the first time is my (relatively short) career, I have been asked to review a paper that is very bad.

We all know the 'Reviewer 2' meme, and have probably heard of the research about the damaging effects overly that negative reviews can have on people - and I don't want to be that reviewer!

So, I'd love some advice focused specifically on how to write negative journal reviews without being too harsh in the tone. How does one best communicate a negative verdict without going over the line?

This is an excellent question, and I'd love to hear readers' answers. Although sometimes it can be hard to avoid being 'reviewer 2' (particularly if, as in the OP's case, the paper one is reviewing seems really bad), here are a few tips that I have found to be useful:

1. Use neutral, non-inflammatory language: As this comparative study found, something like 30% of reviews of philosophy papers contain inflammatory rhetoric. To avoid doing this as a reviewer, I've found that it can be useful to write one's review and then read through it multiple times to see whether there are any unnecessarily hostile words or phrases that I might replace with more neutral ones. 

2. Focus on detailed, dispassionate commentary rather than 'meta-commentary': Is the paper's argument confused? Instead of just saying it is confused (a meta-commentary), along with perhaps a short toss-off comment on why ("The paper commits all kinds of fallacies"), focus instead on showing the error(s) in question. Say, for example, "On p. 1, the paper claims X. On p. 2, it claims to derive Y from X. Yet Y does not follow. Here's why...". My experience as an author is that when one receives these kinds of explanations, a reviewer report can at least feel fair-minded--whereas if report merely says "the paper is confused", it can come across as unfair and needlessly hostile.

3. Set the paper aside for a week or so: Although as a reviewer one should presumably avoid unnecessary delays in completing one's review, it's amazing in my experience what a bit of 'distance' from a review can do. When you read your own review 'afresh', as it were, things can just pop out at you (like, "Wow...yeah, I should really tone that down").

4. Try to say a few laudatory things: Although it may seem hard to do in cases of especially bad papers, my experience is that papers that pass the desk rejection stage usually have at least some virtues. So, if there are any, mention them, particularly toward the beginning of the review.

5. Remind yourself of reviewer fallibility: I assume you have received referee reports that you thought were unfair, right? Well, those reviewers were presumably someone like you: someone who thought they were reading a 'really bad' paper. For this reason, I think it's helpful to consciously remind oneself of just how fallible we can be as referees. Try reminding yourself of this study, for example, which found a high number of classic (and 15 Nobel prize-winning!) economics papers to be rejected from journals, often for the most spurious of reasons (such as being 'trivial'). Or remind yourself of this story by Jason Stanley. Personally, I've found that reminding myself of how fallible we can be can really help one tone down a review. 

6. Read this discussion of bad reviewers by Michael Huemer: Don't reject a paper simply because you disagree with it, or even ultimately find the argument unconvincing. Your task as a reviewer should not be to expect perfection--especially given that, as we all know from graduate school, even the 'most important' papers in the field are usually flawed (or at least dubious) in all kinds of ways. Ask yourself instead, 'Is the paper interesting enough and good enough to publish, or at least give an R&R to?'

7. Imagine receiving the review yourself: I've heard people say they dread hearing from journals, and I empathize. Like most people I've talked to, I've received my share of harsh, meanspirited reviews. So, before submitting your review, ask yourself what it would be like to receive your review as an author. If your review feels needlessly 'hard to read', try to adjust its tone.

Anyway, these are just a few thoughts. Sometimes they can be admittedly hard to follow through on, particularly when one receives a paper to review that just seems incompetent in some way or other. Chances are, many of us will be 'reviewer 2' at some point in our lives. Nobody's perfect. But it is worth trying to do one's best to avoid it. Anyway, what do you all think? Do you have any good tips for avoiding being reviewer 2? If you do, I'm really curious to hear them!

Resubmitting a paper to the same journal?

In our newest "how can we help you?" thread, a reader asks:

Hello! I have a situation that I would like some advice. I submitted a paper to a special issue called for by a journal. The paper was rejected by the editor. However, the two reviews seem to both recommend a "revise and resubmit". I understand that the editor made the decision due to the fact that there is time pressure for the special issue. My question is, once I have the revised version ready, can I send it to the same journal (but obviously not for the special issue)? If so, should I make a note explaining all these, hoping that the editors reach out to the same two reviewers? Or should I just submit it somewhere else? Thank you!

These are good questions, and I have to confess that I've been in a similar situation. Anyone have any insight? Is it okay to resubmit a paper to the same journal as a 'normal submission' after it has been rejected for a special issue? Also, this brings to mind a more general question that I have seen raised from time to time: will journals ever consider a newer version of a paper they previously rejected, if for example enough time has passed (say, a few years) and the paper has been 'completely rewritten', substantially changing and improving its main argument? I've heard this question raised a few times by a few different people, but can't recall what the answer is!

Types of referee reports?

In our newest "how can we help you?" thread, a reader writes:

Maybe we could have a thread on 'types of referee reports'. I am working on a particularly frustrating R&R right now, and could use the entertainment. It maybe could also be instructive (?) on what not to do when writing a referee report?

Here, I'll start:

“You have not cited my book. No one else will tell you to cite my book, but I wrote it, someone published it, it’s on a glancingly related topic, so this lack of engagement with my work is a severe deficiency of your paper. Reject.”

“You have cited my book, but apparently you disagree with me. This means that you have not read my book carefully enough. Read it again. Reject.”

“You have talked about how X is related to Y, but have you considered that perhaps Z is related to Y? I do not know if it is, but neither do you, apparently, since you do not write about it, so … reject.”

“You are here writing about X. But have you considered mildly related issues Y and Z? Apparently you are ignorant of them. I would not consider publishing any paper on X that does not address Y and Z. Revise and resubmit […] This paper is too disjointed. What you write about Y and Z is nuanced and insightful, I am not sure why you discuss Y and Z in a paper on X. Reject.”

“I am just not convinced by this. Reject.”

“This is a great paper, unfortunately it’s on a topic I’m not super interested in. Have you considered rewriting the paper on a different topic, that is more related to my specific areas of interest and expertise? Thanks in advance. Revise and resubmit.”

This could be a rather fun and potentially illuminating exercise to end the week with, as it might help us all to reflect a bit on how we and others around us approach the practice of peer-review. I'll probably share an example or two in the comments section below, but I think I'd just like to open things up for comments first!

Tips for reviewing book manuscripts?

In our most recent "how can we help you?" thread, Kian Mintz-Woo writes:

I have been asked for the first time to evaluate a manuscript. The editor mentioned a 'small fee' or the equivalent in books. I am interested in the topic of the manuscript, know and like the (similarly junior) author, and believe I would be well-placed to comment on it. However, I know it is a massive time commitment and I am sure the 'small fee' is nowhere close to the value of that time.

What is a normal timeline for reviewing a manuscript (this is not very long--the author expects <100k words)? It is worth noting that the editor has mentioned that they can be very flexible abut the reviewing time. Are there other things I should keep in mind? (Should I basically just say no to manuscript reviews until I'm a mid-career academic?) Is there a reasonable range to expect the 'small fee' to be? (I am sure it's not worth haggling or anything like that, but even if I wanted to I don't know enough to do it.)

Is it relevant that I like the press and might pursue them in the future, as I think it is with journals (although, because of subject area, not this particular editor)?

Do I basically review the book like I would articles, just on a larger scale--i.e., track the argument as it unfolds over chapters? Or should I also be thinking about things like audience and sales (!) I would also add some copy-editing in, because I like to do that as well.

I searched through Phil Cocoon for advice on this and--while you have lots of discussions about preparing proposals--you don't (appear!) to have anything about evaluating them. Thanks for any thoughts or advice!

These are good questions that I suspect many early-career people might benefit from hearing good answers to. Here are some quick thoughts...

Should referees ever intervene with editors?

In our newest "how can we help you?" thread, a reader writes:

I refereed a journal submission a few months ago. I thought it was ok, but could use some revisions. I just looked at the submission and saw the other reviewer's comments. Basically, the reviewer is complaining that one paper, which was cited, wasn't engaged with enough, and that another paper that wasn't cited, which is somewhat relevant, should have been cited. That's it. On the basis of that they deem this submission as not sufficiently engaged with the literature. Since this is a niche topic I write about, I can confidently say that this is false. The submission does a decent job of engaging with the literature. My question is this: should I write to the editor to voice my disagreement with the other reviewer?

This is an interesting question. Another reader submitted the following reply:

No ... do not write to the editor to complain about the other referee. The two reports are supposed to be independent assessments. I do not like the practice of sharing referee reports like this anyway. But let the editor do her job now.

I have to confess to being more uncertain. If a referee seems like they are being unreasonable, I can see why another referee might want to intervene. But I also see the point that peer-review reports are supposed to be independent assessments. What do you all think?

Tailoring referee standards to journals?

In our newest "how can we help you?" thread, a reader asks:

I have a question about writing referee reports and matching one's standards to the selectiveness of the journal.

I'm relatively new to writing referee reports and am finding it difficult to know whether and how to match my standards of evaluation to the selectiveness of the journal. To what extent do others try to do this, and could anyone recommend any specific strategies? I feel like I just find myself asking whether the paper adds something important to the scholarly discussion as a whole (and whether it does so in a clear, effective way, etc.), and I don't really have any specific strategies for assessing things differently based on the particular journal's standards.

Great question, and I'm curious what readers think. I've refereed for journals all over the map (top 5 journals, unranked journals, etc.), and I definitely use different standards. I always try to hold authors to standards of soundness and cogency, for obvious reasons. So, for me, the main issue in terms of tailoring my standards to journals is my sense of the piece's overall significance. For example, I tend to be a bit puzzled when I see a paper in a top-5 journal that makes what I take to be a very small move in a well-worn philosophical debate. This isn't to say that small moves aren't worth publishing (surely, they can be). It just strikes me that particularly selective journals should be in the business of publishing particularly noteworthy works that make major contributions to ongoing debates, and that smaller, less significant moves probably belong in other journals. Is it possible for a very small move to make a major contribution? Sure, Gettier's famous paper in Analysis is a perfect example. But papers like this strike me as far and away the exception, not the rule. 

Anyway, that's my main deciding factor for tailoring my refereeing standards to journals. But, having read such journals, my sense is that other referees may tailor their standards in different ways. For example, some particular journals seem to place a priority on formalizations (viz. formal logic and decision theory). Personally, I think formalizations tend to be overused in philosophy--but I recognize this view is idiosyncratic to me. Anyway, what do you all think, particularly (but not only) those of you who have reviewed for different kinds of journals? How do you tailor your standards to different journals? Or, alternatively, how do you think referees should tailor their standards?

When editors ignore your reviews?

In our newest "how can we help you?" thread, Frustrated writes:

I have a question about good practices for reviewers. In the past few months this happened to me 3 times. I was asked to review a paper for a top journal in my AOS. This paper is really really bad. It's bad in the following sense: the topic is important, the thesis is interesting, but it'so poorly written and argued that, if I did something like that myself, I'd get for sure a rejection. However, I recommend major revisions. After a few weeks I got the revisions back, and nothing much has changed. I can see what the other reviewer has suggested, and it's minor revision. I then suggest rejection and the paper is accepted. From this I understand that the paper was either from an established scholar, or from a student of an established scholar. I feel I'm wasting my time by doing my diligent work of reviewer, given that the editor has already decided. Therefore, this is my question: when I have the feeling that something similar is happening again, shall I address the editor directly and ask how I should treat the paper? Or shall I just decline to review the revisions?

I'm glad Frustrated wrote in about this, as I think it's a great issue to discuss. Another reader submitted the following response:

Frustrated: is it possible that your standards are too high? Maybe this series of events has some other kind of takeaway. Three different editors have overridden your negative reports ... perhaps your standards are either too stringent or just out of keeping with what the editors are looking to publish. I've had an editor override a referee on one of my papers. The referee accused me of all kinds of weird things (like I'm *clearly* just a graduate student, and some senior scholar has made some edits to my paper) and seemed really irritated. The editor contacted me with this review and asked if I would make some changes. The paper was then sent out to a third referee, since the editor judged the second referee to have an overly harsh view of the paper. I was grateful to the angry referee despite all the mean-spirited commentary, since I was able to improve the paper. I am not an established scholar, nor is anyone running around wielding influence on my behalf. Did the editor act inappropriately? I don't think so, not at all. These are recommendations you are giving, not dictates.

This is more or less the reaction that I had to Frustrated's comment. If I am reading Frustrated's comment correctly, in just the past few months they have had three (different?) editors (at three different top-ranked journals?) ignore their negative referee reports in favor of more positive reports by other referees. What should Frustrated infer from this? While it's hard to know for sure without looking at the papers themselves, four things seem to me worth noting here:

1. Referees and editors at top-ranked journals tend to have very high standards (viz. high rejection rates).
2. In each of the cases the OP describes, at least one reviewer thought quite highly of the 'really really bad' paper in question, judging the paper to warrant only minor revisions.
3. The associate editors who handle a paper may be specialists in the area.
4. When they receive conflicting reports, these editors may read the paper themselves to decide which report(s) to trust.

Given these four things and given the fact that same thing has happened to Frustrated repeatedly, it seems to me that Frustrated should seriously consider that the possibility that (A) they are writing overly negative referee reports, and (B) that the papers they judge to be obviously really really bad aren't. After all, if the papers really were that bad, one would figure that someone else--either the other referees or the editors who read the reports and then the paper--would see this as well. But that's not what's happening here. In three separate cases, it seems, the editors have read the referee reports and the paper and decided for themselves that the other referees' positive reports are the ones to trust. 

Peer-review: keep, change, or abolish?

Darrell Rowbottom (Lingnan University) has drawn my attention to a new paper of his forthcoming in British Journal for Philosophy of Science in which he defends pre-publication peer-review against a recent argument by Heesen and Bright for abolishing the practice altogether. Readers may recall that over the years, I've tried to stake out a middle ground in this debate: arguing for substantial changes to peer-review, but not its abolition. In brief, my own position has been that philosophy should probably transition toward the publication model practiced in math and physics, wherein it is standard practice to:

1. Upload draft papers to the ArXiv (a preprint server similar to the PhilArchive).
2. Share, discuss, and critique unpublished drafts openly (a kind of crowdsourced peer-review that weeds out bad papers and improves good papers before journal submission).
3. Submit papers for peer-review at journals only after steps 1 and 2.

Although this model to a large extent undermines anonymous review (as people then know who has posted which papers to the ArXiv), my own sense is that the system works well and might help resolve a number of problems with our current peer-review system (including but not limited to: too many half-baked papers being submitted to journals, journal editors and referees being overwhelmed by submissions, etc.). So, then, we really have three options for peer-review in philosophy:

• Option 1 - keep the status quo: stick with the anonymized peer-review system we have.
• Option 2 - abolition: abolish pre-publication peer-review at journals altogether.  
• Option 3 - reform: keep peer-review at journals, but adopt a practice of posting papers on online servers first so that they can be openly vetted (e.g. evaluated, criticized, etc.) before journal submission.

Which option do readers most support, and why - especially, let's say, after reading the papers by Rowbottom, Heesen and Bright? For some additional food for thought, readers may want to check out this story shared over at Daily Nous  by Nathan Salmón (UC Santa Barbara).

What Do You Aim for in Referee Reports? (guest post by Finnur Dellsén)

This is a guest post by Finnur Dellsén, Associate Professor at the University of Iceland

As many people have pointed out recently, the peer review system relies on us accepting several referee requests for each paper we publish. So refereeing is a substantial part of our job as researchers, even if we don't get paid for it specifically. But how do we make sure we write helpful, insightful, and constructive referee reports? How do we improve as referees? We get all sorts of feedback on our own work --- from colleagues, editors and referees --- which helps us improve as researchers. We get much less, if any, feedback on our referee reports.

So when I was named Referee of the Year at the British Journal for the Philosophy of Science for 2020, I thought it would be a good opportunity to get the conversation going on how to write a good referee report (see my Facebook post). Below is a list of things that I aim to do in my reports (but I encourage others to add to the list in comments):

1. I always summarize the paper first in my own words. This lets the authors and editors know I carefully read and understood the paper. It also forces me to focus on the main contribution of the paper as opposed to minor concerns (which is important because of point 3 below).

2. I try to find as many nice things to say about the paper as I possibly can, especially when I recommend rejection. I highlight ideas and arguments that are especially interesting or compelling. This helps the author know what not to change when revising their paper (e.g. for another journal).

3. I try to make the amount of text in the review that I spend on each idea in the paper reflect its importance for the main thread of the paper. Every paper has things in it that I don’t agree with or find slightly unconvincing, but it’s usually not worth getting bogged down with those issues.

4. When suggesting changes, I say explicitly which of those changes I consider to be most important, even dealbreakers, and which ones are mere recommendations for the author to consider. This makes it clear to the author which changes to focus on.

5. I try to indicate clearly, if not to the author then at least to the editors, how confident I am in my recommendations -- not just about whether to publish, but also about individual points or recommendations for changes. In many cases, I am not an expert on all of the things the paper discusses, and when I’m not I try to say so explicitly.

6. Finally, referees can gently point to relevant work by underrepresented groups and/or work that is underrecognized in the field (e.g. because of prestige bias). I admit I haven‘t done this as often as I would have liked in the past, but I think it can do a lot to increase diversity and make philosophy more inclusive.

Those are some points that come to mind. What am I forgetting? What do you aim for in your referee reports?

(Thanks to Helen De Cruz for giving me this opportunity to discuss refereeing practices, and to the BJPS for the award!)

Drop-In Discussion on Refereeing and Publishing Norms & XPhi

Joe Ulatowski (University of Waikato) has drawn my attention to this interesting looking drop-in Zoom session, to be held on September 17th, 2020 at 5:00PM EDT. A quick description:

At a recent XPhi Under Quarantine, participants discussed the possibility of drafting a public statement concerning norms for journals on publishing non-xphi articles that were about xphi, but that didn’t contain any studies.  So, e.g., someone wants to float a hypothesis about what philosophical expertise could consist in but has no data to back up the claim that philosophers actually work that way. Or a paper that sketches an idea for a study, but doesn’t actually do the study it sketches. Or if a non-x-phi paper involves an out-there kind of vignette, can one reasonably ask of an author to go run a study on it? 

Because participants had an interest in talking over these issues, we have decided to host a special edition of X-Phi Under Quarantine. We will host a drop-in discussion session to consider not just the questions outlined above but any such question concerning refereeing norms or publishing norms that affect work in xphi. We ask participants to be prepared to talk about the following questions:

• • What are the norms of refereeing papers in experimental philosophy?
  • Do these norms follow the strict norms of general philosophy journals? If they are different, how so?
  • Is it sufficient for a referee to recommend a paper be rejected because they just don’t believe the data?
  • How much should data & methods be reported on exactly the model of a social science journal, when it is being published in a philosophy journal, especially a general one? (I.e., not Mind & Language, where one can perhaps presume a certain comfort level with such things on the part of the audience)
  • Should general philosophy journals even be publishing methodologically ambitious x-phi? Do they have the refereeing and editorial bench to do it well? one thing we could look to do is make up some sort of database of x-phi-comfortable philosophers and scientists who are willing to serve as referees for such work.
  • Pre-registration? Making data publicly available? Other anti-QRP practices that should be made more widespread?
  • Publishing studies that are adequate empirically but don’t really have anything new to say philosophically? Should null results be published? Should replications be published?

This seems great, and it also drew my attention to the New XPhi Blog, which I wasn't aware of!

How do you decide when to review for journals?

In our new "how can we help you?" thread, a reader writes:

I have been post phd for a few years now. At first, I thought it good to get some referee work under my belt. Now, I am getting a request to review at least once a month if not more. Today, I got two requests 15 minutes apart. My question is, how do people approach the decision whether to accept or reject a request to review? What are the considerations people take into account (besides fit to review)?

Great questions, and I'm very curious to hear what people think! As mentioned in the new thread, to make things more manageable on my end, I've asked readers not to respond in the comments section there--and so there are several comments that I have not approved. However, although I did not post them, I will still try to quote them anyway in new posts when relevant. So, on that note, Bryce responded:

A colleague once gave me this advice about reviewing: review as many papers per year as you submit. So maybe 1-3? The more you review, the more you will be asked for review. Just say no.

This is a rule of thumb that I have commonly heard too, and it seems to me a good one. As professionals, we should all do our part to contribute to what are, effectively, 'public goods' in the discipline. If you submit a lot of papers for others to review, then you should review a lot of papers. If, on the other hand, you submit few papers, then feel free to review more rarely. What does seem clearly wrong to me is to submit lots of papers and rarely review when asked. That's just free-riding. But let me add a few further thoughts.

The importance of narrative in publishing?

In our August "how can we help you?" thread, a grad student writes:

A question on publishing. I get the impression that papers at top journals usually not only aim to provide strong arguments for some claim, but also to provide some narrative about the literature and the place of the paper in it. (For instance, a recent paper defends the claim that view A is supported by view B, and also tells a story according to which opponents of A have long objected to it by implicitly appealing to B, though it is debatable that they have done so.)

I was wondering (a) how important this kind of story telling is in getting a paper published, and (b) if it is important, what are some strategies for developing a skill for it.

My own sense, for what it is worth, is that it is crucial to tell a compelling but accurate story of how the paper fits into the literature and why it makes an important new contribution. In fact, I've heard this is true across fields, not just in philosophy. I've also reviewed a lot of papers, and my sense is that two errors are common: (1) Authors taking for granted that the problem they are working on is important when it's not necessarily clear to anyone but a hard-core 'insider' how this is the case, or alternatively (2) Telling a false, poorly researched, and/or exaggerated story. As a reviewer, I find both a bit frustrating. If, qua 1, I don't see the point of a paper, then it can be hard to recommend acceptance. However, I find instances of (2) even more frustrating. In fact, it seems to me something of a trend in the papers I've reviewed in recent years. For example, one of the more common story-telling maneuvers I've seen seems to be this: "Everyone who defends theory-type T thinks P. This paper is the first to argue that if T, then ~P." The aim here, of course, is to make it look like the paper is doing something spectacularly new and original. The problem, though, in my experience that it's nearly always the case that the author is ignoring wide swaths of literature of T-theorists who clearly aren't committed to P, as well as at least a few T theory papers that explicitly argue ~P. So, for this most part, I've found this a really poor story-telling strategy. A good story in academic research has to be an accurate one: one that doesn't oversell what it's doing by ignoring relevant literature.

This brings me to a couple of strategy suggestions. First, to tell a good story you have to read. Don't just read 3 Famous T-theorists who defend P and then treat them as representative of the entire T-theory literature. Instead, survey the T-literature and find out who is committed to P, who isn't, and who has explicitly denied P--and show in your introduction that your paper contributes something new that they haven't. Second, I suggest getting to the 'hook' first and to not get lost in endless exposition. Try to start the paper with a bang, stating in the very first sentence or two what the paper does. And while accurately situating your paper in the literature is important, do it concisely. Finally, I tend to think it's really important to spell out early and again in the conclusion of your paper the broader implications of your argument for future research. Does your argument merely make a small intervention with no broader implications (in which case, your contribution is a pretty small one), or does it have implications for how T-theorists should theorize or implications outside of T altogether? Provided again you tell an accurate story, that can make your article come across as important (and thus, important to publish!).

But these are just my thoughts. What are yours?

Questions about journal responsiveness

In our August "how can we help you?" thread, one reader writes:

Have you ever withdrawn a submission from a journal due to a delay or a lack of responsiveness? What was the final straw?

And another writes:

Question about contacting journals and ghosting authors. I had a paper accepted for publication back in April In a top journal and was told I would be contacted by the journal press shortly after. A couple weeks went by without contact, so I dropped a kindly-worded follow up email to the managing editor, who said they would look into the matter and that I should expect an email soon (I.e., two weeks). Well, a month and a half goes by and still nada. I drop another kindly-worded email to the journal managing editor and still have yet to receive a response.

I know the process is glacially slow and I am deeply appreciative of all the work journal teams do. As a youngish, untenured prof, however, the silence is a bit unnerving. Some communication to the effect of, “We are backlogged and Covid slowed us down even further; you will be contacted some time in the next decade”, would help.
1. Is journal silence a common occurrence?
2. Suggestions about how to handle journal silence post-acceptance professionally?

While I wouldn't worry about journal silence in reader #2's case (once a paper is accepted, it's accepted - so don't worry!), there are I think some interesting questions to discuss here--among them:

• When does it make sense to withdraw a paper from a journal?
• How should authors and the profession more generally deal with non-responsive journals?

I'm not sure that I have any great tips here. When I was early-career (i.e. on the job market and pretenure), I took a lot of care to submit to journals with good reputations and turnaround times, which you can find out about here and here. Back then, I didn't have time to wait 6-12 months for journal rejections, so I vaguely recall withdrawing a few papers years ago when the review time was inordinately long and the journal was unresponsive. I'd usually send an email or two asking for an update, and if I didn't get a response after some amount of time I'd just send another email withdrawing the paper. But again, generally speaking, I just tried to stay away from those kinds of journals to begin with. What do you all think?

Questions about journal refereeing

In our August "how can we help you?" thread, a reader writes:

I'm wondering how to deal with the following as a journal referee:

1) The manuscript, which discusses a recent paper, just flat-out misses an argument that that paper makes for a particular premise P. It is clear that they missed it because in the manuscript, the author writes, "It isn't clear what the argument for P is." In fact the author of paper being discussed does argue for premise P. I know not only because I am the author, but *more importantly* because I pulled up the paper to make sure I argued for it. I want to flag this to the referee without making it obvious that I am the author being discussed. How can I do this?

2) The manuscript deserves an R&R. But its tone is perhaps overly polemical, even hostile at times. I would say this even if the paper that the manuscript discusses were not myself. For instance, the manuscript contains asides like, "...the paper, characteristically, does not contain real-world examples of [the philosophical phenomenon]". As a referee, how can I flag that these hostile-sounding asides are unnecessary? Again, without revealing that I am the author of the paper being discussed.

Another reader gave what I take to be good answers to both questions:

1) just cite the relevant passage and refer to yourself in the third person. They might guess it’s you, but they won’t have enough evidence for that to be a great guess (even though it’s true), and your responsibility is to avoid revealing your identity, not to avoid your identity being guessed at all costs.

2) the quoted passage makes it clear that the author strongly disprefers your methodology, but it doesn’t come across as hostile to my ear. (As a reader, I find that these sorts of asides can be helpful, as they make it clear what kind of philosopher I’m reading.) Of course, if it *isn’t* characteristic of your writing (or of the literature? out of context I’m a bit unclear on the charge) that it does not contain real-world examples, then that’s an error you can point out.

Indeed, the passage the referee refers to doesn't seem to me problematically hostile. It could well be that the author the paper is criticizing (the reviewer!) doesn't provide any real-world examples, and this could well be a problem with the reviewer's work. That being said, I do think there are two things here that a reviewer could potentially take legitimate issue with. First, suppose it's the case that real-world examples are irrelevant to the issue at hand. In that case, I think it could be fine for a reviewer to point this out and say that the aside is unwarranted. Second, suppose it's the case that in their work, the reviewer has provided real-world examples and the aside in question misleads readers into thinking they haven't. In that case, once again I think it would be fine as a reviewer to point this out. Still, I think it's important to be judicious here. I don't think it's a reviewer's job to defend their own work in a review. Correcting egregious misrepresentations is fine, as is criticizing errors that affect whether the paper should be published. But suppose the aside in question here is merely an aside, having no real bearing on the paper's main argument, and that the reviewer being criticized really didn't provide any real-world examples in their work. In that case, I think the thing to do is probably to let it go.

In any case, this brings us to a final issue the follow-up commenter raised above:

Here’s a related question of my own. I get that folks think that journal article tone needs to be policed. Even if that’s true, is it the referee’s place to do the policing? Shouldn’t the reviewer focus on scholarly merits? The editor, who represents the journal and its style, can police tone if they like.

For my part, I'm inclined to think that unless a journal gives clear directions on how referees should evaluate papers (which some journals do via worksheet questionnaires), it should be up to the individual referee how they evaluate a given paper. This, I think, is why it is called peer-review. I mean, I personally think the primary focus should be on the quality of the scholarship and arguments. However, we all have different views about what a good paper should look like, and some of our peers can think that presentation and tone can be important. For example, during my time as a graduate student I was often upbraided by some of my mentors for writing with too aggressive of a tone. In retrospect, I'm very thankful for it. Generally speaking (though I do think there can be warranted exceptions), I'm inclined to think that we owe each other 'scholarly respect' (or whatever you want to call it). Fortunately, when I review papers for journals, it nearly always seems to me that failures in this regard converge with scholarly failures--that, for example, haughty dismissals of an author tend to misrepresent them in ways that very much warrant correction on purely scholarly grounds.

But these are just my thoughts and experiences. What are yours?