What should I bear in mind when I receive a verdict from a journal? Guest post by the Editors of Analysis (2 of 4)

This is the second of four guest posts by the Editors of Analysis. The Editors' aims are to "give some guidance to prospective authors and to contribute to public debate about philosophy publishing."

It is hoped that the first three posts (including this and the previous post) will generate some discussion. Then in the fourth and final post the Editors plan to respond to some of the comments received.

Please note that the Editors will not respond to comments in this thread. But please do weigh in and discuss below - the better the discussion, the more illuminating the final response post by the Editors is likely to be!

-----

What should I bear in mind when I receive a verdict from a journal?

By Stacie Friend, David Liggins, and Lee Walters

Being a journal editor is a privilege – even if it can be a time-consuming privilege! In particular, it allows you to see the publication process from a different perspective. In this post, we’d like to draw on our experience of journal editing to offer authors some suggestions on how to understand verdicts from journals, and the time it takes journals to reach a decision. The sorts of journals we are thinking of here are busy, highly selective ones, because that is where our experience lies. Some of what we have to say will already be well known to those with more experience of the profession, but we hope it will be useful to those starting to submit journal articles for publication. We’ll close by proposing that journals should always give authors some explanation of why they chose the verdict they did.

When you receive a verdict on your submitted paper, remember that the verdict is not based on how good the paper is. It’s based on how good the paper is relative to the other papers competing for the same space. Selecting papers is a contrastive process: should we publish this one, or publish this other one? So the verdict you receive when you submit a paper to a highly selective journal reflects the editors’ evaluation of its relative merits: it’s no reflection of the paper’s absolute quality.

In the digital era, talk of ‘available space’ might seem out of date. But many journals are still produced in print; and even for those which are not, turning a manuscript into a copy-edited and typeset publication takes time and costs money! Some journals may prefer to be extremely selective as a matter of policy, in an attempt to maximise the quality of the articles published. Of course, there is room for discussion about how selective journals ought to be, but that’s not our topic here.

Remember that parts of the referee report may have been omitted. It’s common for journals to invite referees to give comments to the author and also allow them to provide further comments that are just for the editorial team to see. Naturally, referees tend to be as encouraging as they can in their comments to the author. But where a paper is weak, it’s common for fairly encouraging comments to the author to be combined with more candid negative comments that are just for the editors.

This means that the editors of the journal will be in a different epistemic position to you, in two ways: not only are they comparing the referees’ evaluation of your paper with other referees’ evaluation of other papers, quite possibly they are aware of more of the report or reports on your paper than you will get to see. (We think it’s better for referees to include their overall evaluation in the main body of the report, if possible: see here.)

We know that journals often take a while to get back to authors with a verdict on their paper. At Analysis, 84% of submissions receive a verdict in no more than 60 days, but sometimes it takes longer, and there are many possible reasons for this.

First, it may have been hard for the journal to obtain expert referee reports on your paper. Not all invitations to referees are replied to promptly. Some are never replied to. (At Analysis, about half of invitations to report are turned down or receive no reply, even after follow-up emails.) If, say, three potential referees in a row decline, or fail to reply, several weeks can easily pass before an expert agrees to take on the job. (We have had a case where 16 referees – all carefully chosen – declined or did not respond to our invitations to referee a particular paper.)

We ask our referees to return their report within 28 days, and the vast majority of referees manage to do so. But sometimes they need more time. Many academics are frequently under huge pressure, so the referee is likely to be a busy person. It’s understandable that sometimes more urgent professional or personal obligations crop up that mean that they have to take longer than the period of time initially agreed.

What if there was no report? This probably means the paper was rejected without going out to a referee (‘desk-rejected’). Editors do this when it’s clear to them the paper will not merit acceptance. That decision may still be based on the advice of an expert in the area: a journal with an editorial team can draw on internal expertise to make better decisions about what is worth sending out to referees. For instance, a paper on metaethics might be rejected on the advice of an associate editor specialising in metaethics.

Every time we desk-reject a submission to Analysis, we provide the author with some explanation of why we chose to reject it. For example, perhaps we judged that the writing wasn’t sufficiently clear, the conclusion wasn’t significant enough, or the paper seemed cut off from the relevant literature. Writing a sentence or two stating that does not take long, and we feel it is fairest to the author to give them some indication of why we desk-rejected their paper.

Not all philosophy journals do this. We have to admit that we find it hard to understand why. Analysis is a busy journal with limited resources: it receives around 800 submissions every year and is run by a small charity. We do not find it very difficult to give authors of desk-rejected papers a sentence or two explaining our decision. We would be interested to learn why other journals cannot do this – or choose not to.

Comments

[Joona Räsänen]

I guess journals do not provide explanations for desk-rejections because that would invite authors to argue against the editors.

If an author receives a desk-rejection saying "your paper seemed cut off from the relevant literature", it is tempting for the author to contact the editor and say "tell me what papers should be cited then?".

Or if the decison says "writing wasn’t sufficiently clear" the author could email the journal and say "can I submit my manuscript again if I clarify my writing?"

[Truth]

Editors,
you are dispelling all the popular (mis)conceptions of the way journals work. Editors, it seems, are not monsters trying to frustrate the lives and careers of every young (nd old) academic. You seem like regular, though highly accomplished, people. Can it really be so?!

[Douglas W. Portmore]

Dear Stacie Friend, David Liggins, and Lee Walters, You write: "We do not find it very difficult to give authors of desk-rejected papers a sentence or two explaining our decision. We would be interested to learn why other journals cannot do this – or choose not to." I have no plans to do this as Editor of Ethics. For one, there are three of you and only one of me. For another, it's rarely the case that it's just one simple thing, such as the "writing wasn't sufficiently clear." Most often, the paper, including its clarity, was, in absolute terms, quite good but not as good as around 50% of the other submissions we receive due to a confluence of factors about the paper. Consequently, it's unlikely, in my judgment, to successfully make it through all the other steps in our vetting process. What's more, as Joona Räsänen mentions, I don't want to invite an argument, especially since an argument would often be warranted if I merely provided one or two sentences of explanation, as it's often impossible for me to adequately explain why a paper was desk rejected in just a sentence or two. I do, however, aim to give a decision within a week or two. So, there's little cost to a desk rejection. And if the paper goes much further in the vetting process, the author will almost certainly receive valuable feedback in the later stages.

[Chris]

As an author and reviewer (but not an editor), I support Douglas Portmore's approach to editing at Ethics, where desk rejections happen quickly but without comments.

[anon]

I got a rejection from Philosophers’ Imprint with no explanation. The email said something like ‘the volume of submissions makes it impossible to provide comments…’. Does this count as a desk rejection? Any insight would be appreciated!

[Rejected]

Anon,

That is, indeed, a desk rejection. It happens often — more often than refereed rejections, in my experience (though others’ experiences might differ).

[imprintreferee]

Actually, no, that's definitely not necessarily a desk rejection, especially at Phil Imprint. Unless things have very recently changed there, they use a two-tiered refereeing system, where referees first take up to two weeks to make an initial "summary" judgment (basically: a no, or an "I am willing to spend more time on this/write a real report about it). The summary judgment, iirc, can be pretty brief and I feel quite confident that the editors at least do not always pass it on to authors (maybe they never do even). Different journals have different systems and no comments does not always mean a desk rejection.

[Peter Finocchiaro]

I'd like to hear more about the stats regarding referee requests.

Friend, Liggins, and Walters say "At Analysis, about half of invitations to report are turned down or receive no reply, even after follow-up emails." Do you know what percentage are turned down and what percentage receive no reply? Have these percentages noticeably changed, especially during the COVID years?

[Assistant Professor]

The editors write that "When you receive a verdict on your submitted paper... It’s based on how good the paper is relative to the other papers competing for the same space."

I appreciate that the editors are engaged in "a contrastive process: should we publish this one, or publish this other one?" but individual referees are not, and this does make it more challenging to say that editors are merely putting papers head to head with each other since presumably they are not making a decision based on their direct evaluation of the papers but on referees' individual evaluations of one paper and a different set of referees evaluations of another paper.

[provide feedback]

Re: the comments from some of the above about not wanting to "invite an argument" by giving justification for a desk-rejection. I find this to be a terrible rationale for not providing such feedback. If you get an argument, deal with it. If someone responds asking to resubmit after making some changes, all you have to say is no. From the perspective of authors, it's much more helpful to get some feedback, even if brief, from journal editors. I have served as a referee on several occasions. It's not hard to write up a short explanation for why a paper is being rejected. Anyone serving as an editor of a major journal should be able to do this without much effort after reading a manuscript. This doesn't have to be a full referee report in order to be helpful. And, once again, if you get an argument from the author afterwards, tough. Maybe in certain instances such an argument would reveal that the editor was somehow mistaken? Isn't this more in the spirit of academic publishing? Editors are, after all, fallible and imperfect judges of philosophical merit last time I checked.

[whatever]

I have only heard whispers, so perhaps some one here (an editor) can confirm. Is it standard practice to receive a course release from your university when you're the Editor of a major journal? If so, then it seems you have plenty of time to dash off a quick sentence or two explaining desk rejections... no?

[different take]

@whatever, not everyone receives this, but also I think you are underestimating the amount of time it takes to edit a journal. (My sense is many editors spend significantly more than the equivalent time of teaching a single course, over a year, on editing! And when I was asked (I declined) to join the main editors of a major journal, the time estimate they gave was over what I spend on a class by a lot, and that was at a journal that distributes the work among multiple (six or so I think) editors and is a well-oiled machine).

As a submitter, I have to admit that I find it kind of strange that people think that the editors owe us any kind of explanation. I don't think there should be any default norm that even if your paper is amazing the editors have some obligation to accept it--they don't! It's their journal! I also think authors who argue with brief feedback like this are confused about what the process is. (And @provide feedback... I don't think anyone ever said or thinks editors are perfect or even unbiased judges of philosophical merit, but again, I don't see how philosophical merit somehow gives someone a right to publication, or standing to argue with an editor--journals aren't machines that pick out the best papers, journals are things run by human beings who make human decisions. If people want something different, start it...)

Please note: I am not part of the journal-industrial-complex. I just have a very different understanding of journals than some of the commentators on this thread do! Namely: editors don't owe us anything but efficiency in verdicts; it's not a collaborative process where we try to convince them to publish our work, they are more like judges at a prize competition that doesn't have any clear standards and where the awarding of the prizes is up to the judges. It would be very strange to argue that a judge was wrong in that case--part of what they are doing is giving their opinion of the quality of the different performances, or whatever. They might be able to cite different factors in what was good (or bad!) about it, but they don't owe anyone such citations. Same with editors.

[Ben Bradley]

In response to whatever: no, it is not standard practice for editors to get course releases. I have never got one, even though as different take says, editing takes at least as much time as teaching a course. There is little incentive for a university to give course releases for anything done outside the university. You would need the journal to pay for the release, and Ergo has no income. (Some journals do have income of course.) Asking a university for a course release without any outside funding source is basically just asking them if you can please have thousands of dollars every year.

It is understandable that people want an explanation when their paper gets rejected. At Ergo our decisions are made by area editors, and one of the selling points when we recruit an AE is that they don't have to provide comments on desk-rejections. We are recruiting new area editors year-round and anything that makes recruitment harder is a non-starter. Many of our AEs provide comments even though they don't have to. As others have noted, this not infrequently leads to pushback from authors; a result of that is that over time, area editors become more inclined not to give feedback at all.

It is good that there are journals that give feedback on all submissions. There is nothing wrong with different journals having different policies about this. If you want to make sure you are going to get feedback, submit to a journal that has that policy.

[ed]

Ben
I managed to get a course release when I took on a editorship, but I had just finished doing a huge favour for the administration at my university. So, it might have been that that helped me. I have since changed jobs, and my new job does not give me a course release. But society journals, like Phil Sci, the PSA's journal, do expect the editor to get a course release. And I believe there is some money for that. But "whatever" probably underestimates the amount of work that journal editing requires. It is not only thankless, it is demanding, and the work just keeps building up.

[Be patient, young grasshopper!]

You guys get verdicts from journals? How does one revise in light of comments like:

"Reports complete" (for weeks and weeks)

or

"Under review" (for months and months)

[cecil burrow]

As a former editor, I totally get not giving comments for desk rejections in order not to "invite an argument". For a while, we sent comments out with desk rejections. Most people of course are good citizens. However, a number of people took such comments as the beginning point for a discussion, occasionally even an angry one, sometimes threatening lawsuits, and so on. Even friendlier challenges of the desk rejection had a way of going on and on in a way which was just too time consuming. As an editor, my time was precious, and in order to protect it, I simply stopped sending out comments when I desk rejected something. ('whatever's idea that a course relief somehow vaguely compensates for this is unfortunately laughable.)

[provide feedback]

@different take: right to publication and standing to argue with an editor are very different things, of course. No one has a right to be published in a journal, no matter how good their work. But I do believe that authors have standing to argue for their work. The final call is, of course, with the editor, but why disallow the process by not providing any feedback? I feel like this is especially important given the de facto reality confronting many academics: publishing isn't a mere prize competition, it is a requirement for remaining competitive on the job market, and it is necessary for promotion and advancement in the field. More transparency in decision-making is called for given the impact that publishing has on one's professional trajectory.

[Douglas W. Portmore]

Dear whatever, I do get released from one course per year for editing Ethics throughout the entire year. This is not at all standard. The administration was reluctant to give me this. I had to plead that editing Ethics is particularly burdensome because of the number of submissions that it gets and the way Ethics runs (e.g., that all papers put forward by an Associate Editor go up for a vote before all the editors, including myself). Consequently, I explained that I would have to turn down the position unless my home institution was willing to do at least this much for me. Now, you say that, given this course release, "it seems [I] have plenty of time to dash off a quick sentence or two explaining desk rejections." That's not the way it seems to me. To give you an idea of the work involved, I started on January 1 and have already vetted over 50 submissions. And vetting submissions is not my only task as Editor, although it is probably my most time-consuming task. For instance, I respond to blog posts such as these, because I believe that it's important to be transparent. People should know that I do get a one-course release (although many of my predecessors didn't) and that I nevertheless do not plan on providing comments for desk rejections that come within a week or two of submission.

[cecil burrow]

People are arguing about course relief as if it is vaguely relevant. I promise you, one fewer course does not even vaguely compensate for the time an editor will spend taking care of a journal.

[The Real SLAC Prof]

The problem with not giving any comments on desk rejects is that then authors have no reason not to immediately send the paper out the the next journal. This makes the paper someone else's problem. Of course no one has a right to comments, but not providing comments makes the refereeing/ submitting environment worse for everyone.

Of course, some people are assholes and they will argue with editors, but a journal could simply have a policy of not responding to such arguments. Other people will turn around and resubmit without attempting to deal with the comments. But people are going to be more likely to resubmit without revision if they have no information about the reason for rejection.

[Richard Yetter Chappell]

I wonder if the costs to editors of providing some minimal feedback with desk rejections could be mitigated by:
(i) ticking common explanations off a checklist (rather than typing it out manually),
(ii) setting a "do not reply" reply address, so authors literally cannot reply, and
(iii) noting that any author who contacts them (via other means) to argue about the verdict will be banned from submitting to the journal for X years.

Seems like this would pretty effectively deter arguers, and the extra info would be very helpful for submitters. Checking the boxes would add a bit more of a time burden on editors, who are already doing a supererogatory job. So I certainly don't think it would be obligatory for anyone to implement this. But I do think it would be a *good* policy to implement, and I'd be happy to see it normalized as a general "best practice" (even if not always followed).

To better compensate editors, perhaps journals could introduce an optional modest fee for authors who would sufficiently value that feedback.

I like the idea of submission fees more generally, to help reduce half-baked submissions from clogging up the system. (And to reward people for valuable service!) Departments could offer a modest research budget to their grad students to help defray affordability concerns.

[disclaimers]

I think The Real SLAC Prof makes a nice point above: journals could provide minimal comments explaining a desk rejection and have a policy of not responding to further communications, avoiding the problem of endless back and forth with authors. Even better, they could include a statement of this policy at the end of their desk reject comments: "Please note that, due to the high volume of submissions, it is our policy not to respond to further communications or inquiries about desk rejected papers." This statement alone might help dissuade authors from responding with an argument.

[Desk worker]

The Real SLAC Prof,
I hear this comment over and over again that authors have no reason to revise their papers before submitting again if they do not get feedback with a desk rejection. I think it is exactly the opposite. They have every reason to believe their paper is substandard, and should not be sent to a journal. What they may not know is why. But they should share their manuscript with an honest friend or colleague who can help them see the light. I may have had two papers desk rejected in my career of 25 years or so. It is a real problem if people keep sending their unworked out ideas to journals.

[Richard Yetter Chappell]

@Desk worker: In my experience, my more ambitious papers tend to get desk-rejected several times, before eventually being published in a top-5 journal. My more dull and predictable papers are more easily published. So, in my experience at least, desk rejection is at most *very* weak evidence that a paper is "substandard" (and possibly evidence of the opposite).

I agree that it's a problem that people keep sending out bad papers to journals. I just don't think you can take desk rejection alone as strong evidence that a paper is bad.

[The Real SLAC Prof]

@Desk worker: I think the signal sent by desk rejection has changed quite a bit over the last 25 years. At one time, a desk rejection might have sent the message that one's paper is seriously flawed, but that is no longer the case. At this point, even a mid-tier journal like Ergo is desk rejecting two thirds of its submissions: https://journals.publishing.umich.edu/ergo/site/peer-review/

Moreover, the boilerplate that comes with these desk rejections is usually something along the lines of "we accept only a minuscule number of submissions, and we are forced to reject many excellent papers."

In recent years, I have had papers desk rejected that would not have been desk rejected 10 years ago. I do not believe Chappell's experience is anomalous.

Given all this, I don't think there is much of anything that can be read into a desk rejection without comments. Under these conditions, authors aren't being irrational when they simply resubmit elsewhere without revision. Yet in doing so, many are making the submission/ publication environment worse for all because their papers do, in fact, need significant revision.