Tips for satisfying referees?

Last Friday, I ran a thread asking readers what kinds of things bug them as journal referees. I've found the thread really informative so far, and hope readers continue to weigh in. In today's post, I'd like to run a thread on the other side of things: namely, things that authors feel like they've learned to be important in terms of getting positive reviews from referees. Of course, having a strong and interesting argument helps--but here I'm thinking of other things: specifically, ways of writing papers that tend to fare better in the review process than others. My own experience here is that this is a lot of what I had to learn early in my career to publish successfully, and I am curious whether other readers who have learned to publish successfully have had similar experiences.

So then: do you have any tips for writing papers in ways that are likely to go over well with referees? If so, what are they? Conversely, do you have any tips for things to avoid--things that don't go over well with referees? I'm really curious to see people's responses in the comments section. Allow me to kick things off with a few tips I think I've learned:

Tip 1 - 'For all x, if x is a possible way to misunderstand your argument--no matter how silly--make sure you take steps to prevent referees from interpreting you as saying or implying x': This may be the single most important publishing lesson I think I've learned. Early in my career, when I wasn't having much luck publishing, I was continually surprised at how easily referees can misunderstand a paper's argument--often, it seems, in the least charitable way possible. I also know that I'm not alone in this. I've heard more stories than I count count where people have recounted things like this: "I literally write in the paper, 'I am not saying X', and reviewer 2 recommended rejecting my paper on the grounds that I say X." How is this possible? The most plausible answer I can think of is that a good number of referees probably try to get their reviews done as quickly as possible, skimming them rather than reading closely or carefully. In any case, ever since learning this lesson, my papers have fared much better in the review process.

Tip 2 (corollary of tip 1) - Head off potential misinterpretations and objections as soon as possible in the body of the paper, rather than in footnotes or an objection section late in the paper: I've had cases of papers, and heard other people recount similar cases, where it doesn't seem like a reviewer even got to the end of the paper. A typical case: the reviewer raises an objection O to an argument on page 5, presenting it as grounds for rejecting the paper--despite the fact that you explicitly raise and respond to O later in the paper (e.g. page 25, in an 'Objections and Replies' section). This happened to me enough times that I learned that it's important not just to address potential misinterpretations and objections, but head them off before they can fester in the reviewer's head. In fact, one thing that I've often resorted to in recent papers--and seen other published papers do as well--is to briefly head off misunderstandings and objections in your introduction. Even if it is just saying something like, "The final section of this paper will address objection(s) X, Y, and Z", my experience is that this can be important for at least getting in your reviewer's mind that you plan to address the objections they will have when reading the paper. Otherwise, again, they may not even read far enough (or carefully enough) to see that you address their 'devastating' concern. Finally, my experience is that in general (though not always), it's best not to leave objections to a final 'Objections and Replies' section of the paper or merely address them in footnotes/endnotes (which reviewers may not read or read carefully). As a general rule, I've found it works best to address any potential misinterpretations or objections in the body-text of a paper as soon as a reader is likely to have them.

I'm probably forgetting a bunch of other tips at the moment. However, these are probably the two most important things I've learned for getting papers to go over better with referees. What are your tips? What have you learned works well/doesn't work well with referees?

Comments

[Untenured]

I want to talk about satisfying reviewers once you have an R&R. I think I have only ever had one initial 'accept without revisions' from a journal. On a number of R&Rs, I have had cases where one reviewer clearly recommended 'reject', but the editor chose to go with R&R, sometimes b/c the other reviewer recommended acceptance/R&R or the reviewer recommending rejection had a sloppy review. Sometimes there is a 3rd reviewer who played tie-breaker, but not always.

I have actually enjoyed responding to R&Rs where a reviewer carefully outlines all the changes that should be made. I've usually felt that the paper was much improved as a result. However, when I have been rejected upon submission of an R&R, it has been because I didn't satisfy that reject-reviewer. Any thoughts how to respond to the reject-reviewer (beyond be kind, take their criticisms seriously, etc.)? Do you acknowledge that the reviewer thought the paper was crap? What do you do if the reviewer is rejecting the paper because they think the whole debate to which you are contributing is misguided? Or they offer some horribly question-begging/strawmanning objection?

[anon]

Maybe you're planning on this already, but I'd also be curious to see a thread where editors share their thoughts about all this.

Avoiding annoying the referee / referee-proofing the paper is ideally all one needs to do. But in some cases I've been in the situation where the referee was never going to agree with me and I needed to convince the editor that the referee was in the wrong.

And while I've been successful in those cases, I'm curious to hear how to make a good impression while doing this, what seems more effective, etc.

[Marcus Arvan]

Anon: I think that's a great idea! I don't know how many editors read the Cocoon and will be willing to comment, but it's worth a try!

Untenured: The kind of case that you and anon describe actually happened to me a few years ago. Here are some quick thoughts. I think that if a reviewer recommends rejecting the paper for really weak reasons (like the ones you mention) and the editor overlooked the review to offer an R&R anyway, then it's likely the editor recognized that it was an unfair review. So, the thing to do is to probably approach your revisions and author's reply with that as a background assumption.

In my case, I said clearly and frankly in my author's response that I respectfully disagreed with the reviewer. I then briefly enumerated the reasons they gave for rejecting the paper, and explained why they were erroneous--pointing to several points in the paper that showed why. Finally, some of the revisions I completed for the other reviewer helped to clarify why the negative reviewer's concerns were erroneous, so I pointed to them too and thanked both reviewers for encouraging me to clarify.

The editor apparently bought the explanation, and from what I can tell they only sent out the revised manuscript to the more positive reviewer, who was wholly satisfied with the revisions. The paper was then published. Although it was only one case, my sense is that being frank but gracious in explaining why the negative review is off the mark is probably a good way to go.

In the case Untenured describes, I think a similar approach could probably work. But perhaps you could also add some brief revisions in the paper to the effect of, "Some may object to this whole debate for reasons X, Y, and Z. While those concerns are well-taken and worth continuing to debate, they are outside the scope of the present paper." I've found that stuff like this tends to work too.

[Wear 'em down]

Regarding responding to R&Rs, I think it's important to be extremely thorough, clear, and above all organized in the reply letter. At the beginning of the letter (after graciously thanking the editor and the reviewers for the chance to revise your submission, blah blah), you should start by summarizing the reviewers' comments (they may have forgotten what they wrote, since it might have been a few months since they submitted their review). Then summarize the major changes that you've made to the ms (so they know what to expect). Then you need to methodically go through every single point that each reviewer raised in their reports one by one, and give some kind of response for each. Sometimes that response might be limited to the letter, and sometimes it's in both the letter and the manuscript. Make sure that anytime you've made a change to the ms, you have explained why. I find it's good to copy and paste new or heavily revised passages into the reply letter, while also highlighting them in the resubmitted manuscript (some journals explicitly ask for this, others don't, but it's always a good idea). Just make your changes easy to find.

A further tip for when reviewers ask for changes that you really don't want to make. In those instances, I try to actually do what the reviewer asked *in the letter*, and then give a really clear rationale for why I am not putting it all in the paper (usually, "beyond the scope" or "word limit" or "disrupts the flow"), and then I also include a footnote flagging the issue in the main text. This is about the best way that I have figured out to say "no" to reviewers without getting myself in trouble.

All this makes for very long reply letters, but I think that's a good thing. It shows reviewers and editors that you've taken the reviews seriously, and that you know what you're talking about. It can also have the secondary effect of wearing out the reviewers' resistance, which works just the same...

[historygrrrl]

1. Make sure you cover all the relevant/recent literature. Especially literature from the journal in which you're trying to publish. And *especially* if you are an unknown author or submitting in an area where you're unknown.

2. If there's any question whatsoever that your view has already been defended by someone else, make it disgustingly obvious that you're saying something different.

3. Find your 'authorial voice.' YOU are the expert. Obviously, you have to engage with what other people said. But, I found things started going better when I made my view the headline, rather than offering my view only after a long discussion and criticism of what other people said.

4. R&R: I agree with others - a detailed letter explaining the changes (page by page, line by line even) can be very helpful. Especially since you might not get the same reviewers as the first round, depending on the editor.

Sometimes, even a cranky reviewer can be helpful. I had one submission in which Reviewer 1 wanted to accept as-is and Reviewer 2 wanted to reject (they basically didn't like my topic). By responding to Reviewer 2's comments, it ended up being a much better paper.

5. R&R, part 2: Sometimes you have to know when to call it. I remember how exciting it was to get my first R&R, and even though I worked hard on it, it was ultimately rejected.

That paper got a R&R from another journal, but the reviewer wanted me to make a ton of changes that I didn't agree with - it would have completely changed the scope and even topic of the project. So, I let the editor know I wasn't resubmitting. I know there's a ton of pressure to publish, especially with the job market the way it is, but I'd rather publish my paper than the reviewer's paper.

[an editor]

I was on a panel of editors at a conference and we were asked about R&Rs.
Editors DO NOT want long replies to referee reports. Let me clarify. We do not want point by point replies going for two or more pages. We want you to digest the referees' comments, and then systematically present how you have addressed them. Put the most important things first - "I added the unstated premise that R1 was concerned about". This will show us that you understand what is important, and what is not.
The editor will read this document with great care.

[Marcus Arvan]

An editor: I honestly find that pretty shocking. For two of my most successful R&R's, both of which involved turning 'major revisions' verdicts into acceptances, my author's replies were 23 pages and 11 pages apiece (both single-spaced). The first one, in which I had to respond to 3 reviewers, was over 10,000 words!

I totally understand why editors might not *like* long author replies. But still, authors just have one shot to convince the editor and reviewers that the revisions successfully address the referee concerns, and to me it only makes sense for an author to err on the side of being thorough. And again, both of the above cases I gave were successful, so... ¯\_(ツ)_/¯

[Wear 'em down redux]

Just to clarify my suggestion in light of what "an editor" says:
The summary of your changes at the beginning of the letter should do what "an editor" says - digest the comments, highlight the major issues and the major changes. This part comes first and is what the editor reads closely.

But while editors may sigh when they see a long reply doc (I know, I've been in that role as well: it's a pain to read an entire ms, a set of referee reports, and a long letter), the fact is that reviewers sometimes give very long reports and make lots of comments, and these need addressing. Not addressing reviewers' comments strikes me as very bad advice, and a recipe for the next report to say, "the author didn't do x, y, and z - reject." It also might not be so obvious to a reviewer how a "digested" synthesis addresses their specific points. So you need to connect the dots.

In short: give an overview of the changes that will be easy for the editor to digest, and then a thorough summary to show your work for the reviewers. Not doing the latter is risky and liable to alienate reviewers, even if it does make for less work for the editor.

Now, sometimes editors ask authors to focus on certain aspects of the reviewer comments in the decision letter, because they think that not all of the objections raised in the reports are equally pressing. That gives authors some guidance about what they should focus on in the letter, and also communicates to the reviewers what they should drop. In those cases, I advise authors to do what the editor asks for. But many editors just use a form email for the decision letter that basically says, "address the reviewers' comments." In those cases where there is no editorial guidance, I think you have no choice but to address the reviewers' comments thoroughly.