Reader query on revising rejected papers

In the comments section of our most recent "how can we help you?" post, Amanda writes:

So I was wondering if you could have a thread on revising papers that were submitted and rejected. I have a hard time with this, because lots of people give the advice to just turn in the paper again with no changes. And lots of people give the opposite advice. I think we all know that when a reviewer points out a serious problem, we should change it. But what about the majority of cases when we ourselves are skeptical about the reviewer's complaints? And for me, it is rarely the case that two reviewers mention the same problem. If they did, then it would be an easy case to go ahead and change it.

I struggle with making changes because I simply am not confident that the changes would help increase the odds of acceptance. And if they don't, then basically I wasted a lot of time that I could have been using to edit different work.

Basically, I am curious about two things. First, have people noticed a correlation between making changes to their paper and having it accepted at a decent journal? And second, do most of you cocoon readers recommend making changes to a paper, or simply just resending it unless there are obvious problems?

Good questions!

I'm not sure in my own case whether I've noticed any correlation. I had one paper that was rejected like 14 times (and seriously revised dozens of times) before it finally got accepted somewhere...but that paper is a total outlier for me. For the most part, aside from major revisions in revise-and-resubmit verdicts, the papers I've ended up publishing haven't been all that different than the initial versions I sent out to begin with (and usually after a few rejections without substantial comments, which obviously give one no real grounds for revising). Then again, I've had something of a tendency to "shoot low" in the journal hierarchy, so perhaps I'm not the best person to ask here. 

What do the rest of you think? What have your experiences been?

One relevant issue is that I've heard many cases of the same reviewer getting the unrevised paper again at another journal, and then simply giving the editor at the same review as at the previous journal (advocating rejecting the paper at the new journal). This practice (which judging from my social media feeds appears to be relatively widespread) suggests that revising your rejected paper may be pragmatically good idea. That being said, the practice itself actually bothers me quite a bit, as different referees often disagree greatly over the merits of a paper and the same reviewer rejecting the paper again and again essentially holds the paper's fate hostage to the (possibly idiosyncratic) judgments of a single person. Indeed, in several cases I've been a part of (as either author or reviewer), I've seen a journal's editors forward all of a given paper's reviews to the author and reviewers (a fairly rare practice I totally approve of by the way)...and what do you know? The same paper receives three radically different reviewer verdicts: one reviewer thinks the paper is worthless, a second recommends outright acceptance, and a third a revise-and-resubmit. 

Alas, this brings me to a real problem I've struggled with quite a bit recently. Suppose you adopt the presumption that revising your paper is a pragmatically good idea for the reasons mentioned above. Here's the problem: most of the time when you revise a paper to address reviewer comments, it requires adding a lot of stuff in. So, what starts out as a 7,000-word paper when you sent it to Journal 1 turns into a 9,500-word paper for Journal 2, a 12,000-word paper at Journal 3...and now, after it gets rejected at Journal 3, since it's 12K words long half the journals in the field won't consider it because it goes over their word limit. :/ Seriously, this is my life (see Fig. 1) Anyway, that's a pragmatic reason not to always revise papers before sending them out again

Which I guess brings them to my only real piece of advice: do what's best for the paper in your own philosophical judgment. If you think a reviewer is right about something, then fix it. If not, then don't...and pray you find the right reviewers (though I might add, if reviewer after reviewer keeps saying the same thing, maybe question your philosophical judgment!). But again, these are just my thoughts. What are yours?

Fig. 1

Comments

[Michel]

I always try to do at least *something* to address whatever my referees' concerns are. Even the few times that I thought the comments were completely out to lunch, I tried to do something about it. Usually, if I don't think the concern is very apt, I just adjust my signposting and disclaimer-footnotes, and try to steer new readers away from making the same mistake the referee did.

When it's a concern that I think is apt, I obviously spend a lot more time trying to deal with it. Sometimes that means substantial re-writing, but usually it just means doing some frenzied research and drafting a few new paragraphs. Once I've addressed everything, I go back through the paper and systematically murder my darlings until I've cut it back to an appropriate size. I do this even if the revisions didn't put it over the word count (although in that case I'm obviously more generous about what I leave in!). If a new referee thinks I've omitted something important, it's really easy to reinstate the material (although it does mean doing some trimming elsewhere).

It's been paying off for me recently, and I can definitely say that over time, the resulting papers are *much* better than their previous iterations. And as hard as the cutting process is, it really helps me to a get a better idea of what's crucial to my paper, and what's not. And some of those cuttings make for decent paper topics in their own right!

But I'm one of those people for whom the writing emerges largely as a by-product of the editing process. I can't just sit down and dash off a usable manuscript, even if I've given it a lot of thought. I start with a skeleton, flesh it out, and take it apart and rebuild it ad infinitum.

[anonymous TT prof]

My advice is to mull it over for some time. Don't simply follow the reviewer's advice ... Rather, think about their reasons for giving that advice. If they made some stupid criticism, maybe it's because they misunderstood the general point of the paper. Or some other small point. Think about how to rewrite a bit to make sure it doesn't happen again. If they tell you to incorporate work by so-and-so, read that work and see if it's relevant. In my experience, most of the time it's not, but sometimes it really is the sort of thing that should be incorporated. It can be as easy as adding a "See Smith (2014) for a defense of this view" or it can mean rewriting a section of the paper to take into account some innovative work someone else has done.

If you find yourself tweaking here and there to the proclivities of the reviewer, or adding an entire section that someone else thought was necessary, then you are not doing it right. If you find yourself rewriting a section or two in response to some misunderstandings, or restructuring the paper to make the narrative more cohesive, you probably are doing it right.

[Trevor Hedberg]

Here are some general rules I've followed in this process (with short explanations):

1. When I get a detailed list of comments, I read them immediately and then stow them away for at least a week. Then I reread them and decide what to do with them. When we first get comments -- especially when we get rejected -- we are often frustrated or disappointed. That's not a good state of mind to evaluate the comments objectively. I have found it helpful to let those feelings pass and kick around the ideas for a while before making any decisions about what to do.

2. If I genuinely believe the reviewer has found a significant weakness in the paper, I make a change. I'd feel intellectually irresponsible if I deliberately resubmitted a paper that contained a glaring weakness in the argument that I knew about.

3. If I am unsure whether a single reviewer's comments have merit, then I generally don't make the suggested changes unless they're very minor.

4. If multiple reviewers find a problem with the same portion of the paper, I change that portion of the paper. I have actually had this happen many times. Usually, the reviewers have different suggestions for the paper but agree that there's a specific problem that needs to be fixed.

5. I generally don't expand a paper when I revise it after rejection. I trim certain sections to expand others or retool already existing sections. Usually, I already have a list of potential venues for the paper and have a word limit I'm working with. Massive expansion to the word count is something I only undertake when I have an R&R in hand.

[Amanda]

Thanks to everyone who has offered feedback. I think I might try responding to more comments than I have in the past (while not overdoing it, of course).