Our books






Become a Fan

« Another kind of attrition rate | Main | New contributor: Peter Furlong »

08/07/2018

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

Reviewer

I do not agree to review a paper that I have reviewed before, for another journal. This is a firm policy with me. I think the person deserves another chance. But there are cases where I have been asked to review the same paper three and even four times by different journals. I did it the first time, and I always say I have reviewed the paper before. One tricky author even changed the title a few times ... and it kept getting sent to me. I think I have a reputation for fast turn around times on reviews (I average about 2 days). I only agree to review when I know I can get it done in the next day or two.

Marcus Arvan

Hi Reviewer: Obviously, I don't anything about your performance as a reviewer in other respects--but in these respects you seem to be an ideal reviewer. Speaking as an author, we should all be so lucky!

Marcus Arvan

This article "How Are the Mighty Fallen: Rejected Classic Articles by Leading Economists" (posted by Ingrid Robeyns on another thread) is particularly eye-opening and relevant: https://www.aeaweb.org/articles?id=10.1257/jep.8.1.165

There are a ton of gems in the paper. Everyone should read it from beginning to end. This passage--about some of the harms that can be imposed on authors--is particularly dismaying:

"However, like Becker, Krugman notes that, even if another journal eventually prints a paper, the delay that initial rejection causes may permit others to beat the paper into the intellectual market. Krugman sent his "Target Zones and Exchange Rate Dynamics" to the Journal of Political Economy. "This time I got two favorable referee reports. The paper was nonetheless rejected ... by [the referee] who thought that the paper was of 'insufficient general interest' for the JPE. The paper didn't come out (in the QJE) until August 1991. By that time the target zone literature, all of which made use of the techniques first introduced in my paper, had exploded, and consisted of at least a hundred published and unpublished pieces; in fact, I had to add a postscript to the QJE version referring to subsequent literature."

Peter

I tend to agree with your main claim. I know that there is already strain on the system, but I think that this is not the place to try to relieve some of this strain. At the very least, I think referees should disclose the fact that they refereed this paper before. In general, though, I think they should just reject the invitation to review.

NS

I think Doron Zeilberger has an interesting take on the matter driving the moral risk. http://sites.math.rutgers.edu/~zeilberg/Opinion107.html Not reusing reviews seems to just be stepping away from the central problem of referees being too harsh.

Steven French

I'm not convinced of the relevance of the examples from the history of science but thats another issue ... I also feel uncomfortable refereeing papers that Ive seen before, for broadly the reasons outlined here. And I'm heartened by the number of referees who tell us they've already seen a particular paper, either giving us the option of asking someone else or explicitly declining.

Tim

Since papers are often rejected outright even if one reviewer recommended acceptance or revisions, it should depend on what you as the reviewer recommended originally. Unless your recommendation was rejection, I see no principled reason never to review the paper again. I don't have a policy on what to do when my original recommendation was to reject the paper (happened only once so far), but I am wondering how to implement the proposal: Marcus, is your idea to decline without giving a reason for doing so or to decline while telling the journal that you have reviewed the paper before? Not telling the journal why you decline is awkward (you don't want to come across as a bad colleague), telling them may harm the paper more than actually sending your full report...

Marcus Arvan

Hi Tim: That's a very good point! I guess I'm inclined to say that if you recommended acceptance or an R&R, then one *should* agree to review again--simply because the moral risks in that case are far less than in the case of advocating rejection (viz. holding a paper's publication prospects "hostage" to your negative review).

Consequently, I think I'm willing to amend my proposal to hold that one should not agree to review a paper again only in the case where one advocates rejection.

In any case, in these cases I think of course it makes sense to let the journal know. I think I would simply say something like, "I decline to review the paper because I reviewed it before and out of intellectual humility, I think the paper should have a shot with new reviewers." I don't see how telling a journal this should harm the paper--at least not if an editor is doing their job properly (in this case, I think the *editor* should express the same kind of intellectual humility as the reviewer, giving the paper a shot with new reviewers).

Tim

As a rule of thumb: If you hint at something negative, it should be as specific as possible. Thus, we shouldn't say "I think he isn't an entirely honest person" (but I'm not telling you why I'm suspicious), but rather "I think he isn't an entirely honest person because it seems that he lied about his GPA". Since the latter is specific, others can decide for themselves whether he actually lied and whether lying about one's GPA makes one a dishonest person. The former, however, will plant a lingering doubt. Analogously: "There might be something seriously wrong with this paper, but please ask someone else" vs. "this paper may rest on a misinterpretation of Davidson, but please ask someone else". For a journal editor the latter should be more useful than the former. They just need to ask a Davidson expert whether the interpretation is defensible. – Maybe there are two conceptions of epistemic humility in play here. Not saying anything ("I just can't tell whether he is a honest person") vs. saying something while hedging it and being open-minded ("I think he's a dishonest person because XYZ, but please correct my impression if yours is different").

Pendaran

I was asked before to review a paper I had previously rejected at another journal. I declined saying the author should have the chance at a different opinion. This is called humility. I didn't think the paper was good enough, but I am just one person. Every philosopher knows how much disagreement there is in the profession. And even if your criticisms are accurate that doesn't mean that someone else might not rightly see something worthwhile in the paper. Perhaps someone else would see how it can be revised to be publishable. We also all have our own biases, as has been mentioned before, and these biases may hinder our ability to assess a paper from a more objective standpoint.

Verify your Comment

Previewing your Comment

This is only a preview. Your comment has not yet been posted.

Working...
Your comment could not be posted. Error type:
Your comment has been saved. Comments are moderated and will not appear until approved by the author. Post another comment

The letters and numbers you entered did not match the image. Please try again.

As a final step before posting your comment, enter the letters and numbers you see in the image below. This prevents automated programs from posting comments.

Having trouble reading this image? View an alternate.

Working...

Post a comment

Comments are moderated, and will not appear until the author has approved them.

Your Information

(Name and email address are required. Email address will not be displayed with the comment.)

Current Job-Market Discussion Thread

Job-market reporting thread

Categories