Refereeing papers that are critical of you?

In our newest "how can we help you?" thread, a reader writes:

I was recently asked to review a paper that engages with some of my work. Since the journal asking for the review made the manuscript available to me, I was able to look it over before agreeing. I quickly discovered that the paper is very critical of one of my papers, and that it also includes a number of serious mischaracterizations and misunderstandings of my arguments (which I found very irritating). On the one hand, I feel the urge to accept the review request so that I can correct these mischaracterizations. On the other, I know that I am not a neutral party in this matter, and so I am also somewhat inclined to decline the review on account of my obvious bias. What do other people do when they encounter this kind of situation? Is it better to decline on account of my bias, or to accept on the grounds that I am an authority when it comes to what I say in my own papers?

These are very good questions. From what I've heard elsewhere on social media, my impression is that editors ask targets of critiques to referee papers fairly often. Further, in one way, this sort of seems like a good practice--for the journal has an interest in publishing good papers, and if anyone is well-placed to prevent mischaracterizations, then it is presumably the author being critiqued. On the other hand, it does (as the OP notes) introduce a kind of conflict of interest into the review process. After all, if anyone has an interest in not being embarrassed or their arguments disproven, it is the author whose work is being critiqued. So, I wonder whether editors have a way of spotting this. When editors receive a review from an author whose work is being critiqued, do they subject it to especially careful scrutiny, reading the paper for themselves to determine whether the referee's review is fair? 

I'm really curious to hear what readers think of this practice, as well as from editors--and on whether the OP should accept the referee assignment. What do you all think? 

Comments

[Nicolas Delon]

I think they should accept and write a candid review. If anything, unless the manuscript is just a hatchet job, the incentive goes the other way: you want your work to be discussed and cited. Help the author get things right about you, but also let valid criticisms be heard—–that's what anybody's work needs!

[someone talked about]

I have refereed papers discussing my work. Some have been critical, but interesting. If they advance the debate, which some have, I recommend acceptance.
If they are based on a terrible misreading then they get the same treatment as any such paper, a recommended rejection. Often editorial manager lets you send comments to the editor only - I generally do not use these (if I cannot say something directly to the author, then I probably should not say it all). This is where you can disclose your potential conflict of interest.

[anon]

I think it's fine if you are the referee in this case, especially if you use the comments to editor to emphasize your position here.

Here's a story about "misreadings", since they're particularly relevant here. I guess the issue with them is that it can be pretty philosophically controversial, sometimes, whether something really is a misreading of a view. I have published a paper criticizing figure X for saying Y. Everyone I could find in the literature interprets X as saying Y, and it's the most straightforward and literal reading of the text. But one of my referees was deeply invested in arguing that I was misreading the target of my paper, and that my paper should be scrapped. Maybe I was - maybe in some deep personal sense, X doesn't really believe Y. But I believe in the intervention I made because it was based on a reading that makes sense.

[an associate editor]

As an author:
I'd like to know if I've misrepresented someone's view, so I'd want you to review my paper.

As an associate editor:
I've already read the paper, so I know what the author says about your work. I still chose to invite you, so I'd want you to review the paper unless you thought you couldn't possibly be fair. But if you're not sure what to do, just explain your situation and ask. (It's also ok to accept the invitation and provide a brief explanation of your concern in a confidential note to the editor.)

[Evan]

I have to disagree with my fellow commentators: you should reject the request and explain why. As you say, you are not remotely a neutral party. Some here have commented that the author will want to know that s/he misrepresented someone's view. But I think the author is likely to conclude that the referee is the one who doesn't understand the view. It might also be that the mis-reading is a reasonable reading, but that you as the author can't recognize this. Irritation, while completely understandable, is not a good basis from which to review a paper.

[TT]

I agree with Evan here. The OP writes, "I feel the urge to accept the review request so that I can correct these mischaracterizations." But it's likely that the irritation isn't just about the mischaracterizations, but is also related to the fact that the OP's work is engaged with "very critically" in the manuscript. Because it's so hard to disengage critical engagement from mischaracterization when one's own work is under discussion, and because the fuzzy space between the two is likely driving the irritation (it's quite natural to be defensive of one's work), I would advise declining the review request. As Evan writes, irritation is not a good basis for a helpful review.

[William Vanderburgh]

The editor who invited you wants your input *because* your work is discussed. Disclose your perspective and trust the process. There will be other referees, too. If you feel your work was treated uncharitably, be especially careful to treat this work charitably, rather than lashing out.

[Tim]

OP is presumably a qualified expert not only on the topic, but on OP's views as well! So I think it is appropriate for OP to review the paper.

But as authors we should also keep ourselves honest in these sorts of situations. I myself use the following test. To show that the submission mischaracterizes the view, do I need to write anything new in developing or describing the view, or can I simply provide quotation to already published material? If the latter, I feel comfortable thinking the submission is simply based on mischaracterizations. But if the former, I begin to feel that maybe it might make a contribution. I'm not sure this is the only or best test. But something like it may keep authors from rejecting each and everything that critically discusses them.

[Evan1]

There are epistemic issues with this since the referee truly doesn't know if the author made substantial criticisms or erroneous mischaracterization of their work.

The referee has to mostly trust the testimony of the editors/journals who requested them. There's also the possibility of emotions and pride clouding the referee's judgments as others have pointed out.

Perhaps a better alternative is getting somebody else who is well-versed in the requested referee's work and topic to referee for the journal instead. Unless one works in an extremely niche area, there could be other competent (or better) referees available for these kinds of cases. Tim is right about expertise but I doubt this approach is the only option on the table here.

[Humanati]

I understand people's worries about bias/being overly defensive when it is one's work that is at issue. But I've been in this position a few times before and have applied a test similar to Tim's. (I'll also note that I've never ended up rejecting the paper.) Here is something else that I think it's helpful to keep in mind when placed in this position (which a colleague and I came up with when discussing it):

Ask yourself: is the interpretation of your work just (a) incorrect, or (b) incorrect *and* unfair?

If (b), then you should be able to use Tim's suggested methodology to help the author to see why the text doesn't support their interpretation--at which stage, it might still be open to the author to target their critique at 'position X' and cite you as one development of this position, but not the development that they're focusing on. At the end of the day, it's ideas that should matter to us--not whose ideas they are. If this paper develops a persuasive argument against position X and position X is of philosophical interest and importance, then the paper may still be worth publishing.

If (a), then you can just encourage the author to explicitly frame their description of your view as (e.g.) 'one among a number of possible interpretations', and perhaps to include a footnote mentioning others as well--such as the one that you prefer.