For the blog's mission and comment policy, see http://philosopherscocoon.typepad.com/blog/2012/05/mission-statement.html On Disclosing Conflicts of Interest as a Reviewer - The Philosophers' Cocoon

Become a Fan

« 2nd CFP: Second Annual Philosophers' Cocoon Philosophy Conference | Main | On the elephant in the room »

02/10/2014

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

Rachel

I've reviewed work where I knew who the author was (I contacted the editor: she if I thought I could give the article a fair review. I did think that, so I reviewed it. I R+R'd it and it got eventually published).

I've reviewed work that was *deeply* critical of my own work. I raised some issues with their interpretations and representations of my work. I R+R'd that too (don't know its status).

Here's another conflict of interest: increasingly, our citation rates matter for our careers. So when one comes across a paper where one is cited, there's an *incentive* to suggest publication. I've had to fight against that urge a couple times. Successfully, I think.

Marcus Arvan

(typo fixed)

Hi Rachel: that's a great point! There certainly does seem to be a conflict of interest in reviewing papers that cite one's own work.

While I think it might be too onerous to have a norm against reviewing all such papers (as one goes further along in the discipline, it might become more and more difficult for one's work *not* to be cited), I do think something like the following norm would be a good one: do not agree to review papers that focus *centrally* on one's own work (either positively or critically).

What do you think? It seems to me that however effectively a reviewer might think they can resist the urge to be biased in such cases, there are good reasons to have a norm against reviewing papers that focus on one's own work (since the urge to promote one's own work *is* a conflict of interest, no matter how well one might be able to resist it. I would also mention here the implicit bias literature, which suggests that people are, in general, very poor at introspecting their biases).

Rachel

As I noted, I've reviewed a paper *deeply* critical of my own work (part of the title involves the paper being against me...by name!). It was a good thing it was sent to me because I noticed a lot of subtle but crucial misreadings of my work, ones that were rather serious errors. They would have been hard to see if it weren't me reviewing the paper. 'Rachel says x.' Uhhh, no, if you look on p 455, Rachel says ~x, actually. But I could easily recognize the errors because I know my own work really well. (OK, I did have to go back and give my own paper a close reading.)

If that paper had been sent to someone else, it's likely that they wouldn't have seen the errors.

Elisa Freschi

Marcus, do you mean "do not agree to review papers that focus *centrally* on one's own work"?

Marcus Arvan

Elisa: yes, that's what I meant. Thanks for catching that!

Marcus Arvan

Hi Rachel: Thanks for your reply. While I appreciate your feeling that you can fairly review work deeply critical of your own work, I worry about this line of thought -- particularly the message it sends.

Here, I think, is an important fact: the mere ability of some individuals to overcome conflicts of interest does *not* erase those conflicts, or give us any reason to allow people to self-regulate. After all, many people who think they are able to self-regulate in a non-biased manner are clearly unable to do so.

An analogy: historically, large businesses have trumpeted their ability to self-regulate. But, this is no reason to *let* them self-regulate -- as there have been far too many abuses.

Another analogy: a trial says she is able to overcome conflicts of interest, and so does not recuse herself from trials in which she has some. Would we accept this line of argument? Should we?

In short, even if you're able to resist conflicts of interest, I think it's a bad policy -- with serious moral hazards -- and so not one we should engage in or allow.

Rachel

Oh, I agree. I'm *not* saying that it wasn't a conflict of interest. It was! But the editor was well aware of it, and I did reflect on whether I could give the article a fair review.

I think there are two minimally important steps for noting a conflict of interest *and proceeding with the review*: The editor is made aware of it, and the referee has a very honest reflection on whether they can be fair. (The latter I suspect will result in many, many false positives...we just know this from a litany of psychology studies.)

I think it would have been improper to do the review if either of those conditions were missing.

I've been a referee in two clear cases of conflict, under our description: the first I was *certain* who the author was (because they had sent me a draft for comments on the section where they discuss me--I didn't read the rest of the paper), the second was where the paper was directly on me and was deeply critical.

In the first case, I let the editor know. We agreed to proceed with the review. In the second case, the editor knew that they were sending a paper on me...to me.

Neil Levy

Marcus, I think you're thinking of reviewing inappropriately. Say that Rachel receives a paper on her views and because she is so attached to them she rejects the paper. As you say, no matter how well-motivated and scrupulous she is, she can't rule out the possibility that her verdict was influenced by her understandable attachment to her own views. Suppose that her verdict is sufficient to get the paper rejected. But suppose that she has also otherwise fulfilled her professional obligations: she gave reasons for her rejection which (while not actually sufficient - we are supposing - to justify the rejection) are real reasons and she picked up problems with the presentation of her own view which others would have found it difficult to see. Now (other things equal) the original author has received a benefit: she is able to improve her paper and send it somewhere else.

You need to think of journal submission as a multi-round process, through which papers are given scrutiny from a variety of perspectives. All perspectives are biased, but - hopefully - the biases are different. When you think of submission like that, biases become less important.

At least that's how it should work. I grant it doesn't work like that, all too often. Some problems are fixable and some are not. Among the ones that are fixable are the horrible turn around times at journals (for which I think referees are largely responsible). The multi-round game works only if the review times are reasonable. Unfixable problems include the fact that some biases are overrepresented in the profession, because some views are seen as pretty obviously wrong. That's a problem, but its the problem of normal science (its a problem that is also a background condition of progress). Somewhere inbetween is the problem of underrepresented biases which are within the intellectual pale. These problems make the picture far from ideal. But in principle biases and conflcits of interest aren't that big a deal because submission is a multi-round process (it's also a problem, I should mention, when unreasonable pro-biases prevent the game being multi-round, by leading to acceptance of a paper prematurely).

Marcus Arvan

Rachel: thanks for your reply. I'm still worried, though. You suggest it's okay to go through with the review as long as the editor knows of the potential conflict and the reviewer honestly believes that s-/he can give a fair review. But then you note the empirical literature on how poor people are at recognizing their own biases, and admit that your schema will likely lead to many false positives (viz. people think they can give fair reviews when in fact they can't). Isn't this ethically problematic? Shouldn't we (morally) prefer a system that is less likely to give rise to so many false positives?

Marcus Arvan

Hi Neil: thanks for your comment. I do appreciate the notion of review as a multi-round process. In fact, although it was difficult at times, the 8-year process it took to get my paper on nonideal theory accepted was *incredibly* valuable. Reviewer comments helped tremendously over the years in developing and improving the paper.

That being said, I'm still apt to think that we should try to *reduce* conflicts of interest in the (multi-round) process. For instance, the reviewer who left his name on the "properties" section of his review of my paper gave no *argument* for rejecting the paper. He just stated that he thought the entire project was misguided because it conflicted with his own *conclusions* on the topic. In my view, no good came from that. All it did was waste four months of my time where the paper could have been under review elsewhere, with a less biased reviewer. In other words, while I agree with you by and large, it still seems to me that the review process tolerates *too many* conflicts of interest.

Rachel

Sure it's morally problematic. I think that's not in dispute. What I think Neil and I are saying is that even though it's morally problematic, it's something like the best of a bunch of imperfect options.

For what it's worth, though, I embed some sort of virtue epistemology into the referee has a very honest reflection on whether they can be fair." The worry of false positives is diminished in that case. Not all evaluators, even when they give an honest go, are create equal. Some are more skilled at detecting their own biases as others.

Verify your Comment

Previewing your Comment

This is only a preview. Your comment has not yet been posted.

Working...
Your comment could not be posted. Error type:
Your comment has been saved. Comments are moderated and will not appear until approved by the author. Post another comment

The letters and numbers you entered did not match the image. Please try again.

As a final step before posting your comment, enter the letters and numbers you see in the image below. This prevents automated programs from posting comments.

Having trouble reading this image? View an alternate.

Working...

Post a comment

Comments are moderated, and will not appear until the author has approved them.