Is it the business of referees to detect plagiarism?

There is an interesting discussion over at NewAPPS about how “blind” (if at all) the peer review process in philosophy is. I’d like to ask for your thoughts, my fellow pupae, about one comment in particular.

Clement writes:

When I perform reviews, I “google” phrases from the paper to test for plagiarism. This might account for some of the phrases coming up in the search history. I think every reviewer should perform such electronic searches. If doing so discloses the name of the author of the submitted manuscript, so be it.

What do you think? Should referees be in the business of detecting plagiarism?

Comments

[matt]

I don't think that reviewers should google any part of the paper because the name of the author will most likely show up. These are supposed to be *blind* reviews for a reason.

[Marcus Arvan]

Matt: unfortunately, as the NewAPPS post explains, blind review isn't so blind. I myself have several times submitted papers to journals and immediately had it reported on my academia.edu page that someone googled my paper title. Given that, in each case, no one had ever googled that paper title previously, the simplest explanation is that referees googled it.

[Lewis Powell]

Marcus, I am not sure how your reply addresses Matt's point. Even if there is widespread violation of the blind-review process, it might well be the case that reviewers shouldn't do that.

[Marcus Arvan]

Hi Lewis: I agree with Matt that they shouldn't do it. I just don't see any way to enforce the obligation. It's a very unfortunate situation...

[Moti Mizrahi]

Thanks for the comments, everyone.

Would all of you agree, then, that "detecting plagiarism" doesn't outweigh "blindness"? In other words, the risk of a plagiarized paper being published does not justify googling paper titles (and/or key phrases) and thereby jeopardizing the "blindness" of the peer review process?

[Lewis Powell]

Marcus,

I think it is important to be careful about saying that we can't enforce it. As I noted on the NewAPPs thread, it seems clear that there is no good way to systematically prevent those who are determined to subvert the process from doing so. That does not mean that there are no steps we can take which will lead to higher degrees of compliance. The enforcement mechanism need not be procedural (or punitive/legislative, as it were), in order to qualify as an enforcement mechanism. I think that at least some of the problem is that many referees are judging it to be a relatively minor infraction, rather than a serious concern. One way to shift that perception is for editors to be pro-active rather than re-active when it comes to assurances of blind review.

The re-active model is to ask a referee to review the paper, and if they say, "oh, I can't, I know who the author is", you then move to look for a new referee. A more pro-active approach would be to ask potential referees "Would you be willing to referee this paper, and if so, can you affirm that you do not know the identity of the author?". The main reason editors are unlikely to spontaneously adopt that approach is that it has a high risk of increasing their workload, in terms of people declining to referee.

That's just one thought off of the top of my head. My point is that, I think there are ways for referees to better enforce the obligation, even though someone who is determined to undermine the process could still do so.

[Marcus Arvan]

Lewis: I hope you're right, and I respect your point-of-view. Still, I'm skeptical. In all honesty, I have never once, when completing a review, attempted to discover the identity of its author. Why? Not because I was afraid of getting caught, but because I thought it would be wrong. I can't help but be a bit skeptical that people who are already morally willing to circumvent blind review would be honest with editors if asked the sorts of questions you mention. I expect that many people would assure editors that they won't look up the author and then do it anyway. This is in large part why I've advocated changing the entire review process into the kind of *public* review process used in physics (see for example my post at http://www.typepad.com/site/blogs/6a014e89cbe0fd970d015434353074970c/post/6a014e89cbe0fd970d016766a94c85970b/edit ). As I note there, the standard practice in physics today is to upload papers to a public repository: the physics Arxiv. As papers are uploaded, they are discussed publicly, on messageboards, by experts in the field. Something like a consensus on the paper's quality emerges on the messageboards, and (as I understand it) the actual review process at journals is something of a formality, given the interest the paper in question has/has not generated online.

Now, this isn't a perfect process. There are clear places for bias to infiltrate its way in. Still, physicists by and large seem happy with the process (a far cry from how philosophers feel about our process!), and articles by "outsiders" are positively evaluated on messageboards and then published in journals all the time. In other words, on the whole, this alternative process seems to work very well.

The way I see it, traditional peer review is no longer realistic. The internet has rendered it obsolete. There are just too many ways to circumvent it, and too many people who are willing to do so. Instead, I believe a transition should be made to the best *realistic* way to reducing bias -- which I think is what the physicists have settled upon. I, at any rate, would much prefer for my papers to be subject to public evaluation by my peers over the roulette wheel that is traditional peer review. But, as usual, I am happy to concede that I am probably in the minority here.

[Karl]

One would think that reviewers of papers in professional journals would be sufficiently expert in their field to know if an idea has appeared in the literature previously. They should not need to Google papers to detect real plagiarism.

[elisa freschi]

I am afraid I am missing a point: do you upload your papers on Academia (etc.) before they have been peer-reviewed? And with the same title? Then, how can you expect referees not to come to know about you? Why don't you wait a few more months, or at least modify the title?

[Lewis Powell]

Marcus, I think you underestimate the power of clearly expressed social norms. I think a lot of people think "lots of people do it, it can't be a big deal." In part, the system isn't structured to convey that it is a big deal. One way for editors to signal that it is a big deal to them, is to explicitly ask about it.

Another step is to explicitly train graduate students in how to referee papers, and reinforce the importance of blind review at that stage. This is also important for training graduate students to be good contributing members of the profession.

The point is not to safeguard against people who are actively attempting to subvert the system. That's not something we can do. But we could certainly attempt to shift the culture so that people take it to be a sizable wrong, rather than thinking it is a harmless infraction.

[Marcus Arvan]

Lewis: Perhaps you're right. I don't have too much faith in the motivating power of social norms. Maybe I should, but I'm skeptical. My experience is that even vociferously expressed social norms -- academic and otherwise -- tend to be disregarded by many people when they find it convenient. Sounds jaded, I know, but too much of my life experience attests to it...

[Lewis Powell]

Marcus, on this issue, I might recommend the chapter of Cialdini's "Influence: Science and Practice" that focuses on the principle of social proof.

[Marcus Arvan]

Lewis: cool, I'll check it out! One example I was thinking of is review times. Editors are continually emphasizing two month review deadlines to referees but 6 month to one year turnaround times are not uncommon at all.

[Marcus Arvan]

Hi Elisa: yes, I do upload them, and for a couple of reasons. First, I was advised by a bunch of people here on an earlier thread that uploading stuff to SSRN and Academia is a good way to avoid getting one's ideas ripped off -- or, at least, a public record of them. Second, I've found it's a good way to get paper comments. I realize this makes it easier for reviewers to figure out who I am, but -- for reasons noted in the NewAPPS thread -- it's already easy for them to do that. I tend to present stuff at conferences, which people can find even if the title is changed. My dissertation is also published online by my grad school, so anyone who searches, say, "nonideal theory" can easily figure out that a given paper is mine, as well. These are the basic reasons I've come to believe that traditional blind review is all but impossible these days. If a reviewer really wants to know who wrote a paper, then usually -- not always, but usually -- they can unblind the process.

[Lewis Powell]

Marcus: I actually think many editors drop the ball on that front in particular. If a journal wanted to convey to me that they took a particular timeline seriously, it would begin with them highlighting that timelime at the outset, reminding me as I was a week or 10 days away from the deadline, reminding me at the deadline, and then "hounding" me thereafter. From my experiences and those that I have heard from others, many journals do not even begin to contact referees until they are already a week or so over the deadline. This is sometimes driven by the fact that the editor is treating the referee as someone who has agreed to do them a favor, and it is bad form to harass someone who is doing you a favor. But, this is unfortunate, because I think a huge portion of refereeing delays come about from a referee, who is legitimately very busy (as we all tend to be), having the refereeing task fall off their radar. The editor asks you to referee, you think, "I have a month, I'll easily find a couple days to referee in that span." Three weeks pass and all of a sudden you have a bunch of other things on your plate, you forget that you are a week away from the deadline. Now it is two weeks after the deadline, and you suddenly remember. But, why weren't you reminded by the editor? In ways that are important to how you will treat your own violation of the deadline, the lack of follow-up is a signal to you, from the journal, that missing deadlines is (a) normal, and (b) a relatively minor problem. Now some journals are better about this than others. My guess (based partially in my own experiences as a managing editor) is that the journals that politely remind referees of the deadline as it approaches, and then continue to _act_ like they take the deadline seriously, bring down their average refereeing time significantly.

[Jenny S]

In my opinion, one of the disturbing features of the NewAPPS blog post comes near the end:

"Yep. It's true. I know. I know because people tell me. They are not shy about it either. They say that that's what they do. They don’t think it will cause them to make biased decisions. They just want to know whose paper they are wasting their time on."

There is a disturbing feature referenced here: the culture of academic philosophy, or perhaps more generally of academic refereeing, not only permitting lifting the veil of blind refereeing as a private practice, but allowing referees to discuss this practice openly uncensured. Telling the journal editor for whom you have contributed 'blind' refereeing labour that your refereeing was not blind? Telling a fellow referee that you know whose paper you 'blind' refereed? Telling a colleague how you googled key phrases? You should be embarrassed. We should all be embarrassed by the violation of the supposed blindness of refereeing. And, in order to change the culture, we should react. Although the non-blindness practice of googling is likely to persist, it should be called out every time it appears in a public forum or discussion or conversation.

There is a cultural shift needed here. It is akin to (although I do not mean to suggest moral equivalence) the cultural shift that we have needed to make it unacceptable to tell racist or sexist or homophobia jokes. One argument in that (much more serious) case was that private opinions would not be shifted by limiting free speech. And the related argument here is that the private practice will persist even if we begin to condemn the public expression of it. But my suggestion is the hope that in at least the cases of the racist joke-teller who does not realize her joke is racist, she will learn _that_ her joke is racist, and moreover that her opinion regarding its harmlessness is not widely shared. In drawing the analogy, the hope is that a public culture of condemnation of non-blind 'blind' peer review will help to effect a cultural shift. This practice is not okay; it does indeed harm some people; and those who search for the author, or otherwise unblind the process should be censured in order to shift the culture.

[Clement]

Clement here.

If an article that happened to be plagiarized were to be published because I recommended publication with a favorable review, then I would judge myself responsible (in some non-trivial sense) for the plagiarized publication. Maintaining my integrity as a reviewer requires that I test for plagiarism prior to recommending publication to an editor. This doesn't seem controversial to me.

The Committee on Publication Ethics (COPE) envisions that referees will report plagiarism to editors. See the flow chart here:

http://publicationethics.org/files/u2/02A_Plagiarism_Submitted.pdf

[Lewis Powell]

Clement,

I think part of why some people were taken aback by the way you phrased the original comment was that you seemed to view it as entirely unimportant whether your safeguards against plagiarism undermined the blindness of the review. Since you are agreeing to referee as a blind reviewer, then, even an acute sensitivity to the issue of detecting plagiarism should seem like something of a conflict in the obligations you have. My question to you is whether you wait until after you have composed your report (or, at the very least, settled on a recommendation to provide the editors) prior to taking actions which might lead you to learn the identity of the author of the manuscript you are reviewing.