Coincidence?

I made a post the other day calling attention to an anonymous editor's frank admission that many, many referees blatantly compromise blind peer-review by googling paper names and/or paper content. Everyone who commented seemed to agree that this is deplorable behavior. Well, to follow up, I have a paper on my website that no one had googled in months. I submitted it for review a few days ago. Today, two different people googled it. Coincidence? I suppose it could be, but I doubt it...

Now, I suppose I could take down the paper -- and that one could fault me for posting it online in the first place. But isn't this a classic case of blaming the victim? Sharing papers online has many legitimate purposes, not the least of which are: (A) getting feedback, and (B) posting for the purpose of the job market (so search committees can take a look at work one hasn't published, but which might be useful for evaluating a person's long-term promise). It seems to me that the duty falls on editors and reviewers not to compromise peer-review. I mean, it's just not that hard. I have never once googled a paper that I have been asked to review.

Comments

[Scott Clifton]

I'm pretty sure I was Google-reviewed recently. Over the last couple of years I've presented versions and parts of a paper at several conferences--two APAs, two specialty conferences, and accepted but not presented at two other specialty conferences. The crowds were knowledgeable about the issues on which I spoke and I got a lot of good feedback that I worked into the polished version that I submitted to a prestigious specialty philosophical journal late last year. The journal is located in a country other than the US and had an average review time of 3-4 months. In January it was rapidly approaching 4 months after submission when I got a Google notification that someone had searched my name from the country in which the journal is located. The title of my paper had remained unchanged since the last couple of conference presentations, so the title was easily linkable to my name online. One week later I got a flat rejection with reviewer comments to the effect that such a prestigious journal shouldn't be wasting its time reading such naive and uninformed work. After the sting of this kind of rejection had subsided, I started asking which is more plausible: that scores of audience members at several different conferences and in different kinds of audiences had failed to see that the paper was crap or that a reviewer had Googled my name, found out that I am an ABD graduate student in a non-pedigreed department, and decided that nothing I could possibly say would be up to the journal's standards.

I worry a lot about this, because you can't get more junior than a graduate student at a non-Leiterific university. Moreover, you can't get a TT job coming out of a non-Leiterific university without at least one peer-reviewed publication. So the deck is stacked against such people doubly--we have to publish in order to get a TT job, but it's hard to publish when reviewers can find out who you are and decide prior to reading the paper that it's not publishable.

[Marcus Arvan]

It's not the first time I've suspected it's happened to me either -- and I've had the same sort of puzzling experience. At just about every place I've presented this one paper, I've had people (some of whom are very well known and respected) tell me the paper should be published. But, time and again, I've gotten what seem to me to be very uncharitable reviewers -- often, just in your case, not long after a google hit or two from the country in which the journal is located.

Given how serious the problem of "google reviewing" seems to be, I'm quite surprised it hasn't become a bigger public issue in the profession. After all, it does seem to be blatantly done and well-known to editors, and a practice that disproportionately impacts the career prospects of those of us who are "lower on the totem pole" -- people who already face enough difficulties (high course loads, fewer colleagues, fewer travel funds) that put us at a disadvantage with respect to other, more well-placed people. I very much wish that someone in a position of influence (Brian Leiter, are you reading this?) would draw more attention to it. It seems to me absolutely scandalous that the practice occurs as much as it evidently does, and that we all share a duty to one another and the profession to see that far more is done to address it.

[Clayton]

I think there's been a discussion of this on Leiter's blog. It's a problem. I've encountered this quite frequently. Whenever this is discussed, there's a chorus of people who say (i) you should just not post your stuff online and (ii) it doesn't matter because referees can still be objective having peeked behind the veil. Totally maddening, but I thought I'd mention (i) and (ii) so that others don't have to. A useful first step would be to have editors tell referees that this is forbidden. A useful second step would be for the APA to set up guidelines so that we have a set of industry standards for good refereeing. A really useful third step would be for people in the profession to not be jerks.

[Matt DeStefano]

This is a bummer. How is one supposed to prove their philosophical chops if such a practice persists? One possibility, although I doubt this is the case for most instances, is that they searched online for your paper *after* making a decision. (I know how overly optimistic that sounds.)

[anonymish]

As an ABD at a not particularly sparkly program, I've worried about this too. I've gotten stuff into good journals in my AOS, but haven't broken through to the top generalist journals yet (but maybe soon: I have a promising R&R!).

I'll tell you: I don't put anything in progress on my CV or website (sometimes, with conference presentations, this becomes unavoidable though), in part (but not in full) because I figure if anyone sees where I'm from, this will bias them against me. Full anonymity is my best chance (this is not so with many others, and this is of course very frustrating).

But one thing you might do is put papers in progress on your website with titles removed to protect blind review. Or whatever else will protect you from google. You could even put a note for colleagues and search committees that you put these papers up for them and not for reviewers. I think the point would be clear enough, and Marcus's main reasons for putting work on-line would be satisfied.

[Scott Clifton]

Anonymish: The suggestions you make might work to some extent, but as the original post says, the reviewer who wants to know the identity of the author is quite clever at it, searching for lines of text as well as the title. I sense that the tenacity in these reviewers is similar to my own when I am trying to find online evidence that a student's paper has been plagiarized.

Clayton: I imagine you're right that reviewers who engage in this kind of thing claim that they won't be biased by the information, that they can recognize a sound philosophical argument when they see one, irrespective of author. But this practice makes me doubt whether these reviewers believe their own claim. If they did, why would they feel the need to know the author's identity? It seems more likely to me that the following is the case: Suppose I am one of these reviewers and I am given a paper to review built around an argument I find to be quite good. I give my decision that the paper should be accepted, after which I find that it was written by a "nobody." I may very well come under fire for letting such work into my prestigious journal. The flip side is probably true, too. I may judge an argument to be bad and then find out later that it was made by a "somebody." And since "everyone" knows that nobodies cannot construct sound and original arguments and that any argument made by a somebody is sound, I'd rather know beforehand who the author is than take the chance of outing myself as someone who cannot recognize sound and unsound arguments. I doubt that these reviewers recognize that this is going on, but I cannot see any other reason they would feel compelled to Google the paper's title to find out the identity of the paper's author.

[anonymish]

Scott-

Fair enough. If referees are that persistent, I'd say that's reason not to post the paper. I'm not really sure how much it helps if search committees see you've got a bunch of unpublished papers on your website. There's also the risk that they aren't as good as they should be. Usually, my papers continue to improve, right up to the point of acceptance (thanks to R&R work).

But that's small fries. This seems to be a widespread practice, and a bad one. It should be brought to light, and there should be publicly accepted norms against it. Will there be? Well, I agree with Clayton that people in the profession should stop being jerks, but I don't see it happening.

[Scott Clifton]

Discussion of this issue has come full circle, by the way. There is a new post on NewApps about it:

http://www.newappsblog.com/2013/02/self-restraint-in-refereeing.html#more

[Roman]

Scott--There could be one other reason a reviewer might want to know an author's identity. Presumably, the paper is in the reviewer's area, and it's interesting to know people who work in your area, even (maybe especially) if their arguments aren't very good.

[elisa freschi]

I might be too old (I am already in my thirties) to fully understand this discussion, but I cannot really grasp what is the problem:

1. If you do not want your paper to be found through google, just don't upload it.
2. If you think that the benefits are bigger than the risks, then upload it.

Personally speaking, whenever I have been asked to evaluate a paper, I never tried to find out its author through google, but it has never been hard to guess who the author was (or at least to infer where s/he had been studying), because s/he would refer to tons of his/her unpublished "forthcoming" articles and/or because one could easily detect a certain pattern of argumentation/a specific interest for a certain topic of author.

[Marcus Arvan]

Elisa: Goodness, you're making me feel old (I'm well into my thirties!). Anyway, you're probably right. I've always posted working papers online, but I'm now seeing that it has probably been a mistake.

[Roman]

I'm in my thirties as well. I think many of us "old" people put things online... Actually, here's a question that's a little off topic, but I've been wondering about it. I notice that many people have their own websites. So, let's say you have your own site and upload papers to it. Do you then also upload them to academia and to philpapers? (And SSRN?) Is there a more efficient way to do this?

[Scott Clifton]

"1. If you do not want your paper to be found through google, just don't upload it.
2. If you think that the benefits are bigger than the risks, then upload it."

Elisa,

This is one way that your identity can be found, but it's not the only way. As has been noted in the other discussions, conferences post titles and abstracts on the web and they stay there forever. If you present a version of your submitted paper at any conferences, there is a good chance that a reviewer will be able to find either the title (if it's similar to the presentation title) or choice sentences from the paper in the posted abstract. So in order to guard against this from happening, one must choose either not to present drafts of the papers at conferences or radically change the title and abstract. As Marcus points out above, it's sad that people are forced to make these choices--the one you describe and the one I describe--because they sacrifice valuable opportunities for feedback for the sake of reviewers doing what they shouldn't be doing.

[Marcus Arvan]

Hi Roman: I've tended to upload them at all three places, though for reasons that are obvious by now I wonder how wise this was on my part. One thing worth noting about SSRN is that (or so I've heard) uploading a paper there is a way to establish priority on ideas (such that if someone else publishes your same idea in a peer reviewed journal, you can still stake a historical claim to "getting there first"). This might seem like a silly worry to some, but of course there are a lot of precedence controversies in the history of science (e.g. The Newton controversy, Einstein/Hilbert controversy on the field equations for general relativity, etc.). I've told by more than several people in the know that even though SSRN isn't peer reviewed, it still counts as a "publication" at least in the sense of establishing priority.

[Roman]

Marcus: isn't it a pain to manage these? I have my stuff on academia and philpapers; adding a third place seems annoying. No? Just curious: who has told you about the establishing priority issue? My understanding was that as long as you have evidence of priority, that's good enough (the old system was: mail it to yourself so you have it in a timestamped, sealed envelope). Isn't any online posting fine for that purpose? Second, this is an interesting issue. While in the scientific cases things are relatively clear, philosophy seems like a field where proving that someone else stole your idea is much much more difficult--after all, the publications in the discipline consist to a very large extent of rehashing existing positions, or taking up others' position and publishing them with tiny alterations. So it seems like proving that somebody "stole" your idea would be extremely difficult.

[elisa freschi]

@Marcus: I did not mean to say "do not upload your papers", but rather "Be aware of the risks/benefits". Then, you might decide to upload them anyway (I have uploaded tons of my work either on Academia or on my webpage).

@Scott: I see your point. But is not it the case that we present papers at conferences (at least also) in order to discuss them? Is not then invariably the case that the final paper differs from the one presented at the conference? Changing title and abstract does not seem to be a problem to me, since I would have changed it anyway, in the light of comments/further thinking about the topic. However, it is very useful to be aware of the problem.