A very influential full Professor at a top-ranked program expressed concerns in my facebook feed about "anonymized" review that I've long shared: namely, that "anonymized" peer-review really isn't, and in ways that plausibly have elitist consequences.
What's the worry? The worry this philosopher expressed had to do with posting papers online and the way in which well-placed people get to present their papers at many conferences or move in circles where drafts are discussed. The concern, in brief, was this. It can be disadvantageous for an "unknown" person to post drafts of their papers online, but advantageous for a well-placed person at a ranked program to do so. Why? Among other things, because of "Google reviewing": the (unethical) practice of journal reviewers Googling paper titles, concepts, or phrases to figure out who the author is.
Indeed, I suspect I've had this happen to me many times myself, and know of many other people who have reported experiences just like my own: experiences of submitting papers to journals only to have my academia.edu analytics suspiciously reveal that the paper's title is Googled several times a few days or weeks later--this despite the fact that the paper hadn't been similarly Googled before submission (indicating, to me, at any rate, that someone--curiously, just after I submitted my paper--decided it would be a nice to idea to find out who wrote it!).
Although some reviewers may say they are "just curious" and that they can "be objective" despite Google reviewing, this is nonsense: we have biases, those biases are often insidious, and chances are, bias works against "unknown" people at low-ranked and in favor of well-networked people at top-ranked places. Indeed, a famous study in psychology, "Peer-review practices of psychological journals: The fate of published articles, submitted again", took articles by well-known authors that were already published in top-ranked journals, resubmitted them with fictitious names, and the vast majority of them were rejected, not because reviewers recognized they had already been published, but merely on account of their content (with reviewers often citing "serious methodological flaws"). Not only that, reviewers in other fields have been shown to be able to deduce authors' identities in anonymized review between 25% and 45% of the time.
And the problem in my view, is arguably much worse than this. The problem isn't merely Google reviewing. It is much more systemic. When I peruse my facebook feed, I see philosophers at top programs--including some early-career people who haven't published much, or at all--regularly moving in rarified circles: attending exclusive (sometimes invite-only) conferences or workshops, giving colloquium talks at other well-ranked programs, and rubbing elbows with some of the biggest names in the field. My noting these things might sound like envy--and maybe there is some latent envy there (who knows!)--but honestly, I wouldn't have a problem with it were it not for the ways in which I worry that it can corrupt anonymized review. For consider the following two people submitting their papers to a journal for "anonymized" review:
Person 1 works at a name program, and has presented their paper entitled, 'X', at numerous colloquia and conferences attended by influential people who specialize on the topic X is about. Many of these people know Person 1 personally, like them, and--if they were to receive a request to review X for a journal--would be likely to recognize the title of X or its general argument, thereby knowing (to a good approximation) that Person 1 wrote the paper. Not only that, let's say Person 1 (who, again, is at a name program) posts their paper online, so that other people who specialize on the topic know they wrote it and are from a top-program.
Person 2 works at a non-name program, hasn't had the opportunity to present their paper at conferences frequented by influential people on its topic, and either posts their paper online or doesn't.
Can we really expect "anonymized" review to function as it is supposedly intended--to ferret out and mitigate bias--in these cases? On the face of it, no. Given that Person 1 knows a lot of influential people--people who know they work at a highly ranked program--there are grounds for thinking that they may receive more favorable reviewers than Person 1: reviewers who know who they are, and are inclined (due to tacit, insidious biases) to judge the work of someone like them favorably.
And things are worse than this still. Suppose Person 1 and Person 2 are working on similar papers, on similar topics, with similar arguments (I know many cases of this)--and indeed, that Person 2 has drafted a paper defending argument X first. Suppose Person 1, knowing they are in an advantageous professional position, posts their paper online, but Person 2, knowing they are not, does not. Person 1 may be more likely to receive favorable reviewers than Person 2, which might help them publish their paper first. Indeed, Person 2 may receive a line of unfavorable reviewers, causing their paper to bounce around at journals for several years. Worse still, since they did not post drafts of their paper online (to "preserve anonymized review", Person 2 does not even have an online record that they actually defended X first. In other words, even though Person 2 defended a similar argument as Person 1 before Person 1, Person 1 may be more liable to publish on it first, and thus, be recognized as argument X's originator, "scooping" Person 2.
Suddenly, "anonymized" review isn't looking so anonymized! Are there any better options? I've long suspected that they physicists have it right (see here and here). Anonymized review doesn't exist. It's a sham--and, given that it's a sham, the most equalizing thing to do is have something like the physicists' arxiv: a place where (1) everyone uploads drafts of their papers before sending them to journals, and (2) everyone is expected to cite papers that appear there, regardless of whether they have been published yet. In physics, in other words, journal publishing is sort of pro forma. It still happens, but the main action takes place on the arxiv: it's where everyone uploads drafts, and everyone is expected to cite papers that appear there.
Although this practice effectively does away with anonymized review--since everyone knows what everyone is working on--it nevertheless works (imperfectly, to be sure) as an equalizer. On the arxiv, whether you are a well-placed person or not, if your paper is out there, there is a professional expectation that your paper will be cited. If Person 2 comes up with argument X first, then, even if Person 1 has an advantage in the peer-review process, it is still expected that people cite Person 2, because their paper was on the arxiv first. The arxiv system is a way--an imperfect way, to be sure, but, I would say a far better way--to mitigate for biases than our present system. Philosophy should consider implementing an analogue. Philosophers of science already have an analogue--the PhilSci Archive--and, just like in physics, it seems (to me, at any rate) to work splendidly.
Or so say I. What say you?