Publishing Practices: Why not do as the physicists do?

Anyone who has experience publishing in philosophy knows how frustrating it can be.  It can take several months for a journal to come to a decision (even at journals that generally have a good reputation for quick turnaround times), and whether one gets reasonable or unreasonable (and sometimes just plain mean) reviewers is a complete roll of the dice. Etc.

Is there a better way?  I keep up with particle physics in my spare time, and I've noticed that physicists do things quite differently.  Their publishing process has changed with the times.  Fifty years ago, they had the same process we have.  Now, however, they've done something very cool. Basically, everyone uploads their papers to a single online database -- the Arxiv -- before sending them to jounals. The Arxiv then functions as a kind of public review process.  That is, people in the profession pay great attention to the Arxiv and begin discussing papers even before they have been published.  In this way, people in the profession begin to determine publicly which papers are "good" and "important"...all before the paper has even been sent to a journal.  In essence (if I understand it right), this process makes the actual review process at journals something of a formality. Everyone already knows publicly which papers are good, important, groundbreaking, etc., so there is a smaller chance that unreasonable papers will be rejected by unreasonable reviewers (viz. the otherwise unreasonable reviewer often already knows that everyone thinks a paper is great, and so will be likely to accept it).

Now, one worry about this process is that it is less than fully blind -- yet I think there are several things to say here on its behalf.  First, blind review in today's day-and-age is almost impossible to ensure.  A simple web search is all it often takes for a reviewer to determine who wrote the paper they are reviewing (note: this need not be any fault of the author, as conferences may post paper titles online, etc).  Second, the Arxiv process seems arguably more fair (to me) than traditional "blind" review.  After all, once again, we all know how unfair unreasonable reviewers can be.  What could be more fair than the profession-at-large determining which papers are good, etc.? Aren't there at least some reasons to think that the profession as a whole is a better judge of quality than, say, one or two reviewers?  For a fun example (albeit with a book), see RM Hare's hilariously dismissive review of Rawls' A Theory of Justice.  Here are a few fun excerpts:

In concluding this not very sympathetic notice, it must be said that a reviewer with more ample patience and leisure might possibly have done better for Rawls. I have taken a great deal of pains (and it really has been painful) trying to get hold of his ideas, but with the feeling all the time that they were slipping through my fingers...The book is extremely repetitious, and it is seldom clear whether the repetitions really are repetitions, or modifications of previously expressed views. I have drawn attention to some of these difficulties, and there are all too many others. Rawls is not to be blamed for failing to keep the whole of this huge book in his head at the same time (the only way to avoid inconsistencies when writing a book); and still less are his readers. He is to be blamed, if at all, for not attempting something more modest and doing it properly.

Many years ago [Rawls] wrote some extremely promising articles, containing in germ, though without clarity, a most valuable suggestion about the form and nature of moral thought. It might have been possible to work this idea out with concision and rigour (Rawls' disciple Mr Richards has made a tolerably good job of it in his book A Theory of Reasons for Action, which is much clearer than Rawls' own book as an exposition of this type of theory). If Rawls had limited himself to, say, 300 pages, and had resolved to get his main ideas straight and express them with absolute clarity, he could have made a valuable contribution to moral philosophy. 

Wouldn't an online, public review process like the physicists' Arxiv be better than leaving publication decisions merely up to individuals? (Note: I do not mean doing away with journal review altogether, but rather supplementing it, as physicists do, with a more public review process).  

One final worry might be that a public review process might favor "popular" people in the discipline over the less popular -- but I don't think this has occurred in the physics world.  Quite the contrary, some of the most widely-discussed papers on the Arxiv have been written by "nobodies", thrusting them into professional discussion.  Finally, the Arxiv also seems to do a very good job of ensuring that bad/incorrect things are not published.  For example, the single most downloaded article on the Arxiv -- Garrett Lisi's "An Exceptionally Simple Theory of Everything" -- generated an immense amount of discussion, but has never been published because it was ultimately shown to be incorrect.

Anyway, I'm curious to see what you all think about all of this!

Comments

[Mark Alfano]

Interesting suggestion, Marcus. I'm all for the reformation of our hopelessly medieval peer-review system, but I'm not sure the physicists' model will do the trick. Here are two large disanalogies between philosophy and physics: 1) it's easier to churn out a mediocre philosophy paper than a mediocre physics paper, since we don't do big expensive experiments; 2) it's harder to evaluate the quality of a philosophy paper, since it's much harder for a philosophy paper to be "ultimately shown to be incorrect," as you say Lisi's paper was. These two facts (if they are facts) will make it much harder to sift through the work produced by philosophers, which would then make it harder for quality papers to stand out. When clear criteria for evaluation fail to do the job, heuristics are likely to be used to supplement them, such as the author's name-recognition.

I'm not sure how strong these reasons are, but they do make me nervous about moving to an Arxiv-type system.

[Marcus Arvan]

Good points, Mark, but I have to disagree with both of them. First, a great many physics papers in the area of "phenomenology" (i.e. speculative physics, of which Lisi's paper is an example -- string theory is another good example) aren't based upon experiments at all. They are actually much closer to philosopher papers than to experimental ones. Second, I don't think it's much more difficult to evaluate a philosophy paper. Many -- and probably most -- papers on the physics Arxiv are not "shown to be incorrect" like Lisi's. Most of them are not obviously correct *or* incorrect (and indeed, don't have clear criteria for evaluation -- note below), and thus have receive a great deal of discussion on physics blogs. Indeed, I'd like to highlight this last point. In part because of the Arxiv, physics *blogs* are really where most of the action is. People are constantly discussing and debating new Arxiv papers on physics blogs, and so the blogs play an *enormous* role in the discipline.

Side note: much of speculative physics (which is a very large part of contemporary physics) doesn't appear to have any more definitive standards or criteria for "good work" than in philosophy. Theories (e.g. string theory, brane theories, etc.) are judged by the profession in terms of a number of desiderata, and there is a great deal of disagreement in the profession -- as in philosophy -- on which desiderata are more important than others. Example: even though Lisi's theory has been disproved, some other top-notch people -- e.g. Lee Smolin and Leonard Susskind -- still think he might be onto something, and have been working on variants of it!

[David Morrow]

Of course you keep up with particle physics in your spare time!

Like Mark, I'd be happy to see changes in the peer-review system, but I had exactly the worries that he expressed about an Arxiv-like system. I'd be happy to be proven wrong, though. How/where does consensus emerge, when it does, about which papers are good?

If philosophers wanted to implement something like this, PhilPapers already has most of the infrastructure we would need. You can upload unpublished manuscripts, categorize them to bring them to the attention of interested people, and even discuss them right on the site. Some tweaks to the interface might improve the discussion, but the big problem is just getting people to do it.

In principle, an online review system could be just as blind as the journal system. We could even create a tradition of posting things under pseudonyms so that it's easier to talk about other papers, although I'm sure the PhilPapers folks wouldn't like it if we started cluttering their database with fake names. I suppose we'd need their cooperation to enable us to flag certain names as pseudonyms.

[Marcus Arvan]

David - I get the impression you don't believe me! ;) If you're interested, here are some great, accessible particle physics blogs to follow:

Matt Strassler's "Of Particular Significance"
Phil Gibbs' "Vixra" blog
Quantum Diaries
Tommaso Dorigo's "A Quantum Diaries Survivor"
And...for a total, but rather amusing crank (and one of the most reviled bloggers in the physics world), Lubos Motl's Reference Frame.

I'm also about to throw down a new interpretation of quantum mechanics that I just finished writing up (no joke!). I'm *very* excited about it.

Anyway, as far as I can tell, consensus arises on the physics blogosphere. Indeed, I can't emphasize this enough: *blogs* are where a great deal of the action is at in physics. The amount of online discussion that goes on in physics is truly fantastic. Basically *everyone* gets in on the discussion and consensus arises rather organically. It's pretty damn cool. A lot of discussion about stuff posted on the Arxiv also happens (or so I hear) at conferences. Anyway, as I see it, it's really great. As an analogy, it seems to me that the physics people have their "iTunes" where we philosophers are still screwing around with cassette tapes. I think it's time we caught up with the times!

[David Morrow]

I do believe you. It just seems fitting somehow that, in addition to all the other things you're interested in, you also keep up with particle physics.

I could imagine that social dynamics could also play a negative role in the online consensus-forming process. Do you know if physicists are generally satisfied with the process? If so, I'd take that as evidence that whatever problems exist are less than those that exist in our peer review process.

[Marcus Arvan]

All of the evidence I've come across strongly suggests that physicists are very happy with their system -- far happier than we are with ours. It's very open, obviously, and so much less dependent on behind-the-scene vicissitudes and luck than our system. I don't sense that personal dynamics play a big role, at least not any bigger than they do behind the scenes in our system.

[Buddythomas]

i find it frustrating trying to get published articles also, im on my final year at university and currently finishing my 10,000 word disertation on kinetic energy, it takes time and a load of research to get everything right, im hoping it will eventually end up published in the library as resource material.
owner of - http://www.kineticenergys.com

[Philosophy Adjunct]

I think the single most relevant fact about publishing in academic philosophy is that the system is completely and utterly broken. I came across this wonderful comment by Hintikka recently (from Jean-Yves Beziau's response to the controversy over his Synthese paper http://www.jyb-logic.org/synthes1) which says it so much better than I could:

"The sad fact is that in our field the referee system has collapsed. (There are undoubtedly some exceptions and your journal hopefully is one of them.) It is bad enough that competent referees are impossible to find in sufficient numbers. The catastrophe is that the referees that major journals rely on do not act responsibly any longer. They do not try to understand the paper they are reading. Instead they are looking for excuse to form a recommendation without having to do any thinking.
Furthermore, those few referees who are using substantial standards normally belong to one of the numerous cliques into which philosophy and philosophical logic has split. The members of one clique do not know and do not care what adherents of the other cult are doing. The standards that a referee is using are those of her or his private club and hence idiosyncratic and ill-educated. The outcome is well calculated to guarantee that no new ideas are published.
I was not surprised to hear that Donald Davidson never submitted any papers of his to the refereeing process.
E-mail of Hintikka to JYB, July 1st 2011. "

This seems to me exactly right, and any proposals for where to go from here should begin with this.

The problem is not the anonymity of the author, it is the deplorable, reprehensible, behavior of anonymous reviewers. It is really mind-boggling the kinds of things we collectively let reviewers get away with. The solution, it seems to me, is to deanonymize reviewers. Reviwers should be required to sign their names to their reviews. Let's make it a matter of professional norm to make public the names and woeful efforts of the bad actors, so that we all know who is responsible for the sorry state of affairs and can treat them with the appropriate opprobrium. I suspect this will quickly weed out those bad apples that have spoiled the bunch.

This should also be coupled with a serious push back against the fetishization of peer-reviewed publication generally. Why shouldn't a piece in some non-peer venue also count toward our productivity brownie points? This is a serious pathology, and one in large part responsible for creating the conditions that enabled the disaster that is our failed reviewing process.