Our books






Become a Fan

« June 1st Deadline! — MANCEPT Workshops in Political Theory, Session on “Ideal and Nonideal Theory” | Main | Is Moral Relativism Worth Taking Seriously? »

05/21/2012

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

Mark Alfano

Interesting suggestion, Marcus. I'm all for the reformation of our hopelessly medieval peer-review system, but I'm not sure the physicists' model will do the trick. Here are two large disanalogies between philosophy and physics: 1) it's easier to churn out a mediocre philosophy paper than a mediocre physics paper, since we don't do big expensive experiments; 2) it's harder to evaluate the quality of a philosophy paper, since it's much harder for a philosophy paper to be "ultimately shown to be incorrect," as you say Lisi's paper was. These two facts (if they are facts) will make it much harder to sift through the work produced by philosophers, which would then make it harder for quality papers to stand out. When clear criteria for evaluation fail to do the job, heuristics are likely to be used to supplement them, such as the author's name-recognition.

I'm not sure how strong these reasons are, but they do make me nervous about moving to an Arxiv-type system.

Marcus Arvan

Good points, Mark, but I have to disagree with both of them. First, a great many physics papers in the area of "phenomenology" (i.e. speculative physics, of which Lisi's paper is an example -- string theory is another good example) aren't based upon experiments at all. They are actually much closer to philosopher papers than to experimental ones. Second, I don't think it's much more difficult to evaluate a philosophy paper. Many -- and probably most -- papers on the physics Arxiv are not "shown to be incorrect" like Lisi's. Most of them are not obviously correct *or* incorrect (and indeed, don't have clear criteria for evaluation -- note below), and thus have receive a great deal of discussion on physics blogs. Indeed, I'd like to highlight this last point. In part because of the Arxiv, physics *blogs* are really where most of the action is. People are constantly discussing and debating new Arxiv papers on physics blogs, and so the blogs play an *enormous* role in the discipline.

Side note: much of speculative physics (which is a very large part of contemporary physics) doesn't appear to have any more definitive standards or criteria for "good work" than in philosophy. Theories (e.g. string theory, brane theories, etc.) are judged by the profession in terms of a number of desiderata, and there is a great deal of disagreement in the profession -- as in philosophy -- on which desiderata are more important than others. Example: even though Lisi's theory has been disproved, some other top-notch people -- e.g. Lee Smolin and Leonard Susskind -- still think he might be onto something, and have been working on variants of it!

David Morrow

Of course you keep up with particle physics in your spare time!

Like Mark, I'd be happy to see changes in the peer-review system, but I had exactly the worries that he expressed about an Arxiv-like system. I'd be happy to be proven wrong, though. How/where does consensus emerge, when it does, about which papers are good?

If philosophers wanted to implement something like this, PhilPapers already has most of the infrastructure we would need. You can upload unpublished manuscripts, categorize them to bring them to the attention of interested people, and even discuss them right on the site. Some tweaks to the interface might improve the discussion, but the big problem is just getting people to do it.

In principle, an online review system could be just as blind as the journal system. We could even create a tradition of posting things under pseudonyms so that it's easier to talk about other papers, although I'm sure the PhilPapers folks wouldn't like it if we started cluttering their database with fake names. I suppose we'd need their cooperation to enable us to flag certain names as pseudonyms.

Marcus Arvan

David - I get the impression you don't believe me! ;) If you're interested, here are some great, accessible particle physics blogs to follow:

Matt Strassler's "Of Particular Significance"
Phil Gibbs' "Vixra" blog
Quantum Diaries
Tommaso Dorigo's "A Quantum Diaries Survivor"
And...for a total, but rather amusing crank (and one of the most reviled bloggers in the physics world), Lubos Motl's Reference Frame.

I'm also about to throw down a new interpretation of quantum mechanics that I just finished writing up (no joke!). I'm *very* excited about it.

Anyway, as far as I can tell, consensus arises on the physics blogosphere. Indeed, I can't emphasize this enough: *blogs* are where a great deal of the action is at in physics. The amount of online discussion that goes on in physics is truly fantastic. Basically *everyone* gets in on the discussion and consensus arises rather organically. It's pretty damn cool. A lot of discussion about stuff posted on the Arxiv also happens (or so I hear) at conferences. Anyway, as I see it, it's really great. As an analogy, it seems to me that the physics people have their "iTunes" where we philosophers are still screwing around with cassette tapes. I think it's time we caught up with the times!

David Morrow

I do believe you. It just seems fitting somehow that, in addition to all the other things you're interested in, you also keep up with particle physics.

I could imagine that social dynamics could also play a negative role in the online consensus-forming process. Do you know if physicists are generally satisfied with the process? If so, I'd take that as evidence that whatever problems exist are less than those that exist in our peer review process.

Marcus Arvan

All of the evidence I've come across strongly suggests that physicists are very happy with their system -- far happier than we are with ours. It's very open, obviously, and so much less dependent on behind-the-scene vicissitudes and luck than our system. I don't sense that personal dynamics play a big role, at least not any bigger than they do behind the scenes in our system.

Buddythomas

i find it frustrating trying to get published articles also, im on my final year at university and currently finishing my 10,000 word disertation on kinetic energy, it takes time and a load of research to get everything right, im hoping it will eventually end up published in the library as resource material.
owner of - http://www.kineticenergys.com

Philosophy Adjunct

I think the single most relevant fact about publishing in academic philosophy is that the system is completely and utterly broken. I came across this wonderful comment by Hintikka recently (from Jean-Yves Beziau's response to the controversy over his Synthese paper http://www.jyb-logic.org/synthes1) which says it so much better than I could:

"The sad fact is that in our field the referee system has collapsed. (There are undoubtedly some exceptions and your journal hopefully is one of them.) It is bad enough that competent referees are impossible to find in sufficient numbers. The catastrophe is that the referees that major journals rely on do not act responsibly any longer. They do not try to understand the paper they are reading. Instead they are looking for excuse to form a recommendation without having to do any thinking.
Furthermore, those few referees who are using substantial standards normally belong to one of the numerous cliques into which philosophy and philosophical logic has split. The members of one clique do not know and do not care what adherents of the other cult are doing. The standards that a referee is using are those of her or his private club and hence idiosyncratic and ill-educated. The outcome is well calculated to guarantee that no new ideas are published.
I was not surprised to hear that Donald Davidson never submitted any papers of his to the refereeing process.
E-mail of Hintikka to JYB, July 1st 2011. "

This seems to me exactly right, and any proposals for where to go from here should begin with this.

The problem is not the anonymity of the author, it is the deplorable, reprehensible, behavior of anonymous reviewers. It is really mind-boggling the kinds of things we collectively let reviewers get away with. The solution, it seems to me, is to deanonymize reviewers. Reviwers should be required to sign their names to their reviews. Let's make it a matter of professional norm to make public the names and woeful efforts of the bad actors, so that we all know who is responsible for the sorry state of affairs and can treat them with the appropriate opprobrium. I suspect this will quickly weed out those bad apples that have spoiled the bunch.

This should also be coupled with a serious push back against the fetishization of peer-reviewed publication generally. Why shouldn't a piece in some non-peer venue also count toward our productivity brownie points? This is a serious pathology, and one in large part responsible for creating the conditions that enabled the disaster that is our failed reviewing process.

Verify your Comment

Previewing your Comment

This is only a preview. Your comment has not yet been posted.

Working...
Your comment could not be posted. Error type:
Your comment has been saved. Comments are moderated and will not appear until approved by the author. Post another comment

The letters and numbers you entered did not match the image. Please try again.

As a final step before posting your comment, enter the letters and numbers you see in the image below. This prevents automated programs from posting comments.

Having trouble reading this image? View an alternate.

Working...

Post a comment

Comments are moderated, and will not appear until the author has approved them.

Your Information

(Name and email address are required. Email address will not be displayed with the comment.)

Open job-market threads