Our books






Become a Fan

« Weekend wavelengths | Main | Ferrin on 'Surviving the PhD' »

08/27/2018

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

Amanda

I know a lot of people are afraid to post papers online because of ruining blind review. Of course, I have argued it is commonly ruined already. However there might be a really awkward phase of starting this when peer review is the official system, as it will make it more likely that blind review will not be blind.

So how is the system you imagine Marcus? After a wide range of people discuss a paper unpublished, then this person submits a paper to a journal and it is reviewed, non-blindly, by one or two people? What is the point of publishing if everything is already online and discussed?

Trevor Hedberg

The bulk of the problem with the current peer review system boils down to journals being overwhelmed with too many submissions. (That is actually the only issue that Velleman discusses in the Daily Nous post of his that you link to.) That's what leads to long turnaround times and most other undesirable outcomes. Any full discussion of reform should consider whether there would be ways to alleviate this problem without transitioning to a full-scale overhaul of the system that's currently in place. Velleman's own suggestions involved essentially not allowing graduate students to publish at all. I echo the thoughts of many commentators on that post in thinking that this solution would be a step in the wrong direction, but it's at worth considering what options for reform are available.

Neil Sinhababu's proposal was to significantly increase the number of available journals. Details here: http://dailynous.com/2016/06/07/2000-spaces-for-10000-papers-why-everything-gets-rejected-referees-are-exhausted-guest-post-by-neil-sinhababu/

I imagine there are also other directions we could go, though I cannot recall any other open discussions about the possibilities right offhand.

Marcus Arvan

Hi Trevor: thanks for weighing in!

I share the concerns others have raised about Velleman’s proposal. But I’m not convinced that Sinhababu’s proposal is a promising alternative. There are already a lot of journals. The problem, it seems to me, is that regardless of how many journals there are, people will tend to swamp the same top journals with the vast majority of submissions.

I think there are two reasons for this. First and most obviously, people want to get their papers into the top journals—to get a job, tenure and promotion, reputation, and so on. But I don’t think this is the sole cause. A person is probably going to submit a paper to a top journal only if they think there is some chance it will get in. I don’t know about you, but if I was very confident a paper would never get into a top journal, I would shoot lower (indeed this is what I sometimes do).. But here is what I think happens in our current system. People don’t get a ton of feedback on their work before sending it out. They think to themselves, “Maybe my paper will get into Mind or PPR, so I’ll give it a shot.” The problem then is, when hundreds of other philosophers think this way, they *all* end up sending their papers to the same top journals—swamping them. So again, I don’t think it is likely to matter how many journals there are. The existing structure and incentives of our current system predictably lead people to all send their papers to the same few journals.

I think a transition to the “open” math and physics system would help solve this swamping problem in multiple ways. First, authors would be much more likely to learn from online feedback to their public preprints whether they are publishable at *all*—and end up submitting fewer papers altogether. For my part, I have learned through anonymized review that some of my papers were unpublishable. In an “open” system, I might have learned that without ever submitting my paper to any journals at all (as I might learn from online discussion that my paper has fatal problems, etc.). Second, online discussion of preprints would plausibly give authors a better idea of whether a given paper is viable for a top journal, or whether they should focus on lower-ranked journals. For example, one might learn that one’s paper isn’t as original as one thought, or has problems that are likely to lead to rejection at the very best journals—but one still might be willing to give lower ranked journals a shot rather than give up on the paper. Third, another reason journals are swamped is because of R&R verdicts—papers that sometimes need to go through two or even three stages of revisions before being accepted. An open peer review system would plausibly reduce the number and difficulty of R&R’s by virtue of authors getting better feedback on preprints (and revisions of preprints) online before journal submission.

Of course, this is speculative—but I think when compared to the alternatives (Velleman’s and Sinhababu’s proposals), there is more to be said for this one. Part of the reason I think this is that I have witnessed my own behavior as an author. There are some papers I wish I had never bothered submitting, only because I learned through peer review that they were unpublishable. In an “open” system, I think I probably would have never bothered submitting some of those papers.

The other part of the reason I think this is my experience as a reviewer—reviewing papers that seem to me obviously unpublishable, and which the author might have learned were unpublishable (or at least might have been vastly improved) if they had received more feedback from more people prior to peer review...which they plausibly would have learned from online feedback.

Trevor Hedberg

Marcus -- you may be right, but I think determining what solution is most practical is complicated. There would be significant transition costs to such a dramatic overhaul (as there always are when making a radical reform). I am particularly uncertain what the impact would be for early career scholars. Would peer-reviewed publications become somewhat less valuable? Would people who had stronger personal connections with big-name philosophers be more likely to get their pre-published work notably discussed, especially before this system became a universally practiced norm? Would this means of getting feedback on one's work disincentivize people from submitting to conferences as a means of getting similar feedback? If so, to what extent would that be a bad thing?

Thus, it's worth considering whether the current system could simply be tweaked or modified to ameliorate the problems. The Velleman and Sinhababu proposals are examples of that since they leave the basic structure of peer review largely in tact. I suspect that a milder change to the status quo would be easier to implement and also be more likely to gain widespread support from the profession at large than a complete overhaul of the whole system.

Peter

I know I am late to this conversation, but I wanted to chime in.

I also apologize for the length of this comment…it just kept growing.

There are some interesting worries raised in some of the comments, but I want to leave off discussing them at the moment to focus on the heart of Marcus’s original post. (This might seem odd since I am setting aside the question of whether we should do something to discuss how we might do it. I think further discussion should take place before anyone takes radical steps to transition things, but that is a matter for other posts.)

Marcus brings up some interesting things individuals might do to nudge the profession toward the model he proposes. So, for example we might start posting our own papers online before submitting them to journals; we might openly discuss the preprints others post; we might draw attention to the fact that we are discussing them; we might begin citing the preprints of others.

I think that each of these could nudge the norms of our profession in the direction that Marcus proposes. I can think of three worries with doing some or all of these:

(1) There is a worry that if we post our own work, we will hurt our chances of having our papers accepted at journals, since editors might not be happy either (a) that the work is already readily available or (b) that the blind review process might be impossible. Alternatively, we might just be worried that if referees knew who wrote our papers, they would take our work less seriously if we are not big shots. Additionally, by talking about the early work of others, we might bring these harms upon them.

(2) That whatever problems our system has, we would make things worse by nudging professional norms in this direction.

(3) That discussing the pre-acceptance work of others brings with it opportunity costs. Any time we spend blogging about the work of others is time we don’t spend writing, teaching, or just hanging out with our families.

(2) involves the issues I set aside here. I am assuming that (1) and (3) are not problems with Marcus’s model, but rather with taking certain steps to help the profession transition to that model. (3) turns on the value we might find in blogging about the work of others and on other aspects of our lives that vary widely (like how desperately we need to fill out our CVs or how little time we get away from work). Importantly, the value of blogging about the early work of others is likely to increase if the new model catches on, so the worry concerning it is greater before the model is widely accepted. (1), I take it, is a worry for many. Many people seem to worry about referees discovering identities by googling parts of their papers, and posting them online would certainly cause problems here. One might upload them with nonsearchable text (as scanned images or something), which would make it harder for referees to find your work, but won’t stop them from knowing your identity if they already read your work.

Whatever the value of these worries, I think they will stop a large number of people from doing as you suggest. I do wonder whether it would be easier to change things if one tried to start changing the norms of a relatively small sub-discipline, especially one with some distance from math and physics. I think that if some of these changes were to catch on even somewhat, and the effects of this change were not too bad, it would be easier to get others to change their practices as well.

Some of our subdisciplines are, perhaps, small enough that even a few people, even early career scholars (as long as they were productive enough) might get this ball rolling. If a small group of, let’s say, people in aesthetics or philosophy of religion or something started to post all their work publically, invite others to do so, link to the online work of others, and actively blog about such early work, I could see others following suit. Eventually this could provide an attractive and relatable model for other philosophers to follow suit.

Of course, one might say that there is no reason we cannot just model ourselves off of physics or math, but there are a couple barriers to this. First, many might worry that there are significant enough differences between these fields, if not methodological, then perhaps sociological, that might give us reasons to think that this model would not work for us at this point in time. Even if this worry doesn’t arise, seeing this model work in a philosophical subdiscipline, especially one that is not closely tied to mathematics or physics, would make it easier for other philosophers to see themselves changing their own practices.

Nichole Smith

Marcus, in response to the problem you identify with Sinhabahu's proposal, might the solution be a move away from general journals? My understanding is that the top journals are mostly generalist, and among specialist journals, the top of them are general within the specialty. Of course, generalist journals have a greater number of papers that could be sent to them, so the best of them get the most.

But what if there was no truly general option? Perhaps a "Miscellaneous" journal or dozen could exist for those papers that really just do not fit into a specialty, but the vast majority could go into specialty journals. And if the top specialty journals are still seeing too many submissions, they could further specialize.

I imagine this falls somewhere between just increasing acceptance rates and a switch to the arXiv model in terms of complexity in overcoming inertia. If there's a need that fully general journals serve that a constellation of specialty journals couldn't, I'd be interested to hear it.

Marcus Arvan

Trevor: you're probably right that the other alternatives are more likely to catch on--but as I've noted, I'm skeptical the more mild alternatives are likely to do much to solve various problems with our current model. In terms of your other questions (viz. effects on early-career folk, the value of publications, etc.), my sense is that in math and physics publications still matter (viz. jobs, tenure, and promotion), but that more important still is the level of uptake with one's work. For reasons I've explained (viz. quicker dissemination of one's work, greater online discussion, etc.), I'm inclined to think the math & physics model is likely to lead good work to be recognized more quickly--something that I think would benefit everyone, including early-career people (who currently have to wait months or years to get their work out there due to 'anonymized' review).

Hi Peter: Thanks for raising these issues. I worry about (1) myself. I'm not as certain about (2) or (3). You're certainly right that there are opportunity costs involved--but of course whether those costs stand against the proposal all things considered is a matter of weighing them against the benefits of the new system (which I've suggested are likely to be considerable, though of course I could be wrong!).

Marcus Arvan

Hi Nichole: I think we've probably seen the number of specialist journals explode in recent years for more or less these reasons. However, I'm not sure they really solve various problems with our current model (which I've argued are endemic to anonymized review in general). I'm also a bit wary of overspecialization in philosophy in general and how increasing the number of specialist journals further might add to that. But this is admittedly a parochial concern of my own.

Verify your Comment

Previewing your Comment

This is only a preview. Your comment has not yet been posted.

Working...
Your comment could not be posted. Error type:
Your comment has been saved. Comments are moderated and will not appear until approved by the author. Post another comment

The letters and numbers you entered did not match the image. Please try again.

As a final step before posting your comment, enter the letters and numbers you see in the image below. This prevents automated programs from posting comments.

Having trouble reading this image? View an alternate.

Working...

Post a comment

Comments are moderated, and will not appear until the author has approved them.

Your Information

(Name and email address are required. Email address will not be displayed with the comment.)

Current Job-Market Discussion Thread

Job-market reporting thread

Categories