Anonymized peer-review and timely moral and political discussion

Readers may (or may not) have noticed that I haven't had anything to say about recent political events on the Cocoon. This isn't because I have nothing to say. Far from it. I have a lot of thoughts I would like to share. The problem is that most of them are under anonymized peer-review. Which brings me to several questions I initially broached last summer:

1. Does anonymized peer-review stifle timely moral and political discussion?
2. What are our obligations as authors and human beings vis-a-vis anonymized review in turbulent political times?
3. Should anonymized review be replaced by the more open form of public review practiced in math and science?

I want to open up all three of these questions for discussion, but let me say a few words first.

On the first question, I do not know of any clear evidence--besides my own anecdotal experience--of whether anonymized review stifles timely moral and political discussion. However, what I do have is my own experience, and I can clearly and strongly say that I have withheld (and continue to withhold at this very moment) moral and political arguments from public discussion because of anonymized review. Because I don't want to break anonymized review right now, I won't detail exactly how this is the case. What I will say instead is that it has been incredibly frustrating--indeed positively depressing--to feel like I have things to say as a moral and political philosopher but cannot say them, at least not without violating the professional obligations I have to the places I've submitted my work to (which require anonymized review). While I don't pretend that the arguments I have withheld from public discussion would "change the world" or anything like that (indeed, they might be horribly mistaken!), the very fact that I have felt like I have to withhold these arguments for months on end distresses me greatly. As we see in world events today, things can happen very fast: few predicted the Brexit vote, and even fewer the Trump election, and the consequences of these votes will probably play out quickly as well (as evidenced by how much has already happened in both cases). Peer-review, on the other hand, is dreadfully slow. A single paper can bounce around at journals for months or even years. Supposing it is a paper on moral or political philosophy (and relevant world events), whatever moral or political points the paper makes--and might have contributed to timely public discussion--may be withheld from public discussion until well after relevant world events (elections, political decisions, etc.) have unfolded. 

This brings me to my second question. What are our obligations as professionals and human beings in this sort of circumstance? On the one hand, some might suggest that if one is a philosopher and one has written on an issue of great moral or political immediacy, one's moral duties as a human being should take precedence over one's professional duties: one should bypass anonymized peer-review, post the work publicly as soon as possible, and contribute to public debate. Alas, I don't think things are this simple. For, on the other hand, given current norms in our profession--in which most papers do not appear to be widely read, cited, or engaged with unless they appear in top-ranked journals--an author in this position faces a dilemma, particularly if they are not at a prestigious institution. If they post the paper publicly (without it passing peer-review and appearing in a well-regarded journal), no one may read or engage with it--and, by posting it publicly, they compromise the very anonymized review-process that their paper needs to pass to end up in a journal where it may get read and engaged with! This horn of the dilemma incentivizes the author to fall on the other horn: they can place the paper under anonymized review and wait for the process to play out. But this just brings us back to our initial problem: namely, that by putting the work under anonymized review and respecting the anonymity of the process, the author withholds their ideas and arguments from public discussion for months, even years--undermining timely and open moral and political discourse.

All of which brings me to the third question: should anonymized review be replaced by the more open form of public review practiced in math and science? I have already argued many times that it should. The time and place for "anonymized" review is past. In the internet age, "anonymized" peer-review is not so anonymous anyway--indeed there are many ways that anonymized review is routinely compromised. So, why do we keep on keeping on with the same old review process? Academic math and physics have demonstrated that a different, more timely public review process can work--one where professional norms are to post work publicly before submitting to journals. 

In closing, I want to be clear that I don't begrudge any of the places I have submitted my work to for requiring anonymized peer-review at the present moment. I understand it is our current system, and so I have worked within it, waiting patiently (at least as patiently as I can) for the system to work. All I mean to suggest once again is that--for many reasons--we may want to rethink our discipline's publishing norms and practices. We live in a fast-moving world, and I think philosophers should move with it. We should want norms and practices that incentivize philosophers to engage openly in timely moral and political debate, not stifle it--and I fear (once again, from personal experience) that our current norms and practices may do the latter.

But I am only one voice, and as always I may be wrong. What do you think?

Comments

[R]

I for one post all my papers to academia.edu and philsci-archive.pitt.edu before sending them out to review. If I feel it important to safeguard anonymous review I change the title, but usually I don't even bother with that. I know plenty of people who do it this way so if you think we should all be doing it, I say be the change you want to see in the world.

[Marcus Arvan]

Hi R: I appreciate the sentiment, but I'm not convinced it addresses the problem I'm raising. I'm not just talking about posting papers to academia or the philsci archive. I'm talking about discussing unpublished articles and arguments *publicly* before and while they are under review at journals. This practice is totally standard in physics. I follow many physics blogs, and it is standard practice to post non-anonymized papers to the Arxiv and discuss them publicly on blogs before and while they are under review at journals. This encourages the timely discussion of new ideas. The same could be true if we had similar norms in philosophy. For instance, the Cocoon has anywhere between 700-1,200+ views per day. If we philosophers felt free to share our unpublished ideas here--including moral and political arguments related to current world affairs--then we would have here and at other blogs forums to disseminate and discuss new philosophical ideas and arguments in a timely fashion. But we don't do that. How many times has unpublished work been discussed here? Answer: almost never. And, as I say in the OP, I am currently withholding from readers a number of moral and political arguments that I think are relevant to public affairs because I am worried about violating anonymized peer-review--because under prevailing norms in our discipline, that's my duty: not to compromise anonymized review.

While I very much believe in "trying to be the kind of change you want to see in the world"--and try to live this ideal as much as I can--the problem with prevailing norms and practices in the discipline is that it is a real *risk* in this context to do so, particularly for untenured people who need to publish to get and keep their jobs. I worked really hard on 3-4 papers that are currently under review. At the present moment, if I started writing publicly on the arguments in them, then (since the Cocoon is a pretty widely read blog) I would run the risk of offending reviewer and/or editors who expect authors not to undermine anonymized review. And, to that extent, I run the risk of not publishing papers I think make valuable arguments. My very point in writing posts like this is to draw attention to just how problematic this is, and how our current norms incentivize people like me *not* to be the change we want to see in the world. I very much want to share my unpublished arguments publicly, but out of a sense of professional obligation I haven't. And I think that is terrible. Math and physics have shown us that this isn't the way things need to be--that academic fields and journal review processes can work *well* with a more public review process where unpublished works are posted publicly and widely discussed on blogs before being sent to journals. My hope is that, at some point--especially given how important timely moral and political discussion is that days--our field will finally follow suit.

[R]

Thanks for your reply Marcus!

I would still say go ahead and discuss your papers publicly, especially if you're convinced that's the way the field should go. It sounds like we're weighing the benefits of public discussion and the risks of compromising anonymous review differently.

I also wonder in what way you think public discussion on a place like here is different from discussion at professional conferences? I assume the papers you're talking about have been presented at one or more conferences before you submitted them, and it seems to me that given the specialized nature of most conferences, this is at least as likely (if not more so) to unblind you to the kinds of experts who are likely to be asked to review your work.

[Philosophy Adjunct]

Many comments in this thread seem to have disappeared, so I am re-posting.

I think the single most relevant fact about publishing in academic philosophy is that our reviewing procedures are completely and utterly broken. I came across this wonderful comment by Hintikka recently (from Jean-Yves Beziau's response to the controversy over his Synthese paper) which says it so much better than I could:

"The sad fact is that in our field the referee system has collapsed. (There are undoubtedly some exceptions and your journal hopefully is one of them.) It is bad enough that competent referees are impossible to find in sufficient numbers. The catastrophe is that the referees that major journals rely on do not act responsibly any longer. They do not try to understand the paper they are reading. Instead they are looking for excuse to form a recommendation without having to do any thinking.
Furthermore, those few referees who are using substantial standards normally belong to one of the numerous cliques into which philosophy and philosophical logic has split. The members of one clique do not know and do not care what adherents of the other cult are doing. The standards that a referee is using are those of her or his private club and hence idiosyncratic and ill-educated. The outcome is well calculated to guarantee that no new ideas are published.
I was not surprised to hear that Donald Davidson never submitted any papers of his to the refereeing process.
E-mail of Hintikka to JYB, July 1st 2011. "

This seems to me exactly right, and any proposals for where to go from here should begin with this.

The problem is not the anonymity of the author, it is the deplorable, reprehensible, behavior of anonymous reviewers. It is really mind-boggling the kinds of things we collectively let reviewers get away with. The solution, it seems to me, is to deanonymize reviewers. Let's require that reviewers sign their reviews. Let's make it a matter of professional norm to make public the names and woeful efforts of the bad actors, so that we all know who is responsible for the sorry state of affairs and can treat them with the appropriate opprobrium. I suspect this will quickly weed out those bad apples that have spoiled the bunch.

While we are at it this should also be coupled with a serious push back against the fetishization of peer-reviewed publication generally. Why shouldn't a piece in some non-peer venue (one with respectable editorial practices, and generally of good quality, whatever that amounts to) also count toward our productivity brownie points? This is a serious pathology, and one in large part responsible for creating the conditions that enabled the disaster that is our failed reviewing processes.

[Marcus Arvan]

Philosophy Adjunct: Thanks for your comment! Your post and other comments didn't disappear - I think you accidentally posted your comment over here: http://philosopherscocoon.typepad.com/blog/2012/05/publishing-practices-why-not-do-as-the-physicists-do.html?cid=6a014e89cbe0fd970d01b8d23aedfd970c#comment-6a014e89cbe0fd970d01b8d23aedfd970c

In any case, I largely agree with you. Although I have had some good experiences with editors and reviewers (and want to make that clear!), as I have explained on this blog before my experience is that the overall fairness and quality of the peer-review process is lower than in other fields I have some experience in. I have published work in psychology and my spouse works in psychology, and in her field (1) journal turnaround times are much quicker, (2) submissions are rarely desk-rejected without any clear justification from the editors, and (3) one does not encounter the 3-4 sentence recommendations without argument one often encounters from referees in philosophy (viz. vague, undefended referee pronouncements such as "This paper is too ambitious and the arguments are unpersuasive").

In short, I agree with you that our peer-review system needs reform--but I don't see why we should simply think there is one "magic bullet" to solve the problem. Why not have the much more timely and open system of math and physics, *as well* as your proposal?

[Philosophy Adjunct]

Sorry about that, obviously followed the link in your post and forgot which thread I was in :).

I agree totally, and I don't mean to suggest that there is a magic bullet to solving the problems with reviewing in our discipline (and that my proposal is it). The anonymity of reviewers is not the only incentive to act badly, but I do think it is the main enabling condition. Making the identity of reviewers known wouldn't, by itself, make reviewers more virtuous, as opposed to `merely' acting more virtuously, but even such `mere' acting would be a step to getting our house in order. My worry is that as long as reviewers can still act in the responsibility-free manner they do, and the need for reviewers is as pressing as it is then an arxiv type system won't provide much incentive for the bad reviewers to act better. If publishing in something like arxiv counted towards keeping our jobs then it would be a huge positive by lessening the need for reviewers, and thereby lessening the pressure on journals to rely on low quality reviewers. So I agree with you that both would be welcome developments.

However, there is the further question of how to address the fact that so many of our colleagues feel it is acceptable to behave so viciously (not all, of course, and not even the majority, but enough). I think that until we get a handle on what it is about our culture and practices that produces and encourages such behavior we won't really solve the problem with our reviewing procedures.