Our books






Become a Fan

« How-to Post #2: How to Recover from Grad Student Traps | Main | Three links for MLK day »

01/18/2015

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

a referee

Hi Marcus,
I understand your concerns about refereeing.
But I do have some concerns with your remarks and proposals.
First, the way you talk of "the discipline" having a conversation is perplexing. There is no entity corresponding to this term. There is the APA. There are the many philosophers, young and old, submitting papers to journals (and sometimes refereeing). There are also editors and editorial board members. Never will the discipline have the conversation you envisage, because there is no such entity capable of conversing. Some of these parties can, but that is a different matter. So be clear about what you are asking for. If you express it clearly, you may find the answer is obvious why there is no such conversation (or perhaps you will find we are having it now, as you were having the conversation at the APA).
Second, the relaxing of the exclusivity requirement would be a disaster, and it would increase refereeing time. I have on numerous occasions been asked to referee a paper I just rejected for another journal. I always alert the editor of this fact, and decline to review it. In one case they pressed for a review at a second journal, I agreed, rejected it, and it was accepted anyway (in fact, in a very good journal).
But imagine how the submissions to journals would sky rocket without the exclusivity requirement, as you call it. Let us imagine instead Mind getting 500 submissions they get 1000, because everyone is sending their paper to 2 journals at once; or 1500 submissions, etc.
Third, too many people send papers to journals that are completely unfinished. Generally, philosophers should not send papers into journals until they have been presented at a (at least moderately) selective conference. Conference presentations allow one to refine one's argument. And the comments are apt to come from someone who is somewhat of an expert in the area of the paper.
Incidentally, I have refereed about 100 papers and I am on the editorial board of a non-philosophy journal.
Fourth, after a young philosopher has published a few papers, then she should aim to be more selective in her publishing, whether she has a tenure track job or not. No one cares if you have many many publications. Search Committees want to see that you can publish in competitive and selective journals. I ****suspect**** that philosophers tend to judge each other by their best piece(s).
You may ask: but are selective journals really better? I would say from experience that yes they are. They tend to get better referees who give more thoughtful comments, and who are masters of the literature being addressed. Of course, there are good papers in lower tier journals, and poor papers in very selective journals. But on the whole selective journals publish better stuff, and it is more likely to get read. And, at the end of the day, you want your work read.

Marcus Arvan

a referee: Thanks for your comment!

Of course there isn't a determinate entity that corresponds to "our discipline." Still, I would have thought the intent of the phrase was clear enough: namely, that (in my view) there should be much more, and more open, public discussion among people in the discipline--people here, on prominent blogs, APA committees, etc.-- about appropriate disciplinary-wide norms for journals to follow. Also, although perhaps I may be a bit dense here (which is always a possibility!:), but I'm not entirely sure why you think that once the meaning of the term is suitably understood, it may be obvious why there's little conversation on the subject.

In terms of your objection to dropping the exclusivity requirement, your suggestion that it would be a disaster presupposes that referees would fail to satisfy the norm in question in relativity large numbers. I believe, to the contrary, that the proposal--particularly if it were added to the first proposal (barring slow referees from submitting their own papers)--would generate great incentives for referees and journal editors to *comply*, in which case the consequences you give (journals getting deluged with papers) would not occur.

Finally, I generally agree with your point that after a young philosopher has published a few papers, then they should aim to be more selective--but I've also found that sheer number of decent publications seems to matter more than you think (each year I've had more publications, I've gotten *significantly* more and better interviews than the prior year--and the hiring data I've presented in the past seems to back this too). So, while I agree it's important to shoot higher after a few publications, there still are substantial incentives and time pressures for anyone in temporary positions to send a proportion of their papers to lower-ranked journals (viz. to at least give search committees impression of continued productivity--which again I've found, personally and in the data, to be important).

referee ... again

Hi Marcus,
One more comment to your original post. It is not really feasible to ban negligent referees from submitting to journals. Then the "right" strategic move is to always claim to be too busy, and then never referee.
It is very challenging to get referees. In my editorial capacity (albeit for a non-philosophy journal), I need to ask between 3 and 18 people before I get TWO referees. Further, among the Journal staff there is an understanding that you generally need to ask 6 or so to get 2.
We cannot afford to reduce the pool even further, by "banning" people from publishing who will in turn never agree to referee.

Marcus Arvan

Hi referee: Fair point, but what about the first part of the proposal, which is (A) that if the reviewer is asked to review for a journal and refuses, then they cannot submit to the journal for some period of time? (Note: this component could be plausibly tailored to not be too onerous for people who really *are* busy--for instance, by requiring them to at least say "yes* *once* per calendar year per journal if asked...which seems to me entirely reasonable, given the relatively small # of journals a busy, well-placed person might be interested in submitting to on any given year).

referee ... again

What you are describing would be an administrative nightmare.
Your heart is certainly in the right place. But your solutions seem heavy handed. I cannot even imagine implementing it.
Incidentally, (not to brag, but) I generally get my refereeing done with 1-6 days after being invited. I am sure that is why I have refereed about 100 papers (most in the last 7 years or so). I can be counted on to get it done.
Also, I am currently waiting to hear from a journal that I submitted to in August.

Daniel

I like the idea of trying to give referees incentives to both (a) agree to requests and (b) provide prompt responses once they have agreed, but I worry that the strategy of banning them from submitting if they don't would have some unintended negative consequences.

Editors don't always know who is qualified to referee a paper. In particular, sometimes people get requests to referee papers that, while broadly in their subdiscipline, they are nevertheless not competent to referee (or such that they would have to do a lot of background reading to become competent). What ought to happen in those cases, and what I imagine typically does, is that the referee turns down the request, and recommends somebody who knows the relevant literature better. Under your system, people would have incentives to accept referee requests even when they're not really competent to evaluate the paper.

Marcus Arvan

Hi Daniel: Thanks for your comment. However, the variation of the policy I just described in the final parenthetical to my last response to "a referee" might address that. All a person would have to to "not be banned" is say yes to a journal once per calendar year if asked. In my experience, editors do a pretty good job of selecting appropriate referees. I've had to turn down a couple of requests, but not many (and it seems to me a quick philpapers search can determine whether an author is qualified to referee a given paper). I don't mean to say that my proposal would be a perfect solution, obviously, but we should bear in mind that the comparison is to a *very* imperfect status quo! We should weigh the costs and benefits of both--and given the costs of the status quo are many (and other disciplines appear to have solved them pretty well), I fear rejecting proposals too quickly on the grounds that they would have negative consequences!

Marcus Arvan

referee...again: Maybe they are heavy-handed, maybe not. The point wasn't to say they are definitely good ideas, but to introduce a couple of broad suggestions for discussion. They may not be the best suggestions, and they may be revisable to be less heavy-handed (I've already suggested at least one way to make the proposal less so). In any case, I do think it's important to balance worries about logistical nightmares with (A) the fact that other disciplines appear to have implemented policies that solve the problems I'm raising, and (B) the costs of proposals should be weighed against costs of the status quo (which I believe to be considerable and weighted against more vulnerable portions of the discipline).

I do appreciate that you have experience working for journals that I do not--but I am also skeptical that philosophy journals can't do better given that journals in other fields clearly have.

referee ... again

I am not so sure that other disciplines have introduced *policies* that we philosophers have not. Rather, I think it is just that they have different disciplinary norms than philosophy does.
The non-philosophy journal on which I serve as an editor has the system set up so that a person's invitation to review a manuscript is cancelled after SIX days if they have not responded. That is, the prospective referee has SIX days to respond; then we just move on.
We also have the system set up so we can RATE the referee's reports. As an editor I assign a score to each referee report, and I can see how various prospective referees have been rated.

Marcus Arvan

Referee...again: I agree, norms are different. But I think policies can give rise to and support norms--and indeed, it seems to me that you just gave an example of two such policies: setting up the referee system to cancel requests after 6 days, and rating referees. Anyway, thanks for brining up the referee rating idea. That was another idea I had that I forgot to post on. Back when I was a musician, some sites had a function where users could rate referee feedback, and poorly rated reviewers were not permitted to submit work for review themselves. I suspect that something like this (done right) might work wonders for journal reviewing. There could be a system for authors and editors to both rate reviewers, such that poor overall score (taking into account both measures) could temporarily bar reviewers from submitting their own work for review.

Helen

Thanks for raising this issue, Marcus. It is painfully important to we untenured philosophers.

I suggest am empirical approach. Do other disciplines with shorter review times just have different disciplinary norms? If so, how could we cultivate those norms in philosophy? If not, what else do they do? I don't know what the best practices are, but copying success seems like a good strategy to get there.

Tom

"if a reviewer is (A) asked to review for a journal and refuses, or (B) accepts but takes longer than 4-6 weeks, they could be barred from submitting to that journal for, say, a period of one year."

This is not feasible, for various reasons. The most obvious reason is that people may be legitimately very busy, sick, in a personal crisis, etc., in which case punishing them for declining to referee by disallowing submission seems absurd. (I suspect that this would raise even legal issues.) And, to demand evidence from referees that they are in such circumstances seems equally absurd. (imagine editors demanding doctor's notes)

You also need to re-think the consequences of the proposal if it were universally enforced. One very likely consequence would be that many referees would then comply by doing the absolute minimum to avoid being punished, i.e. give their verdict without devoting any time to serious reading and comments. The second consequence is that the established people in the field can easily shrug their shoulders, stop refereeing and submitting (since they'll get invited to publish their stuff anyway), and the burden will once more be on the junior folks.

I appreciate that you raise this issue, and I agree with you that there is a very serious issue here. The current philosophy journal practices are by and large reprehensible and irresponsible, especially those at the so-called top journals, relative to the prestige that goes with publishing in those journals.

grad

Why not use some positive reinforcement with referees? For example, if a referee has a track record of good review times, then she gets her own papers handled by editors and sent out to reviewers before those with poor review times. That specific proposal might not work, but you get the idea.

NoNES Philosophy student

I have attended a seminar about journal publication for Humanities students. The speaker was a Modern Language professor. I told her about what I could do if I don't get a report from the reviewer. She was puzzled, asking me why I didn't get a report. I told her this's common in philosophy. Some journals even explicitly have a no report policy. She looked at me unbelievably.

Marcus Arvan

grad: Great suggestion-I am all for something like that! I'd be curious to hear what others think.

No-NES: not surprised! I think most philosophers just don't know how our discipline compares to others. I was absolutely (though pleasantly) shocked by how different my experience was submitting to psychology journals. Getting a decision with comments within 4-6 weeks *every* time? I could hardly believe it.

Marcus Arvan

Helen: Thanks for your comment! I'm glad you agree the issue is important, and I entirely agree with you that an empirical approach would be a good idea. :)

not_my_actual_name

On the topic of incentivizing, rather than punitive, proposals for increasing the pool of dependable and talented referees (which, I think, is the nub of the problem):

A few months ago, one of the Springer journals offered me a $150 book coupon (!) for sending them (what they told me was) a detailed, diligent referee report.

I can say with full confidence that I will referee for this journal again, even if I happen to be quite busy.

Of course, not all journals have the resources to do this. But many do, and perhaps enough do that it would make a difference.

Alex Guerrero

I think the biggest and easiest step would be to simply dramatically shorten the "deadline" for referee reports at all journals, and then send lots of electronic reminders.

I do a lot of refereeing. But I am also very busy (who isn't?). I try to do the report right when the request comes in. I succeed around 75% of the time. The other times, it gets buried, and I come back to it later, often around whenever the deadline is. What I find remarkable is how much variance there is in how much time is given. Many journals give you 1 month. Others it is 3 or even 4 months. That's just ridiculous. They should all be 1 month.

I know it is hard to find referees, but I doubt many would be dissuaded from having 1 month, rather that 3 or 4 (it's the rare person who would think, "well I absolutely don't have time now, or in the next month, but I'm likely to have time in 3 months: sign me up"). And I know it is much better for everyone if the guilt and electronic hassling comes much sooner, rather than later. I would expect near perfect correlation between turnaround times and referee deadline times.

Verify your Comment

Previewing your Comment

This is only a preview. Your comment has not yet been posted.

Working...
Your comment could not be posted. Error type:
Your comment has been saved. Comments are moderated and will not appear until approved by the author. Post another comment

The letters and numbers you entered did not match the image. Please try again.

As a final step before posting your comment, enter the letters and numbers you see in the image below. This prevents automated programs from posting comments.

Having trouble reading this image? View an alternate.

Working...

Post a comment

Comments are moderated, and will not appear until the author has approved them.

Your Information

(Name and email address are required. Email address will not be displayed with the comment.)

Open job-market threads