Our books






Become a Fan

« How to know if you're improving at online teaching? (Guest post by Wes Siscoe) | Main | Tips for coming up with paper ideas? »

09/03/2020

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

Random R1 Professor

I'm fairly senior now and get asked to referee quite a lot -- though I'm also in a small subfield, so "a lot" isn't unduly burdensome. I very rarely say no. (In a typical year I referee four or five times as many papers as I submit, but that says more about how little I submit.) Occasionally I'll see that a paper approaches the subject in a way for which I have no patience, and I don't think it's right to review papers for which I can't muster even minimal sympathy. Otherwise I pretty much always say yes.

I can't imagine doing a good job with 30+ papers a year, and I certainly agree that the gate-keeping worry is a serious one.

Overseas TT

I'll chime in. First of all, for what it's worth, reviewing twice the amount of what you submit might be a better approximation. A submission is looked at by 2 referees in average, so a submission requires around two referee reports. This would justify that we referee two papers for every submission, not one.

Having said this, I get way more requests than the number of papers I should referee per year even by this more demanding standard, so I rule them out on the basis of a few rules of thumb (I'm putting aside obvious cases, like when I identify the author as a good friend).

1. Some journals ask me to referee much more than others. If I already refereed a few papers for them recently, I feel much less pressure to agree yet again.

2. I absolutely never, ever referee a paper if I already recommended its rejection at another journal. I hold myself to this rule religiously. So should you. No matter how terrible I thought a paper was, I strongly believe that it would be an abuse of power for me to give it the thumbs down at two different journals. I consider this something like an equivalent of ballot stuffing. I cannot begin to explain how infuriated I am when I'm on the receiving end of a copy-pasted referee report. It's the height of arrogance, and I'm amazed it's not obvious to everyone that you just don't do it.

2. I don't referee papers on topics on which I'm competent but which I find so boring that I can barely maintain focus while reading on it.

3. A bit more controversially, I'm somewhat less inclined to referee for journals where I feel like the editor is trying to manipulate the outcome. Sometimes I felt like editors didn't simply disagree with my judgment (which is their right) but were softly pressuring me to turn a rejection into an R&R.

4. Finally, I don't referee for editors who were in the past rude and disrespectful to me. This isn't common but it happened, I'm sad to say.

5. I'm not less likely to referee papers whose starting assumptions I reject. I don't feel particularly attached to the views I hold in philosophy, including the ones I defended in print, so this doesn't make it likelier that I'd reject something.

Tim

I go by the at least 2 reviews per my own submission.

Tom2

I basically review whenever I'm asked. I enjoy it. It's one of the more gratifying parts of being a philosopher and it helps me keep on top of the literature. With R&Rs it comes to about 2-3x / month.

Once I was reviewing an article for journal X and got a request from a different editor at X to review another paper. That one I turned down.

I agree with Marcus that it's important to be able to skim a whole paper before agreeing to review. Sometimes I can tell from an abstract that I'm competent to review; other times I can't.

One thing that has struck me as a reviewer is just how much competent work is out there. Even when I reject things, the papers are never disasters. They just have problems big enough to warrant rejection. Almost always there's an interesting and worthwhile nugget which can, with work, be made publishable.

I'm quite down on the profession these days, but I still love publishing philosophy--my own and, in a small and indirect way, by working as a reviewer. I know people complain a lot about the journals, but I think most of them do a great job and are one of the bright spots of the philosophy profession.

other opinion

Let me add a controversial opinion: I don't think you have any obligation whatsoever to review for journals and on the margin I think you shouldn't do it. It would be better for all of us if referees were paid. Why should we supply this unpaid labor while publishing houses make profits? It is not in our interest. We should not fool ourselves into thinking that we owe this labor.

Now others might add that refereeing is contributing to a public good and therefore one should contribute. I don't buy that argument. First of all, journal publications are by definition not a public good, because they are an exclusive good -- you have to pay for access to them. At best this argument works for open-access publications.

Second, even if we think that referees contribute to something good (rather than a broken and exploitative peer review system, as I tend to see it), that doesn't mean that they shouldn't be compensated for it. The people regularly cleaning up national parks should be compensated for it, even though they contribute a public good. The same would be true for us.

Third, you might argue that we are obligated to contribute referee reports because you also rely on other people doing so. I don't find that plausible when it comes to unpaid labor. Assume that I rely on textiles that are produced through unpaid labor, because those are pretty much the only ones available. Do I therefore also have an obligation to provide unpaid labor for producing such textiles? That seems ridiculous! It would just be better if the people producing the textiles would get paid -- and if you are a person who can produce textiles you should go on strike to make that more likely to happen. By analogy, you should not review for journals.

I also think we should get paid for our publications, by the way. I understand that prior to tenure we are incentivised to publish without direct payment, but why would we continue to give our valuable research away for free to commercial publishing houses once we have tenure? Now one might make a special case for open access, non-profit publishing, but that is still not the standard.

On a related point...

On a related point (Marcus - please feel free to remove if you think it not related enough), I'm interested in what people think of this: https://medium.com/@jamesheathers/the-450-movement-1f86132a29bd
The idea of asking for payment for peer reviews is not new, but this puts it in a new (for me at least) way. The author also says that they would only send this to journals that are run by large for-profit companies, so open access/society journals could be exempted from any requests for payments.
My thoughts are that something has to change, though whether this is the right route is another matter.

Original asker

Thanks for the responses. I want to focus the discussion away from questions of duty and obligations towards more prudential realms.

For instance, one of the papers I was asked to review is by an editor at a journal I know personally. I plan to submit a paper to that journal in the next few months. I recently (2 months ago) reviewed a paper for that journal. The paper is in my AOC, but not my AOS. Prudentially, should I just go ahead and do it?

Another paper I was asked to review is in my AOS in a top journal I have never reviewed for before. But it seems like more than I would like to do at the moment (given a few other reviews just completed). Nevertheless, I would eventually like to submit a paper to that journal.

I guess, my question is, given that editors don't know the background, when they get a refusal to review, should I worry about pissing them off? I am sure most are decent and not vindictive, but some are, and some are unconsciously more positively disposed towards those who 'played nice'. How much should I worry about being a 'team player' in editors' eyes?

Original asker

I will just add that one reason I don't want to discuss duties and obligations is that I am pretty sure I am fulfilling those (e.g. over 2 reviews per paper submitted).

Peter Furlong

My own view is that by the end of my career I should have refereed at least twice as many times as I submitted (since each submission tends to require two referees). In the beginning, obviously, I was submitting much more than I was refereeing, since no editors knew me or my work well enough to ask me to referee. These days I am getting too many referee requests to handle, and more than twice as many requests as submissions. (This is partially because I have spent most of my time the past few years working on books rather than articles, so my submission numbers have gone way down. This past year, though, the number of referee requests I receive has gone way up, too.) My policy on looking at a paper I have refereed elsewhere is that I refuse to do so unless the paper has been dramatically rewritten, and even in cases of dramatic revision, I let the editor know the situation.

other opinion

@Original asker

The prudential question is harder to answer because none of us has good data. I presume that none of us randomly selected a bunch of journal for which they rejected review requests and a bunch of journals for which they didn't.

I am personally not too worried about editors being vindicative. My hunch is that they have too many rejections and requests to deal with that they don't keep even a mental list of the baddies. But that is just a personal guess and comes without warranty. It also depends on how risk-averse you are. If you are more risk-averse the calculation might come out differently.

When the journal practices triple-blind peer review than the worry should be zero (assuming the process is not compromised). So if you want to reduce your refereeing load in the future (probably not applicable for the specific cases you describe), my prudential suggestion would be to reject requests from journals practicing this type of peer review.

Overseas TT

Just a quick comment: I agree in principle with "other opinion" that we shouldn't referee for journals for free, at least not for journals published by for-profit presses. But I think this raises tricky questions about collective action. Perhaps if enough of us refuse, refereeing at those journals will someday be compensated. But in the short-to-medium run, the only people whose lives we are making more difficult are the editor (who is normally blameless) and the author, who often struggles to secure a job, or tenure, and may be in need of a quick verdict. Still, I do agree with the policy in principle - I'm just skeptical that individual-level action is an even remotely efficient means of implementing it.

Evan

The argument for getting paid to referee is interesting. I suspect that if journals started paying for referees, they’ll be incentivized to be highly selective in *who* they want to referee or edit for them. I think this is because there can be free-riders who want money and may end up doing poor or biased reviews because they’re primarily motivated to make money.

Another unintended consequence that may arise is that those who really want to referee may not have a lot of opportunities because new checks and balances on refereeing might be created to prevent them from doing so.

I think the rationale is that by not paying referees, they’re motivated not by money, but by a commitment to high-quality or competent inquiry. It’s difficult to assess whether or not a referee is reviewing in good-faith as opposed to doing a poor review in order to make (easy) money in that scenario.

Then again, it’s probably not going to be ”easy money” since journals may require referees to meet other criteria in order to qualify to referee. As a result, this may increase the gate-keeping phenomenon.

Of course, one could argue that it wouldn’t make any difference in the quality of refereeing or articles published if journals started paying since there are poor-quality and problematic refereeing practices that currently exist.

One could respond to this by arguing that payment might *increase* the number of poor-quality refereeing or articles since people will probably just do the bare minimum of what is expected of them in order to get paid.

It really depends on who actually likes refereeing and has a sufficient level of epistemic humility to do so. Just something to think about.

Verify your Comment

Previewing your Comment

This is only a preview. Your comment has not yet been posted.

Working...
Your comment could not be posted. Error type:
Your comment has been saved. Comments are moderated and will not appear until approved by the author. Post another comment

The letters and numbers you entered did not match the image. Please try again.

As a final step before posting your comment, enter the letters and numbers you see in the image below. This prevents automated programs from posting comments.

Having trouble reading this image? View an alternate.

Working...

Post a comment

Comments are moderated, and will not appear until the author has approved them.

Your Information

(Name and email address are required. Email address will not be displayed with the comment.)

University of St. Andrews Grants


Job-market reporting thread

Current Job-Market Discussion Thread

Philosophers in Industry Directory

Cocoon Job-Market Mentoring Program

Categories