Our books






Become a Fan

« Application tips for non-faculty jobs | Main | Recycling dissertation text in articles »

01/13/2020

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

Douglas W. Portmore

An editor has asked you to review this submission because he or she values your expertise and judgment. I think, then, that you should tell the editor that you've previously reviewed the paper for another journal and that it hasn't changed from when you previously reviewed it. You can, then, tell the editor what, if anything, you're willing to do: (1) take a fresh look at it or (2) share your previous report. The editor can, then, decide how best to proceed. Perhaps, the editor will move on to other potential reviewers. Or, perhaps, the editor will want to see your report on the paper.

Anony

I think 10 pages is far too long for a referee report. That's not an efficient use of anyone's time: not yours and not the author's (to say nothing of the poor editors!) It is surely well-intentioned of you to provide so much feedback, but part of me thinks this misunderstands the role of the referee, largely for reasons Marcus gives, and that producing a report of that length is, itself, clogging up the process. Sorry :(

postdoc

I agree with Marcus that you should decline to review the paper again, that is unless your previous verdict was favorable. The reality is that we as philosophers agree on very little, and we agree on even less behind closed doors. You may think your comments are good and should be addressed, but the author, who may be just as qualified as you, or more, may disagree. Likely, there are referees who are also equally qualified who think you're wrong too, or worse.

More often than not after reading referee reports I come to the conclusion that few if any changes are necessary and just send the paper out again. Usually I then get an entirely different set of comments from a different perspective. If I tried to address all the comments the paper would be infinitely long. I have to use my own judgement about which changes to make and which not to make. Sometimes I may even rewrite the paper to focus it more if certain issues keep arising. However, from the author's perspective, even if the paper would be better if rewritten, this is not always obvious given the amount of disagreement in philosophy.

Sam

I find it hard to sympathize with the concern about ever-increasing words counts. It seems to me that something has gone wrong with your process of revision if you only ever slap on ad hoc fixes rather than rethinking and rewriting your paper more substantively. It's a recipe for the Frankenstein-like papers we're all forced to slog through nowadays. In my opinion it's such writing that constitutes a real disservice to the profession. But I see nothing wrong with sending out a paper for a second opinion before undertaking the difficult project of substantive revision. And I think Portmore's advice is correct for reviewers like the OP who seems to think their own judgment infallible.

Marcus Arvan

Sam: okay there’s a reason why it’s a running joke (across disciplines) that peer-review requires authors to turn their papers into unwieldy behemoths - see http://jasonya.com/wp/your-manuscript-on-peer-review/

A lot of times, different reviewers pick on different things, such that no single set of revisions or rewriting the paper will suffice...without adding a significant amount of words. This, at any rate, is my sense of things.

Also, although I respect Doug as a philosopher and editor, I think that editors should be no less sensitive to the kinds of epistemic concerns discussed here than reviewers. Sure, an editor may value a particular person’s opinion, but if the paper keeps getting sent to the same reviewer, then the peer review process for a given paper may be effectively beholden to that one reviewer’s opinion. Just imagine if the Nobel Prize winning papers in economics and the other fields I linked to above kept going to the same reviewers who found “serious problems” with them. The editors at the journals that rejected those papers were experts in their fields, as presumably were the reviewers they selected (given that the papers were rejected from top journals in their fields). Presumably, the editors at those journals wanted to hear the opinion of the reviewers they selected, just as Doug notes. And yet, for all that, the reviewers were wrong. I think we should all—authors, reviewers, and editors—have enough epistemic humility to recognize that even those whose opinions we respect the most can sometimes get things wrong. And so, for these reasons, I am inclined to think that editors shouldn’t decide whether a manuscript should go out to someone who already rejected it. If the paper is bad, presumably another person you trust will give a similar verdict—in which case you will at least have “cross checked” the reviewer gave at the previous journal. Conversely, if the new reviewer has a different view on the merits of the paper, you will have once again learned something...

a philosopher

"I recently heard someone (I think it was Mike Huemer) suggest this is less a problem with authors than it is with how some (many?) reviewers regard their task as reviewers. Many reviewers seem to think that if a paper has 'problems', then they should ask for revisions or reject the paper outright. But, Huemer suggested (and I agree), this isn't how reviewers should regard their job. Every paper, even many of the most influential ones in the discipline, have problems. The question shouldn't be whether a paper has problems, but whether it is worth thinking about and exposing a journal's readers to. Now, of course, if as a reviewer you find a paper totally uninteresting or its argument clearly bad, then sure, recommend rejection. But I don't think we should be rejecting every paper that has problems, or require every problem to be addressed in revisions. Problems are not in and of themselves reasons to reject papers. They may very well be interesting problems for the subsequent literature to find and address (e.g. in response papers, etc.)."

This can't be said loudly or often enough. I am perplexed by why more people don't take it to heart.

"I find it hard to sympathize with the concern about ever-increasing words counts. It seems to me that something has gone wrong with your process of revision if you only ever slap on ad hoc fixes rather than rethinking and rewriting your paper more substantively. It's a recipe for the Frankenstein-like papers we're all forced to slog through nowadays."

It might be that some objections are such that they can (and should) be addressed by some sort of reframing or reconceptualization of the argument which allows for the same point to be made (without raising the objection) in the same number of words. But it's just as often that this isn't the case, no? Often there's good reason to frame or conceptualize a paper in one way and the objection just amounts to a sort of confusion which can only really be addressed by adding on some words which explain why this interpretation of the argument is confused.

I certainly agree that no one should be making *those* sorts of changes, unless it's part of an R&R and the journal is likely to publish if they are made.

Chris

I'm sympathetic to Marcus and other's (Huemer's?) point about what referees should do. However, I do find it irritating when asked to referee the same paper and they haven't even fixed the typos I pointed out the first time around. Even still, I had the following experience a couple years ago: I refereed a paper for a top specialty journal: I recommended it for publication, but gave my opinion and why it was likely not to be ground-breaking etc." They declined to publish it. I was then asked to referee the same paper by another journal (with no changes to the paper). I told the editor I'd already refereed it but he wanted me to referee it again. So I sent essentially the same report but in this case the editor decided to publish it (and they finally fixed the typos).

More and more, when I referee, I try to give the editor my sense of what the paper accomplishes, but leave it up to them to decide what sorts of paper they want to publish. I rarely recommend R and R, in part for the reasons above about what the role of the referee should be, but also because unless the paper has a specific flaw that can be easily fixed, I don't think its my job to give extensive comments on how to improve the paper.

Douglas W. Portmore

I agree, of course, that "we should all — authors, reviewers, and editors — have enough epistemic humility to recognize that even those whose opinions we respect the most can sometimes get things wrong." But I don't agree that editors should never seek the opinion of someone who has already reviewed that submission for another journal. Now, Marcus worries about the need to "cross-check" the opinion of such a reviewer. But there are already several "cross-checks" in place. For one, editors almost never make a decision on a submission without at least two reports. So, the other reviewer's report can be used to cross-check the other reviewer. Indeed, sometimes we create two lists of potential reviewers and ask our managing editors to get one reviewer from each of the two lists, because the lists are of different sorts of philosophers with different points of view or different areas of expertise. And we do this precisely to serve as I kind of cross-check. For another, editors don't just accept whatever a report says even if the report is written by someone whom we have the upmost respect for. Editors read the paper, read the reports, and reach a decision based on those reports and their own reading of the paper. Sometimes, we'll go forward with a paper that doesn't receive two positive reports. Sometimes, we'll seek a third report if we think that a report is unhelpful or unfair. So, although we do seek out the opinions of those whom we trust, the trust is not unconditional. Indeed, we expect reports to justify their opinions and not just to state them. But, all of that said, there are times when I really want to hear from some select group of people and I would hate for them to just automatically turn me down because they've reviewed the paper previously. And, in such a case, it seems that the best way to proceed is to let the editor know that you have reviewed the submission before (without telling me what your verdict was or what journal you reviewed it for) and then let me decide given all that I know about the situation how best to proceed in this particular case.

Marcus Arvan

Thanks Doug, for sharing how things work. Speaking both as an author and reviewer, I think it is super helpful to hear an inside perspective on these things. I was strongly inclined to think reviewers should just decline in cases like this, but now I’m a bit less sure about that—and think I’ll consider the approach you defend. I’m also curious what other people think now, particularly other commenters who were inclined to think that declining to review is the right thing before your most recent comment. Has Doug’s inside perspective changed your mind?

HP

Simple solution: agree to review, but provide the same comments. Maybe give an R&R, and if they are addressed accept it.

But I can't see what would be wrong with just giving the exact same comments if it didn't change.

Overseas TT

I'm also with those who think that one shouldn't re-review the same (unrevised) paper. In fact, I take this so strictly that when I decline I don't even explain my reasons, because I think authors are entitled to have their submissions treated with no prejudice of any sort on the editor's part (including knowledge that the paper was rejected before.)

One of my reasons is what others mentioned, namely, epistemic humility. I often feel very confident about my reviewer verdicts. But I also have plenty of higher-order evidence that this confidence is delusional. Also, on the few occasions when I received the same reviewer report I was quite outraged because I perceived it as a kind of hubris. I usually take reviewer reports into consideration before sending something to the next journal, but sometimes I don't - not out of time pressure or laziness but because after carefully reading the report I judge that it's just bad and doesn't merit addressing. Some of the reports I ignored were very long and were clearly written conscientiously. Nonetheless, I thought they were utterly misguided and simply didn't think that addressing them in any detail would make the paper better. Of course, I may well have been wrong. But that's the nature of our epistemic situation: often the referee thinks the author is dead wrong, the author also thinks that the referee is dead wrong, and there's no epistemic higher ground from which to settle the matter. Therefore, as a reviewer I always tell myself that I'm highly fallible and that the author deserves a fresh look by someone else, even if I was very confident about what I wrote.

The second reason I think one shouldn't re-review a paper is that I consider it a sort of power abuse. Reviewers hold a significant amount of power over the fate of a paper: not over whether it gets published (because with enough patience, eventually everything will get published) but over whether it gets published in a top journal or at least a well regarded journal. Especially in the case of the most tippy-top journals, given that there are so few of them I think it's deeply unethical to concentrate so much power in one's own hand. If you thought the paper was full of errors or even outright sucked, and yet the author didn't change it, it's safe to assume that the author was unimpressed with the arguments you thought were lethal. It happens; in fact, perhaps depressingly, this is the norm in philosophy. But you had your say already. So instead of stuffing the ballot, let someone else chime in.

Greg Stoutenburg

I have been in the reviewer's position a few times recently. My approach has been to read the paper, write the report--which may involve little more than uploading my previous report--and communicate to the editor that I've reviewed the paper previously and given a negative recommendation. As others have noted here, the editor wants the reviewer's advice. And as others have noted here, the inconsistency in reviewer feedback makes it difficult to separate signal from noise. I think the solution is that reviewers should do all of the above: review the paper rather than declining it, and simply tell the editor about the background.

Chris

For those who oppose re-viewing, do you also hold this view when your opinion of the paper is positive, and you recommend publication? I've had several such cases. Some of your reasons sound like they only apply if you're going to recommend against publishing a paper you've reviewed before.

Overseas TT

No, when I had a positive verdict I have no problem with re-reviewing.

someone

I don't see why being annoyed. Just copy-paste your comments and send them to the new journal.

Sam Duncan

I don't really find Portmore's argument or Chris's counterexample all that convincing. I don't think that editors generally have the level of knowledge needed to have the confidence in their decisions that Portmore thinks they can. I'll take an example. If I were the editor of a history journal I'd be pretty confident about my judgments on a Hegel or Kant paper and I'd have a very good idea of who the best people to review that paper were. On say a Locke or Heidegger paper I'd have a lot less confidence in my own judgment. I could probably tell if it were terrible and ought to be desk rejected but I wouldn't trust myself to make any more fine grained judgments. Though I would at least have some idea of who reliable referees were and what dogs they had in which particular fights. On a Wittgenstein or Nietzsche paper I'd honestly not only doubt my judgments of quality but I wouldn't be terribly sure of what referees have what biases. On a paper on al-Ghazali I wouldn't even know where to start. And I'd wager most people who work in history would say the same with the names changed a little. Now imagine the editor of a "generalist" journal (most of which are of course LEMMings as we well know). How many people out there are competent to judge papers on not just Wittgenstein, Kant, Heidegger, Locke, Nietzsche, and al-Ghazali but on topics that range from applied ethics to recondite speculation about possible worlds and the necessary features of objects as well? Even if they send the papers to area editors, areas like "political philosophy", "ethics", "epistemology", and so on are huge. As for the fact that most of us who wouldn't rereview a paper we've recommended for rejection would do so for one we recommended for acceptance... Well the question is which is more likely, that we wrongly recommend papers for acceptance or that we wrongly recommend rejection? If a medical test were way more likely to give a false positive for a disease than a false negative then it would make perfect sense for a doctor to accept a negative result but insist on retesting a positive one. The same policy goes for journal editors and referees. And I think there are all sorts of reasons the system is much more likely to give "false rejects" than "false accepts." For one thing there's the whole mindset of most referees which is to look for a reason to reject or to start with reject as the default with the paper needing to convince them it's worthy of acceptance beyond a level of reasonable (or often unreasonable) doubt. Moreover many journals either explicitly encourage referees to reject some incredibly high and artificially dictated percentage of papers or have rubrics that encourage rejection. Still others push toward reject in other more subtle ways. So given that we referees are almost all likely tests with very high rates of false rejects and a very low rate of false accepts it makes sense for us to take an accept result at face value but send cases where we get the reject result on for a retest.

Chris

annoyed because I spend time on my comments and not even fixing typos suggest they're not paying attention to my comments. Maybe I should only direct my comments to the editors?

Amanda

I find it annoying when people take typos seriously. I really do. The point of publishing and reviewing is to decide if a paper, in this very competitive environment, is worth brining out to the philosophical community. Whether or not someone fixed typos seems to have basically no bearing on that issue. The best I can come up with is that papers with typos are less valuable to the community. I find that an odd stance, since the typos will probably fixed in the editing stage, for one. And two, if it is a paper worth publishing in this very competitive environment, I find it unlikely a few typos would make it not worth publishing (unless, of course, the typos changed the meaning of the argument or something unusual. )

The reasons to reject because of typos (and I have seen people say they reject because of the typos not changed, as Chris is saying, or because they see typos in the original paper and deem it "sloppy.) seems nothing more than either bitterness or idiosyncratic preference for a certain kind of presentation. Bitterness "I wasted time on this and they didn't even change it." simply is not a good reason to keep a worthy paper from the community. It is completely irrelevant. As for presentation, "it looks so much better without typos." Sigh. I just don't see this as being very important if the paper is otherwise worth publishing. Unless, of course, the typos actually influence the argument or understanding of the argument. If they do influence that, but can be easily fixed, then if the paper otherwise is worthy of publication, it should be a conditional acceptance.

I wish reviewers could just focus on the arguments and be a bit more understanding to other philosophers. People might have lots of typos for so many reasons, often having to do with a lack of privilege and resources and cognitive disabilities. I also think a reason just of the kind, "I have 80 million things to do and fixing typos didn't seem a good use of my time" is a fine reason. I would prefer that someone not fix typos and instead spend that time prepping their class or helping a colleague by giving comments on their colleague's paper. Almost anything seems a better intellectual use of time than fixing minor typos. People will say: but that's a false dilemma, they can do both. Eh, maybe, but sometimes someone's life is jam-packed and the 20 minutes correcting typos is substantial. I think philosophers should prioritize things that are intellectually valuable, and typo correction falls very, very, low on that list of value, IMHO.

Sam Duncan

I also agree with Amanda about typos for the most part. The referee reports I've gotten back that fixate on typos have generally been not just critical but quite personal in their hostility. Usually the reviews that do that start by either recommending rejection because I didn't discuss four or five usually rather obscure but supposedly "crucial" or "vital" papers by one single author or lambaste me for being stupid enough not to know that philosophers X, Y, and Z have settled this issue beyond a doubt. In those cases I generally get the feeling that the person isn't honestly refereeing or even considering the paper but is just angry that I don't discuss him or have the temerity to suggest that he and his buddies might be wrong. Focusing on typos in those cases just seems like them fixing on whatever small bits of evidence they can to show that it's not personal. Even when none of this is the case fixating on typos is just snotty and pedantic. Obsessing about typos in a paper in most cases just strikes me as condescending and not showing the author the respect they really deserve as a fellow scholar. In these cases the referee treats the author rather like a student in a 100 or 200 level class.
Having said that, I do try to fix any typos anyone points out. After all, that's usually the only helpful feedback reviews of the sort I mentioned have. And if there are enough of them and they're significant enough that might be an issue for me as a referee. In those cases though the issue is that the typos are a symptom of a larger issue. Those papers are generally poorly written and hard to follow. But even in that case I won't point out every single typo (which honestly I regard as a waste of my time as a referee) but instead make some general remarks on the paper being hard to follow and needing more editing.
One final thought: Before we get too caught up on typos let's just remember that few 20-30 page papers are entirely without one typo, grammatical mistake, or the like. Heck even published books have them. I've seen typos in books from Oxford and Cambridge. And the logic book I picked out for my class, which I'll never use again, has a couple of really significant ones that I've spotted.

Another Philosopher

If a paper is poorly written and filled with sloppy typos, I think it's fine to reject on that basis, and I will continue to do so. Remember that reviewers are doing this for free out of the goodness of their hearts. I'm not reading through a mess of a paper to try to give comments on content.

Nicolas Delon

@another philosopher: you're introducing a confound. Would you reject well written papers that contain typos? If you reject on grounds of poor writing, unclear structure or messy argumentation, the question of typos seems orthogonal. As Sam notes, typos might be symptomatic. But the interesting question at stake is whether there's ever a reason to reject a paper on grounds of typos alone, when other factors are held constant. Even without rejections, referee reports that focus on typos when they don't hamper reading and understanding seems like a misdirection of the referee's utmost charity you describe. You can't say a paper is poorly written just because it contains typos, but many of us have received reports telling us our papers were like rough drafts when we spent dozens of hours polishing them but omitted a few typos.

Amanda

I agree with Nicolas. Rejecting a paper because it is poorly written is fine. The problem is that some who say a paper is "poorly written and filled with sloppy typos" might be assuming the latter because of the former. But those don't follow, except in really extreme cases. I think even in an extreme case where there were lots of typos, like, way more than normal, I would read the paper and if the argument was publishable, I would require the paper be fixed from the typos as part of revision. If a referee has a personal issue with typos, then don't review the paper. Once you notice too many typos, just tell the editor you decided you are not fit to review the paper and give it to someone else.

I am not sure if I call paper reviewing an act that is out of the goodness of one's heart. I think anybody who submits papers to journals is morally required to review them. Sure, we don't have a system that professionally demands it as a condition of employment. But it doesn't follow from this that therefore reviewing is charity instead of doing what is morally owed.

someone

@Amanda. I work in ancient philosophy. It occurred to me recently that I was refereeing a paper with lots of typos in the Greek words. (In fact, not a single word in Greek characters was without typos).

The paper had to be rejected on different grounds (bad arguments etc.), but I couldn't stop thinking that the number of typos correlates with the sloppy arguments

Prof. F

I work on a certain philosopher. There are a group of people in my sub-field who take themselves to have established definitively the basic philosophical outlook of that philosopher. I happen to think it's entirely wrong.

Unless I want every paper to be addressing this wrong view, I have to more or less ignore it. Oftentimes I'll have a footnote or two in which I explain my disagreements. These people review my papers, and say things like "Author doesn't consider that Philosopher X is motivated by such-and-such. She dismisses this possibility in a footnote. She asserts that philosopher X is motivated by __, but there is no reason to think that. This leads to systematic problems in the paper." I can see why they think that, because my approach is different. I oftentimes feel like they didn't even read the paper, since the paper is a sustained argument for my point of view. What they think are systematic problems boil down to a difference in what I take to be the basic problematic of philosopher X's work. So when I get comments like this, I simply send the paper out again, sometimes without any revision at all, and hope for a referee who either is not committed to these views or has enough epistemic humility to recognize a different approach as worthwhile.

I often will completely rewrite a paper in response to a negative review, even if it is excessively rude or over the top. You swallow your pride and do it. But there are other kinds of comments which simply dismiss as wrong views that oppose their own. If you find yourself writing things like "that's just not what X thinks" or "there's no reason to think that" or "X doesn't ever talk that way" in response to quotations in which X talks that way ... maybe you are being unhelpfully dogmatic, and your comments aren't as worthwhile as you take them to be.

Amanda

someone: hmm, well typos in Greek seem a bit different, because it might suggest that the person doesn't know the language well. I don't encourage typos. Even though I think it's wrong, I know that lots of people *will* assume typos correlate with bad arguments. So pragmatically speaking, everyone should try to avoid them, at least for the most part without being obsessive. But I also think that reviewers should make some effort to not unfairly hold typos against someone when the argument itself is of high quality.

Another Philosopher

There is a difference between a few typos or a single word being consistently misspelled and a sloppy paper with tons of typos. These papers usually have many other language problems in addition to typos. If anyone is rejecting a paper because of a few typos, they are behaving irrationally and immorally. However, I suspect that at least some of the people complaining about being rejected for typos in fact were rejected for sloppy writing generally.

Verify your Comment

Previewing your Comment

This is only a preview. Your comment has not yet been posted.

Working...
Your comment could not be posted. Error type:
Your comment has been saved. Comments are moderated and will not appear until approved by the author. Post another comment

The letters and numbers you entered did not match the image. Please try again.

As a final step before posting your comment, enter the letters and numbers you see in the image below. This prevents automated programs from posting comments.

Having trouble reading this image? View an alternate.

Working...

Post a comment

Comments are moderated, and will not appear until the author has approved them.

Your Information

(Name and email address are required. Email address will not be displayed with the comment.)

Philosophers in Industry Directory

Current Job-Market Discussion Thread

Cocoon Job-Market Mentoring Program