What bugs you as a referee?

One of the things I've learned over the years is that some ways of writing philosophy papers seem to go over better with referees than others. I've also heard that there are ways of 'writing like a grad student' that referees can spot, and which tend not to work well. This certainly coheres with my personal experience, as I feel like I had to unlearn a lot of writing habits in learning how to publish successfully in philosophy. Unfortunately, I had to learn these things the hard way--mostly by trial and error. So, sometime soon (maybe early next week), I'd like to crowdsource some tips on both of these things from readers--as I think a list of tips might help our early-career readers. 

Before we get to that, I'd like to crowdsource from our readers who have served as journal referees on things that bug them when they review papers--as I think this might provide greater insight into the tips we crowdsource next week, as well as help us all better understand what kinds of things to avoid. For example, a while back, Trevor Hedberg provided this list of some reasons why he rejects papers. If you review papers for journals, what sorts of things routinely bug you, and why? Also, do the things that bug you lead you to reject papers, or are they just things that you wish authors would avoid (and which might, let's say, lessen the need for issuing revise-resubmit recommendations?).

Comments

[referee001]

Here are some things:
-when a short paper tried to present four arguments - so each argument gets about 3 double spaced pages. Such arguments are never convincing. Write a paper on one of them of topics/arguments
-when an author discusses a view or issue - the topic of the paper - and they do not even address recent papers in the very journal they are sending the paper into
-when an author treats those they are criticizing dismissively, or attributes ridiculous views to them (I see, for example, people attributing stupid views to van Fraassen - of course he can be wrong, but if he looks stupid, then you probably misunderstand him)

[Mike]

Tips:

1. Don't be cute and snarky. Just don't. I hate this, even when I'm refereeing a paper from a proponent of "my side" of a debate.

2. Get yourself clear on your original contribution, get this whittled down to a single sentence, and put that somewhere front and center in the abstract and introduction section. (Also, once you have that statement, do a real lit check and make sure someone else hasn't already made the point.) About half the time I recommend rejection, I do it because the paper is simply not original. What's worse, I normally have to do the work of distilling what's the actual original contribution of the author.

For example, I often see someone give their thesis as something vague like "I develop a new argument for position X". Fair enough. I read on, only to discover that their "new argument" actually pulls together insights or premises already in the literature. Normally the author cites this work, so they aren't trying to hide anything. Still, when you draw so heavily from others' work, the question arises just what you've actually contributed. Sometimes the author is showing a genuinely unnoticed consequence of claims in the literature (great! tell me at the start that that's what you're doing), but other times the argument collapses into a mere repackaging of what was already there in the literature, or the author's "new" premises don't actually do any work beyond the premises borrowed from others.

In either case, as the referee (or reader of the final published paper), I shouldn't have to do interpretative work to figure out just what you're contributing.

This leads to:

3. Don't bury your contribution on the final page while overselling in the abstract and introduction. The referee will figure out your actual contribution, and is likely to just be annoyed that you oversold at the start.

[Overseas TT]

I second the point about not trying to be cute and snarky. It never works. A softer version I've heard from friends is not to be snarky unless you're Fodor. Nope, this is also cringe-inducing in Fodor's writing. Just don't. (I should clarify though that I never rejected a paper for this reason.)

OK, as for my part, the single most common reason I recommend rejection is that the author doesn't know the literature. A lot of people simply don't read enough. It's not enough to read the 10 most famous papers from the 3 most famous contributors. That way you miss out on a lot of important stuff and risk reinventing the wheel.

The second most common reason I recommend rejection (or sometimes R&R) is papers that make very small technical points in 35 pages. It's totally fine to make small literature interventions, they have their legitimate role in philosophy. But the paper's length should be proportional to its ambition.

[Overseas TT]

Also, since the opening question was what *bugs* us, here's something that bugs me (although I never recommended rejection for this reason either): numbered propositions. In that familiar meme this would be a case of lawful evil, when it runs from (1) to (87). It can be made more evil by also having (2*), (82***) and so on.

I mean, come on. On page 24 you mention (2), which was introduced on page 1, and you expect me to remember what it said? As I said I never rejected a paper for this reason, but I always recommended to replace numbered propositions with memorable names. Writing like this is almost as inconsiderate as reading out in talks...

[Mike]

Right, I was listing what *bugs* me (although, as I said, I will and often do reject for lack of originality). So, although I won't recommend reject just because of snark, at best you'll get a report from me where I say the R&R needs to get rid of the snark. Also, maybe it's just the base rate of bad papers being high, but the snarky papers I've seen are usually also unoriginal. Still, I do suspect that if you really know and appreciate the literature, and really understand well your original contribution to that literature, you're less apt to be snarky about it.

Also, I do get that a lot of people are snarky unintentionally. When you're just starting and don't know the lit well, it's easy to think you're being more original than you are, or to fail to appreciate the subtlety of someone's view. In that case you might write in a way that reflects as much, without intending it. I was there. I still sometimes do it, even after more than a decade now. We're all only human, but still, be on the lookout for it.

[Tom]

could someone explain to me what 'cute' means in this context? With examples, preferably? (is it the same as 'snarky'?) Asking for a non-native-speaker friend. All I associate with cuteness are cute cats.

[Andrew]

1. excessive footnotes with tangential arguments in them (generally I take this to be a grad student tipoff).

2. heavy reliance on obscure figures for the interpretation of more commonly read philosophers.

3. pointless translation mines.

[Michel]

Some of mine (although, like everyone else, these aren't things that really influence my verdict; they just put me in a bad mood):

1.) Left-aligned text (justified, please, come on!).

2.) Logic-chopping.

3.) Endnotes (makes it awfully hard to check the notes).

4.) Overly discursive footnotes. One or two is just fine; what bugs me is when most of the footnotes are discursive, and several are just walls of text.

5.) Numbered propositions (like Overseas TT), especially when the proposition referred to just disappears, never to be reiterated despite being referenced multiple time. Too many asterisked or primed propositions, too.

[Tom Cochrane]

I agree that numbered propositions are awful. Also the version of this where a principle is labelled say-'PHP' expecting me to remember what this is. Now I have to hunt back through the paper to find out what that means. Much better if it's something like 'the transitivity principle'. But even then, how hard is it just to use prose? Are you really saving that much space?

Pretending your claim is stronger than the evidence actually warrants. It's ok to be modest!

Just boring writing. And I think it's worth noting that if your abstract is boring, I may not even accept the review request.

[AMB.]

What bugs me the most: when the paper is a history paper. Like, I don't care what Hegel thought, get to the point. I'm supposed to be refereeing a philosophy paper, not history paper.

[Mateo]

Obscure arguments. I find this to be especially a problem in moral, social, and political philosophy. If, after carefully reading your prose, I don't have a clear picture in my head of what your argument is in premise and conclusion form then you have failed to do your job properly.

Often, behind the obscurity is an invalid argument or an implicit premise that is obviously false. Unfortunately many philosophers, including many reviewers, lack the logical discipline to properly interrogate obscure arguments and thus many papers with arguments that are unacceptably obscure get published (again, I am especially thinking or moral, social, and political philosophy).

I agree that most arguments should not be written out within a paper in premise and conclusion form (although some should). However, many authors need to practice the logical discipline of first writing out their complex arguments in this form to check validity and implicit premises before translating them back into prose to make the paper more readable.

[Mateo]

A grad student giveaway is to go through all the possible positions/reasons for/reasons against etc., spending a roughly equal number of words on each rather than ignoring or only briefly mentioning the ones that are uninteresting and unimportant and going into greater detail on the ones that are most interesting, relevant, and important.

[Postdoc]

1. Papers arguing for or against a position without properly defining or characterizing the position. This happens often and I always feel like the author refuses to do the groundwork that is required for an academic paper.

2. I once refereed a paper that concluded that proposition 1 requires the truth of proposition 2 based on the fact that many authors that defend proposition 1 also defend proposition 2. This annoyed me greatly.

[William Peden]

The only time I've been actively annoyed by a paper I've rejected was when the author(s) made a big sweeping claim about the literature that was clearly wrong, and which just reading beyond some of the classic texts would reveal to be wrong. It was analogous to saying "The only reason why anyone is a reliabilist is to solve Gettier problems."

The rest of the paper wasn't that bad, but it's annoying to have claims like that which a thorough literature review would avoid. I suppose the general point is "Do the reading!" Your referees will have spent years reading the literature - time that they could have spent sending off under-researched papers too...

[Wes]

What bugs me the most: when the paper is science journalism. Like, I don't care about all these biological case studies and facts, get to a point, and get to an ARGUMENT. I'm supposed to be refereeing a philosophy of biology paper, not a piece of scientific journalism.

[Overseas TT]

Someone asked what 'cute' meant. I don't think it's the same as snarky, and while I don't know what the original poster meant by it, I can say what I mean by it. Here are some examples of "cute" prose:

- Pop culture (esp. music) references
- Unnecessarily politicized examples in a paper whose philosophical content is far removed from politics
- Bad puns. That means almost all puns
- Evaluative expressions in the names of philosophical positions (this is often, but not always, overlaps with snarky): e.g. "lukewarm X-ism", "abject Y-ism", "hard-headed Z-ism"
- Referring to other authors by their first name (perfectly fine in a conference setting but comes across as trying too hard to appear as "one of the cool kids" in a paper. Incidentally, it compromises anonymous review.)

There are more examples. As I said, I never rejected a paper for this reason. In fact, I never even asked an author to revise a paper for this reason. I think prose like this is meant to look cool and hip, but to me it just comes across as infantile and corny. That said, I don't think it's my job as a referee to prevent people from writing in an infantile and corny way. It's merely something that bugs me.

[Mike]

Everything OverseasTT mentions is an example of what I meant by "cute", but I meant characterize it a bit more broadly. I think of being "cute" (in your philosophical writing) as trying to be hip, or just generally trying to be something (socially aspirational) you're not. Avoid inserting prose into your writing in some attempt to look sophisticated or suave. Example: If you are genuinely up on pop cultural, naturally handle it with fluency, and it's relevant to your topic, then go ahead and insert pop culture references. But if you have to force it, just ... don't. (Same thing goes for high-culture references, history references, mimicking those fancy-sounding turns of phrase you've read cool philosophers use, etc.)

[phil sci guy]

This might be area-specific...

With the overwhelming amount of metaphysicians now working on topics like "grounding" and "metaphysical explanation", one thing that's *really* gotten out of hand is metaphysicians who badly--and I mean, *BADLY*--misunderstand and mischaracterize very basic points that are related to philosophy of science.

These are generally bad enough to warrant rejecting the paper outright. But it is surprising to see how many outright egregious misunderstandings make it into publication.

A while ago I saw a paper trying to [redacted]...The paper mischaracterized [redacted] in ways that any competent undergraduate class covers in detail. Like, my exams for my undergrad phil science classes have questions on exactly these points; and they're not hard questions.

I sincerely wish that metaphysicians writing on topics adjacent to philosophy of science would actually learn what was already well-established 40 years ago before they submit their papers.

[TT]

Personally, I wish more papers were silly and/or snarky. Being silly and/or snarky is a gambit: if the point you're making is a good one and you're competent with prose, your paper becomes a more enjoyable read. If one or the other condition isn't satisfied, you look like a fool. But who can resist Russell's "Hegelians, who love a synthesis, will probably conclude that [the King of France] wears a wig" or Fodor's "Aunty regards Cognitive Science in much the same light as Sodom, Gomorrah, and Los Angeles" or even Davidson's whimsical example discourse "'The moon constitutes a severe threat to our security.' This was asserted yesterday by the Mayor of the Indian Ocean"?

One of the things I like most about our discipline is that we are allowed to present our research in a form which also displays the human personalities behind it. Long live this difference between philosophy and the hard sciences.

[grymes]

I would pick different examples than TT, but otherwise agree completely.

[Evan]

Michel: May you elaborate on what you mean by “logic chopping”?

[Overseas TT]

"logic chopping": Write a 10-page long paper that uses concept X. Say almost nothing to explain what concept X is. Instead, have a 35-page long formal appendix to show that the logic of X is sound and complete. If anyone expresses concern that you didn't really explain what X is, act all confused and insinuate in unsubtle ways that they just didn't sufficiently master the formal stuff.

Regardig phil sci guy's comment: I don't want to engage with this specific example because from descriptions like this it's pretty easy to identify the author, and I'd like to stick with Marcus' rule of maintaining a safe and supportive online environment. But I cannot resist noting that I've see shockingly naive misunderstandings of traditional M&E stuff in philosophy of science papers and talks so often that I got tired of even mentioning them. I think the moral is that we should all try to read and engage more broadly, but we should also cut each other some slack.

[Overseas TT]

Personally, I don't like any of the examples that TT mentions. I have to admit that Fodor in particular hits a nerve with me: I never understood why his writing is considered funny or witty or even just good prose. But of course I realize that at bottom this is a matter of taste.

My five cents is that the history of philosophy is full of wittily presented but bad arguments, and I suspect that those bad arguments would have been taken a whole lot less seriously if they had been presented in plain and matter-of-fact language, in the way we were taught in grad school. So if you can pull of being witty and funny (though I think most people who think they can in fact can't), it's good for your career but bad for the profession.

[Metaphysics lady]

Hi Marcus,

I’d strongly encourage you to delete the example from Phil sci guy’s comment. There are very very few people making proposals of the kind he suggests; many of us will quickly be able to identify a very small number of people it might be, and at least one of these people does not have secure employment. I think his point stands without the detailed example.

[Marcus Arvan]

Metaphysics lady: Thanks for bringing my attention to that. Done!

[Michel]

As I understand it, logic-chopping is unnecessary (and unhelpful) formalism, or formalism used to disguise or obscure an otherwise fairly trivial point.

It used to be referred to as the Rutgers-MIT school of paper-writing, but I don't know whether that was fair, or whether it described training there at a particular moment in time, or what.

But you know the stuff: you picked up a paper that sounded interesting, only to discover that reading it requires you to wade through thirty pages of increasingly subtly different definitions and distinctions, and lots of relatively complex logic. It's a slog. But--and this is crucial!--it's *not* a logic paper.

Overseas TT: If you think the metaphysics/phil. sci connection is bad, wait until you see what happens when people decide, late in their careers, that it's time to turn their attention to aesthetics (without bothering to read up on the field or the subject of the paper, of course)!

[Peter]

Some common things that bug me (and haven't been mentioned yet):

- Machine gun argumentation: when the author gives several extremely compressed arguments for a controversial claim that cannot be simply assumed or "motivated". Machine gun argumentation is often a sign that the paper is too ambitious.

- Unreflective assertions about a claim's intuitive appeal. Every paper appeals to some intuition eventually. But authors sometimes assert their intuition in a way that implies universally normative force. So, for example, an author might say "p is intuitive" when, at best, the author is warranted in saying "p seems intuitive to me" or "many philosophers working on this topic have expressed that they find p intuitive". Psychologists get into trouble for overgeneralizing actual empirical studies, but for some reason lone philosophers are regularly allowed to speak on behalf of all of humanity.

- Unnecessarily demanding prose. Authors often write sentences with seemingly little sensitivity to how they'll be processed by the reader. Here's an example. Generally speaking, readers have an easier time processing a progression of ideas when those ideas are presented in reading order, something like this: "Idea A. Because Idea A, therefore Idea B." But the author will write something like this: "Idea A. Idea B, because Idea A." Yeah, sure, there's no substantive difference between these two presentations. And yeah, sure, in some cases there are good, overriding, reasons to present things in the second way. Yet I suspect in most cases it's done for no reason at all.

[Prof L]

AMB: If you aren't interested in Hegel interpretation, why did you agree to review the paper in the first place?

[Overseas TT]

Good call, Metaphysics lady.

[TT]

Those who wish to depersonalize their intellectual contributions with sawdust prose are welcome to do so. It is, after all, easier to churn out papers that way. I just hope there will continue to be a place in the profession for The Rest Of Us.

Since my last post was about what doesn't annoy me rather than what does: I am annoyed when I receive a manuscript which "explores the consequences" of a thesis which no one has endorsed, which is not intuitively plausible, and which is not motivated by the author. I am *very* annoyed when I ask an author to explain why their contribution is different from that of so-and-so's published paper, and they don't answer the question in the R&R.

[Evan]

I prefer to read papers that are written sophisticatedly and yet accessibly. Unfortunately, many people (some philosophers) assume that writing sophisticatedly means writing pretentiously.

I’m inclined to think my own writing strikes a balance between the two above. Once and awhile, I’ll ask my friends on what they think of my writing style to keep me in check and consistent.

I think part of writing pretentiously stems from the fear of being perceived as unintelligent. For example, Foucault once said in an interview that he wrote badly because French people wouldn’t take him seriously if he wrote clearly. Perhaps some of that sentiment is still around.

I’m glad I rid myself of that habit early on in college. I don’t think I was the worst either. I figured: why repeat such a thing that I myself was subjected to when having read certain philosophical texts myself? These days, I try to set an example of what I want from others in terms of writing clearly. I think (future) students need more examples of clear and good writing to follow and modeled after.

[historygrrrl]

OK, I'll play. None of these have ever caused me to reject a paper - in fact, I've R&R'd or accepted papers that have done multiples of these.

General:

1. Piles of numbered propositions or non-obvious abbreviations. Do me a solid and remind me what TRM and MGDI stand for after ten pages or so.

2. Rabbit holes to nowhere. You make this really great case for a position early in the paper, but it seems to be irrelevant for what happens later.

3. Gender stuff. Try to change up the pronouns a bit...it's the future now. Also, misidentifying the gender of someone you're citing. I know, I too once thought that Hilary Putnam was a girl, but that's when we had slow internet and had to search using Dogpile.

Specific to 'dead-serious history of philosophy' scholarship:

1. Dead Philosopher says X in text T1, and you think it means Y. As evidence, you cite T2. But whether or not Dead Philosopher actually held the same views in T1 and T2 is a super-contentious issue - some people even think Dead Philosopher changed their mind! Please, please, please tell me why you think T2 is okay to use with T1. Even a footnote is fine.

2. Weird translation of a passage, or non-standard translation of a difficult passage, but no original language provided to check. This means, as a reviewer, I have to go find/drag out a bunch of heavy books.

3. Similar: Original language given for terms that don't matter, but missing in cases where it makes a difference.

4. Not being careful with what Dead Philosopher says vs. what they mean. I'm pretty sure Aquinas didn't say anything about neo-reducto-grounding or whatever the hot thing might be. He might have been committed to it, but that's a bit different.

[Mike Titelbaum]

Sorry to be late to the thread, but here's a plea to graduate students: Please don't just take a seminar paper that your professor praised and submit it to a journal as-is. I read a lot of submissions that are 15-20 pages, cite a weird subset of the literature that seems like it was a syllabus, and has a mostly-exposition-then-objection/counterproposal-near-the-end structure. I won't say this annoys me, and I won't reject a submission outright for having these features. But if your professor says you have a good idea, that's reason to put *more work* into the paper, developing it and bringing it into contact with more of the literature. That's not reason to submit it right away.

[Nicky Drake]

I'm curious why Michel objects to left-aligned text. That's part of APA style, and surely APA is a reasonable style for philosophy papers?

[Amanda]

Reading this thread is depressing. It just illustrates what a lack of agreed upon standards we have for what constitutes publication worthy philosophy, which partly explains why the publication process seems to depend so much on luck.

I like cuteness. Snakiness - it depends. I don't think people should be mean or do personal attacks. I know the posters said they don't reject because of "cuteness," but clearly, it influences their perception of the paper. That, in turn, influences whether it is ultimately an accept for reject. Moreover, even if you don't reject because of cuteness but you instead tell them to change things in the R&R - something seems wrong about this. The referee is forcing their style preference on someone else, when it is not relevant to the argument nor does it violate the journal's own professional standards. I don't think journal referees should pressure authors to accept the referee's idiosyncratic, epistemically irrelevant, style preferences.

I'm very confused about the left-alignment comment. I thought that was the norm. Justified? I have never once used that format, so if it is what I was supposed to be using all these years, I sure wish someone would have gave me the memo!

[Evan]

I agree with Amanda on “cuteness.” To be accurate, I would say I can be quite sassy and still make a compelling and coherent argument. The beauty of philosophy is that we are allowed to display a bit of feistiness so long as we can back it up with compelling and coherent arguments and clear writing.

My level of sassiness depends on the humility of my interlocutors. Sometimes, sassiness is a way for us philosophers to make each other humble and keep each others’ “epistemological egos” in check.

[an Editor]

I am an editor. Like Amanda, I think the norm is to send manuscripts in flush left. That is what I want as an editor. And I want double spaced as well. Make it easy for us and referees. And yes, our instructions say that. In the end, when a paper goes into production ALL the formatting has to be removed. So you are just making more work when you put in fancy formatting.

[Foreigner]

As a reviewer, I get annoyed by...

(1) The 'I just learnt how to use Latex!' formatting. You know, Computer Modern typeface, gigantic margins, and so on.

(2) Arguments or claims that have an obvious major threatening objection, which the author doesn't mention until the final 2 pages, but then "I don't have space to deal with this here..."

[Foreigner]

Cuteness I think is high risk. David Lewis could be very cute in his writing, and (for the most part) it seemed to work for him. But most people aren't David Lewis and it just comes across as too aspirational. I'd perhaps say that Bernard Williams is an example of the latter... snarky insult can't be used as a substitute for argument too many times without it becoming tiresome.

[Remco Heesen]

@Foreigner: I'm not so sure the points you mention are good indicators that someone just learned LaTeX. I have been writing all my papers in LaTeX for well over a decade, and I typically don't see a need to change the default Computer Modern font or the default ample margins (especially when sending something out for review as the formatting will have to be changed upon acceptance anyway). If anything I would say it's new people who haven't internalized what LaTeX is all about who are most likely to feel the need to fiddle with fonts or margins.

[capitalization]

Capitalized thesis names. coughmitcough

[Foreigner]

@Remco Heesen: I'm thinking mostly of non-formal work in eg history or ethics, and the default Latex formatting does make large blocks of text hard to read. Many people who work in these areas go through a Latex phase as a grad student before they switch back to word processors.

To make readable (non-technical) prose in latex you do need to change the fonts and margins I think, as well as raggedright depending on preference.

[Foreigner]

@capitalization: not just capitalized, but in small caps so the initial letter of the name is slightly larger! I don't mind this if not overused.