Ought one not respond to bad papers?

In my second year of grad school, I audited a course on Davidson. In reading one recent paper on Davidson’s triangulation argument, the entire class agreed that this individual had CLEARLY misinterpreted Davidson, and that this paper was an all-around bad. I asked the prof if she would write a reply, since she was (and is) working on Davidson’s triangulation argument. She replied that a paper’s being mistaken and one’s having good objections is not sufficient motivation for writing a reply to a paper. Her reasoning was something like this: I only have a limited amount of time and energy to put together papers, and if I reply to bad papers, I won’t have time to focus on the papers that are good. It’s better to focus on quality papers than crappy papers. So, in general, one ought not spend time replying to crappy papers. Furthermore, replying to a paper indicates that it is worth the energy of a reply.

What do you all think? If this is right, what do you do with these papers? Do you just ignore them? When it is appropriate to ignore a paper? Ought we also not cite such papers?

 

Comments

[Eugene]

In my second year, I had written a reply to a paper that many faculty members in my department didn't think merited a response. My reply was then used as evidence during graduate student evaluations that year, that I didn't have a developed sense of what made good philosophical work good.

So, it's worth keeping in mind that this is how many philosophers will evaluate a reply to a bad paper, whether or not it really is the case that such papers deserve replies.

[Michel X.]

I suppose the flip side of the coin is that if nobody rebuts mistaken scholarship, then there's (perhaps) a greater chance of those errors getting communicated (and magnified) down the line. A totally trivial example (and not one worth writing a paper about, but perhaps worth footnoting in a paper on a related topic!): in "Truth and Fiction," David Lewis spends some time discussing The Adventure of the Speckled Band, and he refers to the snake in question as a "Russell's viper". In fact, the text calls it a "swamp adder" (a fictional snake most closely resembling the Indian cobra; Russell's vipers are simply not a good physiological fit). Anyway, the point is that this mistake has been reproduced in a number of articles on the subject. So I could easily imagine people magnifying more serious errors, perhaps especially since philosophers are such bad citers. (For a more serious example, you could just open up much of the literature on Schopenhauer. All kinds of weird ideas that aren't consistent with the texts get bandied about, never corrected, and are magnified down the line.)

I guess it all depends on the seriousness of the error, and on whether one thinks that it's the kind of publication that's likely to lend itself to error magnification down the line. I had a similar conversation to the one you report with one of my advisors recently, but he offered a different verdict: in the case we were discussing, it had only taken him a few hours to explain why a new article (already being cited elsewhere) in a well-known journal was desperately mistaken in its claims. His cost/benefit analysis indicated that a few hours was a small price to pay for the potential gain of a publication. If it didn't pan out, then the loss was a small one.

[jmugg]

Michel X-

"I guess it all depends on the seriousness of the error, and on whether one thinks that it's the kind of publication that's likely to lend itself to error magnification down the line."

I think this might be right. Philosophers tend to have low H-Indexes as it is, and writing on paper replying to (or citing) the bad paper attacks attention to it.

Your supervisor's cost/benefit analysis assumes that writing up and getting the reply published will only take a few hours. Just because a paper is bad does not imply that writing up a response will be quick (though it sounds like in your supervisor's case it was!). I wrote up this post because I have encountered a number of all-around bad papers while perusing the nature of doxastic states literature, and I have good criticisms of these crappy papers. Each of the paper (I think) takes the form of an IBE, by offering a new distinction/taxonomy. To reply would take a lot of effort on my part.

[Marcus Arvan]

Hi Josh: Great post!

I'd like to second what Michel X wrote. In my AOS, I have come across numerous people who have said, of a particular paper, "Everyone knows that's a really bad paper." But I haven't seen *anyone* argue for it in print. Moreover, the paper in question is now widely cited. It's one of the most widely cited papers in the entire area...and its author has enjoyed a great deal of success and influence as a result.

Anyway, as a general rule, I think this might be the way to think. If a bad paper is (1) becoming influential or (2) is written by someone who is influential, or (3) appears in an influential journal, then replying to it would benefit the discipline.

On the other hand, if none of (1)-(3) obtain, you might look as though you're wasting time on stuff that's not worth replying to.

[Clayton]

Hi Josh,

Very nice post.

I don't have much to add except to say that I agree with Marcus' (1)-(3). Some papers' mistakes aren't very interesting and that's one reason not to get too worked up on responding to them. Having said that, if the paper is having influence on the discussion, it would be good to draw attention to the errors to try to check that influence.

[Trevor Hedberg]

Hi, Josh. I've also been told by many professors that you have to be selective about what papers you reply to (or otherwise engage) when do your own philosophy. Everything that I've been told is consistent with the three considerations that Marcus mentions above, so that might be further evidence that the general rule that he's proposing is correct.

[jmugg]

Marcus, Clayton, and Trevor-

Agreed! In both the case from a couple of years ago, and the papers in the nature of doxastic states literature I have in mind, 1-3 do not obtain. Marcus's 1 seems sufficient for warranting a reply. 2 might warrant a reply. I'm not sure 3 is sufficient (on its own) though. Even papers in top journals often (usually) do not get much traction.

[Jason Brennan]

I think this is kind of a softball question. It comes down to 1) will you enjoy writing the paper? and 2) will other people learn something valuable from your response?

As far as I can tell, "Everyone knows X is a really bad paper" in political philosophy often just means "X challenges what everyone thinks". And if they want to dismiss it, it's probably because they don't have an adequate response. Read the Oxford Handbook of Political Psychology, and then ask whether it's plausible that political philosophy is among the most biased of the subfields of philosophy.

[Pierre]

Let me join the others and say: Great post, Josh!

Jason makes a good point: we have to ask, “What is a reply worth?” It may be useful, even if one doesn’t actually reply to a bad paper, to think for oneself: “What reply could I imagine?” or “What flaws can I identify in this argument?” It sometimes happens that a bad argument against X helps us understand where the argument in favor of X could be misunderstood, or stands in need of clarification.

If a bad paper can help us in this task, it is plausible that a reply is in order, though perhaps not a self-standing reply. It could occur in the course of another paper: “It is to be added that an argument against X has been advanced by Y. This argument is flawed, but it helps understanding where the argument for X needs to be clarified or strengthened...”

The main difficulty is that, if we think that a paper is utterly flawed, we may appear not to be charitable enough. If one thinks there is not much to “save” in an argument, the result is quite likely to look like a harsh attack on its author. If there is reason to fear this, and if it seems (close to) impossible to offer a plausible reconstruction of the argument, which we could reply to without it sounding like attacking a windmill (or a strawman), it seems somewhat pointless to write a full-length reply. Otherwise, it may be worth the effort.

[Marcus Arvan]

Jason: I think you're right on the whole. As some of my past posts on the Cocoon explain, I think I have pretty idiosyncratic views on what makes for good philosophy papers, and my views on which papers are good or bad often depart from conventional wisdom (I tend to favor "insight" over "rigor"). So, I'm willing to agree with you that, in political philosophy, "everyone knows X is a bad paper" *tends* to be code for, "X because it challenges what everyone thinks."

All that being said, I daresay you would judge the particular paper I'm referring to as bad. I could of course be wrong, but I would be surprised!

[Ambrose]

The epistemology of philosophy is just so confusing. On the one hand, people think that a paper that is just plain "bad" -- a paper that "CLEARLY" involves a basic misunderstanding, say -- can make it through peer review. Even in the "good" journals. On the other hand, people think that the paper can be reasonably believed to be a bad paper simply because they themselves or some small community of people they know believe it to be bad. And yet they themselves, or anyone in that small community whose opinions are being taken as authoritative, are virtually _certain_ to be _exactly_ the same kinds of people whose judgment determines the peer review process.

[Marcus Arvan]

The more that I think about Jason's point (and now Ambrose's point), the more worried I am about my initial suggestion.

My suggestion, again, was that we should reply to bad papers when the paper (1) is becoming influential, (2) is written by someone who is influential, or (3) appears in an influential journal, then replying to it would benefit the discipline.

Now, it has occurred to me that there are philosophical (and moral) hazards in adopting these criteria. Judgments about which papers are "bad" may be subject to serious forms of bias and groupthink. As many people have pointed out elsewhere, articles by women and minorities are often ignored, whereas articles by an influential few are cited over and over again. It is all to easy for "I/we think paper X is bad" to reflect individual or in-group biases...and we should not reinforce systems of bias that result in exclusion.

All of this now suggests to me that we should *not* ignore papers we consider to be bad, or only reply to them when we think considerations (1)-(3) obtain. We should judge for ourselves, on a case-by-case basis, whether we think *we* have anything interesting to say about a "bad" paper -- which is more or less what Jason said. This, it now seems to me, is a better way to avoid bias than (1)-(3).

[Elisa Freschi]

It depends on what you want to achieve. From the point of view of the advancement of research, I would point out big mistakes, in order to help younger or less experienced colleagues, but perhaps not necessarily in the form of an article. Blogs are possibly a good way to do that (for a small example on my part, see this post: http://elisafreschi.com/2014/09/09/enough-with-the-eternality-of-sound-in-mimamsa/). If the question regards (also) one's chances of an academic career, though, the answer changes (blogs might not be an obstacle to one's career, but they are surely not counting as "publications").