On citation practices in philosophy

There have been several posts over at NewAPPS the past few years on citation practices in philosophy (see here, here, here, and here). Basically, there appear to be reasons to think that citation conventions in philosophy may be excessively lax, in ways that appear to systematically favor (1) men over non-men, (2) the "famous" over the non-famous, (3) articles that appear in "good journals" over those that appear in "bad journals", and finally, (4) "in groups" (i.e. people citing "friends" but not "people they've never heard of").

As the husband of someone who works in the sciences, I can attest to just how different citation conventions are in the sciences as compared to philosophy. In her field, an author is literally expected to cite everything that has appeared on the issue in the recent literature. It doesn't matter what journal the article is in, if it published and at all relevant, you are expected to cite it. If you don't, your paper will be summarily rejected.

This might seem overly onerous in philosophy, but with philpapers being what it is -- where you can literally search for any relevant term and have everything that has been recently published on a topic at your fingertips literally in seconds -- is it really too much to expect? In addition to the disturbing data out there (see the aformentioned NewAPPS posts), I have seen really bad citation practices firsthand. I have not only read several things in the literature recently that seemed to willfully ignore recent papers by "nonfamous" people; I also know a colleague who regularly reports her  work being ignored by people who know her work on the subject, and even if they didn't, should know it (and cite it).

All of this suggests to me that things seriously need to change. Non-males, other minorities, and "non-famous" people in the discipline from small schools have a hard enough time in our discipline without being ignored in the literature. The least we can do -- or so I say -- is to insist as a discipline on more rigorous citation practices (APA are you listening?). Specifically, I want to propose that editors  inform referees that they are expected to do a quick literature review and point out any omissions. In my view, this is not only good practice, morally and professionally; it is also not overly onerous. It is, in my understanding, already common practice in the sciences, and really easy to do (it only takes a few minutes to do on philpapers). It is worth our while. Or so say I.

What say you?

Comments

[Daniel]

I sympathize with the concerns that motivate you, but I have a few worries about your proposal:
(1) Referees are already slow, and work for free. If referees are expected to do more, more people will turn down referee requests, there will be more time between submission and assignment of referees, and it will ultimately take longer for papers to be reviewed. Maybe this is worth it, but it's a cost.
(2) Have you ever read papers in law reviews? They tend to be bloated, 70 page monsters, half footnotes, the first 20 pages of which are entirely lit review. If we adopt a norm that you should cite literally all the recent literature on the topic you're writing on, maybe that's an easy norm to interpret, but then I don't see how we avoid something like what law reviews have. (I think law provides a more relevant comparison than sciences, as I think the level of specialization in law is typically more similar to that in philosophy.) On the other hand, if the norm is that you should just cite the most "important" or "influential" recent work, well I think that's the norm we already have.

I'm inclined to think the best way forward is for individual authors to take it upon themselves to try to cite under-appreciated, relevant work. Especially, e.g., if you notice that you're citing very few women in your papers, you should take it on yourself to think that over, and to ask yourself whether you've overlooked somebody doing good, relevant work (given the way implicit bias works, there's a good chance you have). But I'm skeptical that there's a good way to get this to happen as a matter of journal editorial policy.

[Marcus Arvan]

Hi Daniel: Thanks for your comment! Although I understand your reservations, here are a few thoughts...

On (1): I think you are dramatically overestimating how much time and energy a quick check of the literature takes. I have read some recent papers on topic X where, when I typed "X" into philpapers, several of the first ten directly relevant papers --published in the past few years -- were *not* cited (ostensibly because they were not by "name" people or in a top journal). It literally takes like 10 minutes for a typical paper to see if the person cited every recent paper on the topic. Is that really too much to ask?

On (2): I think you are falling prey to a false dilemma here (viz. either do what philosophers presently do, or do what law journals do). Okay, look, law journals are excessive -- and they are excessive in large part because of the sheer volume of articles relating to similar areas of law. This just isn't the case in philosophy, and there is no similar problem in other fields (psychology, physics, etc.). My wife does psychology. They are expected to cite *everything*. Their papers are not 80 pages. They're 20-40, just like ours. It really doesn't take that much page space or effort to add, say, a couple extra dozen recent articles.

For these reasons, although I understand your concerns, I do not think the best way forward is for individual authors to take it upon themselves to do a better job. Experience and the empirical literature on bias both suggest that self-policing is not a good way to go. People are typically blind to their own biases, and *think* they're doing a good job when they're not. Indeed, I don't think anyone *thinks* they systematically under-cite women, etc. -- but they data show they clearly do. In every other field, the policing occurs within the journal review process. It is enforced by editors and reviewers. And it works well. We should emulate fields and practices that work. Our practices are not working.

[Rachel]

My most recent paper has 60 citations. It's not bloated, it's <8000 words. There are also relatively few footnotes. Name (Year)…40 times, to fill out citations is like EIGHTY WORDS.

E.g., "For example, see Name (Year), Name (Year, Year), Name (Year), and Name (Year)."

Pretty simple.

[Elisa Freschi]

Marcus, I agree with you (as I tried to say also on my personal blog and on the NewApps post you mention). It is just a waste of our and the readers' time to write on X without knowing whether someone else already made a similar point/raised an interesting objection to the point we are making/reconsidered the whole X, showing that it is rather an Z/etc.

The only point I would add is that this practice is *not only onerous* (you did not say that, I am only raising the point in case a reader might think it is so). If I am interested in writing about X, should not I be also interested in reading about X? CURIOSITAS IS, for philosophers and thinkers in general, A VIRTUE.

[Tom]

Like Daniel I'm a bit skeptical about your proposal. There's a conflict here between "Cite more inclusively" and "Cite only what you've read". I fear that we end up citing more women and outsiders, but don't actually read them. That is at most a marginal improvement. Citing practices are a symptom, but not itself the real problem. But maybe I'm misreading your proposal and what you really advocate is a change in what we read and engage with. Concerning citation practice in science I must admit that I loathe it. I once read a psychology paper that had a sentence like "Testimony is also widely discussed from a philosophical point of view" followed by a list of papers defending widely divergent theories. The author could have written "I am a nice guy who googled 'testimony philosophy' for you" instead. My impression is that this happens far too often.

[Bob]

Just a quick thought on (1). There is no way that I could recommend rejecting a paper or demanding that it cite papers just because I find papers on that topic in PhilPapers. I could only in good conscience do so if I had read the papers and found them to be relevant to the argument that the author I am reviewing is making. So that takes a bit more than 10 minutes...so either I evaluate the paper on its merits based on what I know of the topic, or else I budget in quite a bit more time to read the papers I have discovered and assess their actual relevance. While I don't have a principled objection to the latter, it would certainly cut down on the number of papers I review and increase the time it takes me to do such reviews.

[Martin Shuster]

I'm sympathetic to your concerns, Marcus, but I think that this a far broader *cultural* problem, and so your proposal will not really dent it (imo).

What it requires is that philosophers commit to reading more, and, as you can already see from the comments in this thread, that's not an appealing option for most people (whether justifiably or not).

[Marcus Arvan]

Thanks for your comments, everyone!

Rachel: Completely agree. It's *easy* to cite everyone who has recently published on a given topic, and it doesn't take much room in a paper. There is no excuse for not doing it. It's simple, and not doing it is a professional injustice.

Elisa: I agree that reading is a virtue. But the question is, how do we promote that virtue? One way to do it would be to insist -- as I am suggesting -- that people have to cite all recent pieces on the topic they are writing about!

Tom: Although I appreciate your reasons for loathing citation practices in the sciences, loathing something does not make it right or just. Professional ethics sometimes require doing things we loathe to do.

Bob: Fair enough - but here's the thing. Almost *no* papers get rejected in my wife's field for that reason, and for one very simple reason: everyone follows the norm! That is, few papers are rejected for failing to cite relevant literature because authors *know* they are expected to cite everything. Also, if you are not willing to outright reject a paper for those reasons alone, how about this alternative: recommend acceptance to the editor on *condition* that the author cites things they left out!

Martin: Agree. I think we have a cultural problem that too many people -- for one reason or another -- want to downplay. Professional inertia is a hard thing to change. Few people, I assume, *want* to spend an extra 20 minutes doing a quick literature search on philpapers (or, perhaps, they assume it will take longer than it does). But what we want is irrelevant. We owe it to our colleagues -- professionally and morally -- not to ignore their work.

[Bob]

Marcus,

I don't quite get the point of your response or how it addresses my worries. My point is that I would consider it irresponsible on my part as reviewer to issue such a recommendation to the editor unless I had read the papers in question, thus the second half of my worry.

[Marcus Arvan]

Hi Bob: I'm sorry if the point of my response wasn't clear, or if it didn't address your worries. It wasn't intentional! I was trying to address them. In any case, let me clarify.

First, I don't think it would be irresponsible. In most cases, you do not need to read an entire paper to know if it should be cited. I've read several papers lately arguing for X which failed to cite papers which contained "Arguing for X" in their *titles*. You don't have to actually read those papers to know that they should have been cited. Additionally, a quick glance at a paper's abstract on philpapers should usually do the trick.

Second, I do think that if there is a substantial body of papers on a reviewer hasn't read, the reviewer shouldn't agree to review the paper. To me, reviewing a paper one doesn't know the recent literature on is irresponsible. How, as a reviewer, can you know the paper adds anything important to the literature if you haven't read stuff? This is not an imaginary worry, by the way. I've read a number of papers recently that "reinvent wheels" that other people have published on already (without citing the first person(s) who published a very similar argument).

Anyway, I hope this better addresses your worries. In short, I think (1) it's easier to know whether a paper it cite-worthy than you think (nobody has time to read everything, but a quick read of recent abstracts takes only a few minutes), and (2) reviewers should not agree to review papers on topics they (the reviewers) do not know the recent literature on.

Thanks again for your comment, and sorry I didn't address it more clearly the first time. I hope this is better!

[Grymes]

Marcus,

I'm not entirely clear on your position. It seems to me that there is a serious disanalogy between (most) empirical work and (most) philosophical work. In the sciences, every relevant article adds a bit more data to the subject at hand. It makes sense to cite everything in the sciences, because everything competent adds a little bit of knowledge. But philosophical arguments don't seem like data in that respect.

Are you advocating carefully reading [i]everything[/i] relevant to one's paper before publishing? What if I'm writing about, e.g., the mind-body problem?

If you're not advocating that, are you advocating citing papers even if one has not carefully read those papers?

I'm all for citing widely (and certainly for citing people who aren't Lewis and Williamson), but that's because I'm for reading widely. On the other hand, I am against citing a piece one has not taken the time to understand. I like being cited, but I'd rather someone not cite me than cite me under false pretenses because they haven't taken the time to understand my work. In other words: what Tom said.

[Marcus Arvan]

Grymes: Thanks for your comment. A couple of thoughts.

First, I don't think there is a serious disanalogy (a small one perhaps, but not one that materially affects the citation problem I am raising). When I see a paper written defending X that fails to cite several of the very *first* papers on X that come up when I google X on philpapers, basically citing only those papers by "famous" people and ignoring many papers on X in "lesser" journals -- and I've seen this with a number of recent papers -- there is a real problem.

Second, I am not suggesting reading every single paper in the literature. What I am suggesting is that if a person writes a paper on X but then does not cite several of the very first papers on X that come up on philpapers, there is a problem. Whatever the cause -- whether they are not reading enough, or whatever -- it is a problem. At a *minimum*, one should be citing every recent paper in the area, particularly those papers that contain X in their title and whose abstracts make it clear what the paper's position is on X.

Third, I think your parenthetical comment is spot-on. Part of the problem, or so it seems, is that some people do not read widely enough. I've heard people say they only read a handful of journals, because they do not have a enough time to read everything. Fair enough - but if one is going to write a paper on a topic and intend to publish it, one has a professional obligation to go beyond the few journals one likes and be aware of (and cite) the broader literature.

My general point is simply that one way to get people to live up to this professional obligation is to insist, like other academic disciplines, that (A) papers under review refer to all of the recent literature on their topic, and (B) reviewers hold authors to these standards.

Look, I understand all of the worries people are raising here. I really do. But we do not solve problems -- serious professional problems -- by focusing primarily on reasons to be skeptical about solutions, without providing better alternatives. Almost no solution to any practical problem in the real world is perfect. My suggestion, in any case, is simply that the status quo is unacceptable, and that we need to think seriously about ways to address the problem.

[Grymes]

Thanks for the response, Marcus.

I don't think the failure to cite is a problem in and of itself. (I don't think I should cite a recent paper in the area if I don't think my readers have anything to gain from it.) It's just a good indicator of the problem: the failure to engage. And merely fixing the indicator will not fix the problem.

I guess I'm skeptical that there is a feasible top-down solution. (If there is a top-down solution, it's probably a matter of training our grad students to engage widely rather than making our referees enforce citation.) But I am entirely behind you in spirit. The way to fix the problem, by my lights, is to encourage our generation of young philosophers to engage with our non-famous-white-dude peers. It's a slow fix, and hard work. But I think that's what's necessary for a problem that runs as deep as ours.

[Robert Seddon]

If you want citations to act as a form of professional kudos - which I assume is the point of expressing concern that some groups are under-cited - then it's unclear to me how that goes along with the idea that *all* recent and relevant work - or, more weakly, the first few PhilPapers results for a given search - should be unselectively cited. (How does citation-as-kudos work among psychologists?)

Though I suppose if you *don't* think citations should act as a form of professional kudos, spreading norms of unselective citation might be one way of undermining the practice.

[Marcus Arvan]

Grymes: I am glad we are on the same page in spirit. We just disagree on solutions. For my part, I think history shows that few things change in this world without at least some real top-down intervention. Saying things need to change, and hoping that the next generations does a better job, seems to me obviously inadequate. It took the Supreme Court to integrate southern public school. It takes governors to veto discriminatory legislation. And -- although I know the professional injustices here pale in comparison -- I think it takes editors to make policies requiring authors and reviewers to do a better job. People, in my experience, tend to not to want to change their ways. Sometimes it takes carrots, other times sticks. I think in this case there need to be both. People have been pointing out the citation problem for years now, and little appears to have changed. Few people think they systematically exclude people, but the data suggests they do. So, I say, something institutional needs to change. Institutional requirements *have* addressed the problem in other disciplines, and by all accounts, those requirements appear to work. My wife would never dream of submitting something to a journal in her field without an exhaustive list of citations. Philosophers should be held to similar standards.

[Marcus Arvan]

Robert: I think this may be a good opportunity to discuss what citations are for. In every other field that I am aware of, the point of a citation is to simply recognize that previous work *exists*, so as not to imply -- falsely, or dishonestly -- that one's work does something new when (in reality) someone has done work on the subject before. It is not an honorific, or a kind of "professional kudos." And there are good reasons for this. Letting people decide "what is worth citing" leaves a field open to unhealthy (and unjust) biases.

There are plenty of conventions in academia for "professional kudos", not the least of which are journal rankings and secondary literature that engages in detail with an author's work. Letting people decide for themselves what is "worth citing" can only be expected to play into the kinds of exclusionary biases that have been observed in philosophy. Again, this is precisely why other fields have disciplinary-wide standards -- so that exactly these sorts of biases are minimized, and people are not unfairly excluded from the literature.

[Jonathan D Jacobs]

One problem with asking referees to do a search to improve citation practices, an important goal, is that it risks undermining anonymous review. So many philosophers put papers or titles of papers on the web that a search by referees will often reveal the author to do referee. I know this because when I as editor search for suitable referees, it often happens to me and then I have to recuse myself from overseeing these view process.

[Marcus Arvan]

Hi Jonathan: thank you for your comment. Look, there are no cost-free solutions. Anonymous review is already compromised all the time (by "Google reviewing", etc.). Nothing comes for free. So, we must -- we ought to -- weigh the importance of ensuring anonymous review against the importance of ensuring that people cite properly. *Both* are important. It is bad to compromise anonymous review, but it is also bad when bad citation practices result in people (usually, underrepresented populations) being excluded from the literature. The way I see it, if other fields can promote healthy citation practices while preserving anonymous review, so can we!

[Moti Mizrahi]

Hi Marcus,

I also agree with the spirit of your proposal. I'd like to suggest that Big Name philosophers should set an example here (as I have suggested before: http://philosopherscocoon.typepad.com/blog/2013/07/on-the-professional-obligations-of-big-shots.html ). That is, Big Name philosophers are not under the pressure of the job market and/or the tenure clock, and, unlike the papers of unknown philosophers, their papers are likely to be read anyway (given that they were written by Big Name philosophers). For these reasons, Big Name philosophers should set an example and cite and engage with the work of philosophers from underrepresented groups. I think this would go a long way toward changing citation practices in our profession.

[Rachel]

Marcus: Well said in your 10:37 am comment.

[Marcus Arvan]

Rachel: Thanks! :)

Moti: I'm glad to hear it! Although I like your suggestion, my feeling is that big name philosophers may be unlikely to change their ways. My feeling is that people tend to change their ways under *pressure* to do so, and that when people can get away with not changing their ways, they are likely to keep doing what they've always been doing. This, again, is why I think institutional changes are necessary. Expecting people to unilaterally change their ways almost never works -- as there's a collective action problem: each person asks themselves, "Why should *I* take to change my ways when other people don't have to?" Collective action problems, in general, are best solved not be asking people to change their ways, but rather by *making* them change their ways -- and only institutions possess the power to do it by imposing costs for noncompliance.

[Elisa Freschi]

Marcus and Moti: what *I* do in these cases is that I announce publicly (e.g., on the blog) what I am doing (being more curious, hence reading more, and *thus* quoting more) and I repeat it in short in each paper. Further, I demand contributors to volumes I edit/to conferences I organize to behave according to these basic standards. In this way, I hope that many people will get the message and perhaps also be influenced by it. I am quite confident that it can influence younger colleagues, whereas in the case of senior colleagues, I can only count on the strength of my argument for persuading them. But, I hope, if many mid- or early-career scholars will start expecting more citations, everyone will have to adjust (or be "out of the game").

[Karen Smith]

Yup. This is definitely a way the that the gatekeepers maintain the keeping of the gate. Very discipline specific. It's sloppy at best; the opposite of academic diligence and integrity. It should be exposed, and it should change.