Some thoughts on Healy and Bloom's citation data

Kieran Healy and his student Nick Bloom have presented some striking results of citation-data across 2,100 articles published in Nous, Philosophical Review, Mind, and Journal of Philosophy between 1993 and 2013. A lot of the data on gender are particularly striking--namely:

1. Only 12.5% of articles published in those journals were by women.
2. On the average, articles by women are not cited less often than articles by men, but
3. The top 1% cited articles by men are cited far more often than the top 1% cited articles by women.

Healey then suggests that (3) is worrisome because it indicates that men are disproportionately treated as "agenda setters" in the discipline:

The key question is, who gets to be a focal point for discussion? Successful, highly-cited articles don’t simply accrue status rewards in the abstract (or just actual money rewards for their authors). They also become centers of gravity that define what a field is about...But the agenda for the discussion is set by a much smaller panel of people up on the stage—people who started out as audience members. Most of academic life has this structure, from departmental talks to conference panels and plenaries to journal exchanges. A key issue then becomes how work gets selected for attention up on the stage, who gets engaged with from the stage, so to speak, and how this process plays out as new audience members come in the door.

I think this is an important issue worth discussing, and think Healy is to be commended for painstaking collecting, analyzing, and presenting this data. But I also think Healy's data and analysis reveals a bunch of other broader issues that are worth discussing too. For example, Healy writes: 

On the average hardly anyone is getting cited, be they man or woman. If you look at the top half of Figure 5, you can see that above 75 citations or so there aren’t any red dots. By definition very few people make it to the upper end of the distribution.

I've written before about how poor I think citations practices in philosophy compared to other fields (see here, here, and here), and want to suggest that it is how philosophers (idiosyncratically) think about the purpose of citations that is plausibly responsible for the gender disparities Healey discusses--and which must be corrected for if those disparities are to be corrected.

Healey implies that in order to correct for the gender gap in the "agenda setting" top-1% of cited articles, we need to think carefully about who is regarded as "agenda setting" (i.e. not just men!). This may be the right way to solve the problem--but I doubt it. The problem, I think, is in regarding the purpose of citations as "recognizing agenda-setting research." My wife works in a STEM field in which there appear to be no significant gender disparities. In her field, men and women alike are cited alike like crazy. Why? Answer: because people don't regard citations as an "honorific"--as recognition of "good, agenda-setting work." No, in her field, one is expected to cite all recent work on a given topic. Obviously, since one is expected to cite all work, opportunities for gender-discrimination are minimized. If one fails to cite a recent article--by man, woman, etc.--a journal reviewer will call you out on it, and tell you to cite it.

Whenever I have discussed citation-practices, however, people have chimed in and said that the purpose of citing work is to "cite good work." Ah, but that's the problem. Once that's how you conceive the purpose of citations, it open the flood-gates to implicit biases. If people have implicit biases that lead them to judge work by men more favorably than work by women, then the "cite good work" approach to citation practices will predictably result in the very disparities that Healey is suggesting are problematic.

Which presents the question: are implicit biases easier to change, or are citation practices? I'll leave it to you to think about and discuss, but I will say this: the literature on implicit bias seems to me to suggest that implicit bias is extremely difficult to combat--and so, I would suggest (yet again), the right answer is: we need to change how philosophers think about the purpose of citations in general, and change citation practices away from the "cite work you think is good and influences you" to "cite all recent work on the issue, regardless of whether you like it."

Comments

[Eugene]

I'm in complete agreement, Marcus. Too bad a lot of philosophers I've run into "hate" (I kid you not) reading other people's work let alone citing it. Frankly, a big part of the problem is that philosophers don't seem themselves as scholars and think (better: know) that they can by being lazy about citing as a result.

[Jerry Green]

Preach!