Our books






Become a Fan

« Reviewers as ghost authors - a strange worry? | Main | Journal of Experimental Philosophy - expressions of interest »

04/03/2016

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

shane wilkins

I think the premise of your question about prestigious publications is false. I'd be very surprised if the median article at a place like phil. review or nous garnered fewer citations than the median article at a place like the southern journal of philosophy or dialectica. (let me hasten to add that I am choosing the latter two, not because I think poorly of them; to the contrary, i'm hoping that we can all recognize these are fine journals which are nevertheless not "high prestige".)

This should be easy to check. I just went and looked at the first issue of phil review and the southern journal for the year 2010 and decided to look on google scholar for how many times each of the papers included in each issue has been cited. Here's the results:

Phil Review January 2010.

Rachel Briggs, "Decision-Theoretic Paradoxes as Voting Paradoxes", 10 cites.
Jonathan Schaeffer, "Monism: The Priority of the Whole", 313 cites.
John Turri, "Epistemic Invariantism and Speech-Act Contextualism", 39 cites.

Southern Journal, March 2010.

Paul Snowdon, "On the What-It-Is-Likeness Of Experience" 0 cites.
Tad Schmaltz, "Malebranche and Leibniz on the Best of All Possible Worlds" 1 cite.
Robert Bernasconi, "Race and Earth in Heidegger's Thinking During the Late 1930s" 5 cites.
Terry Horgan, "Transvaluationism about Vagueness: A Progress Report", 3 cites.
Rachel Barney, "Gorgias's Defense: Plato and His Opponent on Rhetoric and the Good" 4 cites.
Linda Martin Alcoff, "Sotomayor's Reasoning" 12 cites.

So the median phil review paper in this (admittedly super small, and unscientific) sample had 39 cites and the median southern journal in roughly the same period of time has been cited only 3.5 times.

I'm sure we could find examples of Phil Review papers that don't get cited, but I'd be surprised if we could find many. Further, given that often when we're looking at junior candidates, it'd be hard to judge that candidate's ability based on how many citations they've gathered. Most new TT hires don't have a five or ten year long publication history for us to judge their impact using quantitative metrics like citation counts or h-indexes. Senior hires are, of course, a different matter.

Again the above is just a back-of-the-envelope thing. But it also correlates with what you'd expect. If you take the journal's h-index and normalize by the number of papers published a year, that should tell you how likely a paper is to get cited. And there, the ordering of journals pretty much matches the rankings that come up on Leiter's blog. Somebody did this not too long ago. I'll see if I can find the link.

shane wilkins

This is the link by Kate Devitt that I was thinking of earlier:

http://mnemosynosis.livejournal.com/31341.html

Upon looking at it again, I'm not quite sure I follow her methodology though. She's measuring the *rank* in google scholar per number of citable documents, not the h-index per citable documents.

Marcus Arvan

Hi shane: Thanks for chiming in! I am not at all surprised that the median citation #s are higher in Phil Review than, say, the Southern Journal. But this is not what I was trying to get at in my OP. Healy and Bloom's recent wide-ranging analysis of data at four top journals (https://kieranhealy.org/blog/archives/2015/02/25/gender-and-citation-in-four-general-interest-philosophy-journals-1993-2013/ ) shows that, "Even though these are all peer-reviewed articles published in high-prestige journals, almost a fifth of them are never cited at all, and just over half of them are cited five times or fewer. A very small number of articles are cited more than twenty or thirty times."

I think this bears repeating: if Healy/Bloom are right, a full 20% of papers in four of our highest ranked journals are never cited at all--and 50% are cited fewer than 5 times. These are pretty high proportions, if you ask me, especially when we are talking about our four highest journals (one can only imagine what the proportions are like at lower-ranked journals)! And the real concern in the conversation wasn't so much about prestige: it was about how *depressing* it is that the vast majority of papers we publish are either never discussed, or hardly discussed. The question was raised, "Why do we publish when no one bothers to read or cite our work?", to which the answer was: "Because we need prestige tokens for tenure, advancement, etc." And the thought was that there is something off about this--that publishing in philosophy is less about contributing to and engaging each other in philosophical dialogue, and far more about achieving tokens of individual achievement (for career-advancement purposes). The thought then, I take it, is that maybe this is something we should reflect on and try to change.

And indeed, I've seen how enjoyable a different model can be. In my spouse's field, most papers *are* cited and discussed--and I can only imagine how nice it would be if philosophy were that way, as it can indeed be depressing spending months, and years, toiling away on work that few people engage with (and again, this happens to somewhere between 20-50% of papers at our highest ranking journals).

To sum up, the thoughts of the conversation were something like this: "The way discussion/prestige is doled out in philosophy today is super-depressing. It turns most of us into Sisyphean characters. Most of our published work is never engaged with, and in many ways the publishing game seems more about racking up career tokens than philosophical dialogue. Maybe we should try to change this!" I can't help but be sympathetic with the idea that we should.

selective reader

Marcus,
I am having hard time understanding why you think that every paper published in a philosophy journal deserves to be talked about and cited. The refereeing process is just the first culling. Not everything published is interesting, particularly original, or engages others. Such pieces are apt to be neglected. Incidentally, the psychologist Dean Simonton notes that scientists are not especially good at determining beforehand which of their papers will be frequently cited, and which neglected. We need to publish, and then other readers need to respond. But there is nothing inherently unjust about the fact that citations are distributed unevenly.
There are lots of injustices in the world, and in our profession, but that is not one of them.

Shane Wilkins

Hi Marcus, I think I'm tracking your concern a bit better now. Can you share the comparative data about citation rates in other fields? My gut response to the Healy study you cite is that 80% of papers in a top journal is actually pretty good, but granted I don't have a base rate in mind. At any rate, I'd still take a paper in a high prestige journal as pretty good evidence that the author was doing high quality work as recognized by experts in the field. Not all good work gets recognized, of course. So I wouldn't necessarily hold it against someone that their work doesn't get cited, since citations also track things like who your friends in the profession are. Someone who is good at philosophy but bad at networking might have low cite counts even if doing good work. It's hard to want to hold that against someone.

Marcus Arvan

selective reader: I'm not claiming that every paper deserves to cited, nor am I claiming that these phenomena are any kind of injustice! Rather, my claims are these. The data show that approximately 82% of all philosophy articles are never cited, and about 50% of articles in the top 4 generalist journals are cited fewer than five times. Personally, I think it is profoundly implausible that over 80% of published articles deserve no discussion at all, and only 50% of articles in our top 4 journals deserve more than five citations. And, in any case, my point is not that it is an injustice, but rather that it is simply unfortunate, in the sense of being a depressing state of affairs for people who toil their years away publishing articles that no one reads. There is a big difference between saying something is depressing, and saying it is a injustice. Now, to be sure, I do think philosophers have a penchant of under-citing--but that is another matter. I try to err on the side of over-citing and over-including people in philosophical conversations, because I recognize it matters to people to have their contributions recognized. Finally, I *am* suggesting professional philosophy would probably be a more enjoyable occupation for many if we, as a discipline, undertook to err in the same general direction. That's all: it's not a matter of injustice per se. Rather, for me, it's a question of humanity and decency--of treating each other and our work (within reason) as though they matter. I do recognize that not everyone may have these priorities--but I do, and I like to push for them. I want to work in a discipline where people feel appreciated, rather than left out. But that's just me. (I suspect some of our differences here may also be due to our different estimations of how much published philosophical work is good. I'm of the opinion that much of it is quite good, or at least worthy of inclusion in professional discussion. I'm also of the contrary opinion that a good share of articles that *are* discussed don't deserve nearly as much discussion as they receive. For instance, I think the current explosion of literature on "Ground" in metaphysics is one big blind alley, and that there are many more important issues that could be occupying journal pages these days. Perhaps you disagree with me on these things--and if so, those will just have to be a place where we differ!)

Marcus Arvan

Hi shane: I gave a link to baseline rates for other occupations in a recent post (http://philosopherscocoon.typepad.com/blog/2016/02/ways-to-improve-philosophical-engagement.html ).


Here they are:

(1) More than 90% of published humanities papers have received no citations two years after publication.
(2) About 82% have never been cited five years after publication.
(3) Citation rates in the humanities have been consistently this low across the past several decades.

In contrast,

(1) Over 50% of natural science and social science papers receive at least one citation within two years.
(2) About 70% receive at least one citation after five years.
(3) Citation-rates in both areas have climbed dramatically and steadily over the past several decades.

There are some nice graphs in the original study (http://arxiv.org/ftp/arxiv/papers/0809/0809.5250.pdf ) illustrating the disparities, and showing they are only getting worse.

Finally, on your claim, "I wouldn't necessarily hold it against someone that their work doesn't get cited, since citations also track things like who your friends in the profession are." This is *exactly* what I think is problematic (or at least disappointing) about our discipline. Healy's data all found there to be relatively small in-group citation networks (i.e. "friends citing friends"). But this is not, I think, the way things should be. Indeed, other fields are very different. My spouse works in psychology, and I have published in psychology, and getting cited/discussed is really not so much a matter of who you know. Generally speaking, if you publish in a halfway reputable journal, your work will be cited and/or discussed. For instance, I have *no* friends in psychology, and published a very simple correlational study in an out-of-the-way journal in 2013, and I have already received 23 citations on that article (mostly by psychologists). In short, psychologists go out of their way to read and cite new things in the literature.

This is something I believe should change in philosophy, both for the sake of better philosophical research, and for the sake of contributing to a better profession. For some reason, there seems to be a kind of obsession in philosophy for deciding whose work does and does not "matter." I think a *lot* of work matters, and is worthy of discussion--and I am often dismayed by what receives discussion in philosophy, compared to what doesn't (by my lights at least, some work receives far more discussion than it deserves, and much other work not nearly enough.

shane wilkins

Hi Marcus, Thanks for the link to the paper. I'll check it out when I have time. I'm not sure I share your concern. Citation practices in the natural sciences might simply be different than in the humanities for reasons having to do with the nature of the subject matter.

I agree that the obsession with who "matters" is misplaced, and that some work is discussed far too much, other work far too little. But I suspect we'd fall into disagreement just as soon as we tried to get down to cases!

Let me also, add, however that I don't actually think there's anything necessarily wrong with citing the work of people one knows. (There's nothing necessarily good about it either!) Being an active member of the scholarly community should lead to personal connections with scholars interested in your topic who cite your work because they are familiar with it.

Justin Caouette

Nice post, Marcus! I had a similar experience at this year's APA. I hope this trend continues.

Verify your Comment

Previewing your Comment

This is only a preview. Your comment has not yet been posted.

Working...
Your comment could not be posted. Error type:
Your comment has been saved. Comments are moderated and will not appear until approved by the author. Post another comment

The letters and numbers you entered did not match the image. Please try again.

As a final step before posting your comment, enter the letters and numbers you see in the image below. This prevents automated programs from posting comments.

Having trouble reading this image? View an alternate.

Working...

Post a comment

Comments are moderated, and will not appear until the author has approved them.

Your Information

(Name and email address are required. Email address will not be displayed with the comment.)

Open job-market threads

Categories