As a follow-up to my previous post, in which I questioned whether dominant philosophical standards are exclusionary, in this post I will present a tentative, very rough first pass proposal for how journal refereeing norms might be revised so as to promote inclusivity, where this understood in terms of both of the goals Elisa mentioned in her post (including underrepresented people and underrepresented ideas). Again, just to be clear, I offer up the proposal only as a very rough first pass. My main hope is not that the proposal is adequate, but that it will generate further discussion of how we might improve things.
I want to begin by reflecting on how the review process typically works with philosophy journals. Almost all of the journals I have reviewed for give few, if any, guidelines for how referees are supposed to proceed. Basically, the paper is given to the referee, and the referee is to write up a review justifying a recommendaton (viz. accept, reject, R&R, etc.). Then, in light of referee comments, the editor makes an editorial decision.
The most striking thing about this practice is the sheer amount of latitude it gives reviewers (and editors) to judge papers by reference to their own preferences/biases. In my experience, referee comments tend to read as though the referee was primarily asking themselves something like the following questions:
- Do I find this paper's argument persuasive?
- Will the dominant readership of this journal find its argument persuasive?
- Do the author's premises (A) cohere with "commonsense", as well as with (B) premises commonly appealed to in the literature in question?
These are natural enough questions to ask. The first thing I do when reading a paper is ask myself whether I think its argument is any good. But I think this is a problematic way for journal referees to approach their task, and for the reasons I outlined in my previous post. If most referees come from the same background (i.e. not only racial, gender, etc., but also philosophical background, viz. prioritizing "rigor" above all else), the practice of referees asking themselves what they think of the paper threatens to have unintended consequences: namely, most referees with end up rejecting things because the paper they are reviewing contradicts the "received wisdom" that they have internalized as philosophers. In other words, if a vast majority of reviewers come from one background X, and they tend to ask themselves, "Do I find this paper persuasive?", the end result will tend to be: rejecting papers that conflict with their background -- which I have suggested excludes alternative ideas and perspectives. And now notice how innocent it all is! Individual reviewers may not regard themselves as philosophically biased, and yet, if reviewers are given complete latitude to judge "for themselves" whether a paper is any good, the expected end result can be the systematic exclusion of underrepresented perspectives!
I think that this has happened in professional philosophy. Professional philosophers prize rigor, but unfortunately, they have also (as I argued in my previous post) defined rigor relative to the "commonsense intuitions" of an insular few. So, when someone with different intuitions -- or wanting to defend a different (e.g. feminist, racial, non-Western) conceptual framework -- the average reviewer at a mainstream journal thinks to themselves, "This doesn't sit with my intuitions or the intuitions of people who read this journal. Reject." I think this is bad. I think it leads to overly conservative, exclusionary philosophy. I almost don't want to say it, but I have a hard time reading some "top" journals these days. Some of them seem so narrowly focused, and certain debates so far down a deeply mistaken garden-path, that I find the work completely boring. There, I said it. :) This isn't to say that I think all work in analytic philosophy is boring, or even that most of it is. All I am saying is that too much of it is -- and does anyone really disagree with this? I hear it said all the time!
How, then, can philosophy become more inclusive and less boring? To get started, I think it is worth noting that a few journals present reviewers with a very different task than the traditional one I laid out above (viz. "Reviewer, decide for yourself, according to your own unguided judgment, whether this paper is any good!"). What is this alternative? Well, some journals (I can't think of which, but I've reviewed at least a paper or two for one!) instruct reviewers to rate the article under review along several dimensions. Very roughly -- and yes, I am paraphrasing very roughly off the top of my head, some journals give reviewers a form that reads something like this:
Rate the following paper on a scale from 1-10 (1 being "poor", 10 being "excellent"):
- How clear is the paper's argument?
- How rigorously is the paper's argument defended?
- How innovative is the paper's argument?
- Does the paper deal adequately with potential objections?
I think this is good because it prompts the reviewer to rate the paper in question on matters that the reviewer might not have thought of or even care about but which would enable a good editor to prioritize things that the reviewer doesn't. Here are some additional questions that might be included so as to enable an editor to select articles that challenge conventional ways of thinking:
- How far does the paper challenge conventional thinking on the issue examined?
- Can this paper be expected to appeal primarily to a crowd sharing certain intuitions, but not appeal to people not sharing those intuitions?
- Can this paper's premises, even if they do not seem "plausible" to you, be reasonably expected to seem plausible to someone from a different background or vantage-point (e.g. feminists, non-Western philosophers, members of other underrepresented backgrounds)?
- Will this paper be remembered in five years as making a major contribution to the debate, or does it merely make a "small point"?
I don't know quite how these questions might be formulated -- and yes, I do realize that my attempt to formulate these has been pretty ham-handed -- but still, it seems to me a promising way to focus reviewer attention and judgments on criteria other than the conventional, conservative, potentially exclusionary ones they might otherwise be drawn to (i.e. their "own intuitions" and the "commonsense" of those similar to themselves).
If such questions/prompts were formulated well and then given differential weights by clever editors (for instance, since I think really innovative stuff is hard to pull off rigorously -- see e.g. Kant's Groundwork -- as an editor I might come up with a formula or something to discount rigorous work on "small problems" and prefer somewhat less rigorous work with particularly innovative insights), then, I think, we might have a mechanism for encouraging reviewers, and editors, to publish more inclusively (i.e. not excluding ideas/arguments simply because they do not appeal to a narrow in-crowd).
Anyway, although I'm not quite sure how to formulate such questions properly, that's the (very tentative) proposal. I'm curious to hear all of your thoughts!