Our books






Become a Fan

« Reader query on impostor syndrome | Main | Chapel Hill Public Philosophy Workshop »

03/06/2018

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

NK

Marcus,

I share your concern, but I'm not so sure about your proposed solution. I think there's a lot of value in developing lines of thought from premises that don't initially seem all that plausible. In my own reviewing, I've had the thought: "This seems totally wrong-headed to me, but I think it's good that someone is trying to defend it, because it seems worth seeing where it leads." And, in general, I'm pretty willing to grant the author the basic premises of their paper, on the assumption that the ultimate plausibility of those premises depends on the plausibility of the whole view they're developing. And if I prevent them from publishing the first steps, they'll never develop the whole view.

Put slightly differently: I worry that your proposal enforces an attitude towards philosophical debate that I find problematic, namely, one in which every philosophical argument must begin from premises that all parties to the debate find reasonably plausible.

My problem with that attitude is that I'm not convinced that theory-construction works (or should work) this way. Sometimes a premise (space-time is curved? the speed of light is constant?) only becomes plausible once you work out the whole theory and then test *the whole theory* against the available evidence. And I think that, if we don't acknowledge this fact, we risk doing the equivalent of rejecting foundational claims of relativity theory on the grounds that they're just obviously false *by the lights of Newtonian physics*.

bc

Don't journals already implement essentially this policy by assigning Referee #2 to review each paper?

Jonathan Swift

Great idea!

By similar reasoning, let's also require that at least one vaccine denier referee every submission on epidemiology, at least one oil industry lobbyist referee every submission on climate change, and at least one Holocaust denier referee every paper on 20th century European history.

After all, earlier arguments against these positions in their respective literatures *may* have become accepted in part because of self-selection results in the past!

Marcus Arvan

Jonathan: I don’t think that’s a very charitable rendering of the proposal or the rationale I offered. As I noted in the OP, the sciences have systematic controls for observer bias and selection effects. This is precisely what we don’t have in philosophy. Which is why I think the proposal makes sense in a philosophical context but not a scientific one.

Marcus Arvan

The comparison to history is inapt as well. In historical studies, there are (A) clear standards for what counts as reliable historical evidence, (B) a wide body of established historical facts, and (C) multiple lines of converging, mind-independent evidence (photographs, diaries, eyewitnesses, etc). None of these things are true in philosophy, where standards for argumentation are constantly contested (see meta philosophy), there are few established philosophical facts (see the philpapers survey), and the only measure of which premises are “plausible” are philosophers themselves.

Jonathan Swift

Even if it is conceded that there are no 'systematic controls for observer bias and selection effects' in philosophy -- a BIG if, and an assertion for which you have made no argument -- I can continue listing fields of inquiry in which there is no controls with respect to experimental design, statistical analysis, replicability, randomized controlled trials, and so on.

I already mentioned one: historical inquiry. What is your response in this case?

One could also add mathematical inquiry. Should we require that every manuscript in mathematics be evaluated by someone who believes that the only numbers there are can be counted on our fingers and toes? If you can dismiss that type of referee via controls having to do with experimental design, statistical analysis, replicability, randomized controlled trials, and so on, I'd be mightily impressed.

Marcus Arvan

NK: I think that’s a fair concern. But let me push back a bit. Jonathan seemed to take my proposal as advocating an editorial practice directly akin to the Tenth Man Rule from World War Z—namely, selecting at least one reviewer who *opposes* the position being defended. Just to clarify, this was not the proposal at all. Although I mentioned the Tenth Man Rule to begin, my actual proposal was simply to include at least one “outsider”, i.e. someone who simply doesn’t specialize in the topic, and so may or may not have any view on the topic at all (and so will plausibly be a more “neutral observer” than people who specialize in the topic).

Here’s why I think this matters. I agree with you that it would be problematic to stifle lines of inquiry too prematurely. We should by all means encourage people to publish things that may be quite wrongheaded. That being said, I think it’s important to have *some* independent check on theorizing, so as to provide some outside check that a given debate hasn’t become so insular that its members develop a false sense of how attractive their theories’ premises really are.

In other words, the idea behind the proposal isn’t that we should be able to convince everyone that a view is worth pursuing. The idea is that one should be able to convince someone beyond those with a horse in the race—that is, at least *some* outsiders—that the dialectic hasn’t gone off the rails somehow.

Does this address your concern?

Jonathan Swift

Let me take the points you make in your comment of 03/06/2018 at 05:21 PM in turn.

(A) What reason do we have to believe that "there clear standards for what counts as reliable historical evidence"? Clearly, historical study doesn't establish that. Equally clearly, if we should believe that that's true, it's only because we have good reason to believe some specific *philosophical* view about what counts as (historical) evidence.

(B) What reason do we have to believe that there is "a wide body of established historical facts"? Again, clearly, historical study doesn't establish that. Equally clearly, if we should believe that that's true, it's only because we have good reason to believe some specific *philosophical* view about what counts as an established (historical) fact.

(C) What reason do we have to believe that there are "multiple lines of converging, mind-independent evidence (photographs, diaries, eyewitnesses, etc)"? Clearly, historical study doesn't establish that. And again, equally clearly, if we should believe that that's true, it's only because we have good reason to believe some specific *philosophical* view about what counts as a mind-independent (historical) fact.

Thus rather than providing reason to believe that historical inquiry (and the literature it produces) is on better epistemic footing than philosophical inquiry (and the literature it produces), in fact the former presupposes the latter. Thus I ask again: shall we now require that at least one Holocaust denier referee every paper on 20th century European history?

NK

Marcus: Two thoughts.

(1) I wonder whether the reviewing process is really the best place for this kind of "check-up" on the debate. It seems plausible to me, on the face of it, that early-career people would be especially likely to have papers rejected by the outside referee, in part simply because they're especially likely to be deeply embedded in specific debates.

(2) I'm not sure that the outside referee would necessarily be able to tell whether the debate has gone off the rails. Looking for implausible premises, as I was trying to say in my earlier comment, doesn't strike me as an obviously good way of doing that. Take the grounding debate. I've read a bit of this, as an outsider (to metaphysics in general, with one exception). I'm inclined to think it was never even on the rails. But, for one thing, I can't identify a premise in the literature that I think is mistaken. The premises seem okay. It's the project itself I don't get. But, for the other thing, for that very reason, I'm not at all willing to say that the project has definitely gone off the rails. Instead, I just think: "I don't get it." But I don't get lots of things. ––So I guess maybe I agree in principle that it would be good to have some kind of independent check on the debate. I'm just not sure I can even imagine a good check––aside from just letting the debate play out, anyway. Which is to say: I'm not sure I'm as worried as you are about a debate going off the rails. Such is inquiry, really.

Actually, one more thought: maybe the solution to your problem is just to encourage more philosophers to be generalists, so that they can put different debates into conversation with one another, and raise questions about them that way, which the specialists can then respond to and try to hash out using all their specialized powers. (Of course, I'm very much a generalist, so I *would* think that.)

Anthony Fernandez

I think I'm largely in agreement with NK.

It seems that the reviewing process isn't the place for this kind of check. My main concern is that—while many of us have diverging philosophical presuppositions—explaining why someone's presuppositions are illegitimate is a hugely difficult task. I take it that many of the major texts in the history of philosophy (e.g., Kant's first Critique, Heidegger's Being and Time) are precisely objections to illegitimate premises in contemporary debates.

In light of this, I don't see how we can ask reviewers to decide whether the premises of an unfamiliar debate are plausible or legitimate. I see two likely outcomes from asking reviewers to do this:

(1) Reviewers don't give reasons (or at least not good reasons) for discounting the premises of a debate. They simply let the editor know whether they find the premises plausible or implausible. Adding this reviewer to the process just adds another arbitrary element: you get a reviewer who finds your premises plausible, or you don't. And the perceived plausibility of your premises says little about your individual submission. (It's an assessment of the debate.)

(2) Reviewers articulate strong reasons for discounting the premises of a debate. In this case, rejecting the submission might be legitimate. But it doesn't do the *debate* any service to have these reasons confined to an individual referee report. If the reviewer has good reasons for discounting the premises as implausible, then these reasons should be published as an article engaging with the debate (at a meta level).

In short, if this is a genuine problem, then the solution seems to be encouraging people to engage in debates outside of their hyper-specialized area (as NK suggests), or for more people in the hyper-specialized area to reflect on their premises (which I think happens quite often). Having this check implemented at the reviewer level doesn't seem to contribute substantially to the debate or the discipline.

Marcus Arvan

Anthony & NK: I guess that makes sense to me. Thanks!

Verify your Comment

Previewing your Comment

This is only a preview. Your comment has not yet been posted.

Working...
Your comment could not be posted. Error type:
Your comment has been saved. Comments are moderated and will not appear until approved by the author. Post another comment

The letters and numbers you entered did not match the image. Please try again.

As a final step before posting your comment, enter the letters and numbers you see in the image below. This prevents automated programs from posting comments.

Having trouble reading this image? View an alternate.

Working...

Post a comment

Comments are moderated, and will not appear until the author has approved them.

Your Information

(Name and email address are required. Email address will not be displayed with the comment.)

Job-market reporting thread

Current Job-Market Discussion Thread