Our books






Become a Fan

« CFP: Marc Sanders Prize in Metaphysics | Main | Cholbi's on attention books (do not) receive »

11/18/2015

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

anonymous

I think you are missing:

5.) Review the paper, ask the author to address, (i) criticisms of the methodology extant in the literature, and (ii) the consonance of the author's own position, given that methodology.

I don't see why those aren't two perfectly reasonable considerations for reviewers to raise. It will be up to the editor of the journal, of course, whether the issues raised merit rejecting the paper, inviting it to be revised in the light of the criticisms, etc. I take it that the job of the reviewer is simply to make the author and editor aware of such potential problems and I don't see any problem with doing so in this case.

Marcus Arvan

Hi anonymous: Thanks for your comment! That's sort of what I was going for with (3), but the problem is, I'm assuming there's no way the author could adequately address the methodological issue in the paper (as the issue is at least an entire paper in its own right--indeed, an issue that many other papers have belabored in great detail). At best, the author could only address the issue in a cursory/dismissive way, which is precisely what (I'm supposing) one is tired of as a reviewer (e.g. one is a bit fed up with the literature citing the concern only to go on continuing to use the methodology).

Derek Bowman

What is the purpose - or what are the purposes - of peer review in philosophy?

Without at least a sketch of an answer to that question, I don't see how we can begin to answer the one that you've laid out.

If it's to help filter the content of journals to promote the advancement of knowledge, then surely publishing articles that rely on unreliable and discredited methodologies would be counter to that goal. Of course other philosophers - presumably including the one whose article you're reviewing - may well disagree about whether the methodology in question is unreliable and discredited. But you're being asked to review the article based on your expertise in the relevant field, so why shouldn't you use your judgment?

That seems to tell strongly in favor of option 4, though it might allow for something like option 1 or 3 in cases where you recognize that your assessment of the methodology is not shared by others you recognize as fellow experts, or by the editorial board of the journal.

Option 2 only makes sense if you assume that the purpose of peer review is something else; perhaps measuring some kind of procedural competence (or even excellence) using established philosophical techniques, irrespective of whether the methods used are likely to advance knowledge in the relevant areas.

This might be the appropriate standard if you thought either (i) there is no reliable knowledge to be had about the subject matter and/or (ii) the purpose of peer review in philosophy is more sociological than epistemic. For example, one social function of peer review is to determine employment and promotion within the field.

Richard Yetter Chappell

Hi Marcus, does it make a difference that the aspect of the paper you disagree with is it's *methodology*?

A generalized principle (4*) strikes me as pretty clearly wrong, if used to "take an author to task" for participating in a tradition (Kantianism, say) that the reviewer takes to be hopeless, and demonstrably so.

It's just not the place of reviewers to be making those kinds of judgments. Reviewers should judge whether a paper is a good exemplar of its kind, i.e. given the kind of project it is engaged in (which I think includes the kind of methodology used). It is utterly inappropriate for reviewers [who are not themselves here subject to the checks and balances of meta-peer review!] to (ab)use their powers to punish authors for pursuing projects of a type (incl. a methodological type) that they personally happen to disagree with.

No real good is achieved by this, even supposing that your goal of "purifying the literature" (so to speak) were a good one. If it's a good paper for what it is, then it will be accepted at a subsequent journal. All you're doing, by rejecting them now for your idiosyncratic reasons, is screwing the author around in the meantime.

In short: if you have objections to a whole style of philosophy, make them known in your own work (which may include papers, blog posts, etc.). Try to convince others with the force of your arguments; but use no other force than that. The peer review process is not the place for such blanket rejections. Articles for review should be assessed in their own terms, and not rejected just because of the type of philosophy that they engage in.

Marcus Arvan

Hi Richard: Thanks for your comment! I hope I didn't give the impression I actually favored (4), or would go with it. I share your concerns, and am more split between (1)-(3).

Still, I think (4) raises some interesting questions. Presumably, at some point it was appropriate, say, in physics for reviewers to stop taking seriously papers on the luminiferous aether; for academic psychologists to reject papers on Freudianism based on mere speculation; etc. Obviously, these are different cases than philosophy--as they are cases where basically everyone in the field came to recognize a particular methodology/theory as illegitimate (and philosophy, in general, is not quite like this). But still, I think it's an interesting question.

We are allowed as reviewers to "judge the arguments on their merits", right (that's supposed to be our job, no)? But, if we think an entire form of argument is terrible--and we think it has been shown terrible in the literature--why exactly shouldn't we be able to judge its merit in that respect? You suggest no real good would come of it, and that one should simply continue to argue for one's favored approach in one's own work. But suppose, to follow up on a previous issue (research and citation practices), a large body of literature hardly ever engages with the critique in question (this isn't the case on the example of intuition-mongering, but I can think of cases where this seems to me to have occurred). In that case, some good might come of rejections based on reviewers taking a certain methodology to task. Such rejections--if they started to be more common thanks to more reviewers rejecting the methodology--might lead to the members of the relevant literature to finally address the critique(s) in question, rather than continuing to ignore them. Which would be a good thing, no?

Anonymous

I think the answer to this depends on the answer to a couple more questions. Does the paper do anything substantially new or novel with this methodology? If so is the problem it addresses one that someone who does not subscribe to X, would recognize as a real problem? If the answer to those questions is "Yes," then I would be inclined to note my objections to the method and then put them aside as in 3. But if the paper makes a technical point that's only really important if you already buy this method in the first place, then I'd go for 4 and feel no guilt about doing so.
I'll give you an example of what I have in mind. If say I was given a paper that somehow used Frankfurt examples to establish a new and radical point about distributive justice, then I'd probably opt for 3 in reviewing it even though I think that Frankfurt examples are blatantly question begging. On the other hand, if it simply addressed an objection to Frankfurt examples (or worse an objection to an objection to an objection) then I'd opt for 4 and feel fine about doing it.

Brad

Marcus,
I think you should be discussing this issue with the editor of the journal who invited you to review the paper. The editor, author, and journal are entitled to confidentiality while you are handling their manuscript.
Most of your readers would be distressed to see you discussing THEIR own manuscript, which they sent to a journal, on this blog.

Marcus Arvan

Brad: I appreciate the concern, but the topic of this post is hypothetical, the questions it raises are general, and it does not discuss any paper/journal in any identifiable way.

Joe

I'm with Marcus. I am actually very confused about why anything other than (4) is the correct option, given the way the question is posed. How often are we forced to revise and re-revise our papers because we haven't cited and dealt with some relevant literature? All the time. Why does it matter that the relevant literature concerns our methodology, and not the move from P3* to C2?

I would also suggest that Richard Chappell is using language that makes this proposal seem less reasonable than it is. Traditions aren't methodologies: there is the discussion/evaluation of Kant, and then there is the method of deploying transcendental argument. If the method is flawed, it will infect anyone's transcendental argument, no matter what the topic. That's a huge problem, and since that method has been subject to many criticisms in the literature, at least a paragraph on those criticisms seems perfectly warranted (very odd for Chappell to call this "idiosyncratic", since that is by definition untrue of this situation).

He also assumes that (4) involves rejecting the paper... but clearly, a good reviewer can recognize that the paper would be outstanding were its methodology any good, and ask for a revision that defends the method. I don't see how this is any different from the usual gymnastic contortions we require of published authors.

Verify your Comment

Previewing your Comment

This is only a preview. Your comment has not yet been posted.

Working...
Your comment could not be posted. Error type:
Your comment has been saved. Comments are moderated and will not appear until approved by the author. Post another comment

The letters and numbers you entered did not match the image. Please try again.

As a final step before posting your comment, enter the letters and numbers you see in the image below. This prevents automated programs from posting comments.

Having trouble reading this image? View an alternate.

Working...

Post a comment

Comments are moderated, and will not appear until the author has approved them.

Your Information

(Name and email address are required. Email address will not be displayed with the comment.)

Open job-market threads

Categories