Our books






Become a Fan

« Ambivalently going on the market? | Main | Dating seriously while in temporary jobs? »

03/03/2020

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

Trevor Hedberg

I don't have a solution to this quandary, but I do want to clarify one thing about my prior post: I don't think any of the reasons for rejection that I mentioned are incompatible with adopting the "Does this add to the discussion?" standard. I think that all of the concerns I raise point toward ways in which a paper would fail to meaningfully add to the discussion around the issue. Most of my concerns involve presenting one's argument clearly, taking proper account of other people's research on the subject, and not misrepresenting your own conclusions or the views of the authors you're engaging with. Those seem like prerequisites to making a worthwhile contribution to the ongoing discussion (although I can imagine that not everyone will agree with my assessment on that point).

It's also possible for a paper to avoid all those problems and still leave me quite unconvinced that its conclusion is true. But in that case, the judgment should be "This is a good paper that I disagree with" -- not "I disagree with this paper so I'm going to reject it." I wouldn't advise rejecting a paper merely for the reason that you disagree with the paper's central conclusions.

Marcus Arvan

Hey Trevor: Thanks for chiming in. I didn't mean to suggest that your approach to reviewing isn't consistent with the "Does this add to the discussion?" standard. Rather, I meant to draw attention to the relative differences you and I seem to have in arriving at editorial recommendations.

You wrote in your post that you, "thought it would be very difficult to differentiate between papers that warranted R&R verdicts and those that should just be rejected. I was wrong...The reason I was wrong is that most papers I review feature glaring problems..."

I used to have no problem making editorial recommendations...when the standard I used was, "Does this paper have serious problems? If so, reject!" The problem is that in my case, the alternative standard ("Is the the paper good enough add to the conversation?") seems far, far more nebulous.

My point was that I am finding it increasingly difficult to differentiate between recommending rejection, R&R, or acceptance precisely because, when I adopt the latter approach, it's just not clear to me how serious enough a problem with a paper has to be for the paper to properly fall into one category or the other. In my own case, at least, the "Does this add to the discussion?" has made me *much* more tolerant than I previously was, and far less sure of what recommendation to make in any given case. That's what I'm struggling with.

The other Marcus

I think the "is this worth discussing" standard is right on, and to deal with the concern about journal standards, I think you can use something like Trevor's criteria:
1. For a paper to be worth discussing in a top journal, the argument must be very clear, have considered a greater deal of the relevant literature, etc.
2. For lower journals, not being as widely involved with the literature, or having (parts of) arguments that are not extremely clear and precise may be fine.

Alternatively (or alongside) you may also track journal quality by the "how worth discussing is this paper?" - i.e., does it have the potential to make a significant contribution in the relevant literature? Or does it focus on a smaller issue only interesting to a tiny sub-field? Both can be worth discussing, but only the former may be worth discussing in a top journal.

In terms of considering an R&R vs an acceptance (or minor revisions, etc.) you may think about the paper's consideration of objections. Obviously, for reasons initially suggested, we should not expect a paper to consider all possible objections - leave some of those to later papers. But, perhaps a paper meriting major revisions is one that has not considered a quite well-known or obvious objection.

I agree this is a hard thing to figure out, and I don't think (by and large) we can or should want an algorithmic process, but certainly having clearer criteria would help.

Robert Zink

Reviewers don't reject papers, they advise editors.

Anon

I particularly wonder about cases where one is asked to review an essay in which one's own work is criticized. I'm not sure how often this happens, but it seems to happen fairly frequently that, if Smith's theory of X is criticized in a paper, then Smith will be invited to review that paper. (At least it has happened to me that I have been asked to read papers where I am named and criticized, and other friends have confirmed having the same experience.)

In those cases, I really hope that reviewers don't hold to the standard of "is this convincing". In fact, I think that would be pretty bad professional practice. Right?

So, clearly in these cases, whether the paper is worth discussion is the better standard. Right?

Assoc Prof

I would imagine a kind of trade-off between (a) how much a paper is worth talking about and (b) how much a paper is well-researched and argumentatively refined.

If a paper is really, really worth talking about, then some argumentative omissions might be tolerable. Or if a paper is just moderately worth talking about, then we would want the paper to be very well-researched and refined in argumentation.

Since most journals are quite selective, I guess I usually look for a 9 or 10 out of 10 on both criteria to recommend acceptance. But if a paper is really extraordinary in one respect (an 11 out of 10, so to speak), then maybe a lower score in the other respect is OK.

a philosopher

"If a paper is really, really worth talking about, then some argumentative omissions might be tolerable."

What do you mean by "argumentative omissions"? As Marcus points out, *every* paper (even the best, most influential ones) contains conceptual confusions, unclear passages, and either invalid inferences or substantial undefended premises.

Take quite literally any paper you like in philosophy and stick half a dozen philosophers who work in that area in a room together with that paper for three hours. They *will* walk out having identified 2-3 major flaws and be at least slightly confused about what the author was even trying to say.

So it seems to me that "argumentative omissions" not only *might* be tolerable, but in fact *must* be tolerable, else nothing would ever get published.

Perhaps by "argumentative omission" you mean only *serious* issues, but then the issue is that two philosophers rarely agree on how to weight the argumentative, conceptual, and philosophical problems in a paper. Many times I've had someone read a paper and say "oh, well, you don't really discuss objection X, but that's not a big deal", only for the next person to take my lack of discussion of X to erase all interest in the paper.

I know I'm to some extent jumping all over you (Assoc Prof) for what was probably an off-hand remark. It's just that I think it's *really* important to push back as hard as one can against this standard that we should recommend reject unless a paper "convinces" us, or recommend reject whenever there are "argumentative omissions". In practice these are not enforceable or inter-referee consistent standards, and they easily devolve into ad hoc justifications for rejecting papers we simply don't like. (Since every paper has "argumentative omissions", there's always one there for your subconscious to find if you don't like it and want to recommend rejection.)

Assoc Prof

Yes, fair point a philosopher. I guess by "argumentative omission," I meant an omission of the kind that would normally warrant rejection (however we want to fill that in).

My point was that, in the case of work that is very insightful, creative, and "worth talking about," it could make sense to slightly lower one's expectations for argumentative rigor and thoroughness.

Verify your Comment

Previewing your Comment

This is only a preview. Your comment has not yet been posted.

Working...
Your comment could not be posted. Error type:
Your comment has been saved. Comments are moderated and will not appear until approved by the author. Post another comment

The letters and numbers you entered did not match the image. Please try again.

As a final step before posting your comment, enter the letters and numbers you see in the image below. This prevents automated programs from posting comments.

Having trouble reading this image? View an alternate.

Working...

Post a comment

Comments are moderated, and will not appear until the author has approved them.

Your Information

(Name and email address are required. Email address will not be displayed with the comment.)

Job-market reporting thread

Current Job-Market Discussion Thread

Alt-ac jobs discussion thread (2021)

Philosophers in Industry Directory

Cocoon Job-Market Mentoring Program

Categories