Our books






Become a Fan

« Informing your department you're on the market? | Main | Contacting EICs in peer-review delays? »

11/23/2021

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

Douglas W. Portmore

As an associate editor, the things that I most care about in order of their importance are: (1) you respond to requests to referee ASAP (and certainly within a day or two), (2) you get your report in on time, (3) you respond to emails from the managing editor immediately when you get emails telling you that your report is overdue and that we request an new estimate as to when we can expect your report, (4) your report makes a clear judgment as to whether the paper should be accepted as Is, conditionally accepted, given a revise-and-resubmit verdict, or rejected, (5) your report makes clear what the reasons are for that judgment, and (6) you provide some useful feedback for the author. I'd say if you do all these conscientiously, then you're an excellent referee. Unfortunately, far too many fail simply with regard to 1, 2, or 3. And some fail with respect to 4, 5, or 6. As far as knowing whether your reports are "good" and not just conscientious, keep in mind that if you were asked to referee, then your opinion is valued. So, if you are being asked to referee and you get your informed opinion along with your reasons for that opinion to me in a timely fashion, then you're an excellent referee. I rarely have a problem with the content of a report. Of course, that's not to say that I always agree with the referee's verdicts and the reasons for their verdict, but this doesn't mean that I think that the referee is bad. I asked them for their honest, informed, and reasoned opinion and if that what they gave me, then I'm very grateful. So, I think that it's wrong to worry whether your report is good in the sense of being the same thing as what others would say. If you're being asked to review, then your opinion is valued. All you need to worry about is being conscientious and providing your informed and reasoned opinion in a timely fashion while addressing any correspondence from the journal ASAP.

Bob Hartman

I like John Greco's advice to referees that was published as a blog post here and at the Daily Nous. I would recommend reading it!

In my view, a good referee report is a good argument. The conclusion is your recommendation to the editor. The reasons for your conclusion should be stated plainly and with clear support relations. And then you can stand back and ask the same questions you ask about your own papers: are my reasons lucid? Do they support the recommendation? How strong are those reasons, and do they adequately support the recommendation? Have I written these reasons charitably? (We are writing to other human persons to which we owe certain things.)

Whatever reasons you offer to help the editor make the decision will also be helpful to the author as they re-think their paper.

Good referee reports, then, are those that the referee can provide good answers to the above questions.

ref

Hi Marcus
I do not like the idea of journals sharing referee reports. These are supposed to be independent judgments of the paper. And seeing what another referee wrote undermines the independence, or at least threatens to.

Chris

Like Marcus, I like it when journals share referee reports. Since you only see the other referee's report after you've written yours, how does that undermine (or threaten to undermine) the independence of the reports?

anon

"When authors get an R&R, they are not dismissive of my comments, but this is hardly good evidence (as an author, being nice is part of the game!)".

Well, I think there's some evidence to be found for the quality of the OP's reports here. Many of my (successful) responses to R&R's involve arguments explaining why I think the referee made bad recommendations and why I won't follow them.

Douglas W. Portmore

Ref and Chris: Those journals that share reports (at least, all those that I know of) only share the reports after a final decision (acceptance or reject) on the paper has been reached. It's not just after the report has been turned it, as that could comprise independence were the manuscript revised and re-submitted. .

in a (temporary?) non-academic position

One thing I think to be important is to review the revisions you've recommended revision. And if it is already a revision, try not to reject a paper because it did what other reviewers requested.

Another thing is that, as have been stated in different places from my recollection, referee reports shouldn't be response pieces. I have recommended accepting papers that I deeply disagree with. The recommendation was due to my belief that the paper advances the discussion. I personally think having a different philosophical position by itself is an insufficient reason to reject a paper, but unfortunately, this sometimes happens.

Evan

(Wasn’t sure if this comment went through. But if it did, please disregard it)

How do you know you’re being a good referee?

A comparative approach is a fruitful epistemic means to arrive at such an answer as Marcus said.

However, a comparative approach can still be highly subjective since not every referee agrees what constitutes “good refereeing” in the first place. Whether or not you agree with Marcus really depends on what you consider to be the appropriate and inappropriate; necessary or unnecessary functions of refereeing.

There was once a post a while back outsourcing what different referees like or dislike in a paper and how they go about refereeing. The lesson to learn is that different refereeing methods will have different values depending on the person.

If you want an objective account of what is good refereeing vs. bad refereeing, then you need to figure out what the aim(s) or telos of refereeing or peer-reviewing is/are. Then the second step is figuring out what are the instrumental and appropriate means to achieving such aim(s) or telos.

We can even ask further: What do we mean by “good” here? Do we mean respectful, detailed-oriented, wise, effective, competent, generous, knowledgeable, fruitful, etc? What should referees be and do? And why? What shouldn’t they be and do and why?

I don’t have much of substantial answer because I’m not a referee. But I do hope to stimulate further reflection so that people can arrive at non-relativistic answers to these sorts of questions. A teleological approach may be more apt than a comparative approach in this case.

Verify your Comment

Previewing your Comment

This is only a preview. Your comment has not yet been posted.

Working...
Your comment could not be posted. Error type:
Your comment has been saved. Comments are moderated and will not appear until approved by the author. Post another comment

The letters and numbers you entered did not match the image. Please try again.

As a final step before posting your comment, enter the letters and numbers you see in the image below. This prevents automated programs from posting comments.

Having trouble reading this image? View an alternate.

Working...

Post a comment

Comments are moderated, and will not appear until the author has approved them.

Your Information

(Name and email address are required. Email address will not be displayed with the comment.)

Job-market reporting thread

Current Job-Market Discussion Thread

Philosophers in Industry Directory

Cocoon Job-Market Mentoring Program

Categories