Our books






Become a Fan

« On Gilles Deleuze's birthday: on the philosophical significance of co-authoring | Main | Writing good responses to reviewer comments? »

01/22/2024

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

Richard Y Chappell

> "Accept the invitation even if you have refereed the paper (or an earlier draft of it) previously."

I think this is a bad practice, and risks obstructing especially novel or iconoclastic papers that may prove polarizing. Such papers are often very worth publishing, despite only a minority of expert referees initially recognizing this. If you encourage the same gatekeepers to re-apply their conservative negative judgments for each new journal the author applies to, this reduces their chances of finding the fair-minded referee who will recognize the paper's importance.

> "it runs the risk of being published in spite of its poor quality"

I just don't think this matters all that much, compared to the risk of failing to publish really good iconoclastic papers that too many are initially biased against. (But I also think that journals should shift to online-only, so that "taking up space" is not a constraint that prevents them from publishing other papers judged to be sufficiently high in quality.)

Anonymous

Could triple-anonymity disproportionately affect disadvantaged groups? Although identity might not bias referees through name, it may come through the writing (e.g., native vs. non-native English writers and gender differences).

delete or expand

I understand complaints about length. Still, deleting sections in response to the reviewers, as opposed to expanding to address their comments, often feels more difficult, and like more work. I think this is partly because one can have the sense that doing so disrupts the flow of the argument.

'So I just deleted section 4, but it doesn't seem like I can now transition from section 3 to 5 by just adding a sentence; they're very different. Section 4 was the link! So now what?' This is the kind of thing that I find happens if you try to shorten in light of comments rather than expand, and it involves a lot of intellectual labor to get the paper back into shape.

I suppose if editors want shorter papers, then referees should make sure they are not asking for revision work that will be naturally taken as a request for expansion. That, or we should get more direct comments: 'Section 3 can go and the paper will be fine'.

Crisis manager

As the editors suggest, certainly part of the crisis is a function of the massive number of submissions that should not be submitted, but are submitted again and again. Contrary to what has been suggested in other conversations, there really is a law of diminishing returns. More - that is JUST more - publication does not lead anywhere. Indeed, I once heard an excessive early career publisher referred to as a bottom feeder. I was taken aback. But I came to understand what was being said. After you have published a few things, aim higher ... so write better papers. That will serve you well in the long run.

Bill Vanderburgh

Has anyone ever done an "inter-rater reliability" study on philosophy refereeing? Something like: have 20 people review the same paper and see if there is any consensus in the details of the praise and criticism it receives (then do that another hundred times or whatever the statistics require). I rather suspect that although referees are often very confident about their judgements, there isn't much uniformity between them.

If I'm right about this, then we are kidding ourselves with low acceptance rates. (As I recall, philosophy has one of the lowest journal acceptance rates of any academic discipline: https://dailynous.com/2018/05/24/insanely-low-acceptance-rates-philosophy-journals/.) And if that is right, there will be no measurable drop in quality if we double the number of papers published. (Tiny differences between papers, unreliably judged, currently make huge differences in outcomes.) This would reduce the refereeing burden, since papers that would have been sent serially to several journals before eventually being published will instead go to just one (thus using two referees instead of four or eight).

I predict this move would also reduce referee report length (less need to find enough reasons for rejection), and thus reduce the corresponding burden of refereeing, making more people willing to referee more.

grymes

"Consider whether you really need to submit your paper to a journal. Would it be preferable to publish it in an edited collection, or just put it on your website? If the paper closely resembles your earlier publications, is there sufficient new material to justify publication?"

Material realities aside, this is excellent advice. But who is it advice for? Junior people need to publish every piece they journal article they can, and usually are not in a position to pick and choose what (if anything) they put in edited collections. Senior people are in that position, and often don't face the publish-or-perish pressure that their junior colleagues do (though in a time of collapsing universities one can never play it too safe). But, to be frank, way fewer senior people read this blog--and, of those that do, I find it pretty unlikely that (if they currently value publishing in journals) they're willing to change their established publishing practice in order to make things ever so slightly better for others.

grymes

@crisis manager

Many universities conduct regular (in the case of my R1, yearly) reviews of faculty, and want you to publish something substantial--in some cases more than one thing--every year. I would love to spend a few years writing a truly excellent paper; my employment conditions simply do not allow it.

Shen-yi Liao

>> "Don’t feel you have to adopt every suggestion made by the referees."

Do the editors have data on the percentage of revisions submissions that (1) did not address some concern of a referee and (2) received a reject from that referee, and nevertheless were accepted by Analysis with the editors overruling the referee? How does this percentage compare to submissions that (1) did address concerns of a referee and (2) received an accept from that referee?

Shen-yi Liao

>> "Consider whether you really need to submit your paper to a journal. Would it be preferable to publish it in an edited collection, or just put it on your website? If the paper closely resembles your earlier publications, is there sufficient new material to justify publication?"

Assuming that the editors have been involved in the process of hiring candidates. All else being equal, how might they evaluate a candidate who has a paper published in Analysis versus a candidate who has just put that paper on their website?

David Thorstad

It's quite right that addressing every referee suggestion does not improve paper quality, but I worry that this trend is sometimes blamed on authors when what authors want to see is support from editors.

It's well known that referees hate being argued with and like seeing their comments addressed directly in print. Many authors also have the sense that referees who recommend rejection at leading journals will usually get their way. This leads, understandably, to authors avoiding argument with referees and addressing too many comments in print.

Could the editors provide authors with a credible guarantee of support for standing up to reviewers? The last time I stood up to a reviewer (for example, by suggesting that there isn't much cause to fret over the possibility that AI might soon murder us all) I lost a paper over it. I have to say I am not inclined to repeat the experience.

jakub

For me, Analysis epitomizes everything that is wrong with the current journal publishing system and with philosophy in general. These faults include: striving for shorter articles, competing for space, and an analytic style of thinking.
I'm forced by my employer to publish in journals like this. If there were no external pressure, I'd opt for books and edited collections. Most of my papers published in journals are worse than the original manuscripts that I submitted. One half of the paper is just to please the gatekeepers - that is, the editors and reviewers (proper argumentative style, connecting to current debates, citing trendy authors). And one half is missing because I was forced to remove all connections to other ideas. After getting a paper accepted, I'm tempted to add an acknowledgment: "Sorry for the occasional nonsense, but I needed to get this published."
Moreover, the idea, mentioned in these series, that people compete for space in a journal is preposterous. Journal publications are like prizes. Why so? We know from recent controversies that it is not because publishers restrict space. Quality and originality should be the goal, not competition.

commenter

so the suggestion is for reviewers to help fix the problem? Honestly this seems a little ridiculous. The problem is that there are too many good papers and too much snobbery about publishing. The obvious solution is for journals to publish more of these papers. I honestly don't understand this attitude of "we have to guard against the terrible papers that are coming!" as a reviewer myself I rarely find papers that should be outright rejected and feel pressured by journals to find some reason to reject the paper. Makes me less inclined to review until the journal editors do more (advocate for more papers in their journal, do more actually referring themselves, etc).

State of the field

Commenter
I think the field is really divided on this one issue (and many more of course) - unlike you I am not seeing many great papers when I referee. I have refereed 200 manuscripts for journals - many for the leading journals in the philosophy of science. I am not recommending rejecting papers because I have a concern for lack of space. I am recommending that unfinished, uninformed, and poorly argued papers not be published. But it seems that one person's journal half empty is another person's journal half full.

Matt Weiner

I was looking forward to seeing what the editors of a top journal had to say about how journals could help resolve the crisis in publishing--part of which includes philosophy journals' absurdly high rejection rates. But, even granting that you have no systematic solutions, this article seems to be entirely about what authors and reviewers can do to make life easier for journals. I hope that part 4 will make some suggestions what journals can do (and are doing) to help.

On the rejection rates, obviously I side with commenter; I often see papers I think are unworthy, I also often see papers that I think should not be published in their current form but that I would like to see developed. And I agree with Richard that the risk of not publishing good papers is higher than the risk of publishing bad papers; in philosophy it's not like people will accept faulty conclusions as true just because they're published (this follows from a more general principle about philosophers accepting other people's conclusions). I would say that going online does not completely solve the space/cost problem; Ergo is online-only, but our largest expense by far is still per-page costs for proofreading and typesetting.

I would also like to agree with grymes and Shen-yi that the advice to publish in edited collections and on your website is not helpful. Many philosophers would prefer to avoid the journal system, but do not get invited to edited collections and do not get professional credit or exposure for publishing on their websites.

Michael Kates

I wanted to echo Prof. Chappell's comment that it's bad practice to review a paper more than once. I understand the point the Editors of Analysis are making (especially about the problem of finding suitable reviewers in small subfields of philosophy), but I think that consideration is outweighed by the problem of gatekeeping. I don't have a monopoly on good judgment about the quality of a paper in my subfield, and neither does anyone else. Indeed, I think the inclination to review a paper more than once displays a certain form of intellectual arrogance. I've recently turned down two requests to review papers for precisely that reason, and I think it's good practice to question the reliability of one's own judgment on these matters.

Assistant Professor

Just here to amplify some great points raised in the comments:
- Accepting to review a paper you already reviewed (and rejected) elsewhere may be okay sometimes, but may also lead to one idiosyncratic cranky perspective tanking a perfectly good paper.
- Not submitting your work for peer review has real professional consequences so the advice about alternatives for people not-yet-established in the field is not useful.
- If editors don't want us responding to every piece of peer review feedback then editors please weigh in and help give us direction, and, you know, edit.

constant reviewer

I also agree with Richard and Michael--we should not be reviewing papers that we have already reviewed! The only exceptions I've made to this were when I initially had a fairly positive view of the paper, or where a quick skim of the paper suggests that I am likely to recommend acceptance or R&R for this version of it (I recently reviewed a paper that had been improved based on my comments at an earlier journal (where I had recommended an R&R but the editor had rejected the paper), and had no qualms about it because I recommended straight acceptance this time around--the paper was much improved. Other than those somewhat unusual cases, I think this is terrible practice. (And unlike many of the other commentators here, I have otherwise mostly agreed with what the editors have posted here, and as an aside just want to thank them for these illuminating posts!)

Early Career

To weigh in on the point about reviewing a paper you already reviewed. I personally sense a strong asymmetry on what I'd be willing to review a second time - one that tracks the point that it is worse not to publish good paper than to publish bad papers.

If I rejected it the first time, I would not accept the invitation, but if I'd recommended acceptance or revisions then I would be more willing to do so. I don't want to obstruct papers I think are bad, as I might be wrong. But if I found value in a paper then my turning down the invitation to review might unfairly work against that paper, especially if they get another reviewer who misses the value I saw.

replies to refs

Referees: Do you read the replies we write in response to your comments? Do you have a preference for the formatting of such reports?
Editors: how do you treat these in the revision process?

Non-publisher

Have to agree with jakub. Some people may just like seeing others competing crazily for a place in the journal they edit.

Verify your Comment

Previewing your Comment

This is only a preview. Your comment has not yet been posted.

Working...
Your comment could not be posted. Error type:
Your comment has been saved. Comments are moderated and will not appear until approved by the author. Post another comment

The letters and numbers you entered did not match the image. Please try again.

As a final step before posting your comment, enter the letters and numbers you see in the image below. This prevents automated programs from posting comments.

Having trouble reading this image? View an alternate.

Working...

Post a comment

Comments are moderated, and will not appear until the author has approved them.

Your Information

(Name and email address are required. Email address will not be displayed with the comment.)

Subscribe to the Cocoon

Job-market reporting thread

Current Job-Market Discussion Thread

Philosophers in Industry Directory

Categories

Subscribe to the Cocoon