Our books






Become a Fan

« Call for additional mentors | Main | New Call for Featured Early-Career Authors »

10/14/2015

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

Elisa Freschi

Marcus, I agree with all suggestions regarding the peer review system, and wholeheartedly support 2.

However, I am not convinced by 7 ("PhD programs should accept less students"). I am convinced that students should be explicitly warned about the fact that they the odds of finding an academic job are low or minimal. But I would not want to reject applications from people whose purpose is just different, e.g., older people who want to study philosophy just because they are passionate about it. Currently, I have three such students (one is a retired chemist, I still do not know about the other two), and they are a pleasure to teach, exactly because they can focus on learning instead of grades)!

Marcus Arvan

Hi Elisa: that's a great point, and I had a similar concern. If admissions were drastically reduced, it might effectively "price out" of the profession all but a very small number of privileged people. So, I think you're right: far better to insist that programs be clear about their placement rates, attrition rates, and median time to degree, so that prospective students can make an informed choice knowledgable of the risks.

Referee

I object to (8). Some journals do ask if they can share the referee reports I prepare with other journals they publish. I think the authors of the manuscripts deserve to start with a fresh slate. I hope they alter their manuscripts in light of the comments I provided to the journal. But I think the fate of their papers should be in the hands of new referees. More than once I have been asked to referee the same paper by a second journal. I always alert the editors to the fact that I have refereed it before, and they ought to find a new referee.

Marcus Arvan

Hi Referee: Thanks for your comment. However, what's your argument for objecting to (8)?

You say that authors "deserve" to begin with a clean slate--but this doesn't address the reasons given the original post for not having authors begin with a clean slate [see below]. Similarly, while you might hope as a reviewer that authors revise their manuscript be, this isn't itself a justification for them having to do so. Whether a manuscript should be revised is a matter of whether the criticisms offered are good ones [and, as many of us know, sometimes reviewers offer terrible arguments for their critiques]. Further, if an author receives a good review from one reviewer, that shows that a peer-reviewer thought they shouldn't revise it all.

Here are the reasons the original post gave for not having authors begin with a clean slate every time:

First, making authors begin with a clean slate makes the reviewing process more of a "lottery"--where one's fate is determined by luck. Suppose I get a good referee at Journal A, who gives a good justification for conditional acceptance--but Journal A rejects. Now suppose I get an incompetent reviewer at Journal B, who gives incoherent reasons for rejecting my manuscript. This makes whether my manuscript is rejected at B largely a matter of luck [getting an incompetent reviewer]. If I could send the first review--from Journal A--to journal B, then that journal could weigh the good review against the incompetent review, thus lessening the likelihood that the decision will be based on luck. It will provide the editor at Journal B with useful information.

Second, part of the point of the original post is that making authors "start over" every time with a new journal also serves to clog up the system. It requires more reviewers to do more work. Allowing an author to submit reviewer comments to a journal from a previous journal could lead the editors at the second journal to send out the manuscript to only one additional reviewer [as opposed to two or three], which would plausibly cut down on journal turnaround times.

Are these poor justifications? Why?

Ed

On point 5, having briefly worked for a journal I can say that there is real pressure from publishers to have a very low acceptance rate. One carrot that get dangled is funding/resources but another is page limits. 'Want longer issues... get tougher/be more impressive to us.' I guess they think low acceptance rates are a sign of both high submission numbers and high standards. I object to all this and think it is all the more reason for going open access, online, etc etc. Ditch the publishing houses. But since that won't happen anytime soon, it is worth being aware that editors are pressured to behave this way by the very people that get their issues into the hands of readers, manage their online submission systems, dictate page limits, and so on.

Postdoc

I just wanted to respond to Elisa.

I agree that there should be room for older people and I guess rich people to do PhDs who just want to learn philosophy and don't care about getting a job. However, young people who expect a career in philosophy should not be admitted unless the department is confident a job can be secured, dependent on the applicant's performance of course.

There could be two PhD tracks. One for people who just care about learning philosophy at a high level, and one for people who also want to do philosophy professionally. There would be no difference between these tracks except that the professional track would include an expectation of employment and the university working hard to find you employment. Preferably, the professional track would include at least guaranteed part-time employment (conditional on the applicant performing well in the program) at the university for a number of years until a tenure track job was secured there or elsewhere. This may be a pipe dream though.

Philosopher Brown

It is obvious that these suggestions were written from the perspective of a younger person seeking a job or tenure rather than someone who does much refereeing.

I'm sorry to say that as a referee I see a lot of very bad papers. I mean really bad. I suspect that many of them are seminar papers from early stage graduate students. And these are papers that have made it past the editors; I can only imagine what the unfiltered pile looks like.

I assume that all this not very good work makes it difficult to edit a journal. It is hard to find willing referees and important that much of this work remain anonymous, so referees will never be given their proper recognition.

I don't know what can be done. I wish super senior people who don't need to publish in journals would stop, and very junior people (i.e., grad students) would also stop submitting. Perhaps some journals should limit submissions to those who already have a Ph.D.

In any event, I understand the frustration, but I think there are pressures on journals and referees that you may not fully appreciate.

Postdoc

I wanted to add a 9th thing to my list.

9. Job applications in English speaking countries at least should be standardised. Too many jobs require you to create a special application just for them.

Some want a 1000 word research proposal, some want 2 pages, some want 2000 words. Some leave it unspecified!

An Oxford postdoc wanted a research proposal where the online system would only allow you to write x amount of words, but x was never revealed to the candidate!

One job in the US I applied to required a diversity statement.

Some jobs require that you propose a course on some subject they want you to teach.

Some jobs (at least in the UK) require you to answer a series of redundant questions which basically all address what's already on your CV and cover letter. But answering them somehow seems to take forever!

Some jobs require odd orderings of materials, or very particular orderings.

It seems every postdoc research position requires some special document or another.

Come on people! These materials aren't helping you make informed hiring decisions. At least they definitely aren't needed to narrow down a short list. If you must, ask for additional material from shortlisted candidates.

At the initial stage all you really need is the following:

1. A COVER LETTER

2. A CV

THAT'S IT!

You don't EVEN need references up front. Wait till the short listing! You are wasting tons of people's time to make huge piles of materials you're going to throw out. It's dumb and unnecessary and immoral.

The way things are set up right now, applying for jobs is a full time job.

Derek Bowman

Philosopher Brown: Good luck getting grad students to stop submitting papers so long as publications are needed in order to stand out on the job market. Journal acceptance has become part of the basic certification for (non-adjunct) entry level employment.

No doubt there are very many poor submissions. But, as is also clear from the earlier thread that lead to this one there are also many papers that are rejected (even desk rejected) from one or more journals only to be accepted by other, often comparable journals. http://philosopherscocoon.typepad.com/blog/2015/09/stanley-on-peer-review.html

Philosopher Brown

@derek bowman:
Of course graduate students aren't going to voluntarily stop submitting work; I'm suggesting at some point some journals might need to make having a doctorate a requirement for submission.

I don't think this would solve all the problems or be without costs (tenure track faculty produce plenty of low quality work, and some grad students produce great work) but something has to change. There aren't enough referees to deal with the (often weak) submissions, and under these conditions it makes sense that you will have more rejections. If I'm an overworked referee, rejection is the safer choice. Obviously I think this is a bad state of affairs, but any potential solutions need to take into account the facts on the ground.

Grad student

I am unsure how a PhD. program is supposed to be able to assess whether or not they can land a student a job based on their application. Is the point that in general, department's should be thinking something like, "OK. We have the time and resources to really, REALLY try to help 4 students get jobs. So, lets admit 4 students" or is the point that department's should be able to tell based on application alone that this person will have a chance at a job in the future. If the latter, then that is obviously a stupid rule. Admissions programs miss-fire enough when it comes to admitting/rejecting competent graduate students. I'd certainly question their ability to judge the likelihood that a student gets a job based on the contents of that student's initial application to the program.

Postdoc

Brown, isn't the solution just for editors to do a better job desk rejecting low quality submissions? I think a lot of editors just send everything or almost everything for review. I've received pretty poor papers to review too.

Regardless, not allowing grad students to submit papers won't help. They'll just submit a ton all at once as soon as they get their PhDs. So, the system will be just as clogged I think.

Neil

I've been saying the same thing over and over again for a long time, and no doubt anyone who has paid attention is sick of it. Guess what? I'm going to say it again. I really think that what is needed is much less revolutionary. Here's the real problem in philosophy wrt journal submissions: referees take such a long, long time. Here's what to do about it: stop taking so damn long. It's not a fact of nature that it takes 6 months to referee a paper. Other fields don't tolerate this. Philosophy is not harder than these other fields. Editors tolerate this behaviour because its a collective action problem: there just isn't a big enough pool of referees who will agree to submit reports within 4 weeks.

Speed up journal review time, and the other problems shrink dramatically. Unfairly treated by an incompetent referee? Oh well, that's only a few weeks lost and there are plenty of other comparable journals. Conversely, if you wait 11 months for your incompetent report, the opportunity costs are huge (especially for the non-tenured).

I know for a fact that some of the people complaining about journal practices sit on papers for months themselves. They can get stuffed, as far as I am concerned. Until they change their behaviour, they have no right to complain. When enough people change their behaviour, the problems won't be solved, but they will be greatly diminished.

Anon

Even as a job seeker I have to agree with Brown here. I've been sent some real stinkers. I'm honestly as sympathetic a referee as you'll ever get. I actually read papers trying to find reasons to accept it and not vice versa but I've only recommended acceptance on maybe one of the five or six papers I've been asked to referee. Most of the others just reinvented the wheel. They weren't terribly written for the most part but they made a mediocre versions of arguments that someone else had developed in a much better way elsewhere. Most of them struck me as pretty good graduate seminar papers that the authors had just sent off, though one turned out to be an established academic trying to basically publish the same idea twice. (I would usually never try to find out the identity of an author but in this case the piece was so close that I'd have marked it down as plagiarism if it hadn't turned out to be the same person.)
I don't think we should bar graduate students from submitting but I think there ought to be some meaningful cost for submissions. I understand why people send in weaker papers. I've sent in my share of papers I wasn't sure about. And the reason is that the cost is very very low (an hour or two tops if they have a weird house style and a balky submission system) and the reward is very very high a publication or a much improved paper that has a better chance of acceptance elsewhere if one gets good comments. Honestly, I know everyone freaks out about it but the $20 submission fee at Imprint strikes me as a thoroughly good idea. In fact, I'd suggest something higher that actually had some bite. Maybe a staggered system with $50 for grad students and adjuncts, $75 for non TT, a $150 for TT faculty, and $200 for tenured faculty. I know those numbers no doubt strike some people as absurdly high but that's the point. It needs to be enough money that it's an actual disincentive. Allowing people to only submit one paper at a journal every year or even one every two years might also be a good idea.

Marcus Arvan

Anon 9:59: I appreciate the spirit of your suggestions, but they seem to me extremely heavy-handed, imposing costs in precisely the wrong place: on authors.

The idea of imposing a $150 cost per submission on TT faculty is incredible. It took me 14 tries to publish what I believe to be my single best paper. On your proposal, that would have cost me $2100 to publish that single paper. This is an unusual case, but many papers are rejected multiple times before finding a home. And imposing such costs on authors would not only place financial hardships on people who are already not well-off [many tenure-track positions do not pay that well, not to mention student loans many have to pay]. It would probably deter good research.

Further, I want to emphasize a few things Neil said: philosophy is nearly alone in having these types of problems. I know people in other fields where authors produce far more work, where [A] journal editors ALWAYS provide rationales for rejection, even desk-rejection, and [B] reviewers routinely do a good job.

There is simply no excuse for anyone to take 6-12 months to write a one-page review of a 30 page paper. If a paper's bad, a reviewer should be able to read it, write their review, and do their job. But they don't. In my view, following Neil, the evidence is that this is a cultural problem. Our field, unlike others, tolerates reviewer behavior that other fields do not.

grad

Marcus,

You say: "On your proposal, that would have cost me $2100 to publish that single paper."

I don't think this is true. If the reasons your paper was rejected all those times was a result, at least in part, of an overburdened peer-review system, then the cost would be likely be lower if the system weren't as overburdened. Also, weren't a lot of those submissions from before you were TT?

Postdoc

I am sure charging people to submit would lower submission rates. The problem is that it would increase the financial burden on young philosophers. Any amount high enough to deter submissions would seriously affect some people's ability to submit their work and have heat in the winter or healthcare (in many US states). Having a trust fund is already a prerequisite for surviving the long struggle of fixed-term contracts without ever having to live below the poverty line. Let's not make that worse, please...

Marcus Arvan

Grad: You're assuming that the problems are due to the system overburdening people. This is precisely what I'm questioning. Other fields are just as burdened as we are, yet I've seen some of their review systems work first- and second-hand, and they don't have the same pathologies as our field. In my view, it's a culture problem, not a problem of overburdening.

You're right that the publication in question happened before I became TT, but I'm still paying back tens of thousands of dollars of student loans now. I just have a hard time accepting that people like me--people trying to make their way in the profession--should have to pay thousands of dollars to publish journal articles.

The way to fix this problem is not to impose greater costs on authors. It's to insist on--and incentivize--better behavior by reviewers. Other fields have accomplished it without imposing extra costs on authors. If they can, so should we.

Postdoc

Hear Hear Marcus!

I agree it's a culture problem. In general, I think the culture in philosophy is bad.

I do think a big part of the reason has got to do with the job market. We train PhDs, who spend many years of their lives preparing to be professional philosophers, but when they graduate they're thrown into one of the worst job markets on the planet. Young PhDs are seldom prepared for this due to faculty being ignorant and tad bit dishonest.

The PhDs who do not drop out after a year or so often spend many years moving around for 1 year contracts or worse. Many in the US are just adjuncting. These positions do not even provide 1 year guaranteed employment. Many are hired for pure teaching positions or positions with very little research factored in. But they have to spend time writing and publishing articles anyway if they want a permanent job. This is, in effect, unpaid work.

How can you expect young PhDs in these positions do be good articles reviewers? They aren't paid for it. Many are already writing journal articles unpaid. They have no guarantee of future employment. They don't really feel part of the profession.

Anon

9:59 again: Given that referees do a service for the author (if they do their jobs at least) I don't see that it would be wrong to make sure the author isn't wasting their time. But I take your point that it's likely possible to improve journal practices without those sorts of disincentives.

I do want to add on an entirely different note that the list leaves out entirely two of the biggest things that would improve journal practices. Truly blind review and making sure that the editor who reads a paper is actually minimally competent in the subject area of that paper. Editors usually know who the author is and we all know that editors can put their thumbs on the scale in all sorts of ways over and above deciding what even goes to referees in the first place. (i.e. sending a paper to someone they know to be sympathetic or hostile to your thesis, sending it to reviewers who are known to be particularly harsh or particularly forgiving and so on). Further, I think a lot of the arbitrary desk rejections can be traced back to the fact that editors aren't competent to make judgments about many of the papers they read and also come to them with huge biases. John Q. Lemming likely isn't going to take any paper in applied ethics or continental philosophy seriously, and if he can't understand something a political philosophy or ethics paper says his first response is not going to be to think that he might not have the proper background to understand the debate.

Blinding papers isn't that hard to do in this age of electronic submissions so it's really inexcusable that journals don't do it. As for the second point, the distributed editorial system Ergo has, where papers are reviewed by section editors who specialize in the paper's field, is a solution that's both simple and brilliant. I don't know why it isn't the standard for the profession. Not only would this second improvement stop a lot of arbitrary desk rejections for promising work, but I think that it would also do a lot to take care of the problem of mediocre papers getting through to referees.

Adam

Team,

It seems weird to me that 30 journals represent the entire spectrum of the quality of journals available. The top-10 are Tier 1, the next 10 are Tier 2 (the 'lesser' journals, like AJP?) and the last ten ranked journals are Tier 3 (I'm told not even to submit in this range).

So out of probably over 100 philosophy journals, it is common not to"waste ones time" submitting to journals not ranked by some metric to be in the top 30.

Would it be beneficial to re-define the top 30 as "the best", then the next 30 ranked journals as the next tier, etc. so as to get more people published, spread out the number of submissions that each journal gets, possibly increase time to publication given fewer submissions, etc?

Again, it just seems to create a lot of problems that our profession thinks journals ranked (by some method) 20-30 (out of much more many journals) is the ranking range wherein one starts wasting one's time.

Verify your Comment

Previewing your Comment

This is only a preview. Your comment has not yet been posted.

Working...
Your comment could not be posted. Error type:
Your comment has been saved. Comments are moderated and will not appear until approved by the author. Post another comment

The letters and numbers you entered did not match the image. Please try again.

As a final step before posting your comment, enter the letters and numbers you see in the image below. This prevents automated programs from posting comments.

Having trouble reading this image? View an alternate.

Working...

Post a comment

Comments are moderated, and will not appear until the author has approved them.

Your Information

(Name and email address are required. Email address will not be displayed with the comment.)

Open job-market threads

Categories