Our books






Become a Fan

« Reader query on avoiding false social-political impressions on the job-market | Main | On discussing grad school with interested students »

01/14/2016

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

Pendaran Roberts

Here's my 'Long journey's into print' story.

It starts in 2012. I was working on my PhD on the ontology of color and wrote a chapter responding to arguments based on color variation (where two or more experiences disagree about the color of something). Eventually I decided to modify the chapter for a journal and try my luck. It was rejected four or five times (can't remember). One journal took 12 months to get back to me but thankfully with a somewhat positive rejection.

It's now 2014. I have my PhD (got it in June). I decide that my earlier attempts to publish were misguided. The derived paper just tries to do too much to satisfy the ever so picky referees. So, I take out what I think is the most original material and write a new paper arguing that colors are extrinsic (in that they depend on the colors of the surround). I knew this was a novel position, and I knew it contributed to certain debates positively. The paper was rejected five times from top journals and even from more poorly ranked ones.

2015 comes along. I don't know why the paper is being rejected. A lot of the comments show that referees aren't understanding it, but I can’t think how to make it clearer. Sometimes I get comments that just are not fair to the paper. Sometimes I get referees who are clearly incompetent. Other times I get the same referee who rejects the paper without comment saying, 'I've reviewed this before, and I still don’t like it' (or something to this effect). In a state of despair, I send the paper to a rather low ranked journal.

A month passes. One day I realize how to fix something in the paper that had been bugging me. No referee had brought it up, but I knew it was a flaw with the paper. After fixing the flaw, I became more positive. I thought to myself, 'this paper should be published in a top journal.' So, I withdrew it, and sent the revised version to Synthese. Three months later I get an R&R! The referees understood the paper and were positive about it. THANK GOD!

I had to make major revisions, which only made the paper better. The referees were really helpful. I was so nervous about the revisions that I couldn't sleep one night. The idea that after three plus years of trying I might screw up this only R&R was a lot to bear.

In the end, it was accepted! The title is ‘An ecumenical response to color contrast cases’ (DOI: 10.1007/s11229-016-1016-1). I am still waiting on the proofs, but will post a link here when it's online. I have open access through the University of Warwick.

Moral of the story? If you know you've got an original idea don't give up on it!

Best,

Pendaran

recent grad

Though these stories are interesting, they seem to be different from the stories of Clark and Chalmers' or Stanley's paper rejections. Both the OP and Roberts admit that earlier versions of their papers had flaws. The OP's was not original enough, for example. If that's the case, then the stories of rejection--though no doubt frustrating--are quite understandable. Their lesson seems to be: if you like a paper, keep working on it and you can improve it to the point of publication.

Marcus Arvan

recent grad: I didn't mean to imply, by my introduction, that the series would only or primarily feature stories akin to the Clark/Chalmers/Stanley ones. Rather, I referenced those stories merely as interesting anecdotes about the vicissitudes of the peer-reviewed publishing game. Our series merely aims to provide readers with stories of how different, long-gestating works eventually made their way into print. Maybe we'll receive some Clark/Chalmers/Stanley-type stories moving forward, and maybe we won't! In any case, the main intent of the series is to provide readers a "behind the scenes" look into different people's experiences.

recent grad

Marcus: I misunderstood in that case. I do like hearing about how things eventually get into print.

Journals

Marcus, I think that Recent Grad's point is that these are stories that show that the "journal-system" is working well and is in good order. Generally journals do not let people publish underdeveloped arguments. And, I want to underscore that it is NOT a journal's job to make an underdeveloped paper publishable. If anyone has that responsibility it is the author or in the case of graduate students, their dissertation committees.
This is important because there is an awful lot of complaining about how dysfunctional journals are. In my experience, the system does work well. I have published quite a bit, but I have also had my share of rejections.

Marcus Arvan

Journals: And I (mostly) wouldn't disagree with that. While there are a few journals I think arguably have problematic editorial practices (these issues have been discussed elsewhere), and I do share worries that well-placed people can have advantages in the process by posting and publicly workshopping papers prior to sending them out (which I have heard, and believe, plausibly undermines anonymized review), for the most part my experience is that the peer-review system works quite well. I do think there are alternative systems that might be better (I particularly like the way things are done in physics), but on the whole, my own experience has been that the system works *relatively* well.

Pendaran Roberts

I wish to note first that Marcus did not post my story. I posted it.

Second, I did not intend to give the impression that the review system works.

Think about it. Whether rejections are for good reasons or bad, continually thinking about a paper and trying to improve it over many years is bound to result in a better and better paper, which will eventually get published.

So, though my paper did get better and better due to rejections, most of the time the paper was rejected for poor reasons. I had at least 2 referees who were completely incompetent. I had another who basically had no concerns with the paper but recommended rejection anyway. The paper was good enough to be R&Red a long time ago. (The flaw that I ended up correcting amounted to the adding of a paragraph, albeit an important one.)

What my paper really needed was to find some competent and friendly referees who knew how to do their jobs. The referees at Synthese were great, and helped me improve it further. Most of the rejections were not justified or at least not justified for the reasons the referees cited.

So, yes numerous rejections result in a paper getting better. Yes, often rejected papers can be better. No, the system doesn't work. If the numerous referees who rejected the paper did so for consistent reasons, maybe. But the reasons were all over the place. And for someone who has worked on color for a 5 years, I can assure you they often were not for competent reasons.

Marcus Arvan

Penderan: First, thanks for sharing your story! I've had similar experiences myself.

In terms of "whether the system works", I think there are a few things we need to distinguish. On the one hand, we need to distinguish (A) particular instances, from (B) the system as a whole. Like you (and like most, of so I have heard), I have endured my share of referees who I thought did an incompetent job. But I have also, like you, had referees who did an amazing job, either in terms of providing wel-reasoned grounds for rejection, or well reasoned grounds for improving the paper in an R&R. But neither instance alone is sufficient to show whether the "system works." For that, we need to look at the system's output. Your paper, after all, ended up in Synthese--a good journal. So, it appears, at the end of the day, with your paper, the system seems to have worked. Your paper was well-vetted, and ended up in a relatively prestigious journal.

I also think we need to distinguish "ideal success" from comparative notions of success. No one, I think, would say that the current peer-review system is ideal in a Platonic sense. There are poor referees, and sometimes bad papers end up in high-ranked journals and awesome papers in low-ranked journals. In an ideal world, none of these things would be the case. Still, is the system *comparatively* successful? Arguably, it is. A peer-review system with double or triple-anonymized review, etc., is far better than many alternatives. Again, not ideal--but the relevant issue is, how well does the system compare to *realistic* alternatives?

Pendaran Roberts

I suppose the system as a whole 'works' in the sense that if an author has a novel and interesting idea and sends it around for years, someday the author will luck onto two referees who *get it*. The system does work in this sense, hence the moral of my story: don't give up on a paper if you believe in it!

However, to say this system works is kind of like saying a car that goes 4 mph, is really loud and uncomfortable, and burns you when you touch the door works, because in the end it does get you from A to B.

In my mind a review system that works, should R&R papers that are novel and interesting contributions and reject papers that are not in a timely fashion. Keep in mind that the system only 'worked' in my case because I was stubborn, thick skinned, believed in my work, and didn't give up.

Scott Clifton

In the context of the system overall, one question to ask is whether papers that reach a minimum threshold of quality are being published. But another question would be: are the best papers being published in the "best" journals? If we're going to have a hierarchy of journals (a different question that has already been considered in other places/times), then that hierarchy will be (and is) used to give prima facie indications of paper quality. If it appears in highly ranked journal X, then, prima facie, it's of a top quality. If it appears in lowly ranked journal Y, then, prima facie, it's of a less than top quality (which, hopefully, will not be of low quality). But Pendaran's paper almost fell through the cracks, since it was only Pendaran's realization that the paper needed this tweak and enduring faith in the paper that resulted in the publication in Synthese. Now, how many of us don't have that a-ha moment and, instead, "settle" for publishing what may be a top quality paper in a lower-ranked journal? Who knows? This is the problem with using WHERE a paper was published as a surrogate for HOW GOOD the paper is. Let's grant that only top quality papers are published in highly ranked journals (an assumption not everyone is willing to make, but ignore that). What justification is there for inferring that a paper published in a lower ranked journal is not top quality? And, to make matters more dire, what justification is there for basing some job search decisions about who to interview/hire on the fact that someone's work appears in lower-ranked journals, which is a different phenomenon from basing decisions on the fact that someone's work appears in a higher-ranked journal?

Journals

Scott's many questions point to one important question which I would like to answer, in part: Why does the journal where one publishes an article or the publisher with whom one publishes a book matter? It matters because it affects the likelihood that your work will get read. There are too many article to read and too many books to buy (let alone read). Libraries are more apt to buy Cambridge and Oxford books than books from many other publishers; indeed, faculty are more likely to recommend that such books be bought by their libraries. And a similar argument runs for journals - if you did not cite a single article from a particular journal in your dissertation, I do not know why you would bother sending them a manuscript to consider for publication. You should certainly not expect others in your sub-field to read the journal if you do not read it.

Scott Clifton

Thanks for the reply, Journals. This deals with a question I have wondered about: how many of us could be said to read journals or book publishers, rather than articles or books? That is, do people really wait until an issue of Journal of Philosophy comes out and then read it cover to cover? Or read the latest book in epistemology published by OUP?

Here is my experience, so I am curious if it's wildly different from that of others: I do literature searches on topics I am interested in. I use databases to find articles with keywords in the abstracts or authors whose work I have already read and confronted in bibliographies or titles that have terms I am interested in. I use my library's catalog system to search for books written on topics I am interested in. I run down references in bibliographies of work I am already reading. When I get results that seem appropriate, I download the articles or check out the books. Granted, if the journal or the book publisher is somewhat questionable--say, one I have never heard of or one that does not have a history of publishing philosophy--I may approach with some suspicion. But I judge the work by its quality and its appropriateness to my own purposes. In some cases, I will do a search on the author, see if their work has already been cited or whatever. I doubt if I've ever decided not to read work based exclusively on its publisher, though.

Sometimes, of course, I will just browse some journals and book publishers, but for the most part, I search out works on topics I am interested in, not work that has been published in particular venues. Maybe that's to my detriment--I don't know.

Marcus Arvan

Journals and Scott: I have no doubt that some (many?) people only read papers in journals they recognize--but my own approach is akin to Scott's. I don't much care where a paper comes out. If I see a new paper in areas I work in posted to academia.edu or philpapers, I tend to give it a look, even if it's not in a highly-ranked journal. Sometimes, I see rather quickly that it's not a great piece of work--but, all too often, I find work published or forthcoming in somewhat out of the way journals that I think is good or great. Similarly, I sometimes come across work in highly-ranked journals that I don't think is all that good. Although I do think, broadly speaking, that better work tends to appear in "better journals", this is why I utilize Scott's approach: I try my best to follow whatever work comes out in my area--rather than certain journals--because I want to find good work wherever it appears, even if it takes a bit more effort.

Pendaran Roberts

My personal approach is to search philpapers and google. I could care less about the journal. I'll look at anything relevant. I don't read journals; I read papers. I've noticed that papers at top journals tend to be more polished. However, as far as finding interesting arguments, interesting references, and getting good ideas for my own work highly polished papers don't matter as much. A less polished paper can have better more interesting ideas. In fact, sometimes the highly ranked journal papers are so polished that they can be annoying to read. What I mean is they will have way too many responses to objections, comparisons to other views, clarifications etc. For someone who just wants to know what the view is to write about, this can be a pain.

Neil

All ya got is anecdotes. Anecdotes is perfectly appropriate for the OP. The aim of the OP is, after all, to show that good people and good papers get rejected, so as to encourage those who are going through the same thing and show them that their experiences do not demonstrate that their papers are bad or that they should give up. But you just can't conclude from your experiences whether the review system works or how well it works. Not even accumulating all our anecdotes would do this. Who reports, and what they report, is going to be coloured by all sorts of memory and saliency biases. To find out whether the journal system works, you're going to need proper data: at minimum hundreds of data points gathered in a way to avoid biases. Ideally you would also need a way of identifying papers that ought to be published, though I think you could go a long way with just the first. The idea that your experiences constitute grounds for thinking that the system works, or doesn't work, is just naive.

Marcus Arvan

Hi Neil: you're quite right, and I stand corrected. Like many people, I have a sense of how well I think the publishing system works--but you're right: without carefully collected and analyzed corroborating evidence, we are not in any good position to tell whether any of these impressions are accurate, as opposed to mere reflections of our own biases.

Pendaran Roberts

I agree that anecdotal evidence is only so strong. I didn't mean to imply any general strong conclusions. I was just saying that I don't think my experience is evidence the system works (in the way I think it should at least).

Of course, I have so many personal stories and have heard so many from other people complain. Anecdotal evidence adds up so to speak!

Journals

Just to be clear, I never suggested that people consciously only read articles in J Phil and Phil Review, and Oxford and Cambridge books. (some people might, for all I know). But the journals and publishers with reputations have higher visibility. Again, review your own dissertations and see what sources you used. They will, for the most part, be the "name" journals and publishers.
Incidentally, from my own reading experience (as someone who is not early career), for the most part, papers in lower tier journals are generally much worse than those in leading journals.

Marcus Arvan

Journals: Fair point, and sorry if I misconstrued your position. I wouldn't deny that papers in lower-ranked journals *tend* to be worse than those in higher-ranked journals. I just think good/bad articles happen to appear in both types of journals. I've read terrible articles in highly-ranked journals, and excellent articles in lower-ranked ones. This is why I try to read whatever comes up in my areas in my philpapers feed. I don't want to miss good stuff wherever it appears.

Neil

Pendaran Roberts. The conditions under which anecdotal evidence supports a hypothesis is a very interesting question. We have no reason to believe that these conditions are satisfied in this case, however, because we have every reason to believe that we are getting a biased sample. Again, bear in mind the salience bias, asymmetrical memory searches, and so on. I agree with you that your evidence doesn't support the hypothesise that the system works. That's because it doesn't support *any* hypothesis.

Pendaran Roberts

Neil, I agree there are lots of reasons for concern. I guess I just don't know anyone who's positive about the peer review system, even people successful in it (including me I guess). I think looking at Journal Surveys explains part of the reason. The data shows many top journals are quite slow. This is much less of a problem in other fields. Philosophy review times are worse. Why?

Verify your Comment

Previewing your Comment

This is only a preview. Your comment has not yet been posted.

Working...
Your comment could not be posted. Error type:
Your comment has been saved. Comments are moderated and will not appear until approved by the author. Post another comment

The letters and numbers you entered did not match the image. Please try again.

As a final step before posting your comment, enter the letters and numbers you see in the image below. This prevents automated programs from posting comments.

Having trouble reading this image? View an alternate.

Working...

Post a comment

Comments are moderated, and will not appear until the author has approved them.

Your Information

(Name and email address are required. Email address will not be displayed with the comment.)

Open job-market threads

Categories