How are different types of publications weighted?

Yesterday on social media I came across an interesting question: namely, how different types of publications are weighted, both on the job market and in tenure and promotion. Among other things, it was brought to my attention that different types of publications can be valued more in some countries than others. For example, I learned that in Germany, publications in edited volumes are looked upon very highly, but that in the UK, this may not be true (with exceptions, perhaps, for publications in volumes published by OUP or CUP). Similarly, I heard that public philosophy (i.e. publications in popular press such as Aeon, The New York Times, etc.) can be looked upon highly in the US, but not in other countries. I think it might be great to crowdsource more information about these differences in the comments section below.

I also think it might be good to hear people's thoughts on other particular issues, such as how it looks to a search committee  or tenure and promotion (T&P) committee when most of a candidate's publications are in edited volumes (as opposed to journals). For example, suppose someone has a publication or two in decent but not great journals, but a bunch of publications in edited volumes by top-ranked presses with influential editors. How do people think this looks on the job-market and in T&P? On the one hand, I imagine that it may look good to publish in volumes put together by big name editors for good presses. On the other hand, I've always wondered whether publishing primarily in edited volumes may raise the following concern: namely, that publishing in edited volumes is a more ambiguous measure of research quality than publishing in journals. After all, one of the key things about journal publishing--at least in the present system--is that peer-review is anonymized. In edited volumes, on the other hand, the editors know who you are and (depending on the review process) so too may any reviewers they send your chapter to. Because content for edited volumes isn't selected by a fully (double- or triple-) anonymized process, the worry is that readers (i.e. search committees or T&P committees) won't feel in a good position to know if the work was selected primarily on its merits or as a result of the person having 'good connections.'

For these reasons, I can't help but wonder--at least in terms of job-market success and eventual tenure decisions--whether candidates should focus primarily on journal publishing. Maybe I'm off on this--it's just something I've long wondered about, and figured it might be good to hear from more people about. Anyway, what do you all think, particularly those of you with good experience in these matters? How are different kinds of publications weighted on the job-market and T&P? How do things differ from country to country? And how do things differ with respect to department types (e.g. do R1 institutions weigh certain things more heavily than non-R1's)?

Comments

[K]

A few comments, following up on Marcus' remarks.
I give less weight to pieces published in anthologies. They are mostly invited, and the refereeing is light. I have contributed to such things, and my motives are different than when I write for journals. I publish enough that I do not need extra articles, but there is a type of thing I want to write that fits nicely in anthologies, handbooks, etc.
I work in a country where contributions to anthologies and the like are given less weight than contributions to journals. There is a journal list, and the top 20 % journals count for more than the lower 80 %.
Young scholars should publish in selective journals. This will serve them well, no matter where they go with their careers. Anthologies and such are a treat or perk, but do not think they count for much. Especially for young people who really do not have a reputation, it is obvious that their connections got them the invitation.

[GermanBackground]

This is a great question to discuss, Marcus! Coming from the German-speaking world and having a TT position in America now, I have some experience with this kind of issues. Here are some of my thoughts:
1. I do not think general advice is useful. It depends a lot on your field (and your institution) which kind of publications are acceptable. I would look at CVs from distinguished scholars in your field and discuss the issue with your supervisor. While in most fields journal articles are the gold standard, in some monographs and/or book chapters can be valuable too. The same holds for institutions, when it comes to the TT process.
2. I find it astonishing that many philosophers simply take it for granted that journals have stricter peer-review processes than volumes. While this might be true in most cases, we learnt very recently that one of the top journals (Philosophical Studies) has sent many articles to one reviewer only (http://dailynous.com/2020/06/12/resignation-philosophical-studies-reply-editors/). I have contributed to volumes that had a double-blind review with multiple reviewers and, hence, a stricter review process. There is also the possibility that book series are indexed in Scopus, something that is more common in the natural sciences. Hence, there are ways to make the review process of volumes stricter and more transparent--and some editors do this already. We should acknowledge that.
3. While it is still more common in the German-speaking world to publish in volumes, journal publications become more and more important. I started my career in Germany and was expected to publish books and book chapters. Luckily, I had supervisor who told me that I have to diversify my publications. I have the impression that after I had shown that I can publish in well-respected journals, the books and book chapters became an additional asset of my file, something that distinguished me from others.
4. An important criterion to assess the quality of a book/volume in German is the series in which it has been published. University presses are not very widespread in the German-speaking world. There are only a few publishing houses (e.g. Suhrkamp, Karl Alber) that guarantee the quality of a book. Yet many renowned research institutions edit series for publishing houses and, hence, guarantee that the books are peer-reviewed. This is an important qualification, since German PhDs have to publish their dissertation as a book. Publishing your book in a series means that there certainly was an additional peer-review process by the institution that edits the series. Many PhDs have to fund the publication of their dissertation on their own (often by organizing third party funding). I know that this sounds like vanity publishing for English ears. That is why the place of the publication of the book (publisher, series) is even more important, since these are usually not vanity publications.

[NH]

I value invited contributions somewhere between book reviews and (blind refereed) journal articles. I agree with K that younger scholars are better served by directing their work to journals rather than invited contributions. Invited contributions can nicely round out a handful of journal articles.

[Malcolm]

Strongly agree with GermanBackground. There is no such thing as "how a publication looks," full stop, only how it looks to different audiences. This will depend on your institution and how tenure decisions are made there, as well as your field within philosophy.

Find out how tenure & promotion works at your institution as soon as you are hired. Also look at comparable institutions (or places you might like to get hired at). Compare your CV to others who have gotten tenure recently. Don't look just at total numbers, but numbers by year.

Further, while I am pre-tenure and this advice is only on the basis of what others who have gotten tenure have told me, it's not just the facts, but your narrative that counts. Can you give a coherent account of your work to date and the direction it is headed? Is there a reason you've selected the journals you have? For instance, while university presses like Oxford and Cambridge may be considered as "top" by default at most places, your field might be such that they aren't actually the best places for your publications.

While the answer may be, to an extent, that you've published in X journal because they accepted your paper, you can still say things about why that journal is a good one and relevant for your work.

Whether candidates should focus just on journal publishing or not, then, depends on what field they are in, and what their T&P committee is like, along with their personal goals. I strongly doubt there's a single answer.

[JR]

I am surprised at what GermanBackground said regarding Philosophical Studies and their referee process. I have never thought we should, or that people do in fact, think that journals that use just one referee are not as highly valued than journals that use two. Many respectable journals use (mostly) just one referee nowadays. Analysis and Ratio at least comes to my mind. If you have received rejections from those journals you receive one not two referee comments. (and authors of published papers thank a referee not referees even though the journals themselves does not, for my knowledge, tell how many referees they use per paper). The point of referee process is not that every paper goes through exactly two external referees, the point is that someone checks the paper is of sufficient quality and for that we do not necessarily need two referees (but sometimes we need three for example). Phil & public affairs do not even use external referees but their papers are referee by the editors themselves - yet it is highly valued venue.

Regarding the original question. I assume journals that have the word 'American' are valued more in US than journals that have the word 'European' in their title (and vice versa). Such as Journal of American Philosophical Association or European Journal of Philosophy.

[GermanBackground]

Sorry, JR, but you misconstrue what I say. Marcus argues, in my opinion correctly, that people worry about that the "content for edited volumes isn't selected by a fully (double- or triple-) anonymized process." Hence, people assume that journal publications pass a formally better peer review. I argue that there are volumes that have a formally better peer review process than journals (and yes, I regard blind review by multiple reviewers as formally better than by a single reviewer; and not only me as you see here: http://dailynous.com/2015/02/02/guarding-the-guardians-or-editors/ ). This is not saying that these journals have less of a reputation. It is saying that referring to the review process as argument for their reputation is not always correct. So, you are right, journals with what I see as formally weak peer-review processes have an excellent reputation in philosophy. It is another discussion whether this is good, hence I leave this question aside. My point simply is that what many see as a main "objective" difference between journals and volumes is not always the case. I argue for being more honest, since we are a reputation driven field with sometimes remarkably weak formal standards, as demonstrated here: http://www.adrianpiper.com/berlinjphil/philosophy-journal-paper-submission-policies-2018-table-1.shtml

[JR]

GermanBackground, sorry if I misinterpreted you. Journal refereeing is not always as strict as it seems. I know journal editors who publish their partner's/student's/friend's papers without a proper peer-review process. I admit that I am not sure how strict peer-review is when it comes to edited volumes, for example. Personally, I think one referee for a paper is enough, most likely the referees will just disagree with each other anyway.

I think one should publish in edited volumes in addition to journals because most papers in edited books are invited. If you managed to get your paper in a volume that indicates for others that someone likes your work and you are considered someone inside the field. I mean everyone can publish journals, even without affiliation or a formal degree - if the quality of the work is high enough to pass the referee process. So, I guess someone might be suspicions if one is publishing only in journals and it looks like no-one is inviting that someone to publish in edited books - perhaps it indicates that no-one is paying attention to his/her work?

[K]

JR
With all due respect, you have it backwards. The reason early career people get in invited volumes is almost always because of connections - their supervisor is invited and passes it along. I have pieces in numerous such volumes, and I have pieces in very good journals (PhilSci, Nous, APQ ... and non-phil journals as well). I know the pieces that made it through peer review in journals did so on their merit ... but the ones in edited volumes. I was always asked to contribute. Of course there is some refereeing involved. But it is pretty light weight. When I review files for hiring I give far more weight to well placed journal articles.

[Michel]

My sense is that most volumes are compiled by invitation, and these are usually (though not always) less rigorously reviewed. My sense has also been that, for grad students and early-career folks, many of these invitations come through supervisors. (My own supervisor offered to hand an invitation down to me once, but I had to decline because I knew absolutely nothing about the particular topic, and was in the midst of the final dissertation push.)

So I would usually count that kind of publication for less than I would a journal article. A clear exception that springs to mind are the recurrent 'Oxford Studies In...' collections, which are top venues in their subfields.

I have a couple pieces in edited collections. In both cases, I responded to an open CFP, and my papers went through anonymous review (of some kind). One process was quite rigorous, the other less so (but then, it's a public-facing volume. Plus, frankly, I did an *amazing* job on that one, rather than a merely serviceable job, like with my other one!).

On the strength of the rigorously refereed one and a related conference presentation, I was invited to contribute to another volume, for 2021. I've accepted the invitation. I expect that'll be refereed, too, but perhaps not as rigorously as if there'd been an open CFP.

(Oh, and FWIW I recently published and PhilStudies, and got two referees. IIRC, my Analysis submissions have typically been rejected by two referees, too.)

[Michel]

Oops. I meant to add one more thing:

Even though I think invited pubs usually count for less on the CV, I often value them more highly, as far as research is concerned.

What I mean is, I've found that edited volumes are often invaluable resources for research. Often the work that's published in there is work that would clearly struggle to be published elsewhere--not because of its quality, but because of the topic, or its narrowness, because it's a quirky idea, because it's off-the-wall, etc. It's often exactly the kind of thing that referees can get nasty about, but also exactly what I want to be reading on my deep dive into some topic. I've found that the work in these collections tends to be super interesting and super imaginative, so I think they fill a real and valuable niche.

(Maybe that means that I'm wrong to assume that they count for less, on a CV. I don't know. I guess what it points to is the difference between using venues as a heuristic, and actually looking at the content and quality of the paper itself.)

[JR]

K said: "The reason early career people get in invited volumes is almost always because of connections - their supervisor is invited and passes it along."

This is what I mean. Getting invited is a sign that the person is considered to be a part of the academic group. And I think academic jobs for instance go for people who are considered being a part of the academic group. If one is just publishing in journals, one's work is good. But it is not a sign that one has connections, has friends at other universities, is considered a member of the academic group, anyone is paying attention to that work etc.

Michel: thanks for your points. Maybe the number of referees is not always the same with the same journal. I have received rejections with 3 ref comments from journal that normally use just 2 referees.

[anon]

To add to JR's point that invitations can carry a lot of weight, it's just not true that editors who send out invites to edited collections will accept the original senior invitee's grad student/some other junior person in their place. Editors are also concerned with the prestige of their table of contents. For a junior person to have your work appear in an OUP volume alongside the top names in your field is a big deal, the same is not true if you're in an invited volume alongside lots of grad students or junior folk, no matter how good their work is. Most senior folk also won't accept an invite to a volume that's mostly grad students and/or junior folk. Part of my advice for accepting such invites is to consider who your work will appear alongside. Just as you want your work to appear in the most prestigious journal, you want your name to appear alongside the most prestigious in a book's table of contents.