Our books






Become a Fan

« Journal of Experimental Philosophy - expressions of interest | Main | A blog milestone »

04/07/2016

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

C

I think it's real and worth combating, but I don't have a ton of good ideas for fighting it. Let me just mention two small things we're doing now (I'm leading a search committee). First, I discuss this with my colleagues. I try to make the case that there really is such a thing as prestige bias and that it would be a mistake to let prestige weigh against more important factors in hiring decisions. My hope is that raising awareness is a step towards minimizing the effect of the bias. Most of the people I take to agree that it would be silly to let things like pedigree enter into our assessment of candidates. Second, I've been trying to get people to come up with quantitative assessments of candidates on the basis the things that we do think matter and try to use those as the basis of discussions of candidates when ranking them and deciding who makes it to the interview stage. The discussions tend to be based on notes and scores contained in a spreadsheet where there's just not any information about things like where the person got their PhD and my hope is that having lots of information present that has little if anything to do with prestige helps make the ranking more rational than it might be otherwise.

Jerry Green

One thing that I've heard in defense of prestige bias is that institutional ranking really does track student quality, so its not really a bias (i.e. the best schools tend to recruit the best students). And of course people aren't going to combat prestige bias if they think its justified.

Two reasons I suspect this line of thinking doesn't work:
1) There's very little reason to think that there's an innate philosophical talent that you manifest in your senior year of undergrad (let alone in high school when you applied to the good undergrad that helped you get into the good grad program). Thinking in terms of innate talent is bad in a number of ways; for a start see http://dailynous.com/2015/01/15/raw-intellectual-talent-and-academias-gender-gaps/
and
http://dailynous.com/2015/02/05/more-on-innate-talent-and-philosophy/
2) Folks at the best schools get a lot of benefits that you don't get elsewhere. Funding tends to be better, which means more semesters on fellowship rather than teaching. Its easier to travel to a bunch of conferences, either to present or just to attend and network. You have more access to more professors, both in your own faculty and with visitors. So, if anything, we should expects the people at the top departments to have more impressive CVs: more publications, more presentations, etc. But at least in my experience its the reverse: a PhD from a top department with no publications will beat out a PhD with publications from elsewhere.

This suggests two concrete steps in response [and kudos to the OP for thinking in terms of concrete steps rather than just complaining]
i) Stop thinking in terms of innate talent, and allow for growth, the value of hard work, etc.
ii) Expect *more* from applicants from top schools, not less. If you have more time, more resources, and more personal connections, then you should be outperforming those without these benefits.

Ed

First pass thought: break hiring up into rounds (as I assume most departments already do). Let round 1 be based on materials that do not allow those on the committee to see the PhD granting institution - redacted CVs, overviews of teaching evals but no original copies that name the school, writing sample appropriately blinded, and so on. In later rounds - when a committee is down to say 15-20 candidates - allow in letters and institution identifying information if it is deemed valuable by the department. This comes at the cost of not seeing letters early on and letters can contain helpful information, but for whatever it is worth, many programs in the UK request letters only later in the process. I have no reason to believe all those departments are at a loss to make good hiring decisions. There are costs, but perhaps they are outweighed if they level the playing field a bit when the field is big. I'd be very glad to hear if readers have worries or objections to this since it's something I'd like my own department to consider doing. One worry is that it is extra work for candidates, though I think it is pretty minimal since redactions are very easy to make in various programs such as acrobat.

Grad student

I meet criteria a and b, but not c. I'm a grad student interested in overcoming the negative effects of prestige bias on my own opportunities. I'd be interested to hear, from hiring committees, about cases where they did choose to interview or hire candidates from less prestigious programs over more prestigious ones. Which factors played a role in taking the application of the candidate from the less prestigious program seriously? Is it primarily publications (quantity or quality)? Letters from well known figures in the field? Particularly engaging application materials?

I'm not asking for any universal criteria for getting someone an interview or hire, or for negating the effects of prestige bias. Rather, I'm just trying to get a sense of which kinds of things have (and hopefully will) get someone from a low ranked (or unranked) department seriously considered as a job candidate.

from a search commitee member

To grad student:
I give careful consideration to candidates from any school. I want to see: publications in good journals (the type of journals that make a top 20 list); and evidence of effective teaching, including a range of courses taught, with thoughtfully constructed syllabi. IMPORTANT: I do not fetishize "highly ranked" journals. I have refereed for more than 30 journals (more than 100 papers), and the better journals publish better paper, generally. My preference for candidates publishing in ranked journals is due to the fact that it shows good judgment, maturity as a scholar, and that the candidate can stick with it, and revise a paper until it is worth publishing in such a venue.
But I have sat on hiring committees where a colleague's only accomplishment was the fact that they went to an Ivey League school. Such people inevitably privilege applicants from prestigious schools. It is almost hopeless pushing for a candidate from a lower ranked school with such a colleague. That is my experience.

Grad student

Thanks, search committee member. It's really helpful to have this kind of insider view on the process. It's nice to know there are actions we can take to be taken seriously on the market. But, at the same time, a bit disheartening to hear that some search committee members simply won't be swayed because of their own background.

recent grad

Mark Lance or ES said once over at NewApps that if two candidates' CVs are equal, they think more highly of the one from the more lowly ranked school. The thought is that they did more with less, as JG above suggested.

a philosopher

I agree that there is a tendency for better papers to appear in better journals. However, I want to remind folks of what has been said before. There are certain publishing advantages for those from prestige places. One is that papers are often presented at conferences, and then the reviewer knows the author. Here is another. One of my advisers was working on a paper for a long time. But before sending it to a journal, he sent it to 10 of his very famous friends for feedback. Because of the paper's topic, it is very likely one of those friends will also be the reviewer. (And I know many who will review a paper even if they know the author.)

from a search commitee member

In reply to "a philosopher";
My remarks were directed more at assessing candidates for a junior position. The advisor example you mention, I assume, involves a senior person. And of course certain people have advantages others do not have. But imagine my position on search committee. I cannot say, "I know they have not published in a ranked journal, and I know they have not graduated from a ranked PhD program ... but let us hire them over all these others (who have had all sorts of advantages)". Even if I said that, I doubt I could persuade a search committee to follow me. At one point, people have to do the best they can to make up for the disadvantages they have.

a philosopher

a search committee member,

I agree with everything you said. And I am glad you are trying to do what you can to fight prestige bias. (I know many that do not try). I guess I was just expressing some frustration. Sure I was talking about a senior person. However, there could be similar things going on with junior folks. With the already very high bar to publishing in top places, the other small barriers make a difference. The things that those from less prestige departments must over come can be overwhelming. But yes, those with disadvantages just have to do the best they can. And so those that do publish in top journals have to overcome a lot and should be rewarded accordingly.

Another minor compliant: Those at top schools, both senor and junior, often come off as clubby. Many seem to know each other, interact regularly in reading groups and conferences, and have special access to the famous people within and without their department. One thing I find irritating is when at a reading group or conference, persons refer to a well-known philosopher by his/her first name. (even when the philosopher is not in the room) For instance, "John tries to argue X, but I point out Y. He does not like my reply, but I told him...) This is more me emoting than anything else. But here is another true story:

Me: Send paper to journal and I wait over a year with no reply. I send multiple emails and no answer. I ask a friend/famous person to write to the journal on my behalf. They respond within four hours and give me a R&R. Take that for what you will.

Finally, in spite of what I have been saying, publishing seems a more fair way to evaluate candidates than many others. I think I would be in favor of the algorithmic approach Marcus has advocated.

anon

to counter not only prestige bias but also gender and race bias, it seems pretty obvious to me that writing samples, CVs, and the dossiers in general should be anonymous and that a first cut should be made on the basis of these anonymized materials. Some SCs appear to already be doing this (I wasn't on these SCs, but I applied to a couple places in the past few years that required anonymization).

Of course, reference letters are a bit harder to anonymize. I think the gender/name of the candidate should be carefully obscured in reference letters, but they can be a give away as to the candidate's institution, so long as the letter writers' name is not obscured. I think these should be used only in making a second cut -- perhaps before the longlist?)

I do think that if anonymization is employed, it has to be followed very strictly. I.e., it should be impossible for a SC member to "peek" even if they're so inclined. Since googling could probably reveal identities, there should be a very strong social & ethical prohibition on this. Otherwise, we get the worst of both worlds: the *appearance* of anonymous evaluation combined with all the biases of non-anonymous evaluation.

from another search committee member

To grad student:
Our committee did not do anything explicit to combat prestige bias. However, our department did pass over candidates from more prestigious institutions in favor of candidates from less prestigious ones. We also did not have any members push for candidates from particularly prestigious departments. I give careful consideration to candidates from any school. I look carefully at the entire dossier. I care particularly about publication record, writing sample, and evidence of teaching effectiveness. I want to hire someone who does interesting work, teaches well, and will get tenure at my (teaching oriented) institution. Because tenure at my institution does not require particularly prestigious publications, I generally look positively on both top 20 publications but also those out of the top 20. However, top 20 publications are still more impressive and a publication record primarily in 'vanity' journals or journals that are essentially conference proceedings is a big negative for me. To me, writing samples make a big difference. Some writing samples were spectacular, others less so, and a spectacular writing sample can put someone at the top of my list, regardless of institutional prestige. A strong teaching record is also crucial for my school, and I would be wary of hiring someone from a prestigious department with little teaching experience or evidence of teaching success.

postdoc

A friend and me were looking over the appointments section on philjobs the other day. We were amazed by how many candidates from prestigious departments got jobs and good postdocs with CVs that were not that impressive or not impressive at all.

As it was not hard to find these examples, I can only surmise that this means there is a big problem. In fact, there is more than just anecdotal evidence for this, which I believe Marcus has posted on: that top 10 schools account for like 50% of TT appointments, or something like that.

It makes me wonder whether the majority of PhD programs should just close down. If the community cares mainly about where you got your PhD, the idea that you can publish your way into a job is unrealistic.

anonymous grad student

Do people genuinely think we should be evaluating job candidates purely based on their CVs?

That's a serious question. I've always thought that hiring departments don't get enough information about a candidate to make a decision--and they get lots of information. But I would imagine, if it were me, that I would want the writing sample to play the primary role in my decision (and maybe also the content of the job talk, but there are a lot of things that can throw that off). If most search committees were to do that, there would be no reason to think that the CV would be a guide to hiring decisions in the first place.

I find it mystifying that a bunch of people on the internet are so convinced that they can judge the quality of job candidates, and know that there is unjust bias towards high leiter ranked departments, with so little information about job candidates or about how departments made their decisions.

Also, I should disclose that I am a grad student at a high ranked department, but that I used to be one at a much lower ranked department. The difference between the quality of grad students at these places is pretty marked. There were a few people at the place I used to be at that seem to me just as good as the grad students where I am now. But the grad students where I am now all seem to me to be consistently good.

Of course, everyone is now going to yell that this is just my own prestige bias, and maybe so. But I've talked to many people (e.g. faculty who have moved from lower to higher ranked depts. or vice-versa, etc.) who have a similar impression. It would be interesting if we were all so consistently deluded about this. Maybe it's true. But maybe not. I don't think we really have evidence one way or the other, and since we don't know how search committees are making hiring decisions (but they almost certainly are not making them based purely on CVs), we really don't know how much unfair bias towards prestigious places there is. I'm completely open to there being lots, but this is (a) an empirical question and (b) one that seems nearly impossible to settle.

Anonymous VAP

Dear anonymous grad student,

Your comment reads like (a) and (b) at the end are taken to settle the issue -- that is, that you think the fact that we can't be sure whether there is prestige bias means we oughtn't be worried about it. But I'm not sure why this should follow. It might be there. Several of the other commenters seem to be pointing out that there are relatively straightforward steps to take to combat it if it is there. It seems to follow that, absent the steps having undesirable consequences, that they should be taken. Or am I missing something?

anonymous grad student

No, I agree that we ought to worry about it--I was responding to 'postdoc' above, and other people who continuously seem to think that one can "prove" prestige bias by glancing at cvs of people being hired. I completely agree that we should be worried about it and that we should take the straightforward steps, just as we should do what we can to combat race and gender bias, etc., even if we didn't have evidence of those. Sorry to not say that clearly. I just think we should *also* not jump to conclusions about why particular individuals get particular jobs and think it's a little weird that people think they can do a quick CV check and determine that the reason was prestige bias. So yeah, agree with most of what is going on in this thread, just strongly disagree with that.

postdoc

I guess it's not hard to 'prove' it If you see someone with no publications who got a great postdoc and they happen to be from Princeton.

What other explanations for this do you have? I can think of some.

1. Connections

2. He/she have an amazing research proposal.

Connections are also something that comes with prestige and seems likely related to prestige bias. But anyway, I am happy to say that connections are part of it.

Regarding 2, do you really think that the proposal can be so freaking good that he/she genuinely looks to be a better applicant than the 50-100 other people with amazing publication records and (at least) good proposals from less prestigious departments?

No rational committee would think that a two page proposal is worth more than 50 plus pages of published research. haha! So, It's not a big leap to think that something is going on here. It's either prestige bias or connections or both. I don't see any other rational explanation.

anonymous grad student

Yes, I can think of more, for example, that people read writing samples (this seems to me the most important one). That people who the committee trusts sell the candidate's work in a compelling way (I don't mean just "say great things about the candidate", I mean show the committee that it matters and is important). That the candidate clearly explains their own work and why it matters. That the candidate's research compliments the research of people in the department well.

Given what I know about how searches work at various research depts, I'd be much more worried about people getting thrown out in the first round based on what school they got their PhD from (a form of direct, obvious bias that seems really unjustified and terrible) than being unfairly discriminated against in later rounds, because in later rounds, at least at the schools I'm familiar with, writing samples really do get read.

Also, I've read an awful lot of crappy papers published in journals, even "top" journals. I bet you have too. There's no reason that search committees should trust that because a paper is published, it is good. So I'm not all that sure why they should be that concerned about publications.

Anyway, I'm all for taking steps to remove prestige bias, but CVs are just not how job decisions are made, and they should not be how job decisions are made.

Sam Duncan

So I appreciate the comments from search committee members here and I'm glad to see that at least they are going with factors like publications that have more claim to track qualifications than does pedigree. But I did have a question and possibly a concern about the statements about top 20 journals. How would you, or anyone else on a search committee for that matter, rate publications in very good specialty journals. I take search committee members point that, "My preference for candidates publishing in ranked journals is due to the fact that it shows good judgment, maturity as a scholar, and that the candidate can stick with it, and revise a paper until it is worth publishing in such a venue." But I think that the same could be said for publishing in a lot of specialty journals. Take history. My feeling is that for historians even though say BJHP or Archiv doesn't quite make the top 20 they're as good as say publishing in a top 20 generalist journal. It certainly shows good judgment and maturity as a scholar to get something published there. (Okay maybe I'm biased since I published in BJHP and have always had great experiences even with rejections, but I doubt many historians would disagree with me. Feel free to if you do.) A lot of very good papers in history that get published in BJHP or Archiv probably wouldn't get published in generalist journals due to some rather arbitrary judgments about what has "news value" or "general interest" but they nonetheless make substantial contributions to their fields. So I think specialty journals should be factored but the problem is then that I wonder how good search committee members are at knowing what specialty journals are comparable to top 20 if they don't work in that area. This is complicated by the fact that with specialties the group of relevant good journals can be very small and the drop off in quality from one spot on the list to the next can be very steep. So do committee members factor in good specialist journals? And how do they figure out which are good if they do? Do they just look at say Leiter's rankings for the relevant specialties, ask people who work in the field, or something else entirely?

An anonymous philosopher

Just two comments in response to 'anonymous grad student', if I may.

First, it should be no surprise at all that people with neither publications nor a complete dissertation from high prestige departments have high prestige professors expending quite a lot of effort making sure that their students have top-notch writing samples, and research statements that appeal to high prestige professors at other departments. This is, I take it, precisely the kind of 'finger on the scales' that many object to, and that the practices that 'recent grad' alluded to are supposed to counteract.

Second, the fact that 'anonymous grad student' notes that we have all read a crappy paper or two in a top journal is of course quite weak evidence that a search committee should discount or in some other way not be "concerned" whether a job candidate has published in a top journal. (Would 'anonymous grad student' agree that since we have all encountered a crappy product of a high prestige department, we should discount or in some other way not be "concerned" whether a job candidate comes from a high prestige department? Somehow I doubt it.)

JR

Just want to second Sam Duncan's comments about speciality journals vs. generalist journals. This seems especially important for search committees hiring for supposedly "open" positions, where one's specialization is (presumably) not supposed to matter as much as just being a "good" philosopher (however this is defined). While they do publish articles on a variety of topics, in my experience most of the "top generalist" journals aren't really generalist; they tend to favor papers in areas like M&E and ethics, especially those on topics and in styles generally associated with *ahem* prestige departments. (How often does PPR actually publish research in phenomenology, for example?)

So, if publication in "top 20" journals is used as a major metric, for someone on the market working in an area that doesn't fit the mold of these supposedly generalist journals for topical or historical reasons, not only are they at a disadvantage in that there are typically fewer jobs for them to apply for; they are even at a disadvantage as candidates for supposedly "open" jobs. In my experience following searches and resulting hires through my several recent years on the market, "AOS/AOC open" has most often turned out to mean "open to 'core analytic areas' as defined by prestige departments."

I don't want to deny that there is a lot of great work being published in "top" generalist journals, but search committees would do well to remember that there is a lot of work that does not get accepted for publication in such journals not because of its quality but because of its style or subject matter (and I would guess that quality work of the latter sort comes disproportionately from applicants from non-prestige departments, which tend in my experience to be open to broader conceptions of what counts as good philosophy).

Of course, a good writing sample should allow an applicant to overcome such bias (at least in the case of open positions), but only if the candidate gets far enough along in the process that her sample is actually read.

from a search commitee member

Sam,
I am the one who made the comment you quote.

Let me quote you quoting me: "My preference for candidates publishing in ranked journals is due to the fact that it shows good judgment, maturity as a scholar, and that the candidate can stick with it, and revise a paper until it is worth publishing in such a venue."

I did not say top 20 generalist journals. In fact, I do not work in a core area, and I count some of the specialist journals in my area in the top 20 (Philosophy of Science, BJPS; Synthese, for example). I also count some history of philosophy journals in the top 20. Indeed, I have a deep appreciation for history of philosophy. But people on search committees value different things.

from another search committee member

Sam,
To answer your question (and concern), I treat top-ranked specialty journals similarly as 'from a search committee member.' I am well aware of the top specialty journals in different subfields (philosophy of religion, phil science, ethics, political, feminist philosophy, etc.), and treat publications in them roughly on a par with similarly ranked generalist journals. Other search committee members I know seem similarly minded, so I think the concern here is probably misplaced.

from a search commitee member

to "from another search committee member":
You sound as sensible, and level headed as me. I wish you were serving on my search committee rather than another search committee.

Sam Duncan

"from a search committee member", I didn't think you mean top 20 generalist journals but I wasn't sure. I'm glad to hear that you and "another search committee member" think the worry is probably misplaced.

Verify your Comment

Previewing your Comment

This is only a preview. Your comment has not yet been posted.

Working...
Your comment could not be posted. Error type:
Your comment has been saved. Comments are moderated and will not appear until approved by the author. Post another comment

The letters and numbers you entered did not match the image. Please try again.

As a final step before posting your comment, enter the letters and numbers you see in the image below. This prevents automated programs from posting comments.

Having trouble reading this image? View an alternate.

Working...

Post a comment

Comments are moderated, and will not appear until the author has approved them.

Your Information

(Name and email address are required. Email address will not be displayed with the comment.)

Open job-market threads