The Secret Lives of Search Committees - Part 17: how decisions are made

One thing I often wondered about as a job-candidate is how decisions are made: decisions about who to interview, who to invite to campus, and finally, who to hire. From an outsider's perspective, I suppose it's natural to think that at each stage, search committees come together, agree upon which candidate(s) are best (as researchers, teachers, interview performance, etc.), and then choose to move forward with whichever candidates are ranked 'the best.' Having served on three search committees now, this now strikes me as a vast oversimplification of what actually goes on. In reality, decisions about who to interview, invite to campus, and who to hire are complicated by a variety of other factors. Allow me to explain.

Because my experience on the hiring side of things is limited to my department and my university (a rapidly growing 'teaching' institution that cares increasingly about its research profile), I don't mean to imply that all of my experiences generalize. So, as usual, I think it would be good to hear in the comments section from other people who have been on the hiring side of things. Hopefully, if we hear from enough people, job-candidates reading this can get some idea of how generalizable my experiences are, as well as a picture of the kinds of things that can affect decisionmaking in general. For simplicity's sake, I'm going to deal in this post with decisions at a general level, mostly abstracting away from differences between decisions about who to interview, invite to campus, and hire. While there are some differences in decisionmaking between the three levels (a few of which I will draw attention to), my own experience has been that decisionmaking is pretty similar at the different stages.

At any rate, here are some things I've learned about how decisions can be made:

One little thing can make all the difference: As I mentioned above, it's natural for candidates to believe that when hiring committees select who to interview, invite to campus, or hire, they select 'the best performer' (i.e. the best researcher, best teacher, best interviewee, etc.). However, in my experience the reality is more often like this: the committee ranks candidates on the basis of 'performance', but there are many candidates who are essentially tied with each other. So, for instance, the committee may agree by consensus that candidate A is the best performer, but rank candidates B and C as tied for second-best. Suppose then that the committee can only invite two candidates to campus, so they have to choose between B and C. What sways the committee's decision about who to select? In my experience, all too often it is one little thing, like the fact that candidate B has experience teaching one class that the department would like to have taught, or the fact that B has taken more initiative working with students outside of the classroom. In other words, often enough a candidate 'wins out' over another candidate at a crucial point not because of performance (as they are 'tied' in the committee's eyes), but simply because there is one little thing the one candidate has in their favor that the other candidate does not. This is one of the biggest reasons why, as I mentioned in my post on 'fit', I think the best job-market strategy for candidates at places like mine is to simply 'do more': teach a broader variety of classes, get more involved with students, etc. 

Particularly dramatic differences between candidates can make a difference: One of the more interesting things I've experienced is that sometimes search committees can make decisions partly on the basis of how different candidates are from each other. Here's what I mean. Suppose candidates A and B are similar in many respects (viz. their research profile, teaching style, etc.), but candidate C is very different (their approach to research is very different, their teaching style is very different, etc.). Now suppose that in the committee's overall ranking of candidates, A is ranked '1', B is ranked '2', and C ranked '3'. You might expect that the committee would invite candidates A and B to campus...and you might be wrong. Why? Here's one scenario: the committee might agree that if they had to choose between A and B, A is the clear choice, as A and B are very similar in most respects but the committee thinks A is better. In that case, the committee might see little point in inviting B to campus, as they might perceive it to be very unlikely that B will do anything in the on-campus to "flip" their ranking between the two. On the other hand, even though C is currently ranked '#3', the committee might have an interest in seeing whether the dramatic differences C has compared to the others may flip their ranking during the on-campus. In other words, even though C is currently ranked 'lower' than B, the committee might invite A and C to campus because they are interested in seeing whether C's dramatic differences from A and B might show up and make a huge difference in the on-campus. 

'Coin-flip' cases are common: I would be very surprised if committees ever actually flipped coins to decide between two candidates who are otherwise “tied.” However, my sense is that there are quite a lot of cases where committees find it 'impossible to decide' between multiple candidates, as the committee may rank two, three, or four candidates as basically tied, all-thing-considered. Often, what happens in these cases is that someone says in jest, "Can we just flip a coin?" No, a coin isn't flipped then, at least not in my experience. The committee finds some way of deciding between the candidates - but quite frankly, at this point the committee may have to search so hard for some way to make the decision that it might as well be luck. 

'Who we are likely to get' can make a difference: when it comes to who to choose to fly out, job-candidates might once again think that the 'best candidates' are always chosen. However, there is a problem here. University administrators may place limits on how many candidates can be flown out to campus - usually two or three. Suppose then that your limit is two, and administrators might get upset if neither of the two candidates invited to campus 'work out' - such as might occur if offers are made to both of them but both candidates accept job-offers elsewhere. This may put some pressure on the hiring department to invite to campus at least one candidate who is judged to be likely to 'accept the job' if it is offered. So, for instance, suppose candidates A and B are judged to be the best performers, but there is some real worry that A and B are likely to get offers for other jobs (since their applications are very strong). Suppose then that candidate C is ranked just behind A and B, but there are some reasons to think they are less likely to receive competing offers (perhaps they have been on the market a while and haven't been able to get a job). In that case, a committee may invite A and C to campus to 'play it safe.' Even though they might have liked candidate B a bit more than candidate C, they might invite C to campus instead, simply because they want to maximize the probability that at least one of the campus invites will accept the job if it is offered. 

Perceived job-happiness/flight-risk can make a difference: I've talked to a lot of people about this, and my sense is that people are very split on it. On the one hand, some think committees shouldn't try to judge which candidates might be flight risks or 'unhappy in the job' (due to, say, a high teaching load). On the other hand, I know plenty of people who think it would be foolish not to take these things into account in hiring (because, as I have said many times, hiring someone who leaves or is unhappy in the job once hired can be equivalent to a 'failed search' from the committee's perspective). Whichever side of this debate you fall on, my clear sense is that these kinds of perceptions can play a real role in committees' decisions. In fact, I was told as a candidate by more than one search committee member that this played a role in their decision regarding me. So, for better or worse, if you want a job as a candidate and you might be perceived as a flight risk or potentially 'unhappy' with the job in question, try to dispel those perceptions in some way in your file and/or interviews.

Committees may have to conduct interviews they don't want to: This may sound absolutely bizarre, and indeed it is in my view one of the most regrettable features of the academic job-market. Some universities require search committees to schedule X-number of first-round interviews (say, 12 of them). Alas, the search committee doing the hiring might want to interview significantly fewer people than that. Indeed, the committee might really have their eye on only five or six candidates, making the other five or six people they have to interview people who don't stand much of a chance for the job. I personally think this should change - that committees should only interview as many candidates as they really want to - but my sense is that in all too many cases, it's something committees cannot do anything about.

There isn't always agreement: One would like to think that committees do everything by consensus. However, while committees sometimes do agree by consensus--for instance, that candidates A and B are the two most promising at a given stage--decisions may not be made by consensus a fair amount of the time. Committees can be very split on certain candidates, not just when deciding who to interview but even at the stage of who to invite to campus or who to hire. 

Anyway, these are just some of the impressions I've gotten about how decisions can be made. Again, I'm not sure how generalizable they are, and would love to hear from others who have been involved in hiring - but I hope you all find these remarks helpful.

Comments

[Amanda]

Wow that is really interesting that search committees are forced to interview a certain number of candidates. I'v definitely had interviews where I got the strong impression I was number 12 on the list, and that there was nothing I could say that would change anybody's mind. This is not only a waste of time but pretty immoral to the candidates. Interviews take a ton of time and emotional energy, and it is pretty messed up to lead vulnerable people on like that. Obviously, if there is nothing search committees can do then that is that. But there should be some effort to change it. I am not sure how I feel about going straight to flyouts. For schools that fly out 4 people that seems good. But if you only fly out two that is a different story. I was actually surprised to learn that even big research schools sometimes fly out 3 candidates. I was under the false impression that all schools flew out 4!

[Chris]

We (an R1) almost never fly out 4, because our admin will only pay for 3. We could fly out 4 but we have to come up with the money to do that, and we have on occasion only flown out 2 if we think they're clearly above the rest. But 3 fly outs is the most common. We don't usually do preliminary interviews, either. Most of my colleagues think they're mostly noise.

[Marcus Arvan]

Our university policy is to only fly out 2. It is very rare to fly out more than that, and that only occurs (or so I have heard) when both of the candidates flown out either turn down the job or are judged unacceptable by the hiring committee.

[slac chair]

We do not do first round interviews. Nor do we have the search committee pick the finalists. We have the search committee pick a long list, then the entire department looks over the dossiers of those on the long list and votes to bring a few to campus. I've noticed that many candidates think they have been chosen by the committee, which is not true. So - not every department works the same way.

I would really emphasize Marcus's first point. And the lesson here is that it's nothing about you, the candidate. There is so much randomness to this process.

Most of Marcus' other points match my experience, except for the "who can we get" one. We realize that the job market is horrible, so we bring in whomever we want. Sometimes they're from a good but not top-10 program, sometimes they're from the top 3. Sometimes we lose our preferred candidate to an Ivy or whatever, sometimes we don't! (We're a bit lucky in that if we don't fill a line, we know we can get the search again, so we might be an anomaly.)