Notes from both sides of the market - part 6: interpreting interview results

As someone who was on the job-market for a number of years and who now mentors job-candidates, I know how tempting it can be to want to interpret interview results. You thought you had a good Skype interview...but you didn't get invited for a fly-out. Then, for another school, you thought you had a bad interview...but did get a fly-out. What gives? Given that I've now been on both sides of the job-market--as both a candidate and two-time search committee member--allow me to try to provide a bit more insight.

One of the things I really didn't know about the hiring side of the process as a job-candidate is how committees rank candidates prior to initial interviews. Sometimes, members of a search committee can all be on the same page, ranking candidates similarly, in some cases even agreeing that two or three candidates in particular are at the very top of their list by far. Other times, committees can be much more split. In some cases, one or more committee members may have candidates at the top of their list that other committee members don't have anywhere near the top of their list--leading to substantial disagreements in the committee about who to interview. Indeed, committee members may even be so divided on candidates that they might strike deals with each other (viz. "I'm fine with interviewing your favored candidate X if we also interview my favored candidate Y").

These different types of situations can plausibly have effects on deliberations before and after first-round interviews. From what I have heard, search committees typically extend first-round interviews to anywhere between 6-12 candidates. So now consider the two different types of cases I mentioned above:

• Case 1 (agreement): the committee largely agrees that two or three candidates are at the very top of their list.
• Case 2 (disagreement): the committee is pretty split between their candidate rankings prior to interviews.

In Case 1, it may be difficult for candidates lower down on the committee's list to help themselves much in the first-round interview. Let's say a given candidate is toward the bottom of the committees list of 6-12 interviewees. Then suppose that candidate has a very good interview, but other candidates higher on the list have good interviews too. In that case, even though they interviewed well, they might not get invited for a fly-out--largely because the committee was already pretty sure who their highest-ranked candidates were. Case 2, however, is very different. In a case like this, where a committee is split on candidates heading into first-round interviews, a good or bad interview can plausibly make a much larger difference in terms of who gets invited to campus.

For these reasons--and others to be mentioned momentarily--it can be very difficult to read into "interview results." You may interview well but not get invited for a campus visit because you weren't the highest on the committee's list. Conversely, your interview might not go so well but you still might get invited to campus because you were so high on the committee's list to begin with (and they might figure you just don't interview well). Finally, it can be hard to read into interview results because you don't know how well other candidates interviewed. Because you are competing against many other very accomplished candidates, there's a pretty decent chance that if you interviewed well many of them did too! Although these remarks may be a bit discouraging, I think they explain in part the oft-made claim that the job-market is a "crapshoot." There are just so many moving parts--including search committee members with different priorities--that it is hard to read too much into anything. Still, allow me to end this most with a more encouraging set of thoughts.

In my seven years on the market, I did notice a few clear trends. From my vantage-point, I thought I nailed about 10-15% of my interviews, did fairly well in about 70-80% of them, and then totally bombed about another 10-15% of them. As they say, we all have good days and bad days. Still, my experience was this: although my interview results were a "crapshoot" in the 70-80% of cases where I thought I interviewed "pretty well" (in some cases I got fly-outs, in other cases not), the other two categories were much more clear-cut. If I thought I totally aced an interview, I almost always got a fly-out; and conversely, I never got a single fly-out for any of the interviews I bombed. These trends cohere with my experience on the other side of things (the hiring side). Many candidates interview well--and it can be difficult to choose between them when they do. But when someone has a spectacular interview? Then they may be hard to pass up. I still don't think we should be doing first-round interviews at all--but, be that as it may, these have been my experiences on both sides of the market.

Comments

[Recent hire]

This is very interesting. I was on the market this year (first time), and our placement officer said that once we passed to the interviewing stage, we should regard it as a blank plate --- that a school wouldn't interview someone whom they are not very seriously considering to hire.
But it does make sense that there is a ranking already; after all, it's the same committee who had read your other materials already.

[The other postdoc]

I think the advice Recent hire received is right. Even if most hiring committees rank the candidates prior to interview, one should still go into the interview as if it were a blank slate. There's no way to know where you are on the rankings, so it's probably best to go at it as if the interview is now all that matters for getting to the next stage—but, on the other hand, don't beat yourself up if you don't make it to the next stage, since that might not have been within your control.

[Marcus Arvan]

The other postdoc: I agree - it's the right attitude to have even if it's false!

By all means, treat every interview as a blank slate, going into every interview optimistic with the aim of knocking it out of the park. As I mentioned in the OP, if you do nail the interview, you may well receive an invite for an on-campus, regardless of how the committee might have ranked things heading in.

At the very same time, if you feel like you interviewed well but don't get an on-campus invite, don't beat yourself up: it could well have been because, on the hiring side, it wasn't a blank slate after all!