What should search committees remember about the market?

Last month, Trevor wrote a really illuminating post on his job-market experience last year. One of the many things that stood out to me was a conclusion he drew about interviews:

Job interviews are bullshit. This is not a new observation, but it was reinforced throughout the year. I was always the same job candidate, and I always engaged in roughly the same interview preparation – same attire, same amount of time researching each school and department, the same interviewing location (with a few exceptions). Yet my interview performance varied dramatically and usually due to factors that had nothing to do with how well I could do the job in question. For this reason, you shouldn’t be discouraged if an interview goes badly, but it’s difficult to have that outlook when they can have such a huge impact on your career and life more generally.

My experience on the job-market was similar. Of every ten interviews I had, I'd say about 3 went really well, 3 went badly, and four went so-so. It was an incredibly frustrating experience, as just like Trevor I felt that they just weren't reliably reflective of who I was (and am) as a professional philosopher. No matter how well I prepared, my interview performances felt like a crapshoot. They seemed to vary depending on everything ranging from how well I slept the previous night to how well I could hear the committee's questions on Skype, to whether I drank too much coffee, to how sharp I felt on that particular day. Readers may also recall that I've also shared some of the science on hiring, including just how sensitive to job-irrelevant factors (attractiveness, height, weight, race, gender, etc.) interviews are known to be.

Why, then, do committees stake so much on interviewing? Having served on four search committees now, the answer is fairly clear to me: on the hiring side of things, interviews seem to have probative value. When a candidate answers questions very poorly, it can seem hard to imagine hiring the person. Or, when a candidate 'kills it' in an interview, it can be very hard not to come away impressed. The problem, though, is that the science doesn't support any of this--and a little commonsense reflection can help us remember why. Consider Tom Brady, by most estimations the best quarterback ever to play NFL football. Tom Brady impressed roughly no one in his combine try-out for the NFL, which is why he was drafted in the very last round of the NFL draft. Tom Brady has also had some miserable games during his long career in the league--just like every sporting great has ranging from LeBron James to Venus and Serena Williams. Conversely, here are some players who impressed just about everyone in their NFL tryouts: JaMarcus Russell, Ryan Leaf, Joey Harrington, Tim Couch, Kyle Boller...all players who were drafted in the very first round of the NFL draft who nevertheless went on to have miserable careers. What all of these cases show (including Trevor's experience) is something that should be obvious to anyone: namely, individual, one-off performances can vary tremendously in ways that make them horribly unreliable measures of future promise. Indeed, on that note, it's worth noting that I've seen some very famous and otherwise brilliant philosophers give bizarrely bad talks! Given that even Tom Brady can have miserable games, we should always remember just how fallible judgments based on single performances can be.

The problem, it seems to me, may be that search committee members simply forget this--that is, that they are so far removed from actually being job-candidates that they forget just how many so-so or bad interviews they gave as candidates, and how frustrating it is to prepare hard for an interview only to see it not go well when another interview you prepared just as hard for goes splendidly. Anyway, this got me thinking about what else search committees might forget about what it's like for candidates on the market. Here's another one that occurred to me: what an awful waste of time it is for candidates to have to create application materials (beyond the cover letter) specifically tailored to the job--such as a research vision indicating how you would specifically contribute to X, Y, and Y. These kinds of things can take forever for candidates to put together...and for what? 95% of the time, it will all go for naught--a committee may not be impressed enough by a candidate's CV to even spend the time to look carefully at the tailored materials the candidate spent so much time on!  

As someone who spent a ton of time on the market as a job-candidate and then served on several search committees, it occurred to me how important it may be for people like me to not lose touch with these facts about the market. Which brings me to my question for current and recent job-candidates out there: what do you think search-committees should remember about being a candidate on the market? What sorts of things seem just "obvious" to you as candidates that search committee members may have forgotten or otherwise be inadequately aware of or sensitive to?

Comments

[Still on the market]

Invisible disabilities, particular linguistic disorders, massively effect a candidate's ability to articulate clearly all the things that they can in response to a question.

On a number of occasions now, my own disabilities in this regard have meant that I've not expressed points in interviews using the terms that I've later been told that I should have mentioned. I've made the same point, but just in different language - panels need to be more flexible around what is needed to give a good answer to a question in light of these kinds of (often) invisible disabilities.

[placement director and recent job marketer]

One reasonable choice (it's not the choice I would make, but I think it is definitely a reasonable one) for search committees to make is to stop doing first-round interviews, and perhaps to stop doing interviews altogether.

One thing that search committees should remember if they do this, though, is that so much of job market stress comes from not knowing anything about where you stand except by receiving interviews (that indicate that you are on a short list) or job offers (that indicate that you did great). So, I would strongly encourage search committees who e.g. choose to not conduct first round interviews to contact whoever was on their "short list" (if they had one) and let them know that they did well in the search, the committee was impressed by their application, etc.

This can also be valuable info for people trying to gain information in order to decide whether to continue searching for an academic job.

[anonymous grumpy philosopher]

One thing that drove me nuts on the market was unclear deadlines. E.g.: why have a "soft" and a "hard" deadline and claim that all applications received before the hard deadline will be given equal consideration? From a candidate's perspective it's hard not to read this as a lie--why else would you supply a "soft" deadline? Another thing that happened frequently (not sure if it is still happening frequently) is mismatch between the information given on the actual job posting on a university's website and the Philjobs (or other) job ad. (I remember mismatch between what materials were requested, for example.)

I don't know if this is normal or not, but in my department our chair writes drafts of our job ads and then we all read them and dedicate part of a faculty meeting to tweaking/making sure that things are as we want them to be. I think that this kind of process is important. But I also think that junior faculty, who are most likely to remember issues with job ads, can be hesitant to speak up in full faculty meetings. So one concrete suggestion is: have your own junior faculty vet your job ads, not necessarily for content (that should be discussed by the whole faculty), but for "what would make this clearer for applicants and what is making things unnecessary complicated or confusing" kinds of considerations.

[VAP]

Search Committees should remember that applicants are almost certainly applying for dozens and dozens of jobs. There are a number of ways I think SCs forget this.

1) If you request another specialized document--say, a statement of how the applicant's values align with your department or whatever--make sure you really, really need it. And please, please reflect on whether that has to be a separate document, or if it can just be folded into a cover letter. (Also, what are you actually going to gain from this kind of thing? The mind boggles.) If, upon reflection, it is not essential, then ditch it. You're actively making applicants' lives much, much harder for no good reason.

2) Try not to read into an applicant's motivations on the basis of the extent to which they've done uber-customization of their documents for you. This is most easily done by those in a comparative place of privilege--i.e., low teaching loads, fewer obligations, tons of support, etc. The rest of us are trying our best to signal that we're interested, but time doesn't always allow for the extreme level of detail that some offer you. Trust us that we're applying because we're interested. Just because we didn't mine the depths of your department's website does not mean that we wouldn't be awesome colleagues.

3) You should consider giving public deadlines for information. E.g., "First round requests will be sent out by Dec 10". This helps those who are juggling a lot of different applications across different deadlines to clearly whittle down our lists.

[a philosopher]

I second "Still on the market". I'm often not good on my feet anymore, as often in the moment my mind is simply blank or words don't come to me. But if given a few hours, or a day or two, to think things over, quality ideas will come to me and I'll be able to articulate myself well.

[hey it's another philosopher]

"why have a "soft" and a "hard" deadline and claim that all applications received before the hard deadline will be given equal consideration?"

I think it's to give the hiring department the option of considering someone they were super-excited about, if the application came in after the "soft" deadline, while still being able to proceed in an orderly fashion with the great bulk of the applications that arrive before that -- and to be able to circular file any less-than-really-spectacular ones that come in after the "soft" deadline.

Basically, you should always treat the "soft" deadline as the real deadline.

[hey it's another philosopher]

I totally agree with VAP's first point -- though it would not surprise me if, in many such cases, it wasn't the department's doing but a requirement imposed by some other university office.

Regarding their second point, though, I think this is maybe misreading the mechanism of how this works. It's not that the committee will think, "Oh, candidate X isn't really interested, since they just gave us a pro forma letter". Rather, what the committee will think is, "Oh, candidate Y is really excited about being here, look how great their cover letter is!" And then when it comes time to draw up short lists, if the ceteris are paribus, of course Y is going to get invited over X. Since it's a competition, X can still be disadvantaged merely by Y being advantaged, even if X is not being _penalized_ per se.

Having said that, I do take the point about how this intersects in a highly undesirable way with the unequal distribution of time & other resources among the applicant pool. One thing we should think about, is are there other means we could institute in order to facilitate honest signaling by candidates about what departments they'd really really like to be hired by? The APA could, as an institution, create a system where each applicant has a scarce number of "prefs" (or whatever we would want to call them), and that information could be conveyed by the APA to the relevant departments, i.e., that candidate X has burned one of their small number of prefs to indicate to department D that they are a highly desirable department for them. (One thing: it would be best if prestigious departments would refuse to accept prefs, knowing that they are already desirable, and not put candidates in a position of needing to burn some of this scarce resource even to have a long shot at a job where there's no question about someone's wanting to be there.)

I'm just sort of thinking out my backside here, but to the extent that there's a _signaling_ value in lavish cover letters, there should be some way of making that happen where it wouldn't place such a demand on applicants.

[Anonymous McGee]

I do wish I could tell people that, when I refer to myself as an 'independent scholar', it only means I've been unlucky on the job market for a little while, and not that I am a war criminal and been disowned by all my loved ones (or what have you).

[that guy]

Quibble: Brady was drafted in round 6 out of 7 in the 2000 NFL Draft.

[philosopher in search of time]

I hope the following (which I take to bolster some above points) is remembered.

Even slightly increasing how long it takes someone to apply for a given job (or how many people apply for it) may well lead highly capable people--many of whom are in their intellectual prime, overworked, and at risk of or suffering from mental health problems because of what competing on the market demands--(collectively) to perform dozens of hours of fruitless labor that would have otherwise been devoted to activities such as teaching, research, relationships, exercise, and sleep. Likewise, even slightly decreasing how long it takes to apply for the job at your school may well have the opposite, positive effect. And attempts to decrease the amount of time it takes to apply that have only a small probability of succeeding may have quite significant expected value--asking HR department not to require forms for information already on applicants' CVs may fall in this category.  In all likelihood, you won't directly observe any of the positive effects for applicants that you generate through efforts to decrease what your application requires of them, and  your efforts will be thankless.  But regardless, these effects and efforts matter.

[Amanda]

I've been offered 6 jobs over 3 years on the job market: 3 research postdocs, 1 VAP at a R1, one TT job at a teaching school, and one TT job at a R1.

I am also horrible at skype interviews, or at least, I think I am. The probabilistic evidence suggests I might be correct, as not a single one of those job included a skype interview. 2 of them had NO interviews at all. That's right - they went straight from the CV and other application materials to the offer. 2 others went straight to flyouts. And the remaining 2 had a phone interview and then a flyout.

I say this because I just think it is one more story of the negative impact of interviews. How people assessed me after reading my CV, or meeting me in person, was completely different than their assessment during a skype interview. But as Trevor says, I was the same candidate every time. I hope more and more schools go toward skipping first round interviews. I am less sure about totally skipping interviews, as I think on campus might be different, but I am prepared to change my mind on that.

The only thing I would say about what else search committees should remember is that please pay attention to time zone differences. Asking someone from Oregon to go to an 8am meeting (5am their time) after a total of 7 hours of flying the day before is just cruel.

[Amanda]

Okay I actually have two more suggestions:

1. Please go out of your way to figure out how to pay the costs for the candidate upfront. I know they might be in the minority, but for some candidates, having to front costs and wait for reimbursement is a *major* financial burden, if not impossibility.

2. Some schools actually have 2-day flyouts! This is just unnecessary and inconsiderate of a candidate's time. Often candidate's have to work very hard to manage a flyout with their teaching schedule, and to add an extra day onto things can again but incredibly difficult, if not impossible.

[on the jyerb market]

Skype interviews are terrible, but one thing I appreciated (I doubt all departments do this) was the departments I did get skype interviews for gave me their list of questions ahead of time, so at least I could prepare ahead of time.

One thing I'll say, at least we don't have to travel to the APA East every year now and can spend time with our families.

[Job marketer from years past]

About the APA east: at least one department is still making candidates fly out there! Shame on you, Lake Forest!

[Trevor Hedberg]

"on the jyerb market" mentions departments that give you info about the questions they will ask in advance. I do think that makes things easier in some respects, but I will mention one case in which this worked against me: I was told by the search committee chair that the focus of the interview would be on my teaching and in particular how I could teach the courses specified in the job ad. So naturally I prepped for that. When the actual interview came, I was asked 6 questions. Only 1 of these questions was about teaching specifically, and none of the questions pertained to how I would teach the courses mentioned in the job ad. Needless to say, the interview did not go well. So, if committees are going to do this, be truthful about what you're actually going to ask the candidates.

I'd also highlight the need for transparency with the process. Keep us in the loop regarding your search -- don't make us send you awkward followups 1-2 months after an interview or wait 4-6 months for a form letter email rejection. Also, don't just flake out and not notify candidates at all, which has happened to me with roughly 40% of my job applications. In the age of mass email, it's unacceptably lazy. It's also disrespectful to candidates (because of the time it takes them to apply), and it causes job candidates unnecessary stress since they often don't have a clear picture of their situation.

[Nicolas Delon]

Do not ask: 'How would you teach class X?' It cannot be answered meaningfully in a two-minute snippet. Rather, ask candidates what classes they would like to teach, both among those the department offers and new ones they could offer, whether they'd feel comfortable teaching in this or that area, etc. Or ask concrete questions: what textbook would you use for X? How do you address interdisciplinary questions or controversial topics in X in the classroom? etc.

[sick]

Amanda
I had at least one FIVE day - and this is within North America. It was quite challenging. I did have a supportive job at the time (though contingent).
But five days ... and then to discover at the interview that they had an inside candidate (who did get the job). Out of curiosity, I just looked ... he is still there after many years ... and it looks like his most cited piece has about 5 citations - I have 1000+ citations, and over 20 papers with 20+ citations. The market is sick!

[Amanda]

Five days???? Yeah, that is nuts. I can't even begin to imagine justification for that.

other advice: It is beyond unacceptable to invite someone to a flyout and then never get back to them. Treat people with decency. I really think those who get skype interviews (if there are any) should also get a personal rejection letter. You can assign one person in the department to do this, and it really will take very little time.