A past search committee member who would prefer to remain anonymous writes in (thanks to Helen De Cruz for obtaining the submission!):
I’ve served on tenure-track search committees at two universities, both large, public, land-grant schools, and both classified by Carnegie as RU/VH (“very high research activity”) institutions. At the first school, our department had a (not very highly ranked) PhD program; at the school I’m at now, we have an MA program, but no PhD program. I’ve served on more than a dozen search committees for tenure-stream hires within my discipline, a few search committees for tenure-stream hires in other disciplines, and a few search committees for administrative positions at the dean-level or above. As the latter are very different in many key ways from faculty searches, my experiences with those searches is probably irrelevant, and I won’t discuss those below. I participated in a few searches for tenure-stream faculty in a rather different role when I was a Department Chair, and in that role I was also a big part of the process of hiring fixed term (“adjunct”) faculty. But again, for the below, I’ll focus on my experiences as a member of search committees for tenure-stream hires.
There are (what are considered by some people to be) ‘best practices’ for academic hiring that are, alas, followed by at best relatively few committees (to the best of my knowledge). These include at least: developing operationalizable screening criteria clearly related to the job that the candidate is expected to perform before beginning the search; developing a search strategy likely to successfully encourage those people with the relevant skills, especially including people from traditionally under-represented groups, to apply; the implementation of an initial screening system designed to identify those candidates who meet the previously developed screening criteria, in a way that minimizes the effects of implicit biases, while following all relevant state and federal laws; and a way of fairly selecting a reasonable sub-group of those applicants identified by the initial screening as highly qualified for further evaluation. (Readers interested in contemporary best practice guidelines for minimizing implicit bias in hiring should consult the extensive literature on the subject.)
I was introduced to some techniques for achieving those goals over the course of a two-day training session my current university runs. I’ve in fact gone through the training twice, and got to see how it had changed between the first time I took it (soon after it was first developed) and when I took it next, about 5 years later. Most of the changes involve the incorporation of new techniques for achieving the above goals, and updates based on more recent empirical evidence on what works and what doesn’t. Some searches for tenure-stream faculty at my university require that someone serve on the hiring committee who has completed the training mentioned above, and who is not a member of the hiring department / unit, in order to try to ensure that the committee at least tries to follow those practices. The process is still, however, a little rough around the edges, and I have served on relatively few committees, in any role, that have done even a reasonably good job of meeting all of the above goals, though my impression is that more committees (both those required to have someone with the training onboard, and those that aren’t) are taking seriously that at least some of these goals are reasonable desiderata.
While my impression is that more committees at my university are trying to meet at least some of the above goals, the most common set of techniques I’ve seen deployed by committees fail on pretty much each of the above counts. Usually, committee members start the screening process knowing only that they are trying to hire someone with a particular AOS and (perhaps) some set of AOCs, and only the vaguest criteria for selecting among the candidates (e.g. “we want someone who is already a successful researcher, given the stage of their career”). Often, the criteria are not even discussed ahead of reviewing the applications, and it is simply assumed that “everyone knows what we are looking for.” When faced with the applications, a committee will generally have either each member read (skim) every application, and pick, say her or his top 10 or 15 candidates (based on whatever impression they form from skimming the materials they choose to look at) to bring back to the group, or break the stack up, so that every application is read (skimmed) by 2 committee members (say), each of whom again brings some number (again, say, 10 or 15) back to the group. With a committee of four members (say), you’ll end up (given that some candidates will be selected by more than one member) with perhaps 25 or 30 candidates to discuss. At this point, the applications are read rather more carefully, and then discussed. From the standpoint of selecting candidates to interview (or for further evaluation) in a fair way, about the worst technique one could use is the one that’s actually used most often, where people on the committee just say whatever they think about each candidate, picking out strengths and weaknesses that happen to strike them, in no particular order, etc., and arguing with each other until a final list of 10 or so candidates emerges.
Far better would be a multistage process, where first committee members apply (perhaps some subset of) the (pre-established) screening criteria to each application, and select those candidates that they feel are well qualified, given that (subset of the) screening criteria. Those applicants are then discussed by the committee in a group, applying the full set of the official screening criteria to each candidate’s application, where the candidates are taken in a pre-established order, with the committee taking sufficient breaks (with food) to prevent the kind of poor decision-making that comes from being tired (and/or hungry).
All that said, from the candidate’s point of view, the above kinds of options for search committees probably don’t make much difference, on average, in what kinds of applications have the best chance of being selected; the exception here is that criteria that many committees would, on reflection, decide were not particularly important are more likely to sway decisions under the ‘usual’ way of doing things than under the more careful methods. But these are things candidates have little control over (whether the graduate school you attended is thought of highly by one of the committee members; how famous your adviser was, and whether someone on the committee knows them; whether someone happened to have read one of your papers, etc.), and so probably aren’t worth worrying about. (Or if they are worth worrying about, it is too late to do anything about them by the time one is submitting applications.
My impression, given the current state of the market, is that for tenure-track jobs at decent places where the job listing isn’t absurdly restrictive, many highly-qualified candidates will apply. A screening method that picks out candidates on the basis of reasonable, operationalizable criteria that are relevant to performing the job well, will tend to pick out more candidates than can reasonably be interviewed (or otherwise further evaluated). Such a method will exclude many people of course – people whose areas don’t quite match the position’s, whose research agendas have not (yet) been particularly successful, who lack other relevant experience (e.g. teaching, service, grant-writing, interdisciplinary collaborative research, etc., depending on the position), etc. But it will fail to exclude too many people, from the standpoint of the committee trying to come up with a reasonable cut. At this point, other methods of selecting candidates will need to be deployed. It is best for a committee to have thought through this problem ahead of time, in order to prevent criteria that the committee members think, on reflection, should not be used from sneaking in.
The process of moving from a list of too-many highly qualified applicants to a list that is appropriate for initial interviews or other further screening is, I would argue, ultimately somewhat arbitrary. By the very nature of the problem – there are many more highly qualified applicants than can be selected for further screening – no solution will be fully adequate, if by ‘adequate’ one means something like “has a good chance of picking out the candidates who are really ‘the best’ – that is, most likely to succeed as colleagues.” The failure of any system to do that may come from it being an impossible task – of the candidates identified by high-quality screening criteria as very well qualified, there may be nothing in the applications that could be used to meaningfully distinguish them further with respect to their chances of succeeding in the jobs for which they are being considered (or at least nothing that ought to be used for that purpose). Further, I would suggest that most of the candidates initially selected on the basis of being highly qualified, in fact are more or less equally likely to succeed – many candidates, if selected, would in fact become excellent colleagues.
The obvious implication of this – that there is not a good way to further narrow the field – is hard to accept. Accepting it would tend to imply that those of us who were lucky enough to be hired were just that – lucky, and/or that the criteria in fact used to pick us out from among other highly qualified applicants were, in one way or another, bad criteria (criteria that we would not endorse as reasonable or ethical). But certainly, any thought that we were really “the best” person for the job should be undermined by our experience with the actual practice of picking candidates for further evaluation. Rather, we should think that, at best, we were one of a number of well-qualified applicants that no reasonably endorsable criteria could have much better distinguished.
This is not terribly encouraging to someone applying for jobs, except perhaps in this way: that you didn’t make the first-round cut for a position does not provide much (if any) evidence that you are not very highly qualified for that position, nor that you would not have succeeded at that position, nor even that there are any objective criteria according to which you are a worse candidate than those people in fact selected. The hiring process is a bit of a lottery – reasonable excellence is the price of a ticket, but after that, it’s all a little murky. That you made the initial cut means, in part, that you got lucky (and, very likely, that you are in fact highly qualified for the position – as were a number of other people, some of whom will also have made the initial cut, and some of whom won’t have). That you didn’t make that cut likely means little more than that you didn’t get lucky.
Comments