The Job Application Materials Arms Race (guest post by Shane Ralston)

Shane J. Ralston is currently the Dean of Wright College, Woolf University.  He was formerly the founding Human Resources Director at the American University of Malta and a tenured Associate Professor of Philosophy at Pennsylvania State University.

The Job Application Materials Arms Race

How many times have you read a job announcement, decided to apply, read the required application materials and then felt your heart drop with disappointment?  In the current job market, there is an arms race in required applications materials. (Besides an arms race, I’ve elsewhere called it a ‘paper chase’.)  Fail to submit one of these required items and you’re immediately eliminated from the candidate pool. 

One common complaint against application material inflation is that search committees and human resources professionals are insensitive to the applicant’s time. More time invested in assembling and submitting job applications means less time spent with family and friends, conducting research and prepping for courses.  Why don’t human resources departments or search committees cap the number of required materials in the job announcement?  Why not request the bulk of the materials only after the first or second round cuts?   

Here’s a list of materials typically required of applicants in the initial round approximately ten years ago. (I assembled this list from my own job applications when I was on the market in 2008-9.)

1. Cover letter (usually tailored to the institution and the position, explaining how your qualifications make you a good fit).
2. Unofficial transcripts
3. Teaching statement
4. Writing sample
5. List of references with contact information

Ten years later, the list has expanded considerably to include (or calls to include in places like HigherEdJobs):

6. Letters of recommendation
7. Teaching portfolio (including sample syllabi and student evaluations)
8. Statement of diversity and inclusion
9. List of courses taught
10. Research statement
11. Statement of antiracism and educational justice
12. Evidence of experience with inclusive teaching
13. Leadership statement
14. Service work or public philosophy statement
15. Religious faith statement (exclusive to sectarian schools).

What is the rationale for increasing the number of required application materials?  Who makes the decision?  Based on my experience, both serving in HR and as a search committee member, it is usually (but not always) a collaborative choice to expand the number and kinds of required application. In some cases, expanding the number of required materials is intended to reduce the size of the applicant pool. In other cases, search committee members genuinely wish to secure more data points about applicants who would become their future colleagues.  Given that the person selected for the position could spend upwards of forty years in the department, search committee members tend to be risk averse.  They don’t want to mistakenly select the wrong person for the job. So, the more data, the better. Thus, the more application materials, the better. 

What should we make of these rationales for expanding the number and type of documents that job-applicants must submit? On the one hand, a growing list of required materials promotes applicant fatigue, nudging the applicant to give up on applying for a job because the perceived costs (or time and effort spent applying) outweigh the perceived benefits (the likelihood of securing the position). On the other hand, a long list of required materials might be excessive for a Visiting Assistant Professor position, while perfectly reasonable for a Tenure-Track position.  In order to rank candidates, data must be put in a comparable form relative to an agreed-upon list of criteria, so that collective decisions can be made about which candidates will advance to later rounds (e.g., phone interviews and on-campus interviews). However, at some point, the amount of data collected potentially exceeds the time that the committee members are willing to invest sifting and operationalizing that data.

What I have found, unfortunately, is that search committees overwhelmed by large applicant pools, excessive dossier materials and too much data tend to deploy arbitrary criteria.  (In some ways, it is the flip side of applicant fatigue, but instead of giving up, search committee members take an illicit short cut.) Arbitrary criteria make elimination decisions easy. However, the outcome is often just as arbitrary as the criteria deployed.  I’ll give you an example. When I served on one search committee member, the committee voted to eliminate a candidate because she misspelled the city where the institution is located.  (I was outvoted.) Their rationale was that failure to correctly spell the city name indicated a lack of care in preparing the job application, and thus revealed the candidate’s disinterest in the position. (The committee was concerned that many candidates did not wish to live and work in the area, though they would accept an offer, later using the position as a stepping stone to secure a position at a more attractive institution). What the majority of committee members failed to consider is that the misspelling could have been an honest mistake. In other words, a busy person who works full time, has responsibilities to loved ones and applies for jobs in the limited time remaining outside of work and family life should not be eliminated from a search based on a simple spelling error.

The other uncomfortable truth that few human resources professionals and search committee members publicly acknowledge is that the job market is an insider’s game.  ‘Insiders’ I define as anyone with a connection to one or more members of the hiring committee, usually through a relationship with an advisor or mentor. Advisors and mentors assist job applicants by reaching out to committee members. The proliferation of job application material requirements might just be another way to easily eliminate outsiders or those who lack these prized connections. Applicants who decide to eliminate themselves due to application fatigue free up institutional resources that would otherwise be spent accepting, acknowledging, reviewing and eventually rejecting their application.  The insider still must jump through the same hoops (or endure the same difficulty of preparing all the required application materials) as any outsider, but she does so with the confidence that her application will receive greater attention (or will have a greater chance of success) in virtue of her inside connection. In other words, the expected benefits justify the costs.

In terms of required application materials, how much is too much? Should there be something like a standardized list of application materials?  Would such a list help avoid replicating information that a search committee should be able to cull from other materials? For example, diversity and leadership statements seem particularly redundant to me. If an applicant is committed to diversity or leadership, evidence should show up in their dossier—for example, in their teaching materials and CV. Have they served as a Minorities and Philosophy representative? Have they participated in other diversity related activities? Have they led university committees? What does a diversity or leadership statement add to a dossier, other than serving as a kind of performance? I’m curious to hear what readers think, particularly job-candidates but also search committee members and administrators. Is the job application materials arms race a bad thing? If so, what can/should be done to address it?

Comments

[Manny]

It verges on criminal for a department *that is in control of the application requirements* to ask for anything customised. It also verges on criminal to ask for materials that, in all likelihood, are not going to affect the decision either way. (I wonder, for example, how often top schools have rejected a candidate because their...research statement wasn't all that exciting.)

CV, writing sample, three letters, and (if you must) teaching portfolio (although, I reckon that a teaching portfolio, for many jobs, violates the second crime, above).

[Mike Deigan]

"In other cases, search committee members genuinely wish to secure more data points about applicants who would become their future colleagues. Given that the person selected for the position could spend upwards of forty years in the department, search committee members tend to be risk averse. They don’t want to mistakenly select the wrong person for the job. So, the more data, the better. Thus, the more application materials, the better."

Even assuming that requiring the extra materials does help avert the risk of hiring a bad colleague, why not wait until the shortlist stage to ask for it? Instead of imposing x hours of work each on 400 people who have on average a ~1/400 chance of getting the job, wouldn't it be better to impose it on 20 people who each have a 1/20 chance? This gets you the same information for the person you're actually gonna end up hiring, so if you weigh applicants' interests at all, it seems clearly better.

If what you really want to do is cut down the number of applications to consider, why not just do an actual lottery? You can do it after a very quick first overview to exempt the obvious gems or make a first cut based on some simple criterion that takes 30 seconds per candidate to figure out (at least one publication, has taught such-and-such amount, whatever). But this would be an easy to implement way to cut the pool in half (or however much) and would impose no burden on applicants. Maybe some would worry that this would make the process too dependent on luck, but I doubt it would make it any more luck-dependent than it is already.

Here's another question: would it matter if some job-candidate here comes up with some great idea for an improvement, even a Pareto improvement? Is there any way this would affect what HR and search committees do? I doubt it, at least beyond a handful of searches. But it would be nice if this is too pessimistic. In any case, perhaps it's worth talking about whether anyone has ideas for how to bring it about that search committees do what they obviously should do (like notify eliminated candidates).

[philosopher]

I have seen ridiculous requirements even for single-course adjuncting contracts, places asking for a full dossier that you'd expect to see in a TT search.

It strikes me that you can't complain about all the forms your university makes *you* fill out, while agreeing with the HR department that you should make *others* fill out more forms.

Regarding the "we need more data points to sort candidates", I would ask whether faculty make full use of the data points they already have. Do most search committee members carefully read and thoughtfully consider each of the hundreds of CVs they get? A CV also tells you a lot more than is on the page. For example, do you actually go and look up the candidate's webpage that's listed on their CV, or actually read some abstracts of the listed papers (which will be on philpapers and/or that website) --- for each CV?

Better and full use of available data points seems preferable to *more* poorly used or underutilized data points.

There's another point to raise here too, which is that basically every other industry makes due with hiring based off a couple-page resume. Yes, most industries aren't hiring for a "forty year colleague", but these industries are trying to succeed in cut-throat markets with millions at stake. The simple point is that if Google doesn't need a fifty page dossier to hire a lead programmer or market analyst or whatever, No-name U in southern Arizona doesn't need one to hire a professor (whether visiting or TT).

Finally, I worry that more dossier materials = more ways for committee members to rationalize away their subjective choices (whether pro or con). If you get enough data from everybody, you’re guaranteed to be able to find points which will justify almost any decision, for any candidate.

[an ESL philosopher]

"What the majority of committee members failed to consider is that the misspelling could have been an honest mistake. In other words, a busy person who works full time, has responsibilities to loved ones and applies for jobs in the limited time remaining outside of work and family life should not be eliminated from a search based on a simple spelling error." -- In addition to this, misspelling-based elimination discriminates against non-native speakers and against people with learning disabilities such as dyslexia. For example, my wife is, in my opinion, a very promising philosopher (and fortunately has landed a TT job recently), but she has dyslexia and constantly makes spelling mistakes (including in her application materials), some of which are detectable/detected by Word & proofreading apps, but quite a few are not.

[Marcus Arvan]

One particularly ridiculous kind of case that I encountered multiple times as a job-candidate were ads requiring *syllabi* for the exact courses they wanted taught in the department.

I understand that this may be one way of dramatically cutting down the applicant pool to people with previous background teaching the courses in question. But given how competitive the job-market is, in practice this can lead job-candidates to spend many hours (or even days) putting together multiple syllabi for a job they unlikely to even get an interview for.

This struck me as deeply unethical as a job-candidate--not, again, because a committee has a no interest in such materials, but rather because it seems profoundly insensitive to the reality that job-candidates face. If you want detailed syllabi for a job, ask for one later--at most after first-round interviews, if the candidate is seriously being considered for a campus invite.

[nonny mouse]

"Advisors and mentors assist job applicants by reaching out to committee members" How is this a thing???? Is this common? Is this even allowed? It seems deeply unethical.

[syllabus]

@Marcus Strongly agree with the hyper-specific syllabus requirement. I forget what it was but last cycle I applied for a job that asked for two such syllabi. Of course I spent several days putting them together and then, predictably, crickets. I think one was a class half of which was on Montaigne. Not to throw shade on Montaigne, but I am *never* going to use that syllabus.

@nonny mouse It happens. I'm not sure that it is unethical, it strikes me as in the vicinity of what letters of recommendation do and people seem generally okay with those (I know there are exceptions). It does seem unfair, though. Only some candidate's advisors do that or even be in a position to do that.

[UK lecturer]

I agree with the view that it is ridiculous and problematic, for a variety of reasons. It is also worth noting that universities in other countries do not require anything like this level of documentation. At least, UK universities do not. Despite this, their hiring process seems no less effective / successful (by whatever criteria you use to determine that).

[CV only]

To echo Philosopher's point above: committees should need nothing more than a CV.
--The "writing sample" can be obtained by looking up publications on the CV.
--Letters of recommendation most likely have little positive value to outweigh their negative effects.
--Teaching portfolios demonstrate little of value (it is very easy to say the things that "good teachers" say, so their evidential value is nil).
--If a committee wants someone with diversity or leadership cred, that will be in the CV, too. And by scrapping all of these things, except for the CV, you can more effectively separate those candidates who say the right things whether or not they are true.
--Finally, by looking at the actual accomplishments of a candidate, committees can correct their criteria for determining "fit", which so often is a euphemism for "we'll hire whoever we want, standards be damned".

[Michel]

Having applied to hundreds of jobs (with minimal success), I'm entirely sympathetic.

That said, if we're just asking to have the number of materials reduced in the first round, it's worth observing that this doesn't really help candidates. After all, you'll still have to submit the teaching portfolio, research or diversity statements, etc. in a later round, should you be so lucky as to get that far. Which means everyone still needs to prepare them, even if they won't be submitting them far and wide. That doesn't cut the applicant's work down by any appreciable amount.

Simply not requiring them all, of course, would be much more helpful. And not requiring bespoke documents would be even more helpful.

The other applicant-side task that takes forever, however, is filling out all of the educational, reference, work history, etc. details for HR. And that clearly DOESN'T matter, except for reporting purposes, and all of it duplicates information on the CV.

While we're at it: the most egregious bespoke document request I've seen was the creation of a course website for an online course, ownership of which would be immediately transferred to the hiring department upon submission of the application.

[philosopher]

Some people will drive circles around the large parking lots of big-box stores, looking for spaces near the entrance. These people, with some obvious exceptions, are wasting their time to avoid a negligible amount of inconvenience (walking 100m).

Wading through stacks of 50+ pages dossiers is equally irrational. Asking applicants for even more materials is like enlarging the parking lot and wasting even more time circling it.

The point is that sometimes the perfect is the enemy of the good. There is no "best" candidate, just a lot of good ones. Attempts to suss out the very best, most promising candidate who will be a great colleague and stick around for 40 years are doomed to fail. At the certain point it's like the NFL draft: a lottery.

By adding layers of requirements and procedures you're not actually improving your odds of getting the best candidate, you're just wasting everybody's time.

NFL teams have entire departments of full-time staff whose entire job is just to evaluate talent and fit. If those people can barely do better than chance at making the right draft and free-agent picks, what makes you think five overworked professors meeting half a dozen times over two months can do any better?

The point is that if you throw out the assumption that this isn't a lottery after a certain point, you're left with no reason to ask for all this stuff. It just doesn't help. It is all a weird, well-intentioned performance of self-deception.

[postduck]

The list given above is kind of exaggerated. I've never had to give a separate list of courses taught. In my experience, the "Statement of antiracism and educational justice" is pretty much the same as "Evidence of experience with inclusive teaching" - aka the "Diversity Statement". I have also never been required to write a leadership statement or service work/public philosophy statement. Let's not gild the lily here.

It seems to me like the main source of statement bloat comes from the Diversity Statement and statements that ask you how you fit with some particular program or with the college mission. Both the "religious faith statement" (I've never seen it called that) and the Diversity Statement strike me as morally problematic because - in addition to the fact that they add to applicants' workload - they're effectively ideological tests that give you a chance to proclaim that you are an antiracist or a Catholic or what-have-you. I am also pretty skeptical that Diversity Statements actually achieve diversity (versus say, creating targeted postdocs and tenure lines specifically for people of color). And if you want applicants to discuss their "fit" with the program/institution, ask for that in the cover letter instead of making them write up a whole new statement. Otherwise, required application materials should be standardized.

Also, +1 for Marcus' point about providing syllabi for courses beforehand. I always treat those ads as a sign that they're planning on doing an inside hire.

[teaching portfolios are valuable]

CV only:

--Why shouldn't hiring departments hire whoever they want? (It's not "standards be damned", on one way of looking at it, since the situation with almost every hire is that there are at least 50 or so well-qualified applicants, many more if it's an open area or an area with a lot of candidates like ethics. There's no reason to think that within this pool there are some objective facts of the matter about who "deserves" a job. The idea that you could read those objective facts off a cv somehow seems... bizarre.)

--I think it is straightforwardly false that teaching portfolios have no evidential value. My students have a huge amount of help with placement related stuff, including constructing a teaching portfolio. It is just not true that, even though they know (in some sense) what it is standardly supposed to look like, you cannot distinguish between who is a stellar teacher and who is not on the basis of those portfolios. And by and large my overall sense (as a placement director at a department that places people into mostly at least partially teaching-focused jobs) is that in fact hiring departments track "quality of teaching" pretty well--our stellar teachers get interviews (partly or even largely on the basis of their teaching portfolios). I also think the idea that there are just generic good things to say that make you sound like a good teacher is wrong. Communicating what is unique about your teaching and how you personally connect with students is actually quite difficult for a lot of people. As is constructing a compelling, exciting, novel syllabus (which is what our best candidates do).

--I also hate diversity statements.

--Not all small schools have easy access to every journal, etc. to look up publications. Also I think it puts a weird burden on a hiring committee to have to look up every candidate's published work (and pick which things to read) when it is extremely easy, as a candidate, if you have published, to send along a copy of a paper.

Surely there is some room for middle ground here (I actually agree that applications are too bloated, I just strongly disagree that you can evaluate much at all off of a cv--and if you're worried about bias in hiring, well, in some ways just having the cv is worse for that, since you just have less information, period, and so, e.g., what university a candidate got their PhD at, etc., may end up playing more of a role in your process than it otherwise would.)

[FritzJMcDonald]

Nonny mouse, They called this “working the phones” (by dissertation advisers and other grad school faculty) when I was on the job market (way back in ‘06). I had the impression then that it was pretty common practice.

[Ttj]

Re:teachin portfolios

Let's go through what you said point by point.

-My students have a huge amount of help with placement related stuff, including constructing a teaching portfolio.
--Good for your students! Does it follow from your students getting this help that they're better teachers? Probably not. It does follow that they look like they're better teachers. This point seems to weigh *against* and not *in favor of* the evidential value of teaching portfolios.

-It is just not true that, even though they know (in some sense) what it is standardly supposed to look like, you cannot distinguish between who is a stellar teacher and who is not on the basis of those portfolios.
--Do you have any evidence for this? Or are we just supposed to believe you that you can make this judgment? My experience is the opposite of yours. Lots of folks who are super good at putting together fancy teaching portfolios are, well, good at that and not so much at teaching. These seem like orthogonal skills to me.
-And by and large my overall sense (as a placement director at a department that places people into mostly at least partially teaching-focused jobs) is that in fact hiring departments track "quality of teaching" pretty well--our stellar teachers get interviews (partly or even largely on the basis of their teaching portfolios).
--If by `quality of teaching' you mean `results on teaching evaluations', then how do you square this with well-known facts about the biases of tevals? If you mean something else, then how do you know who the `stellar teachers' are? Is it your (definitely unbiased and totally accurate) gut feelings?
-I also think the idea that there are just generic good things to say that make you sound like a good teacher is wrong. Communicating what is unique about your teaching and how you personally connect with students is actually quite difficult for a lot of people.
--I agree! I also think there's basically no correlation between being able to do this and being a good teacher. Or at least, there's no clear reason to think there's such a connection.
-As is constructing a compelling, exciting, novel syllabus (which is what our best candidates do).
--That is indeed hard! It's also generally worthless. Why do we need to make every syllabus novel when, what's really needed, is a straightforward, accessible syllabus for a practical intro/medical ethics/political philosophy class that you can actually teach while also teaching three other classes, managing to do a modicum of research, and deal with all of one's committee work. I mean, novel exciting syllabi are... well... novel and exciting. They're also usually impossible.

[teaching portfolios are valuable]

Hi Ttj,

--Sorry for being unclear. What I was trying to say about the teaching portfolios is that even with a lot of help, they don't all end up looking the same/being super compelling. This suggests to me that good teachers really do shine through in theirs! (That's my experience.)

--Yes, my evidence is that I've been doing placement for many years and know quite well which of my students are great teachers, and departments that care a lot about teaching tend to interview and hire those students. (And don't tend to hire my other students.)

--I don't mean 'results on teaching evaluations'. Also your post is a bit rude/obnoxious/sarcastic in a way that I didn't think was encouraged here (flagging this for Marcus). It's not my "definitely unbiased and totally accurate gut feelings". We care about teaching a lot in my department. We observe our graduate students teaching all the time. We write evaluations of them. We discuss them in meetings. They meet with each other to talk about pedagogy. I'm pretty sure that there is a fact of the matter about which of them are the best teachers. I'm also pretty sure that me + my colleagues + the other graduate students are tracking those facts pretty well with all these ways of learning about their teaching. I think it's kind of silly to think that these aren't good ways of evaluating teaching, but if you're a person who both (a) wants hard data/numbers and (b) knows that the hard data/numbers that are available (student evals) are useless, then I think we have bigger differences. I think there are other ways to track good teaching. I don't think this should be controversial.

--I strongly disagree that there is no correlation between being able to do this and being a good teacher; as I said I've witnessed this correlation over and over again in my students' materials.

--You can disagree with departments that want to see compelling, exciting, novel syllabi. But my experience is that many departments that care a lot about teaching do want this. If that is part of what a department is interested in looking for, it seems to me to make sense for them to ask for sample syllabi.

[CV only]

I just want to flag that the kind of special interest pleading evident in Teaching Portfolio's posts is the cause of bloated applicant dossiers. TP thinks that in addition to CVs, we need detailed teaching portfolios because they reveal important information relevant to TP. Well, people who want applicants to submit diversity statements do so because they care about that. People who want to see a planned research trajectory ask for a research statement because they care about that. People who want candidates to address X in cover letters ask because they care about X. The arms race ends by agreeing that just because you want something doesn't make it appropriate to demand it. Especially of first-round applicants, where something like 60-90% will be rejected on the first pass.

Even though I agree that there can be some valuable information in non-CV sources, that information is not valuable enough to ask for it, at least not until later.

[philosopher]

To interject into the debate between Ttj and "teaching portfolios are valuable" (tpav):

Part of my points above have been this: Even if we grant tpav their premise that they and their colleagues are good at evaluating their students' teaching, there's no guarantee that that transfers well to their students' post-PhD jobs. Radically change the circumstances -- different student body, different colleagues, more responsibilities, new life circumstances, etc. -- and grad students great at teaching can sink (or leave, or ...), while those poor at teaching can swim (and stay, and excel, and ...). It's the NFL draft thing: lots of objectively better college players are bombs in the NFL, while objectively weaker players rise to the challenge and succeed.

I'm not saying its total chance or that there's no evidential value to someone like tpav's evaluation and the portfolio they help their students put together. But let's say the signal to noise ratio is 50/50 or even 70/30. My point is this: If you're a hiring committee member painfully scrutinizing teaching letters and portfolios trying to separate your #3 candidate from your #7, you're wasting your time. There's too much noise. Too much uncertainty. We all know this, right? I feel like I'm stating the obvious.

My point is that this obvious point has a consequence which seems to get missed. Namely, in general, "more data" (e.g., a more elaborate teaching portfolio) isn't going to help you separate your #3 from your #7 --- you can't reliably make that separation, anyway. The OP asked what we should make of expanding demands for more application materials. I'm arguing that at this point there's never a justification for them. A few high-quality pieces of data, like maybe a CV, letter or two, and a sample syllabus of the applicant's choosing, is about all you can reliably make use of. The evidential value of everything else is washed out by the noise, and is just fodder for rationalizing subjective preferences.

[Evan]

philosopher: First you argued saying:

“I'm not saying its total chance or that there's no evidential value to someone like tpav's evaluation and the portfolio they help their students put together. But let's say the signal to noise ratio is 50/50 or even 70/30.”

But then you conclude with:

“The OP asked what we should make of expanding demands for more application materials. I'm arguing that at this point there's never a justification for them.”

Your conclusion does not follow from the premise above. Either there is *zero* evidential value to teaching portfolios or evaluation, which supports your conclusion that it is *never* justified to require applicants to provide such data OR there are *some* evidential value to them, which is occasionally or always justified in requiring applicants to provide such data. Pick one. Otherwise, your argument will collapse.

Second, to your other claim that “The evidential value of everything else is washed out by the noise, and is just fodder for rationalizing subjective preferences.”

I fail to see how this is fodder for rationalizing subjective preferences if the university’s aim is to provide high quality teaching or education for their students in the first place. At an R1 it may be debatable whether such an aim is intrinsic or necessary to their institution since the bare minimum that philosophy professors at these schools do for teaching is lecturing and most students are just seen as numbers.

Whether or not providing high quality education to students should be an aim of R1 universities is still debatable and controversial simply because of the size and because many professors in them see themselves mainly as researchers than teachers. Students there are usually just seen as a number and whether or not they had a good education is up to their own responsibility.

At a teaching focused small school, the aim of providing high quality education and hence hiring high quality teachers is justified since most students who attend them and parents who pay for them expect that to be one of the aims in the first place. If the function and aim of these schools are to provide high quality education, then I fail to see how their requirements for teaching portfolio is subjectivistic.

[Assistant Professor]

@nonny mouse as FritzJMcDonald said: it is definitely a thing. I know of various people to whom it applied this current hiring cycle, whose mentors advocated for them directly to hiring committees or to friends in the department to which their student(s) were applying. For those still forming their committees, these practices might make you bristle, but also might give you reason to consider how connected your chosen mentors are and if they are willing to make calls on your behalf, because having well-connected ones who will reach out to their network will benefit you in the long run. Yes, it is another among the unfair advantages in hiring processes.