Best practices for choosing external reviewers for tenure applications?

In our April "how can we help you?" thread, a reader asks:

Are there any best practices or advice for how to choose external reviewers for a tenure application?

I wouldn't know, as my university doesn't use external reviewers. But I'm curious to hear from people at institutions that do! 

Anyone have any helpful tips to share?

Comments

[elisa freschi]

Hi Marcus,

Q to you: I did not know that some institutions didn't rely at all on external reviewers in tenure reviews (my University uses about 7 external reviewers). Does this not make the whole process resemble a "sympathy context" in which amiable candidates are confirmed and "awkward" ones may be refuted tenure? (I am sure I am missing context, hence the Q).

[when teaching is most of what matters]

Hi Elisa,

I'm not Marcus but I'm familiar with tenure practices at a variety of institutions, and many more teaching-focused or less research-intense institutions in the US do not use external reviewers, at least in part because if the main criteria for tenure is teaching and service, it's hard to see how external reviewers could say much of substance about that compared to the evidence from within the candidate's institution. I'm not sure about the sympathy context stuff, but I do think that most institutions where teaching is the most important criterion for tenure at least try to have some standards about what that means.

[K]

I also worked at a college where we did not have external reviewers for tenure - our principal concern was teaching and service. And the state would probably not want to pay even meagre compensation to external reviewers. But my department was research strong, so we started to solicit external letters - sent directly to the head of the promotion committee. In one way, we were just starting an arms race ... raising the bar for all, at least tacitly.

[Chris]

Dear TT

I have been at an R1 with a large philosophy department for many years (and previously at a different R1), and so have seen many tenure reviews. As has already come up, what kind of institution you're at can make a difference here (even within the "research" category). So take all of my advice with a grain of salt.
Each school will have its own (slightly or significantly) different standards. Consequently, different schools and departments are looking for different things in the letters.

Some suggestions:
1. Ask your senior colleagues for advice! Your external letters usually serve a couple different purposes: (i) to give the colleagues in your department info on the quality of your work, and (ii) to give "higher ups" who aren't philosophers info about the quality of your work. Sometimes one of these can be more important than another. Maybe everyone in your department thinks you're great, but you want external letters to give independent validation, etc. Maybe you work in ethics, and 5 other people in your department do so, and they all have their own opinions about the quality of your work. Or maybe you're in logic, and no one else in your department understands your work, so they're relying on the letters as much as the higher ups in other disciplines will to determine the quality of your work.
The ideal letter (like certain grant applications) makes you look good to both philosophers and non-philosophers. Some philosophers know how to write good tenure letters, and some don't. Some people are known assholes in the profession. Your senior colleagues might have an idea of whom to avoid.
2. Depending on your University, you might also need to think about the quality of the referee's institution. For many places, you want letter writers who are at institutions that are viewed as "peers" or better. Again, some places care more about this than others. In some cases, it is OK if the person writing has a fancy research position (e.g., endowed chair) even if their institution isn't quite considered "peer". (Some places ask their letter writers to say whether you would get tenure at their institution, but some don't. Find out what kind of institution yours is).
3. Find out whether it matters if the people writing your letters only have to have tenure themselves, or whether they should be "full" professors. If unsure and you have the option, better to err on the side of asking full professors.
4. Look for people that have cited your work positively. Sometimes, of course, in philosophy, people discuss your work critically and it can be a sign of respect. In other cases it means they think your work is crap. But sometimes it is a straightforward "for a nice discussion of X, see Y". Those can be good people to ask.
At some universities, it can be a strike against you if too many external letters say "never heard of X, but their work is great". That is, they want to know that you already have a bit of a national or international reputation. Try to pick people that you think have heard of you. Maybe you had a positive interaction at a conference.
5. How many external letters do you need? I've been at institutions where you only needed 4, and one where you needed 10. There are pros and cons to each. If there are only 4, one bad letter can have a bigger impact than one bad letter out of 10.
6. Usually the candidate (you) has some input in letter writers, and usually your department has the option of their own input for say, half the external letters. If the department supports you and knows what they are doing, they will find a way to informally consult you about "their" choices (e.g., they will also choose people from your list, unless they know other people whom they know will write fair letters, etc.)

I'm sure there are more suggestions others can add. Good luck with this!

[Chris]

I forgot to add: look for people who have used your articles in their teaching. This is less common for junior people, but it does happen, and if that person also meets the other criteria (at a peer institution, full prof. etc.) they can be good people to ask. You don't usually assign a paper in a course unless you think there is something good about it.

[Marcus Arvan]

Hi Elisa, I'd second what the others have said so far. Insofar as institutions like mine prioritize teaching and service, it's not clear how external reviewers might comment on these parts of a tenure file.

Additionally, our university's standards for what constitute merit in teaching, service, and scholarship are clearly spelled out in the faculty handbook (e.g. publishing work in peer-reviewed journals). So, again, it's not clear what external reviewers would add that would be salient.

My sense is that R1 schools solicit external letters because their aims in evaluating tenure cases are quite different: they want evidence not just that a person has published, for example; they want to know whether a tenure-candidate is a "leading scholar in the field" (or whatever), and while citation-rates, etc., might be some evidence of that, the general belief is that external reviewers (i.e. other experts in the field) are necessary or at least helpful for judging that.

Having served on T&P committees, my experience is that they approach the process with integrity, and have to justify their recommendations by systematic documentation of merit as defined in the handbook. This strongly mitigates, in my experience, a "sympathy context" or bias against "awkward" candidates. Moreover, if any such things were to occur, there are appeals and other processes in place to address them.

In fact, although I may be misguided in thinking this (in part due to never having participated in a T&P process at an R1), I personally worry much more about the kinds of biases that *external* letters incorporate into T&P decisions. What is more subjective, after all--and more liable to personal, moral, and/or political biases--than appealing to individual, handpicked judgments of whether someone is a "leading figure", "rock star", or "does excellent work"? If I have to judge someone, I'd prefer to judge them on their objective accomplishments!

[Rosa]

Two more things you should ask your department about:

1. How "close" to you the letter writers can be. I believe you are generally not allowed to have co-authors or thesis supervisors write you letters. But whether you can have, for instance, people who have written you letters for grants, the job market, etc, write for you varies from institution to institution.

2. Whether declined requests are held against you. At some places declined requests are treated like negative letters. At others (especially if the person declining makes it clear they are declining due to being busy and nothing else), it doesn't matter if folks decline. If it *does* matter at your institution, then you probably want to avoid very fancy people or people who you doubt have heard of you.

[Tenured SLAC]

From my experience on T&P committees at a SLAC, we use external reviews for context. Though teaching and service are the priorities, we are also interested in how a candidate's scholarship fits in their field and how it is perceived by peers. Where I work, many departments are so small that colleagues aren't overly familiar with the subfields their colleagues work in. Getting external letters--in my experience at least--will often give a T&P committee a deeper appreciation for the quality of the scholarship our colleagues do.

So, in terms of advice, it is important to get to know people in your subfield who work at like institutions. If you are at a SLAC, talk to other people in your field who work at a SLAC. This may help you select your reviewers. As well, if you are at conferences where scholars respond to each other, take note of who gives honest but generous feedback and who doesn't. Finally, when putting together your T&P file, offer context. If you are worried that an external reviewer may look down at work published in X journal or presented at Y conference, offer the context for why your work appears there. T&P committees understand that we can't all publish in top-tier journals. But they will want to know that your work is making an impact in places that matter.

[elisa freschi]

Thank you, K, Marcus and "when teaching is most of what matters", for explaining!

[Mike Titelbaum]

Marcus, my experience with R1 tenure letters has been that bias largely isn't a big issue. (It's much better than with job recommendation letters.) Most letter-writers do a careful, thoughtful job explaining the candidate's work and putting it into context. Then they speak to the promotion criteria provided them by the candidate's university, which is usually something more like "national reputation" than something like "leading figure" or "rock star".

HOWEVER, there are a handful of miscreants in this process, and it can be hard to predict who they are. So I'd recommend to the OP that they consult someone (or a number of someones) whom they trust in their subfield to find out which tenure letter-writers are absolutely to be avoided. Are you still close with your dissertation advisor? Have you developed any mentors in the field? Have them ask around to find out what you need to know.

With that said, it will perhaps be reassuring to know that I'm familiar with a number of cases in which one bad letter did not sink a tenure case. So don't panic, but do be conscientious!

[AnonymousT]

I work at an R2 and we require three external letters. I don't know how much they matter within the department, unless the individual's research is in an area less known to other members of the department and the letters provide context. I think the letters are helpful to people outside the department and are part of making sure that candidates do serious work along the way. Knowing that you're going to have to have external letters from your peers in six years tends to make people get their work done (since there's no avoid it). But letters can have other uses as well.