Tips from Search Committee Members: How committees evaluate teaching portfolios

The first four posts in this series examined how hiring committees read and evaluate CVs, cover letters, research statements, and teaching statements. In today's post, I'd like to ask search committee members to answer some or all of the following questions about teaching portfolios:

1. Which parts of a teaching portfolio are you the most interested in?
2. What do you think belongs in a good teaching portfolio?
3. What do you think does not belong?
4. How much weight do you assign to student evaluations?
5. Should candidates provide student comments? If so, is it better to provide curated or non-curated/complete comments? 
6. Anything else you think candidates should know about how you read and evaluate teaching portfolios?
7. What type of institution do you work at? (R1? R2? Highly-selective SLAC? Non-highly selective SLAC? Community College?)

Really curious to hear your answers!

Comments

[Bill Vanderburgh]

As search chair and committee member several times in the last five years, at a teaching-focused large regional state university (R2), I've read thousands of applications.

When we evaluate a candidate's file, we are looking for evidence of (a) strong fit for the job as advertised, and (b) high likelihood of success in earning tenure at our institution. The teaching parts of (a) and (b) are what I'll mention here.

Ideally, committees want someone with a proven track record in teaching. In our case, this means teaching in the AOS for lower and upper division majors, in general education philosophy classes (we teach more of those than we do major classes), and to students like ours (the details of what "like" means will vary across universities, but they all care about this). A proven track record in these areas means we don't have to guess or infer from other factors whether the person, if hired, would build a teaching record sufficient to earn tenure with us. Of course, we know that few/no newly-minted PhDs have experience this extensive and exactly focused. It is an ideal, not an expectation.

As discussed in a recent post elsewhere on Cocoon, teaching statements are not that significant. So what is important? Experience. We would like to see evidence of having been the (sole) "instructor of record" for undergrad classes of about the size we offer, preferably for students like ours. Even better, several such courses over several years. Ideally, with good and steadily improving teaching evaluations.

Again, almost no one has that kind of record fresh out of grad school. This is where folks who have been VAPing, adjuncting, and teaching elsewhere on the tenure track may have a competitive advantage. (But not an unbeatable one.)

How can you make up for not having this experience? I look for evidence of having participated in teacher training, having given presentations (professional or public outreach), being a TA who did more than just grading (led discussion sections, etc.), and related things.

How do we judge the quality of the experience, however extensive it is? We know their flaws and limitations, but student teaching evaluations still come into it. These scores are most informative when contextualized with brief comments like "x/5, where y/5 was the average across all courses in the whole department, and z/5 was the average for other graduate student instructors." I tend to prefer selected comments over photocopies of pages and pages of handwritten student remarks, or at least a single page with the comments typed out in a space-efficient format. We know student comments are biased and uninformative in all kinds of ways, but they can sometimes be useful. (I think it is perfectly fine to leave out the egregious comments that are ill-informed, biased, nasty, etc. It is the general tenor of comments across students that matters, not the outliers.)

More important than the student comments and ratings, we like to see a teaching letter from someone who has mentored or at least done classroom visits for the applicant.

Perhaps most important for us are the sample course materials (syllabi, reading schedules, handouts, assessments, etc.). I suggest giving one full set of these (either a course you have already taught or one in the AOS of the job). Then for other courses taught (or projected), just give a half-page outline of topics, readings, and grading scheme. Here we are looking to see that the candidate is competent to run their own class, knows not to try to jam too much material into a course, knows roughly what numbers and kinds of assignments are appropriate, etc. We know all this is coachable, and tunable to local needs, so we don't take it overly seriously, but it is one of the best indications we can get of someone's teaching ability.

When you list your courses taught, make sure the level (upper or lower division), type (major, GE, other), subject area (especially when not clear from the fancy course name), duration (full semester, a few weeks, or whatever), and number of students enrolled are clear. Especially, make your role clear: "instructor of record; responsible for all course design and materials," vs. "instructor of record; used a common syllabus and exams across all sections, with some discretion to pick readings," vs. "did grading, held office hours, and led three class meetings under Prof. X's direction." (I sometimes see candidates try to list TA experience ambiguously so as to leave open the possibility we think it is a course they have taught themselves. This strategy is not going to help, it is going to get you eliminated from consideration.)

Because we are reading at least many dozens of files for every job, presenting your materials in a reader-friendly way matters quite a bit. Use legible fonts, sensible headings and organization, sign-posting, white-space, etc. Teaching packages that are overly long are not likely to be thoroughly read. A chart on the first page that lists all courses taught with student evaluation scores is very useful. A table of contents (with page numbers) of what is in the teaching package is also a good idea.

[Would-be R1 reader]

When I have been on committees, the teaching portfolio did not receive much attention. This was not because teaching was seen as unimportant, but because nothing it could provide was thought to give much in the way of useful information.

To the best of my knowledge, nobody read any teaching statements except for the very final candidates already selected for interviews on other grounds (and those played no role in further selection-- people were just reading throughly at that point.)

I myself skimmed teaching scores, but only to make sure they were not egregiously terrible. I skimmed teaching comments to look for consistent concrete statements (did many students mention something specific.) I gave unedited full class comments a bit more weight, but even there I was only looking to see whether many people consistently mentioned a concrete specific terrible thing (I don't think students can assess teaching excellence, I think at best these things tell us if the person has a serious problem.) This received very little weight, but more than teaching statements (if a person had a bad teaching statement I wouldn't have noticed unless they said something bizarre--- I did consciously look for whether someone had stand out terrible teaching scores, but nothing more than that.)

I didn't (and I think the other readers also) didn't super care if the person had teaching experience in the exact area, or even that much teaching experience, on the grounds that they very rapidly would.

I did look at syllabi - honestly, not for teaching assessment per se but to get a sense of what kinds of things the person thinks are clear or interesting pieces of philosophy in the relevant area. I kind of think that tells me a little about the person intellectually.

Mostly, my view is the best way of assessing teaching is the job talk. That tells you if the person is pretty good at presenting information and personable in presentation style to an audience.

I suspect it matters for this that I was assessing teaching at a low-ranking "aspirational R1" where the candidate would be teaching a relatively small load of small to medium sized classes. If I was assessing for a person to teach A LOT of courses in a way that would prove grinding, I would have used different criteria.

[taught]

When I worked at a typical state college, we did look at teaching portfolios. Here are some dos and don'ts. Too many people submit too much information - we cannot read it, and we won't. And when there is too much information it is often hard to sort through because it is often not systematically organized and clearly labelled. So files that look unorganized are usually just set aside; it suggests that you cannot communicate effectively. So, think of the teaching portfolio as telling a story. And keep tight reins on the narrative. What we actually talk about in our deliberations is: (i) have they taught the courses we need taught; (ii) have they taught a full teaching load; (iii) have they taught in circumstances like ours (class size, student body, etc.); (iv) do they provide us with clear examples of two or three well designed course outlines (we do not want to see "the ultimate course" - we want to see a course that is reasonable in its expectations, and that can be taught effectively by someone with a full course load and other obligations); (v) is there evidence of effective teaching (it could be a few course evaluations (weak evidence); a letter from a chair where one has taught discussing your teaching and evaluations (stronger evidence); or better still a letter from someone who has actually observed you teaching).
This is how things played out at a four year state college. We will also have you do a teaching demo - try to avoid fancy stunts ... things that wow the students but really could not be sustained through a course, and really do not enhance student learning. The best teaching demos get the students talking, saying sensible things on topic, and addressing each other. This suggests that the teacher knows how to get students working and learning, and can manage a class. Few teaching demos rise to this level.
I am now at a research university, and to be frank with you, it seems my colleagues do not really care about effective teaching at all. Of course, they say they do, and they might even think they do, but research will always trump teaching here. ALWAYS