In the comments section of the teaching portfolio entry in our Job-Market Boot Camp, Grad Student Z writes:
Thanks for this (and all the other) helpful posts. Do you think there is a reason to include unedited student comments as opposed to a selection of student comments along with a website link to the original evaluations (including student comments)?
I'm also wondering how to approach the quantitative data in the following cases:
(i) You are slightly below university and department averages (e.g., less than a 1 point difference with 5 being the maximum) for a particular semester (e.g., your first time teaching), but above those averages for subsequent semesters. Is it worth including all of the data with the the hope that committees will interpret this as exhibiting improvement over time?
(ii) You are above university averages, but slightly below department averages (e.g., less than a 1 point difference with 5 being the maximum). Would you recommend only including a comparison to the university averages (and excluding the department averages)? Or could that come across as if you have something to hide?
(iii) On some metrics, you are above university averages, but below department averages; on others, you are above department averages, but below university averages. Should you include everything? Nothing?
Good questions! I don't think there are any hard and fast rules for any of these things, but want to suggest it is probably best for candidates to try to (A) frame themselves in the best light, while (B) not looking like they are trying to hide anything. Let me explain.
To address Z's first question, I think it is entirely fine to share 'select student comments' in your dossier...provided you do link to complete and unedited comments on your website. What one shouldn't do, or so I've argued before, is merely include select comments and leave it at that (that makes it look like you are trying to hide something!).
With respect to (i), I think there are two good options. One is to include all of one's data. Here, I think one's teaching trajectory is very important. Even if your first semester evaluations weren't great, that's not a big deal. What people are looking for, I think, is a trajectory of improvement. However, what if you have a more recent semester of not-very-good scores? To be honest this happens to everyone! One thing you can do here is, again, just include everything and let committees chalk it up as an outlier (again, not a big deal). Another thing I think you can do is to just include all of your evaluations from a single very-good semester. I don't think I would balk as a search committee member at a dossier like that at all. In a sense, in may be "trying to hide something" if you only include one semester of evals, but I'm not sure it looks that way, since my sense is that it is what a lot of people do.
My answer to (ii) is similar. I don't see any reason to include comparisons to department averages. Some places don't even provide them to instructors! So, here again, feel free to frame yourself in whatever way looks best. If your evaluations look favorable comparable to university averages, just include that. It won't look like you're trying to hide anything, because again many candidates don't even include that much.
Ditto for (iii). The summary page in my own teaching portfolio only listed comparisons on particular items that I thought reflected well on me. I just included all items in the actual evaluations following the summary page so that it was clear that I wasn't trying to hide the fact that some scores were lower than others. There is nothing wrong, I think, with accentuating the positive!
Anyway, these are just my thoughts, and they could well be wrong. I'm curious to hear whether other people agree. One final thing to bear in mind, I think, is that an increasing number of people appear to be skeptical of the evidential value of student evaluations. If your scores are consistently way below average, then that may be seen as a red flag, especially if the scores are accompanied by worrisome comments (e.g. "The professor was never prepared for class"). But, in general these days I think student evaluations are taken with a real grain of salt!
What do you all think on these matters?
I included a very truncated summary of my evaluations (like, only included the 'rate this class' question), representative comments, a narrative explanation of the evaluations, and said the full evaluations would be available on request.
I don't think it's reasonable to ask people to include their full evaluations. I've seen evaluations which comment on a professor's dress, looks, personal bearing, hygiene, sexual orientation, and so forth. Some are meant to be complementary, some helpful, some are mean-spirited. All these things are inappropriate and not something I would want to include in a dossier. Nor is it something I would want to post online on my website. I said full comments would be available on request, and listed sample or representative comments. No one every asked for the full comments, but of course I would have provided them if they had been requested, and did try to make the representative comments representative, and used the narrative to interpret those comments and talk about my response to them, things I thought I did well, things that I was committed to improving, ways I had already improved.
A search committee will see that you are hiding something if your scores are low and your selected comments are glowing. But if you have reasonably high scores and comments to match, I don't think that's a problem.
I do think it's important to include evaluations for all semesters taught. An upward trajectory will count in your favor (here I think a narrative is important), and the more recent evaluations will count more.
Posted by: Prof. L | 10/17/2019 at 11:39 AM
I agree with Prof. L, mostly - I'm less sure about their last point: that one should include evaluations for all semesters taught. If you have a lot of teaching experience, I think its OK to only include all of the last couple of years evaluations, for example. It can depend a bit on what you've taught (if you taught different courses early on, and those are the courses that the job ad is looking for, then it might be wise to include those.)
Posted by: Chris | 10/17/2019 at 12:22 PM
Thanks to Marcus for sharing my comment!
After reading Marcus's post, I went through my teaching dossier with the goal of presenting myself in the best light while avoiding the impression that I'm trying to hide something. Here's what I noticed:
- Eliminating comparisons to department averages made me look much better.
- My first semester teaching my own course is below average with respect to a number of items. However, for the items "overall course" and "overall instructor," line graphs help to visually emphasize that my ratings went up the following semesters (while university averages remained constant).
- I experimented with a "summary" chart (similar to Marcus's) that lists my averages with respect to various items for the three semesters that I taught my own courses. Unfortunately, that first semester brings most of my averages below the university averages. So, I decided not to include this chart. Once I've taught more courses (and my averages go up), I suspect that it will be to my advantage to include it.
My dossier currently has the following structure:
(i) Quantitate evaluations as instructor (3 courses)
(ii) Quantitative evaluations as a TA (5 courses)
(iii) Selection of unedited student comments together with a link to my website where complete evaluations can be downloaded
I'm a bit conflicted about whether to keep the TA evaluations. Most of them are above university averages with respect to almost every item, which is why I included them. On the other hand, there is a case to be made that they reinforce the impression that I am a graduate student.
Marcus wrote:
>>
Another thing I think you can do is to just include all of your evaluations from a single very-good semester. I don't think I would balk as a search committee member at a dossier like that at all. In a sense, in may be "trying to hide something" if you only include one semester of evals, but I'm not sure it looks that way, since my sense is that it is what a lot of people do.
<<
Would you recommend doing this even if you have good scores for more recent semesters?
My two main options appear to be (a) including complete quantitative evaluations for the three semesters that I taught my own courses (showing improvement over time) and a selection of student comments together with a link to my website or else (b) including complete quantitative evaluations for my most recent semester (all above average) together with complete student comments (mostly positive).
Posted by: Grad Student Z | 10/18/2019 at 06:04 PM
Prof. L mentioned including a "narrative to interpret those comments and talk about my response to them, things I thought I did well, things that I was committed to improving, ways I had already improved."
I've looked at a number of teaching dossiers, and this is my first time encountering this approach. I'm curious to hear more about it. How long is the narrative? Can someone provide a concrete example of how this might look in practice?
Posted by: Grad Student Z | 10/18/2019 at 06:36 PM