Our books






Become a Fan

« Citation rates and journal rankings | Main | Should more journals do 'targets and replies'? »

07/22/2019

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

Daniel

It has been a little while since I was on the market, but if I recall correctly, I provided selected student comments but (right there with the selected comments) noted that full comments are available on my website for download and also via request. I assumed that this would contravene the impression that I'm trying to hide something, because if I were trying to hide something I wouldn't put them all available online for anyone to download. Moreover, it made the portfolio a much more manageable length and more readable, since complete comments from even one class would take up a lot of space. Thoughts on this sort of approach?

postdoc

Echoing Daniel's comment, since I was confused by the same part of the post. I've seen plenty of dossiers with both a selection of top comments and raw evaluations complete with comments for several courses. Surely an approach like this doesn't come off as though the candidate is trying to hide something? I take it that providing selected student comments would only be problematic if that where all the applicant included in their dossier.

Marcus Arvan

Daniel: I think that is a fine idea! As long as you draw attention to the fact that complete and unedited comments are available on your website or by request, it shouldn’t look like you are trying to hide something.

Postdoc: thanks I’ll clarify the post. I should have said that *only* including raw numerical evals and select comments is the issue (and it does seem to me to be a pretty common mistake). As long as complete and unedited comments are somewhere in the portfolio or otherwise available (as Daniel’s approach does), I think candidates are fine.

Amanda

I strongly agree that including only the raw, unedited evals is a big mistake. It is often impossible to read, and meaningless without really understanding the context. Canidadtes should (1) summarize their numbers for each class in a way that is easy to read. For example,

*Below is a list of my average score for each of the mentioned questions. My university scores on a system of 1 to 7, with 7 being excellent. (if your university includes too many questions, only include the most important 6 or 8, and always include the "overall evaluation of the course/instructor questions."

While it is fine to include the raw data at the back of the portfolio, I also think it is just as good to include a link to this data, since it takes up so much space and few search committees will look at it initially.

I have mixed feelings about what Marcus says about select comments. I personally would not want to see select comments, but I know many people who have been successful on the market with this method, and many search committee members who don't care. So if you have some very negative comments, it might be best (strategically) to do select comments. This seems especially so if all it takes to satisfy the "all comments included" search committee member is a link to a complete set of comments. Most search committee members won't bother looking, so what will be in their head is whatever comments you included.

A table of contents that makes everything easy to read is very important.

anon

Question then:

If I decide to include both positive and negative student comments, this would seem to ward off the impression that I am hiding something, right?

But at the same time, including negative comments might also hurt me? But presumably everyone has negative comments, so maybe not?

So I wonder if the best approach is just to not include any comments at all and just go with quantitative data but make sure you indicate that full data is available upon request?

Marcus Arvan

anon: I think the answer probably depends on what your student eval comments are like.

I included all of my student comments because the vast majority of them were positive, and I reasoned (correctly, I think) that search-committee members would look at the big picture. My sense is that it is usually possible to find at least *one* recent class one has taught where the student comments are positive on balance--so all things being equal that is what I would suggest: including all student comments for your best recent class.

On the other hand, if you don't have a single recent class with comments that you are comfortable sharing complete and unedited, then I think the best thing to do is to probably only include quantitative information.

guy who was looking for a job

Kind of a n00b observation here, but one thing I noticed last year was that some teaching portfolios I saw were fairly short, as in, only around 8-10 pages. I did not submit one that was shorter than 24 pages, and some of mine were as long as 30 (and I've seen ones that are longer than that from other people). Part of the issue here is that having a thicker teaching portfolio is going to make you seem more experienced, as long as the stuff in there isn't filler. And student evals, either written or quantitative, are not filler.

For what it's worth, I had 4 pages of written evals, and 4 pages of quantitative evals. For the written part, I included every single comment I had received for the courses I included, and noted that the comments were "complete and unedited." This meant that I included a few comments which blasted me for various things, but I don't think that matters. I doubt the committee members who looked at my portfolio read many of the comments. It's much more likely that they are going to skim through a few, see that they are positive (since most of them are), and then just move on. (You can be strategic about the order you put them in, even when including all of them--the very first written comment listed on mine was, "Absolutely take the class with [my name].")

For quantitative evals, again, a n00b observation here, but I have been surprised at how many people just include a list of numbers. This is *not* a good way to present data! You must control the story that you want your numbers to tell. Quit being lazy and design some attractive-looking bar charts or other graphs that will represent your numbers in a format that is much easier to understand.

And again, making the charts allows you to be strategic about the way you present it. As any good scientist knows, you can play with the axes on the charts to exaggerate differences between you and a comparison group (assuming that you are better). It is well understood by now that people are very prone to cognitive biases in situations like this, since the way most people read the chart is by looking at the distance between the bars, without paying much attention to what the scale is, or where the axis begins. This way, you can make a two-tenths difference in mean values seem very large indeed. Of course we can all agree that means separated by two-tenths of a point in teaching eval scores are basically the same; the error bars would be huge, were you to include them. But again, this is not how people read the charts.

I guess I would echo something that Amanda said in an earlier thread: you must imagine that you are the person reading your portfolio, and then you must present the material in the way that will make YOU look the best. And this includes exploiting every cognitive bias of theirs that you can along the way.

Amanda

I am not sure a "thicker" portfolio will make someone look more experienced. First, almost everything is online, so the physical thickness will not be apparent in an obvious sort of way. Second, it is just so easy to look at someone's CV and see how many classes they've taught. This is typically noticed before delving into the teaching portfolio, so I suspect search committee members have a picture of how experienced a candidate happens to be before they look at the details of the portfolio.

Exploiting cognitive biases in the way "guy who was looking for a job" suggests might be helpful, all things considered equal. Whether it is worth it, strategically, seems to depend on how long it would take you to put this together, and how much of an increased benefit would result.

FWIW, my got to format was to always include evals from 4 courses. I thought this was more than enough to give a picture of who I was as a teacher, and not too much as to overwhelm search committee members. it is relevant, though, that showing these 4 courses evals conisted in, (1) summarizing the scores from 6-8 numerical ranking questions, and (2) including complete, unedited comments. Also key: I never taught classes with more than 40 students, so the complete list of comments was pretty short. If you teach giant courses, I wouldn't have all evals be from giant courses (hopefully you also taught some smaller ones.) If you have only taught giant courses, than I wouldn't include more than 3 full sets of evals. This is just my best guess on what is likely to show a full picture of your teaching without overwhelming search committee members. Lastly, make sure the evals you choose are recent, i.e. that unless you have been on teaching leave, all should be from the last 4 years, and at least one from the most recent year

Verify your Comment

Previewing your Comment

This is only a preview. Your comment has not yet been posted.

Working...
Your comment could not be posted. Error type:
Your comment has been saved. Comments are moderated and will not appear until approved by the author. Post another comment

The letters and numbers you entered did not match the image. Please try again.

As a final step before posting your comment, enter the letters and numbers you see in the image below. This prevents automated programs from posting comments.

Having trouble reading this image? View an alternate.

Working...

Post a comment

Comments are moderated, and will not appear until the author has approved them.

Your Information

(Name and email address are required. Email address will not be displayed with the comment.)

Job-market reporting thread

Current Job-Market Discussion Thread

Writing Service


Categories