Visualizations of PhD program placement data

Charles Lassiter (Gonzaga) writes in:

I've been working on a project for some time now. I've downloaded placement and graduation data from Philjobs, cleaned it up, did some analyses, and visualized the findings. The result is a site where visitors can find job placement rates for PhD programs. There's also a searchable drill-down table where users can see where graduates of a program have historically placed. You can find it here: https://charleslassiter.weebly.com/placement-data.html

Lassiter adds, "This is an ongoing project and I plan to update every 6 months in September (for job seekers) and March (for students headed into PhD programs)." He also notes that, "The biggest issue is that the data is crowd-sourced from Philjobs' placement page as well as PhilPeople profiles."

 

Comments

[Mike]

Sorry, I don't see explanations for most of the labels (am I missing something obvious?). What exactly is "mean placements per graduate"? Is that just (number of placements)/(number of graduates)? If so, how can it ever be more than 1? And, what are the ranges of those number, i.e. number of placements per year? for a given year? etc.

But, anyway, thanks for all this hard work!!

[Daniel Weltman]

@Mike, if a program graduates 2 students, and each of those two students has two positions over the years covered by the study (say, a postdoc then a professor job, or a professor job then they move to another school) then mean placements per graduate would be 2. If you scroll over to the last thing in the chart ("Into the weeds") you get details about how everything is calculated.

I think the use of this is mitigated a bit by the typical issues with data. For instance, it's impossible to discriminate between the kinds of jobs, so a school that sends someone off to 6 sequential 1 year adjunct positions before they drop out of the profession and become a consultant would end up doing way better in the data than a school which places everyone at a bucolic small liberal arts college where they buy a comfortable cottage and live the rest of their life in bliss, never once considering moving jobs until they get asked to be chair and even then they only fleetingly consider it. And the data is always quite spotty in the first place. And so on.

Plus, with numbers come rankings and with rankings come doom. But I think overall this is more helpful than hurtful and if it encourages a norm of adding one's information to PhilPeople then that would be good, since I think PhilPeople is the best thing to happen to the profession since chopped logic. Thank you, Charles Lassiter!

[Mike]

Thanks Daniel!

[Charles Lassiter]

Thanks, Marcus for sharing! I'll be making additions and tweaks when I update every six months. If there's anything you'd like to see or any criticisms of the project you'd like to pass along, please don't hesitate to email.

[C-3PO]

I took a look - I sometimes do related research. I think these tables and charts lack a theoretical underpinning. Consequently, very odd things are being grouped together, and differences that matter are not accounted for. Consider the problem Daniel raises - read his message. As a result, this sort of "information" is not useful for job candidates. It has the veneer of science, but it is really unclear what conclusions we can draw from the data.

[Mike]

@C-3PO, [Y]ou're dismissing the project as useless, when some relatively minor changes in the presentation would meet your criticism. After all, job type is in the data Charles is using. So, presumably he can add options to see placement rates by job type.

Also, my sense, for what it's worth, is that people mostly only advertise their placement on philjobs if it's fancy(ish). How many people are posting up their 2-course, 1-term adjuncting gig at Northwest Central South Dakota Community College? So, some self-selection is probably turning this data set into placement rates about desirable (or, at least, not terrible) positions.

For what it's worth, I looked at the data for my own school, and I think it reflects well their placement rate into fancy(ish) or desirable positions. I'd be inclined to (roughly) trust this data as a reflection of that kind of placement -- and that *is* useful information.

[Moderator note: comment lightly edited to conform to blog's supportive mission].

[R]

I really don't want to dismiss the valuable time and effort that goes into a project like this (thank you Charles!), but looking at this I'm inclined to agree with C-3PO that it's very difficult to extract useful information about job market success from this.

Just looking at the place I did my PhD, there are two academic placements recorded for 2011-2020, whereas I'm aware of at least 19.

That's already not great in itself, but the real problem, it seems to me, is that this dataset has no way of capturing how many graduates my program (or any program) produced, so the denominator used in these calculations is potentially way off. If I'm reading this correctly, it is effectively assumed that one person graduated from each program for each year for which there is no data. Regardless of whether you then use the "optimistic" or "pessimistic" assumption, that will massively skew the results for programs that have far more or fewer than one graduate per year.

(Note that at the top of the page the aim is stated as "to paint a picture of programs' academic job placements per graduated student". It's the "per graduated student" part I think is problematic here, in a stronger sense than just "there's some missing data".)