Query on grad program attrition

Mike Titelbaum (University of Wisconsin-Madison) wrote in by email:

I am a big fan of The Philosophers' Cocoon, and occasionally jump into the comments section...[But] There's something that occasionally comes up on Cocoon threads and elsewhere that bothers me...

Often when we counsel students comparing graduate schools, we suggest that they look (among other things) at the percentage of admits who complete the program. The underlying implication seems to be that higher completion rates indicate a better program. But this seems to me out of step with another discourse floating around the internet in general and the Cocoon in particular—that deciding as a PhD student that you don't want to become an academic isn't a failure in any sense.

Sometimes a department has a low completion rate because they're unsupportive, are chasing out certain students, or have a toxic environment. But PhD students leave programs for many reasons, including that the program has done a good job of revealing to them what the academic life is like, and they've decided it isn't the life for them (even though it might be available to them). If a department was pressuring people to stay and finish their degree despite having had such a realization, that would enhance their completion rate but it seems to me would be the kind of department that I would discourage a student from joining.

There's also presumably a big selection effect here. Different departments wind up with different kinds of students.  Some populations might be more inclined to decide that academia isn't for them and leave their program before completing. That would hurt a department's completion rate, but perhaps indicate nothing bad about how the department treats students once they arrive. In fact, if a department is more willing to take a risk on certain types of students who might not finish, that might be a good sign about that department's atmosphere and goals.

As I said, this is just something that's been bothering me. I have absolutely no empirical data on any of this, and not even much anecdotal data. But if you're measuring a department by its completion rate, then you're assuming the sole goal of a PhD program is to create PhDs (and possibly then academics). Perhaps instead the first goal of a PhD program should be to help students figure out whether writing a dissertation, completing a PhD, and then becoming an academic is the right path for them?

I think Mike raises really good points here. In email discussion, I suggested that perhaps the best thing to do is to compile and make more transparent both qualitative and quantitative data regarding attrition and job-placemtn.

Offhand, all of the following facts about attrition seem directly relevant to prospective students vis-a-vis making an informed choice about whether to enter a grad program:

1. The proportion of students who enter a program but do not finish.
2. The amount of time that students who leave spend in the program before choosing to leave.
3. The qualitative reasons they choose to leave.
4. The quality of job-placements obtained by those who choose to leave.

For example, there is a huge qualitative difference between leaving a program after one or two years, compared to leaving after 7-8+ years. At least anecdotally, I've heard some programs tend to have the former (encouraging struggling students to leave early), whereas other programs allow struggling students to flounder for many years. As a prospective student, I would absolutely want to know which kind of program I'm entering. By a similar token, I'd want to know why students who leave choose to do so. Is it because they have good non-academic non-prospects (which might be the case if the program is at a particularly prestigious university)? Or, it is because their program has a poor climate or offers poor mentoring? Again, as a prospective student these are things I would want to know. Finally, I would want to know what kinds of jobs people who leave the program end up getting. Are they well-paying non-academic jobs, or low-paid entry-level jobs with few benefits and little opportunity for advancement?

I'm also inclined to think that it would be good to gather and transparently disseminate similar forms of qualitative and quantitative information about those who do finish their grad programs:

1. The proportion of graduates who pursue academic vs. non-academic jobs.
2. The proportion of those who seek academic jobs who obtain permanent jobs.
3. The kinds of non-academic jobs obtained by those who don't.
4. How long it takes the program's graduates to obtain permanent jobs.
5. Etc.

In our exchange, Mike said he liked the idea of making these kinds of details more transparent. However, he noted he wasn't sure how this would work. After all, grad programs already have trouble tracking and reporting these details reliably, and perhaps in some cases incentives not to. Here, though, is my suggestion: if grad programs are not well-positioned to gather and make this kind of quantitative and qualitative data transparent, then perhaps some other institution can. If I recall correctly, the Academic Placement Data and Analysis project is engaged in an ongoing process of collecting and reporting these kinds of details--at least for recent graduates of PhD programs. Although I don't know whether they are gathering data on attrition, I imagine something like this could work. For example, suppose there were a site that all new MA and PHD students were encouraged to register at upon entering a grad program. During the registration process, these students might be informed that they can update their profile at any time, answering survey questions such as whether they are still in the program, whether they left without graduating (and if so, after how many years), what kind of job they obtained if they left (including the job's salary), and so on. I imagine that if this kind of site and database were well-designed and monitored--and if it made its findings transparent--then it might go a long way to giving prospective students accurate and useful information for making informed decisions.

As someone who has known grad students who very much regretted joining their program, I think a project like this could be a very important service to many people. I'll never forget a fellow I met on a flight to the Eastern APA many years ago. He had been in his (highly-ranked) PhD program for 8+ years, and he was absolutely at the end of his rope. He told me that his program was very dysfunctional, that hardly any of its students ever finished, and he just seemed hopeless. And he's far from the first grad student I've met who felt that way about their program--whereas I know others who loved their programs and felt like their program put them in a good position to succeed. These, I think, are the kinds of details that prospective students should know. Further, public reporting of qualitative and quantitative information like this might also function as an important accountability mechanism--as few programs, I think, would want to become known as "that program" that fails its students.

But these are just my thoughts. What do you all think?

Comments

[Trevor Hedberg]

I almost feel like this inquiry is addressing me specifically since I have endorsed both those views in written posts on the Cocoon -- both (1) prospective students should consider attrition rates in their choice of grad program and (2) it is not a sign of failure to pursue a non-academic career (even if that means deciding not to finish the PhD).

My thought behind (1) is that prospective grad students usually have the goal of finishing their PhD, and so it makes sense to go to a program where that is more likely to happen, other things equal. But their values could change along the way to the PhD (a journey that is likely to last 6-8 years), and in that case, I'd say it's okay for them to switch trajectories. So that's the support for (2).

But Mike's right that there's a potential tension in these views, since programs with low attrition rates might have them because they are less supportive of students pursuing non-academic careers. I suppose overall you would want a program that has a low attrition rate but where you would still be supported by the faculty if you elected, say, to take a terminal MA and pursue a non-academic career. But given the information that's available, that combination could be hard to find.

I'd say prospective students should still aim for programs with low attrition rates -- in part because I think leaving a PhD program after with no degree after many years of study is the least desirable outcome. Even if one doesn't use the PhD for an academic job, I still think it's worth getting for its own sake. (Perhaps I am influenced by my father's time as a part-time student pursuing an MA in history -- over 30 years later, he still regrets not finishing the degree.) I'd say prospective students should try to inquire with current or former grad students about how the department deals with cases of attrition and how it's perceived. I suspect getting accurate information about that through reported numbers and metrics (such as those listed above) would be difficult even if they were implemented across a lot of programs, but I have generally found that grad students are willing to respond to emails and share their experiences pretty regularly.

[Amanda]

I think it makes the most sense for the current project to expand itself, and ask more questions. If multiple organizations are sending emails asking students to fill out forms, it is more likely people will ignore the emails and we may end up with very incomplete information, which can be even worse than no information. I actually switched jobs twice before updating my profile, so I guess I'm worried about how complete the info is, while at the same time I support the project and think it's important.

On the point about attrition rates - whether they are good or bad - I have lots of mixed feelings. I think Mike makes great points, and agree that many grad students drop out for good reasons, and that says nothing bad about their graduate program. On the other hand,I do think that programs which tend to have high attrition rates, tend to be less supportive and don't foster a good sense of community. But this is just what I observed, not statistical.

[Peter Furlong]

This is an interesting issue. I have long worried that paying a lot of attention to completion percentage could be a problem and for the very reasons that Mike mentions. I also worry about some of the more specific issues Marcus points to. So, for example, he mentions both of the following as important:

The kinds of non-academic jobs obtained by those who don't.

The quality of job-placements obtained by those who choose to leave.

I worry that the answers to these questions may tell us next to nothing about the quality of the program and tell us much more about the sorts of students these programs admit in the first place. So, for example, if school A does much better in these questions than school B does, that might have nothing to do with the quality of education at these two schools and much more to do with the demographics of the students they accept.

Even if we take this info as merely one piece of information, I worry that it will be a misleading one.

Average time to completion is probably a little more reliable in terms of reflecting something about the program rather than merely something about the students they tend to admit, but it brings with it its own problems. First, it seems to me that certain specializations tend to correspond to longer completion times than others. Perhaps I am widely off-base on this, but it has seemed to me (and I have heard others say the same thing) that people focusing in the history of philosophy often require a bit more time than average (perhaps because of the need to gain a command of multiple new languages). Second, students from certain demographics may tend, on average, to have more distractions from outside of their academic work that will lead to longer graduation times. Finally, there is the issue of whether grad programs should be given an incentive to force out students who have taken a bit longer than usual. I know that many programs have policies on the books about requiring students to finish within a certain amount of time, but I also hear that these are widely ignored, at least at some programs. I firmly believe that programs should support and encourage students to finish quickly, but I am less sure whether it is overall beneficial to provide an incentive to cut off students who are struggling at the dissertation stage. Some of these students probably could have been identified earlier and approached about their future ability to finish the program, but in my own (very limited) experience, those who struggle at the dissertation stage are not always those who struggled earlier in the program.

Perhaps there is some value in knowing the average success rate and the average completion rate, but if there is, I don't think it is very high. I would like to see a list of completion times (perhaps something that said "in the last five years, we have had 6 graduates take 5 years to graduate, 8 who took 6 years, 5 who took 7 years, and 1 who took 13 years). I think this would be more useful than averages and would give less incentive to schools to simply cut off those who take more time in the writing phase.

As for other issues, I think that the only way to get information on how supportive a given department is (both in terms of supporting those who choose to leave and guiding students who stay through every stage of their graduate education) is to contact current students and talk to them. Many schools list their graduate students and provide their email addresses online. I doubt most would mind if prospective students sent them a short note asking about these issues.