Our books






Become a Fan

« The Problem of Contingency for Religious Belief | Main | Development of a Grad Student Report »

09/24/2014

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

Matt

I think that this sort of survey would be extremely useful for prospective applicants. One thing that might be worth considering is whether or not to restrict participation to grads that have been at their programs for a certain length of time (a year, say, or maybe restrict it to ABDs). The idea being to eliminate overly-optimistic responses from students that are still excited from getting into a program or who have not yet met with the "endurance-based" difficulties of grad school yet.

I would suggest that if you do run such a survey, it would be advisable to conduct one for terminal MA programs as well, given the increasing rate at which students are choosing to do an MA before applying to PhD programs.

Joe

It's a well-motivated proposal, but it has a basic problem. I know of a top 10 department full of graduate students who are very negative about their department. This in spite of the fact that it does *comparatively* very well in most of the metrics above, from placement to climate. I think grads are generally quite isolated from realities at other departments, yet it is the comparative data that is of interest to us.

Also, there is always the threat of retaliation. If Faculty at a given department are already grouchy or distant or inaccessible, a published report by the grads calling them out is likely to exacerbate tensions. Grads have (or will perceive themselves to have) strong personal reasons to stay quiet about these things.

I'm 100% in favor of programs being accountable for how they treat their grads, but there are severe epistemic and practical barriers to developing an objective ranking via questionnaires. This is why the Jennings Rankings are so valuable: two fairly 'hard' numbers are compared: prestige and job placement. Grads should, as a rule, avoid departments where it looks like a low Jennings ranking isn't going to improve.

Alex Guerrero

For some of the reasons mentioned above, there might be good reason to survey not just (or not even!) current grad students, but also recent program alums. This gets logistically tricky, of course. But those people are less likely to fear retaliation, will have a better sense of the field (having likely moved on to other places, with other climates, problems, etc.), and will be somewhat less invested in gaming the rankings either positively or negatively. That also helps increase the survey pool, so that there is greater anonymity (some programs are very small, particularly in terms of the numbers of current female and minority students).

Another important element, I would think, is to include in the survey people who have left the program before getting a PhD. But again, that proves logistically tricky.

And you'd want to talk to someone with serious surveying training, so as to do things in a methodologically appropriate (I'd worry a lot about response rate issues) and ethical way.

I think this kind of thing could be useful, but I worry that the logistical difficulties might be considerable. I'm not convinced that bad data plus caveats is better than no data at all... If nothing else, it would be interesting to see how departments self-conceive...

Carolyn

I suggested a graduate survey on my Ideal Rankings post (http://www.newappsblog.com/2014/07/an-ideal-ranking.html) and aim to try to implement one in the next year or two, with the help of others (http://www.newappsblog.com/2014/09/placement-data-news.html). If you have any insights on this, feel free to reach out to me by email.

Marcus Arvan

Thanks for the comments on this, everyone.

Matt: Thanks for the apt suggestions!

Joe: I appreciate the concerns but I agree with you on the value of the Jennings rankings, and am now in touch with Carolyn (see below).

Alex: I think data from recent grads is a good idea, but you're right, the logistics are tough.

Carolyn: Thanks for drawing my attention to your NewAPPs posts. I look forward to talking to you about these issues more!

jmugg

I am concerned about conflict of interest in reporting. Isn't in my interest for my grad school to do well? (It is for this reason that Leiter Rankers do not rank their current department nor the department where they earned their PhD).

Nick Byrd

My only concern is that this lacks some kind of baseline. E.g., one and the same placement record my be rated differently by two different grad students. So it seems that one might also want to gather the actual placement data (and other easily measured metrics) alongside these self-reports.

That being said, I heartily support the idea of gathering climate data.

Marcus Arvan

jmugg: I appreciate the concern, but I'm not sure about it. Speaking as someone who has known people in bad situations, my experience has been that these people tend to *want* people to know about the problems in their program, so that the program faces pressure to improve.

Suppose, for instance, your program had a poor placement rate. Would you want this to continue, or would you want your program to face public pressure to improve its practices? Or, consider a program with a bad climate that is making you miserable. Wouldn't you again want your program to face pressure to improve?

In any case, my experience has been that people in bad situations tend to want change, and change only tends to happen when there's some kind of public pressure to do so -- the kind of pressure this sort of grad-student survey might lead to.

Marcus Arvan

Nick: Absolutely! It's presumably best to collect objective and qualitative data. Objective data misses the qualitative elements, and qualitative data can come apart from objective facts. Both kinds of data together give a fuller picture.

anon

We have implemented this at the University of Miami, after the McGinn scandal. I believe we only did a Grad Student Survey the year of his departure, not in the year since, but it may be scheduled again. What we found:

1. Faculty retaliation.
2. Censorship: Whether to share it with the Deans or not was a very heated discussion. Certainly we would not share it publicly, nor with prospective students.
3. Some very clear, specific changes were implemented regarding the process of running the Grad Programme: The qualifying exam process was tweaked, the weekly talks were transformed into workshops, the Faculty received confirmation that McGinn's replacement ought be a woman (it was). Logistically, regarding the operation of the department, these were small yet meaningful corrections.
4. Regarding the climate of the department, it may have sent Faculty and Grad Students off to their own corners, Faculty licking their wounds. But in the wake of our scandal, we were already factioned.

There needs to be a commitment to changing the environment. Graduate Students asked that Faculty to hire an outside mediator to come in and work on integrating the Survey results *and* reconnecting Faculty and Grad Students in a hospitable way, but that particular method has not happened, nor is it likely to. For now students still rely on getting a Dean to step-in when hostilities arise.

jmugg

Marcus-

I think you are right about 'bad' situations, but my worry is more so about the 'in between' places. That is, situations in where the department is not really bad, or really great, might get a bump.

Anonymous Assistant Professor

Just want to point out a potential complication that arises when doing this kind of climate survey.

When I was in graduate school the climate for women in my department was never that great, but it went through a few years that were pretty bad, then it got better because some new grad students came in that were more aware of climate issues in general. During the bad years, everybody would have said we had a great climate, because the grads were almost entirely made up of men who were responsible for the bad climate but they had no idea. During the good years, people would have had a more mixed response, because they were becoming aware of climate concerns they hadn't been aware of before.

Given the extreme demographics of so many departments this can't be a unique thing. Just remember Michael Tooley's claims that he had never witnessed any sexism at UC Boulder so he couldn't believe it was happening. I suspect many grad students are similarly blissfully ignorant.

AC

I agree that graduate student evaluations of their own departments would be useful. However, restricting survey participants to those students who have graduated or are ABD would undoubtedly lead to unrepresentative results. In particular, such a survey would likely omit most students who leave the program - regardless of their reason for leaving. For example, a student might leave because he or she (1) had a health issue, (2) had a problem with the faculty, staff, or another student, (3) found the department unsupportive, (4) couldn't effectively study his or her desired area of expertise within that department, (5) ran out of or was never offered funding, (6) became overly discouraged by the field or its job prospects, (7) switched to another (presumably more favorable) program, or (8) had another personal or professional reason for departure. Gathering information about most of these issues could prove helpful to prospective students and should not be selected against based on participation prerequisites. I do agree, however, that there be a requirement that participants have been a member of the department for at least 1 or 2 semesters. This might be easily accomplished by conducting the study in January each year since most graduate philosophy programs (in the northern hemisphere anyway) only accept new students in the Fall.

I think it is worth noting that, while this sort of survey is likely to produce a lot of unfavorable results, there will also be positive results. Furthermore, even if the results are overall negative, general negativity across all departments would put most departments on an even footing. It could also have the benefit of warning/preparing prospective grad students of the harsh reality that is grad school in philosophy.

anon

the attrition question is especially important, and it's great you're including this.

my phd program has a great placement record, *for those who finish.* but it also has substantial attrition which I suspect to be partly due to a very poor climate. i honestly can't say whether the amount of attrition in this department is typical or below or above average. *because I dont have that data for other departments*

the ghost of philosophy past

I was in graduate school in the 1990s. It was common then, in Canada and the USA, for Ph.D. programs to have a 50 % attrition rate. A friend who was at Cornell said the attrition rate was similar there in the 1990s. If all these people weigh in on your survey, then all programs will be damned.

I think surveys are fraught with problems. Notoriously, people's opinions about the value of things often run at odds with the true value of them. Course evaluations do not reliably track genuine learning (and, yes, I do have strong evaluations).

Anonymous Assistant Professor

Let me just add that the WORST things that happened at my grad department with respect to climate I have never told anyone outside of my department. Other grad students and professors knew but did nothing, which was part of the problem. But the person affected by the worst things was my friend and I promised that friend I wouldn't talk about it. This is not something I could even post about on what it's like to be a woman in philosophy because the smallest details will potentially give away the people who I'm talking about. And for the same reason if I were taking any kind of climate survey I couldn't talk about it either. I would only be able to mention it in the most oblique way, for example, *something* bad happened.

That doesn't mean you wouldn't get useful information with a climate survey, just that we should always let prospective students know that there is no such instrument which will perfectly measure climate.

Marcus Arvan

Anonymous Assistant Professor: Thanks for sharing your story. At this point, I think the plan is for the survey to be purely *quantitative* (i.e. responses to items on a 1-5 Likert scale), rather than qualitative (sharing experiences). This would prevent people from being identified.

Marcus Arvan

ghost of philosophy past: Thanks for your comment. A few thoughts.

First, it might be good for "all programs to be damned." People entering PhD programs should *know* what they are getting into--especially the fact that they may only have a 50% chance of finishing (and a far lower chance of receiving a tenure-track job).

Second, even if many departments fare badly in some respects (i.e. attrition rates), some may be much better than others--which again, would good information to know.

Third, such negative information would put much needed pressure on programs to not let this sort of stuff happen. Sorry--there's no excuse for 50% of your students failing to finish. There may be a long tradition of that, but it's a bad tradition. Programs have an obligation to see to it that their students flourish. Not every student, of course (no program can achieve perfection)--but surely, many programs could do a *whole* lot better...and a Grad Report would put pressure on them to do so.

Fourth, the Grad Report could make other important differences very clear (some departments have *far* better climates than others). This would not only help prospective students decide where to go. It would also, once again, put pressure on programs to improve.

Finally, I think the analogy to student evaluations is poor. First, we will be collecting (as the APA does) objective information on placement rates. Such information can be used to judge how accurate grad-students perceptions are of their programs' level of success. Second, we're not talking about "measuring learning"--a notoriously difficult thing to measure. We're measuring whether grad students are miserable, whether they feel supported by faculty, whether they have to go to school walking on eggshells everyday, etc.--in other words, things that deeply affect people's lives, and which grad students are more than capable of judging in their own case.

Verify your Comment

Previewing your Comment

This is only a preview. Your comment has not yet been posted.

Working...
Your comment could not be posted. Error type:
Your comment has been saved. Comments are moderated and will not appear until approved by the author. Post another comment

The letters and numbers you entered did not match the image. Please try again.

As a final step before posting your comment, enter the letters and numbers you see in the image below. This prevents automated programs from posting comments.

Having trouble reading this image? View an alternate.

Working...

Post a comment

Comments are moderated, and will not appear until the author has approved them.

Your Information

(Name and email address are required. Email address will not be displayed with the comment.)

Job-market reporting thread

Current Job-Market Discussion Thread

Philosophers in Industry Directory

Cocoon Job-Market Mentoring Program

Categories