Development of a Grad Student Report

I am happy to report that I will be collaborating with Carolyn Dicey Jennings and the volunteer committee she is working with to collect and report job placement data to also develop a philosophy grad student report. Although the report is only at the very beginning stages of development--and our aim at this point to see whether a sound methodology can be developed for such a report--its prospective aim will be to report grad student evaluations of their graduate programs on issues such as:

• Overall department climate
• Climate for women
• Support from faculty
• Job placement
• Attrition
• Etc.

In addition, if feasible, the report may also include evaluations by previous grad-students, both (1) early-career philosophers who recently graduated from their program, as well as (2) individuals who left their programs without finishing their degree.

Although readers raised some concerns about such a report in the comments section of my previous post, there appears to be significant support for the proposal (see here and here), provided it is done well. Furthermore, my own first-, second-, and third-hand experiences strongly suggest the importance of such a report. Seeking a graduate degree in philosophy -- especially a PhD -- involves great risks and personal investment. Prospective students who want the kind of information a grad-student report might provide should, I believe, have access to such a report...again, provided the report is done well.

Our first aim, then, is to try to develop a sound methodology -- and this is where we, the Cocoon, come in! One thing I think our readers can do is help us work through the development of a sound methodology together. Hammering out potential methodologies openly is, I think, not only good in terms of transparency, but also in terms of developing a sound one.

So, here is what I would like to do today. I would like to:

1. Present for discussion some programmatic, possible solutions to concerns readers raised in the comments section of my previous post, and
2. Solicit reader suggestions for survey items (i.e. what should be measured, and how?)

Here, in brief, are some concerns readers raised about a grad student report (both publicly and privately to me by email):

• Potential for retaliation/increasing department tensions: since the report may paint some grad programs in a negative light, there is a serious worry about retaliation, both by faculty and fellow students. As one reader noted, negative climate evaluations of their department pit faculty and grad students against one another.
• Potential for "gaming" the report: since grad-students have an interest in their program's reputation, the results of a grad student report could erroneously measure how students want their program to be perceived.
• Difficulties in measuring climate for women & underrepresented minorities: since men vastly outnumber women and underrepresented minorites in almost all departments, measures of climate could primarily reflect the interests and judgments of the dominant majority.
• Difficulties measuring overall department climate: since a department climate could be "poor" for all sorts of relatively innocuous reasons (i.e. grad students/faculty living far from campus and rarely socializing), data on climate could be highly misleading, suggesting serious problems where there are none.
• Problems classifying and perpetuating relatively innocuous climate problems: on a related note, data suggesting a department has a poor climate in relatively innocuous ways could scare away prospective students who might significantly improve those elements of the department.
• Problems with baselines: as one commenter in my previous post noted, some students may dislike their department and/or rate their department's placement rate negatively even though their department does a great job educating and placing students.

I am optimistic that all of these concerns can be sufficiently addressed. Here are some of my thoughts:

• On the report fostering retaliation/tension: The report could include items relating specifically to retaliation ("Members of my program would respond to negative data from this report in a positive, productive manner" [Agree/disagree]). This, I believe, might seriously deter faculty and grad students from engaging in it, and at the very least put public pressure on programs to avoid it/cut it out. Furthermore, it might be possible to secure departmental commitment not to retaliate, and prevent retaliation by faculty and students, relating the basis of the report. Finally, although retaliation and tensions are bad, it's not obvious to me that their existence is worse than the alternative, which is for no grad-student report to exist and serious departmental problems (which seriously affect students) to go consistently unaddressed.
• On potential for "gaming" the report: we could have some sort of reporting mechanism in the survey itself logging respondent concerns about gaming the report ("My department has exerted pressure on students to reflect positively on the department in this report" [agree/disagree]). Although grad students who might be complicit in trying to game the report might answer "disagree", it is, I think, highly unlikely that every grad student would do so, as there are unhappy students in most, if not all, programs who would be apt to give genuine answers. Significant differences across departments in "agree" rates to such a survey question (even if only a small number of students in a given department "agree" with the item) might, therefore, indicate attempts to "game" the report, suggesting to readers of the report to consider that department's scores with some suspicion, as well as deter the practice within departments.
• On difficulties measuring climate for women & underrepresented minorities: I would like to suggest that this issue can be dealt with by controlling for differences in departmental representation. For instance, even if men vastly outnumber women in a given department (say, 29 men to 3 women), the survey's item on climate for women could give male respondents and women respondents each a "50% share" in their department's score on the item (where, in the current case, responses by the 3 women in the department would comprise a full 50% of the report's average score on this item). Normalizing the data in this way would give women and men in a department equal influence over their department's score on the item, amplifying the voices of individual women in minority positions in comparison to individual men in a majority position--and, I think, rightly so.
• On problems classifying and perpetuating relatively innocuous climate issues (i.e. little socializing in the department): There should, I think, be survey items on these climate issues ("There is a lot of socializing in my department" [agree/disagree]), and, while a department's faring poorly on such items might harm the department in the short run, leading prospective students who enjoy socializing to avoid the program, (A) this information would be very much in the interest of prospective students of that sort to know, and (B) the survey could lead departments to make concerted efforts to improve their scores on these items to attract students.
• On problems with baselines: of course there are always issues with separating "perception from reality." Grad students in a given program might rate their department's placement rate poorly even though, by any objective measure, their department's placement rate is stellar. But this, I want to say, is a problem with all "perception" surveys, and it is all the more reason to present such surveys alongside more objective measures (of actual placement rates).

Anyway, these are some of my initial thoughts about these issues. I'm not saying that I'm right about all of these things, and I'm more than happy to receive/discuss critiques. Again, as far as I'm concerned, my aim--in taking part in the development of this survey--is to see whether the project can be done well, and with sound (enough) methodologies. And so I'd really like to solicit your feedback!

Comments

[Matt DeStefano]

A colleague and I developed a similar idea a few months ago, which I posted about here on the Cocoon and on NewAPPS(http://www.newappsblog.com/2014/02/philosophical-climate-survey.html). Although there was some positive feedback, we were in correspondence with Jennifer Saul (who said people that worked with the climate in the APA and elsewhere had thought about surveys like this before) and were told that this could hugely backfire. The reasons are that the people who would report negative instances of climate would be easily identifiable - and this would either leave them open to retaliation, or make them hesitant to report for fear of retaliation (therefore making the results unreliable). The other idea was that many people who would be reporting on climate would not be in positions to adequately recognize problems. The idea seemed to me that the upshot of a survey like this would probably be negligible, while it could seriously harm minorities in unintended ways.

[Marcus Arvan]

Hi Matt: Thanks for relating those concerns. Although I appreciate them, I am not convinced that they are sufficient reasons not to pursue the project. Allow me to explain why.

Fist, I'd like to suggest that if a sizable body of graduate students -- particularly members of vulnerable groups -- chimed in and said they did not want a report, that, in my view, would be a good reason not to go forward with it. However, this hasn't been the case so far, either here at the Cocoon or over at NewAPPS. Quite the contrary, some mild methodological concerns aside, the response has been pretty uniformly positive.

More generally, I think we need to carefully compare (1) the prospective costs/benefits of action, with (2) the *known* costs of inaction. And my experience has been that the costs of inaction here are enormous. Too many grad students work in bad situations--in bad climates, with unsupportive faculty, etc. Many of these students suffer immensely, both in grad school and beyond. Indeed, some never finish their programs, the costs of which are enormous to them.

The most salient question is whether a Grad Student Report would benefit grad students on *balance* (particularly the most vulnerable). I believe the net benefits of a Grad Report for grad students are likely to be considerably positive. First, as of now, grad students have no real mechanism to address department issues besides essentially making their concerns public--which very few do, for obvious reasons. A Grad Report would like put public pressure on problem-programs to improve their programs for current grad students. Second, it would put pressure on programs to improve for *future* grad students--students who are not in the program yet. As someone who has known people who have suffered many years in problem-programs, both of these would be considerable benefits indeed.

I am also not yet persuaded by the specific worries you raise. For example, it is unclear to me how an anonymized, quantitative Grad Report would make people easily identifiable. In my experience, a primary reason dissatisfied grad students are easy to identify is that, as of now, their only recourse is to make their complaints *known*. A Grad Report would make doing so far less necessary. People could safely report their concerns in the survey, without making themselves known. In any case, although retaliation is always a possibility, it needs to be weighed against the alternative: namely, the status-quo (which, as I have suggested, in some cases is very, very bad).

At the end of the day, we live in a nonideal world in which we have to choose between morally nonideal options. Currently, grad students have very few (if any) means available to them to address issues in their departments--and the means that *do* exist already put them in extreme jeopardy of retaliation. While a Grad Survey might have some costs, the real question is whether the benefits outweigh them for grad students and the most vulnerable. And, following my work on nonideal justice, I would submit the following: we should let grad students speak for themselves. People in comparative positions of privilege may have their own views about the costs and benefits of a Grad Report. But we should let grad students--particularly the vulnerable--decide what costs *they* are willing to bear for the sake of change.

[anon]

great, great project and idea. You and Carolyn deserve thanks for doing this.

[Kristina Meshelski]

Good on you for taking on this project! When it comes to social science I'm better at pointing out problems than I am at solving them, so I'm unsure how to assess your proposed solutions at this point. But it looks like a start.

Are you in touch with any of the people who were involved with the controversial climate report for the Pluralist's Guide? I never really understood all the issues surrounding that, and I know many people were very unhappy with them, but it was hard to sort out the legitimate critique from the usual ranting about feminists. But perhaps they have some insight to share? Even if just what not to do? (Maybe I'm just repeating what Matt DeStefano said, I'm not sure if these are all the same people.)

[Marcus Arvan]

Thanks, Kristina! I think I have a pretty good idea of what went wrong with the Pluralist's Guide (their survey methods and data collection were seriously flawed), and am optimistic that we can avoid those issues. First, we are going to construct the survey in a very transparent way, seeking feedback from people here at the Cocoon and elsewhere. Second, unlike the Pluralist's Guide, which had third-parties rate the climate of departments other than their own (which resulted in some departments with very bad climates erroneously recognized as "good climates", and vice versa), we will be having grad students rate only their own departments--a fundamentally more sound methodology. Finally, I'm also fortunate to be married to someone who does quantitative survey studies and such for a living!

[Marcus Arvan]

anon: Thanks for the kind, positive feedback. :)

[Ambrose]

My mind is blown by the proposal that, if there are 60 male grad students and one female, the opinions of "women" in the department -- that is, all one of them -- should have an "equal" or "50% share" in determining the score. At least, if the hope is to get scores that have some connection to reality, this system would work only on the assumption that in such cases the opinion of that one woman is likely to be _radically_ epistemically better than any the judgments of all of her peers together. And that would have to be a reasonable assumption regardless of whether that one person
(a) is less honest than some of her peers
(b) has incentives for skewing her answers that her peers don't have
(c) is not as well-informed than some of her peers
(d) has strong personal or ideological biases that skew her perceptions of the "climate"
(e) happens to have had personal experiences that are just unusual (i.e., that most women would not have had even if there were more of them in that department)...
I could go on. I guess the other way to make sense of the assumption is to hold that conditions like a-e simply don't obtain, or aren't likely to obtain very often. But that also just seems really silly.

Suppose there are 3 women in the most profoundly sexist department in the world. As it happens, they are all radical sexists and traditionalists who like it that way. As it happens, there are 15 male feminist graduate students who are the only people in the department capable of recognizing what's wrong with it. Under your system the voices of the 15 (male) feminists would be drowned out, and perhaps ONLY because it just so happens that some of the least perceptive or reasonable people answering your survey are women. And therefore, naturally, their opinions are just better and more important. In that kind of case, I assume you'd agree that something is badly wrong. Explain to me: why is this a reasonable methodology when it happens that the distribution of opinions by sex is different?

[anon]

as to the concern that grad students will not be honest in surveys because of fear of retaliation, I think that is less a worry than you might think. i have been involved (in different capacities) with three graduate surveys, conducted internally in two different departments. i would have the thought the results of the surveys would all be generally positive, since grad students would be too afraid to complain, even behind the veil of anonymity.

i was wrong! in all cases, the grad students seemed to have no qualms about voicing their discontent and explaining their concerns in some detail.

surveys that are *publicly* available might be a different matter. but philosophers are an independent and opinionated bunch. also grad students are powerless and i think this makes them all the more keen on grabbing their chance to speak once given it. given the slightest safety (via the shield of anonymity) to voice their opinions, I bet they will. they might not give details of the kind that could identify them -- but my guess is they'll generally answer numerical questions honestly.

further, if there is indeed a chilling effect and grad students are hesitant to speak out, i would expect that to hold more or less across the board. in that case, a smaller percentage of grad students will speak out but probably a few will at each department. we could still obtain meaningful inter-departmental comparisons.

[anon2]

I'm very happy to see a project like this, but am I correct in reading you to plan on reporting grad students' evaluations of attrition and placement rather than reporting the actual attrition and placement? Or is the plan to report both?

I hope the plan is to report both, though I suspect that many departments will balk at providing actual numbers. Is there any plan for encouraging departments to provide these numbers?

[Marcus Arvan]

anon2: Yes, both are going to be measured!

[Marcus Arvan]

Ambrose: Thanks for your comment. But I don't know why your mind is blown. You're using what seem to me to be completely unrealistic situations to raise concerns about the proposal. There is not, to my knowledge, any philosophy graduate department with 1 female grad student and 60 males. The other hypothetical case you describe is even more unrealistic. We are dealing with the real world here, not possible worlds.

In the actual world, grad-student populations tend to be something like 20% female, 80% male--in which case I think it is *entirely* appropriate to give that 20% of women an equal voice in determining "climate for women" scores as the men. One lone woman might have a skewed, dishonest, or ideologically-loaded view of her department's climate--but an entire group of women? It seems to me that we have *overwhelming* moral and epistemic reasons to give a sizable minority who *experience* how they are treated (as women!) a 50% share in determining their department's climate for women score. To suggest otherwise--to suggest that entire populations of female grad students might be ideologically motivated, dishonest, etc.--seems to me morally and epistemically wrong. If a sizable minority of a given department expresses the view that life for them in their life is terrible, that's generally a very good reason to think that it is the case. Given the discipline's well-known climate issues (which I do not think can be reasonably denied), we *should* lend added weight to women's perceptions of their departments, not dismiss those perceptions as dishonest, ideological, etc.

[Ambrose]

Hi Marcus,
In your own example you imagined a department with 29 males and 3 females. My example was slightly more extreme, but the idea that 3 people should be treated as an epistemically equal block in relation to 29 seems to me only very slightly less bizarre than the (slightly exaggerated) scenario I discussed. And it doesn't seem that much of a stretch. If you endorse some underlying principle in the case you imagined, it seems that the principle _should_ hold in my case too. Whether the world is actually more like your case or mine is beside the point (unless standard philosophical reasoning doesn't apply here).

In any case, let me address your claim that it's unlikely that any sizable group of women -- even just 3 of them? -- would be worse off epistemically than any group of men of the same size who disagree. (That is what you need to hold, anyway, given your argument.) You think that's unlikely because the people in the group "experience how they are treated" while others don't. Of course that claim is ambiguous: it could mean simply that they are the ones how subjectively experience being women. True enough, but then subjective experience of that might not accurately reflect the thing you want to measure, the "climate" of the department. Men who hate feminism are the only ones in a feminist department who subjectively experience "what it's like to be a feminist-hating man" in that department. But the mere fact that they have an epistemic advantage in that respect does not make them any more likely than others to judge correctly whether men (or people who hate feminism) are indeed treated badly or unfairly there.

Maybe you mean that only women have any experience _of_ the treatment of women in the department. If that were true it would of course show that they have a major epistemic advantage but it's untrue. Others do also have experience of that, and it could be that in many cases 3 women (even 5 or 10) interpret that shared experience in ways that are no _better_ epistemically than 3 (or 5 or 10) men who interpret it differently.

Finally, I don't think it is at all absurd or even improbable to worry that _some_ sizable groups of women may have ideological biases that skew their responses. As a general point, sizable groups of people -- even female ones -- very often do have such biases as a result of acculturation or whatever. And surely you've noticed that in our society feminism is a real political force, especially in the universities. A young woman entering an American university at any time in the past 30 years is pretty likely to have heard a LOT about the problems of insidious systemic sexism, etc. The glass ceiling, etc. Modern media promotes these kinds of ideas pretty relentlessly. You seem to assume that none of this is real, or that none of it is likely to have any influence on how people interpret or experience things. Why is that so improbable?

Notice that all I am suggesting is that there MIGHT be such bias (along with other possible epistemic defects common to humans). And that therefore it's unreasonable to expect that groups of women are in general epistemically SUPERIOR to groups of men with respect to these topics. (I allow that they might in general be epistemic equals, of course.) You think it's epistemically and morally wrong to suggest this? Even though it's blindingly obvious that feminism is (a) an ideology and (b) almost universally and officially endorsed by the academic establishment over many decades?

I don't get it. Even if, as you say, there are these undeniable "climate" issues that alone is no reason to accept the epistemic assumptions you're working with. (On the contrary, it's easy to imagine how such issues might induce yet other biases or misperceptions.) The fact that women tend to be treated unfairly (if that is a fact) doesn't make it any more likely that any group of n women stands in this kind of epistemic relation to any group of n men.

[Wesley Buckwalter]

Rankings based on placement would be a tremendous service to students. I also definitely support the idea that a diversity of rankings is a good thing, and am interested to help brainstorm.

There is so much variation in individual experiences, in say, support offered by working with faculty, or the complex ways that many important climate issues impact individuals, that it seems like it is going to be incredibly challenging to collect judgments about these things in a survey or capture them representatively. An honest question since you’ve asked for discussion about possible methodology: perhaps there are other more effective ways to share this information with perspective students besides ranking survey responses? For instance, perhaps your ranking could have a separate page of organized links about student faculty grants, collaborations in the news, and conversely, public accounts of faculty misbehavior students should be aware of when considering the program.

For a positive suggestion about the metrics included, since you’re interested in giving students a picture of what the graduate student environment they will soon enter into is like, why not include things like: average times to completion of degree, funding support, and teaching expectations? Those things seem really relevant for student experience, especially those with families, and more straightforward to collect.

[Marcus Arvan]

Hi Wesley: Thanks for your feedback. Although I think public information about "faculty grants, collaborations in the news, and conversely, public accounts of faculty misbehavior students" is useful, this kind of information leaves out a lot. In my experience, some students have positive experiences in programs, whereas others have very negative experiences--and I don't see how the kind of public information you mention would track these very real, and important, differences in grad student experiences. I'm also not sure why you think it will be incredibly challenging to collect judgments about these (more personal experiences) in a survey, especially before we've presented a draft of such a survey for feedback! I'm optimistic that the survey we're constructing will measure grad students' perceptions of their programs in a valid and representative manner. In any case, I would suggest to wait and see! I think it is important not to shy away from difficult but important endeavors preemptively.

[Wesley Buckwalter]

Thanks for your response Marcus. For the record, I think it's admirable and ambitious, just also brainstorming about possibilities and challenges. Some of the challenges seem orthogonal to the materials. For instance, one is that you get a volume of responses to yield an accurate picture of programs you wish to evaluate given variance. Another is that given the range and diversity in personal experiences ("some students have positive experiences in programs, whereas others have very negative experiences") it might be difficult for prospective students use an average ranking based on surveys, even with really complex experiential items, to make inferences about the experience they may be likely to have at that program in future. In other words distilling that down to a number to be ranked might be masking a lot of information, which led me to wonder before about alternative ways beyond ranking per se to share the information. At the same time, perhaps the distributions revealed by your work would be very illuminating of programs.

[jmugg]

Marcus-

Do you have a way to deal with low sample size? Not all departments are big, and this is not necessarily a bad thing. I know of a small Canadian school that admits only one PhD a year. My department has only admitted around 5 new PhDs each year (at least for the last 6 years). You would need every current PhD to fill out a survey to get n=30.

Also, I would caution against taking high attrition to be a bad thing on its own, and measuring when folks are dropping out. I would rather see high attrition early in a program combined with a high placement record rather than very low attrition rate but a bad placement record. Or worse--a bunch of people who stay in the program for a decade and never finish. A PhD in philosophy is not for everyone--in fact, it's not for most people. And that's OK. For a lot of folks, getting out sooner is likely better than getting out later.

[Stacey Goguen]

Thank you for putting up this post.

On climates for women:
I support developing this kind of survey, though I've been a bit worried lately about the way we (as a profession) talk about wanting to know what a department's "climate" (singular) is.

As I've talked to many different graduate students from many different depts, I've gotten the sense that at some places, a few different interactions can lead two graduate students who are equally observant to vastly different conclusions about their own dept's climate regarding women.

So if two women can go through the same dept and have vastly different experiences in terms of perceived support and friendliness, that has led me to suspect that depts. might not have a single "climate," and the weather analogy might suggest a sort of unity or consistency that does not necessarily (or perhaps not even often) exist in the thing we are trying to measure.

I'm not sure what a good response to this worry is; I just wanted to register it in case anyone else has an idea of how surveys could take it into consideration.

[Markos Valaris]

Ambrose,

The population of Tibet is 3 million. The population of China is 1.3 billion. Why take the word of Tibetans over that of the Chinese with regard to "climate issues" in Tibet?

[David Wallace]

Mostly this sounds like a good idea, but I'm not sure what the benefit is for collecting data on grad students' subjective impressions of things for which objective data exists and is reasonably easy to obtain (notably, placement rates; possibly also attrition rates). It might be valuable to department X to know that its students are inappropriately pessimistic or optimistic about department X's placement record, but I'm not sure how it would be valuable to prospective applicants (except maybe as some general proxy for morale, which might be easier assessed more directly.)

[Marcus Arvan]

Hi David: Thanks for your comment. I agree that there may not be much benefit for measuring subjective impressions on things for which there is objective data. But this is not what we are doing. We will be collecting objective data, and then be collecting subjective impressions on things for which there is no objective data (e.g. department climate, morale, etc.).

[Ambrose]

Markos,
The vast majority of those Chinese people don't live in Tibet, of course, and will never visit even for a day. But male grad students do spend time in their own departments, do directly observe women there, how they're treated, etc. And it's not absurd to think that (absent other info) the testimony of arbitrary Tibetan T and arbitrary Chinese person C count roughly the same when both of them have lived there for just as long, etc.

Another disanalogy: Tibetans in Tibet are not taught in school or in the media that Chinese occupation is bad, that the Chinese are oppressing them, etc. There is nothing in Tibet even remotely like the pervasive, well funded, state backed and media backed feminism that exists here. Which might just have some influence on how some women perceive things.

So your analogy faces two big problems. Try again?

[Marcus Arvan]

Ambrose: I was once a (male!) grad student, and I would be willing to wager that most male grad students recognize that women are better judges of the climate for women in their department than they are. Why? Because (1) a lot of the bad stuff that goes on is out of public view, and (2) it's hard to appreciate negative treatment when *you're* not the one who systematically suffers it day in and day out.

Second, if you read my paper, "First Steps Toward a Nonideal Theory of Justice", you will see that I give a detailed philosophical argument there for *why* justice requires amplifying the voices of underrepresented minorities.

[Ambrose]

Hi Marcus,
I will have a look at the paper. (Really.) But let me just reply one of the new points you raise here: that bad stuff happens out of public view.

I grant that at least as a claim about some possible bad things that might be both relevant to the "climate" and also such that women in general would be more likely to know about it. If in fact most of the bad things we're considering were like that, or the most significant ones, then given some further assumptions your epistemic principle might be right.

But it seems we need some dubious assumptions:
(a) It not only happens in private, but in private settings where no male grad students are likely to observe it. (Not at the bar, not at a big party, not in someone's office...)
(b) It's the kind of bad stuff that other (male) grad students aren't likely to learn about in some other way (other than directly observing it). (Not by being told by female friends, not by observing Professor P's odd behaviour with Doctoral Candidate Cindy...)
(c) Despite being "private" in this strong sense, the bad stuff in question nevertheless does have an effect on the climate of the department for everyone, or at least for women.
Is it fair to assume that a-c all hold in most departments? Or that point 1 is relevant even if some or all don't hold in many?

So far you haven't addressed a point that seems important and quite obvious: feminism is a very powerful force in our society, in the media and education system, especially in universities. Do you deny this, or do you simply deny that any "sizeable" group of women might have cognitive or other biases as a result? (I'd suspect that sizeable groups of women and men also might be biased by it, so that in this time and place it's only to be expected that lots of people are primed to detect "sexism" or a bad "climate" for women.)

Also your reasoning seems to involve some unacceptable circularity. Apparently you already know that women "suffer day in and day out" from "systemic" sexism and oppression in philosophy departments. But if you already know that, what's the point of doing a survey to collect evidence on that very topic? On the other hand, if it's an open question whether philosophy has a bad "climate" for women, or how bad it really is, you can't privilege the testimony of women on the assumption that the "climate" is so bad for them.

There's a worrying air of boot-strapping too. How do we know that it's so bad for women? On this blog the answer has often been that they say so -- they tell us "what it's like to be a woman in philosophy", etc. Now the proposal is that that same testimony be given greater epistemic weight, on the basis of the presumed reliability of that same testimony. But then we can just reiterate the same argument. Once we give it greater weight, the same old testimony will now count even more heavily than before. Eventually we can get to the point where any woman's claim that her department is a sexist hell totally trumps the opinion of everyone else there it should provided merely that they're all men: other things being equal, we should just totally ignore what they say and take her at her word.

[Markos Valaris]

Sure, my analogy faces these and many other problems, all analogies do. Try black and white perceptions of the US criminal justice system? (https://www.youtube.com/watch?v=InOsF5x1lZw)

If you don't like my analogies, though, here is why I think Marcus's proposal is legitimate.

First, a privileged majority is likely to be blind to the experiences of an under-privileged minority. There are many well-documented reasons for this. Our understanding of the world is heavily influenced by figures of authority, and those figures will tend to be members of the majority. We are all averse to change, especially if the status quo is pretty comfortable for us. The "just world" bias will make us prone to thinking of victims to be whiners rather than genuine victims. And so on.

These are well-known cognitive biases. Even if you think you personally are immune to them, people on average are known not to be. So we should correct for them in drawing conclusions from surveys.

Presumably your worry is we might overshoot in the other direction. But the second point is, what are the costs, even if this happens? We know that the costs of letting mistreatment go unchecked can be quite dramatic --- in extreme cases we might be talking about serious trauma, ruined careers and so on. As for the discipline as a whole, drawing from a talent pool of mostly white males seems like a bad strategy for the future. Even if over-correcting is a serious possibility, I doubt that its costs would come anywhere near to outweighing these.

[Marcus Arvan]

Ambrose: I think that (a)-(c) are indeed likely to hold in departments with bad climates for women. I think in some such departments, there are *rumors* among male grad students of bad behavior towards females, but that the really bad stuff happens out of public view--which is why the female grad students are better judges of the true climate than the men (who might think "it's not so bad" or "boys will be boys!", given that they don't experience the bad stuff first-hand).

Anyway, in terms of your broader worries -- about how powerful feminism is in our society -- I'm pretty doubtful. I think norms of equal treatment are very powerful in our society, but that super-strong ideological feminism is not so powerful. Indeed, I've come across a great many males and *females* who are very skeptical of super-strong feminist claims (re: feminist epistemology and such).

I'm more inclined to think that there's something else that underlies your opposition to my proposal. In the past, you have made no secret on this blog of your opposition to *equal* treatment. You have more or less stated that you consider the entire Enlightenment, and liberal ideals of equality, as wrong. If this is what you think, that's fine -- but given that liberal-equality is widely accepted in our society, I see no reason for our survey to be influenced by anti-liberal ideology.

[Ambrose]

Hi Marcus,
Let me take these points in reverse order.

1. You suggest that my real motivation is skepticism about "equal treatment" (which is a curious idea, in a way, since I'm the one suggesting that men and women be treated equally in this respect). Even if that were true, which it isn't, I am certainly not suggesting that the survey should be "influenced by anti-liberal ideology". Not unless denying your principle that women's testimony counts more than men's is already an "anti-liberal" position. On the contrary, I am arguing from non-ideological, non-anti-liberal premises that I'm pretty sure most ordinary liberals and egalitarians would find plausible. (I used to be one, so I have some idea of what that's like.) So I don't see how your last point is relevant.

2. I never said that :"super-strong feminism" is a pervasive force. I said that _feminism_ is. And it is a completely familiar, mainstream feminist idea that there are "chilly climates". See discussions about "Lean In", see NYT pieces on the "climate" at Harvard Business School or in Silicon Valley, etc. You seem to be responding to a straw man here. If you'll grant merely that (a) the "climate" stuff is not "super-strong feminism", "b) it's pervasive, then you should allow that (c) it might have a significant effect on some "sizeable groups" of people (either male or female).

3. Suppose we agree that under a set of conditions C, such as those I mentioned, your epistemic principle is okay. (They'd also include conditions I didn't mention, incidentally, such as the condition that people who _have_ experienced bad things are not especially likely to _over_estimate how bad the experience was or how representative, etc.) Let's agree too to your new claim that, probably, condition C obtains IN DEPARTMENTS WITH BAD CLIMATES FOR WOMEN. You seem to be going in a small circle. You want to know which departments have bad climates for women. So you're going to ask people to report on their experience. Now for justification to work, you need not only the conditional we agree, but also the assumption that C really does obtain regularly enough that it's likely women in general really are epistemically better off in this way than men. But you could only know THAT if you already knew that some decent number of the departments in question really do have bad climates for women. Isn't that what the survey is supposed to assess, rather than a presupposition guiding the assessment?

[Ambrose]

Marko,
I guess we're setting aside the Tibet analogy then. That is, you're not suggesting that it's _as_ silly to propose that male and female philosophers be treated as epistemic equals wrt "climates" in their own departments as it would be to claim that millions of Chinese people outside Tibet are epistemic equals of Tibetans wrt the "climate" in Tibet. Good! Because it's definitely not as silly as that. So now there other arguments to consider. Let's consider, then...

You write that Marcus's epistemic principle is reasonable because

"a privileged majority is likely to be blind to the experiences of an under-privileged minority"

Be that as it may, who ARE the "privileged" in our world, in philosophy departments? Are you suggesting that a working class man who paid his own way through school is "privileged" in comparison with Paris Hilton? Or is the idea that, on the whole, it tends to be men who are rich and powerful rather than women (including the wives and daughters of rich and powerful men)? And even though it is men who tend to pick up the garbage and work in mines and get sent to the front to have their limbs blown off? Even though women have had affirmative action for 50 years, etc.? I allow that your principle is kind of plausible in the abstract but if we plug in the actual world the output will not be that women are an "under-privileged minority".

You say:

"Our understanding of the world is heavily influenced by figures of authority, and those figures will tend to be members of the majority"

Actually the authorities are very often not members of the majority. Think of the authorities in Tibet, for example. Obviously that depends on how majorities are individuated. (Take your pick.) But in our society you may have noticed that Jews are vastly, vastly, vastly over-represented in the ranks of "authority figures". Do they "tend to be members of the majority"? Well, not if we're individuating by ethnicity or religion. That's actually not at all uncommon. For example it's also true that our "authorities" are vastly, vastly, vastly more liberal and secular than the people they rule over. And so on. So your principle seems false to me on various natural ways of individuating majorities.

But I grant that people's opinions are indeed shaped by authority, and may be biased for that reason. But who ARE the authorities in our society?

In reality there are almost NO recognized authorities who seriously and deeply oppose feminism, and virtually ALL the forces that shape public opinion accept the basic premises of feminism. It's in the schools, on television, in the history textbooks, in Hollywood movies, etc. Are you suggesting that "authority figures" in our radically sexually liberated, post-feminist, post-Christian, post-Eurocentric culture tend to oppose feminism? And that, not only do these anti-feminist authorities exist, but that their influence is not cancelled by the opposing influence of any feminist authority figures? I find that impossible to believe, but I guess we may have such different world-views that there's not much to argue about.