Our books






Become a Fan

« What to do if the hypothesis is true? - part 1: job candidates | Main | Impact and Engagement for Early Career Philosophers: Part I »

02/09/2018

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

Nick

whatever else you think of Wittgenstein he did grounbreaking work but would hardly pass for qualified by Pendaran's standards

Pendaran Roberts

"That seems plainly false, and it is easy to understand why if you consider that most papers published in top journals are not accurately described as *groundbreaking* research. Groundbreaking research is to be understood as research that breaks ground, and ground is broken mainly when important problems are understood in new, innovative, and promising ways, or when those problems are approached in new, innovative, and promising ways."

And where should I go to read this groundbreaking research?

A Non-Mouse

Apparently, we are to believe that Pendaran can't think of a single book or article that counts as presenting groundbreaking research as described. You're joking, right?

A Fat Mouse

A non-mouse,
I am as perplexed as Pendaran. Where is all this ground breaking research? Please list just seven papers published in the last five years that are ground breaking by people who have only had their Ph.D.s for less than three years. I am at a different stage in my career. I will tell you, the peers of mine that have had very successful careers did not do any ground breaking research in the early part of their careers. Much of their research is quiet fine, but hardly distinguished. And most of them have earned the position they have in the field.

Marcus Arvan

Wittgenstein didn’t get hired until well after he published the Tractatus and it made a huge impact in the discipline. Similarly, Einstein had to actually publish the special theory of relativity and make an impact before anyone would give him a job.

I have seen far, far too many hires at R1’s fail to get tenure to believe committees are reliable at selecting for groundbreaking research. In fact, I've known some personally: candidates hired with few or no publications...who then went on to publish very little and get denied tenure. Groundbreaking ideas aren't enough. To be successful, a person must have the ability to *publish* them. And what's the best evidence of that? Demonstrable evidence publishing.

As I have pointed out at this blog before, my spouse is a PHD researcher in a top-5 program in Industrial Organizational Psychology. One of the most robust findings in her field—confirmed hundreds of times over five decades across different job markets—is that (A) hiring committees always think they have a talent “divining rod”, but (B) they don’t: far and away, the best predictor of future success is one’s past record of actual accomplishment.

There was even a study on this in academia just a year or two ago, where an algorithm crunching #’s of publications at time of hire better predicted tenure than the decisions of hiring committees.

There was a book and movie based on this research a few years ago called “Moneyball.” Watch it. In professional baseball, for decades scouts selected prospects based on their “talent.” They routinely labeled people “can’t miss” prospects on the basis of their “groundbreaking abilities.” Alas, despite being sure they could spot talent, all too often the players they selected went on to fail. Then someone said, “Hmm..I wonder what would happen if we actually selected players on the basis of their objective accomplishments: walks, hits, runs scored.” Turns out, just as the decades of above research indicate, this person was *right*. At the time, the scouts protested vehemently, saying algorithms couldn't do better than the decades of expertise they had developed as scouts. The problem is, the algorithms showed them wrong. Now every MLB team uses the moneyball method.

This is not just one case. These kinds of results have been replicated across a wide variety of industries. It is hubris for academics to think we are the sole exception--that we and we alone are the reliable ones at selecting for expertise. Wonder why so many candidates at R1's are denied tenure? The science suggests the answer is because, all too often, people are hired on the basis of a method akin to reading tea leaves.

Amanda

I can buy the idea that many philosophers at top places *believe* what non-mouse says. This was the idea I expressed before when talking to my professor in grad school. Whether or not you agree with it, a point is being made that some of these professors do think they hire by quality when hiring someone without many publications but with what others judge important work in the pipe line.

As for what I personally think, I do think A LOT of work published in top journals is boring and not ground-breaking. So that gives me some sympathies to the idea. But here is where I lose all sympathy. If it were true that philosophers at top departments really just wanted the best work, they would at least *occasionally* hire someone from a non-top department that had this potential. I can buy that people from top places are *more likely* to have ground breaking work. But I cannot buy that this is true 99.5% of the time. And that is the rate at which those without great publications are hired from top departments. If it was 80 or even 90 percent from top departments then maybe it is fair. But 99.5% suggests prestige bias beyond any plausible rate which reflects quality.

A sociological perspective

Adding to Marcus' thoughts ...
sociologists of science have noted that the best predictor that a scientist or scholar will publish a SECOND paper after publishing one paper is that she has published one. And the best predictor that one will publish a third paper is that one has published two papers.

Marcus Arvan

I would also add that there is an internal tension in the perspective being espoused.

The view is something like this:
(1) The top journals are full of mediocre stuff that isn't very groundbreaking.
(2) Top departments routinely seek to hire people doing groundbreaking work.

Here's the tension: if departments are reliable at hiring people doing groundbreaking work, why are the journals so full of work that *isn't* groundbreaking?

Pendaran Roberts

"The view is something like this:
(1) The top journals are full of mediocre stuff that isn't very groundbreaking.
(2) Top departments routinely seek to hire people doing groundbreaking work.

Here's the tension: if departments are reliable at hiring people doing groundbreaking work, why are the journals so full of work that *isn't* groundbreaking?"

That's a good point. There are other tensions too. Here's one. If top journals are so full of mediocre work, why don't these top departments who are presumably so reliable at hiring groundbreaking researchers simply start new journals where decisions are made entirely in-house? Or, in other words, if peer review is worse at determining good research than in-house committees at top R1 schools, why don't we replace peer-review with in-house committees from R1 schools? My only worry is that someone will in fact want to bite this bullet. haha!

A sociological perspective

Pendaran,
Apparently, The Journal of Philosophy was, for a long time, run in house by the Columbia University Philosophy Department. And my understanding is that Philosophical Review was more or less run out of Cornell.

A Non-Mouse

I have to get back to work. So, unfortunately, I can't get to all the comments directed at mine. I'll be brief.

All the complaints I've seen recently, about hiring for groundbreaking work seem to involve confusions.

1. Not groundbreaking work is not the same as mediocre, as some have suggested.

2. Some seem to think that groundbreaking work is ubiquitous, or should be. It's not, and this is because it is not at all easy to do (I'd bet).

3. Nobody (as far as I know) has suggested that R1s reliably hire people doing groundbreaking work (despite their attempts).

4. *If* there has not been much groundbreaking work published recently, and *if* nobody doing groundbreaking work has been hired at R1s, it is because it is not ubiquitous.

A Dud

Non-Mouse
By "not ubiquitous" you mean as rare as hens' teeth. In that case, it is absurd for departments, even the best, to use it as a criterion for hiring. Why not just hire some dud, like the rest of us, who have a number of well placed publications.

Marcus Arvan

A Non-Mouse: I didn't mean to suggest that groundbreaking work is ubiquitous. I did mean to suggest that one evidently common way of trying to predict groundbreaking work--judging candidates by their writing samples and/or letters of recommendation rather than by their publication record--is inductively likely to be a terrible method, given a wealth of science on the subject.

Here's one case-study from a different academic field (I don't think it's consistent with this blog's mission to single out philosophers who might be reading the blog):

A couple of years ago, an R1 department had two finalists. One was from a #1 program with no publications. The other was from a lower-ranked program with lots of highly-ranked publications. The department chose the former candidate on the basis of his writing sample, letters, pedigree, and job-talk (which was supposedly "mind-blowing")--even though the latter candidate had, in every objective sense, a far better CV. The committee judged the latter candidate's publications as "just okay", even though they appeared in top journals.

Fast-forward several years. First dude has still published nothing. The second candidate (the one not hired) has gone on to continue publishing in top-ranked journals, and is now an emerging leader in her field. Is this just a one-off error? No. The department in question has done the same thing in their last *four* hires, none of whom got tenure. In each case, they convinced themselves the person they hired was a genius...and what they ended up getting was exactly what the person's CV suggested to begin with: someone with no demonstrable ability to publish their supposedly "groundbreaking" ideas. In retrospect, many in the department in question think they made a mistake. And yet they chose the person anyway. Why? Because their writing sample was "groundbreaking", their grad advisor from a top program said they were brilliant, and they gave an impressive sounding job-talk. What they didn't do--and still haven't done--is actually publish anything.

Word is there is a number of departments in philosophy who haven't tenured their last X-number of junior hires. Could this possibly be because hiring committees are going about things the wrong way? The problem with trying to judge whether someone is the "next Rawls" (or whomever) on the basis of unpublished work is that there are so many possible confounds. Perhaps you just *think* their writing sample is groundbreaking because in the back of your mind you know they come from Harvard. Or perhaps that one paper really is brilliant but it's the only groundbreaking idea they'll ever have (perhaps because they've been working face-to-face with Parfit the last five years). Or perhaps you think it's groundbreaking but people in their field think the opposite. I don't mean to say that these biases are always at work--but the problem is that they *may* be at work, and this is why (or so I-O psychology systematically demonstrates) objective measures are better predictors.

The thing about publishing is that one has to convince anonymous reviewers, editors, and readers that one has a groundbreaking idea--not just five search committee members who know what program you're from and who your advisor is.

flux

I think talk of a prestige “bias” may be misleading, especially in a discussion about predicting future success (publishing, tenure, etc).

The word ‘bias’ in this context suggests a kind of irrational heuristic: one may think that being from a prestigious department is a good predictor of future publications, but, the critics say, this is a mere bias; a better predictor of future success is publication record (regardless of prestige of pedigree).

But I think this misses the point: those who hire for prestige don’t do that because (they think) prestige predicts something else (in this case, publications). They hire for prestige because they want the prestige. They just want it to say on their department website that their faculty came from Harvard and studied under Kant himself. I don’t think they care about whether the research will or won’t be groundbreaking, and I don’t think they care about predicting who will eventually have the most publications. They just want luggage that says ‘Gucci’ on it, and a shirt that says ‘Armani’ on it.

Is one more likely to have an objectively better or more successful vacation with Gucci luggage? The question is misguided. Some people just want to be seen with a fashionable bag, simply because of the status it connotes. So, sure, other bags might carry more stuff, and might objectively be “better” bags. But Gucci is Gucci, and those top departments wouldn’t be caught dead in anything else…

Amanda

We are confusing two points here:

(1) Whether search committee members *think* they are hiring by quality when they hire people from top departments with unproven records.

(2) Whether the above practice mentioned in (1) is either justified or effective.

As far as I can tell non-mouse was not suggesting (2) but only (1). And I think there is a good case for (1). I also think it is possible for a search committee member to think lots of work in top journals is boring and yet still hire on the basis of a great record. Why? Because not all work in top journals is boring. Some is great. And the job market is so over flooded with talent that one can ignore a lot of candidates with impressive but boring records while still hiring someone they believe is exciting but also has a great publishing record. It is a buyer's market. Given that, I find it hard to justify hiring someone without a proven record.

Amanda

flux I think you are right. At least, a lot of the time I think that is what it is about. But this isn't all of the time. And even when it is about the label, I think a lot of professors lie to themselves and try to convince themselves they are hiring for "real" reasons related to merit. (I think the way we use bias has changed from the original meaning of the word.)

Trust the Process

I agree with Marcus and others that hiring methods at the tippy-top are akin to reading tea leaves, and also that there is often a delusion that the tippy-top are the best at doing this (just gossip with someone on a tippy-top search committee). I'd only add that while some in this sphere do want to hire the best qualified, others really don't - they want to swing for the fences, hoping for the next Wittgenstein. And if they miss, well, they expected that, and there's no pressure to tenure the hire. Look at the tenure rates at places like Columbia or Yale et al - often getting hired there is basically getting a six-year VAP, and those hired often have some sense of this.

There is also the thing that since our profession is status obsessed (even when we pretend we aren't) getting that top gig often really transforms the person, creating reinforcing cycles of prestige. Often someone who is mediocre will get touted as doing the ground-breaking research, when someone at a less prestigious place is doing similar and better and much less recognized work. (I've seen this from both sides of the thing.) (But, in my pessimistic view, there is nothing to be done about this, because philosophers inside and outside of the tippy-top are not good at recognizing quality (unless the signal is long-lasting or crazy-strong or both): hell, the above claims are based on my assessments, which may be way off, although I rate them highly.)

A Non-Mouse

Marcus: I appreciate the "case study." But I am far from impressed by it. This is why. I agree, it *might* be that when departments hire for potential for groundbreaking (GB) work, they are likely not to end up with someone who publishes much. I agree, it *might* be that when this occurs, they passed on someone who would've published a lot of great work. Perhaps it happens even more than we think, and even more than the evidence shows. But this does not show that those departments *should not* hire for potential for GB work.

You and others have been offering support for the claim that departments *should not* hire for potential GB work. It's not clear that any of the criticisms work. Your latest criticism would clearly work if we had a clear idea of their reason(s) for hiring for potential GB work, and the reason were *simply* that they want someone who publishes a lot of great work. I take it, their reason(s) is not simply this.

In fact, publishing a lot of great (but not GB) work might be a good predictor of producing less than GB work. In fact, statistically, we should expect precisely this. After all, GB work is very rare, and most people publishing great (but not GB) work have not published GB work. So, assuming that R1s really do want someone who promises to produce GB work, how do they get what they want?

Statistically, given the above, *maybe* the candidate most likely to do GB work is not the candidate who has published lots of great (but not GB) work. Perhaps it is the candidate whose first project promises to be groundbreaking, but who has published nothing. After all, nothing tells us this person will not publish GB work, and something tells us that this person has potential to publish GB work. Whereas, with respect to the other person, statistically, they are likely not to produce GB work.

Again, I'm not suggesting that this is what should be going on. This is simply a speculative suggestion for what best explains the facts.

A Non-Mouse

Amanda: yes, I was not suggesting (2). Thanks for the charitable reading! And I agree it may be hard to justify hiring someone without a proven record. But as I suggest in my last comment, there is a case to be made.

Anonymous

I've talked to faculty at several top departments about this issue. Many of them feel that they are in a better epistemic position to judge the quality of someone's research program than a random, unidentified journal reviewer. They're reading the work themselves; why should they be swayed just because a better paper hasn't appeared in a journal and a worse one has? Furthermore, as has been said several times already, what makes for one good journal article is something entirely different than what makes for an impressive research program or general body of work (even if it's just a potential body of work at the moment it's being vetted). And finally, many of these candidates are being hired out of graduate school, giving them less time to publish. Even if you argue, "well, what about all the candidates at lower-ranked schools who published 10 articles by their 5th year of graduate school", if what's most relevant is developing an original body of work, focusing on publishing and other miscellaneous CV-fillers (tons of conferences, teaching, and other things that often seem "lacking" in who's being hired) might arguably *detract* from that goal, insofar as they might be distractions for some from the task of thinking uninterruptedly about one's dissertation and developing one's ideas.

Marcus Arvan

A Non-Mouse: fair enough. I guess I am just unconvinced. Historically speaking, the next Wittgenstein (or Einstein or whomever) hasn’t emerged from departments playing the Next-Genius Slot-Machine. They’ve emerged from actually publishing the Tractatus, special theory of relativity, etc. In other words, I believe truly groundbreaking work isn’t discovered by gambling on job candidates who haven’t published anything. Far more often than not, it’s discovered by the groundbreaking work making *itself* known by the person who published it.

A Non-Mouse

I understand why you say this, Marcus. And I don't blame you.

Reply

One thing anonymous gets right is that some people on the market do not realize that after you have published a few things, you should switch your strategy and try to publish in higher ranked journals. So polish your stuff, and produce better work. In my field, there are a few excessive publishers, and their papers are often quite unfinished or under developed. They could benefit from publishing less, and slower.

Pendaran Roberts

"Is one more likely to have an objectively better or more successful vacation with Gucci luggage? The question is misguided. Some people just want to be seen with a fashionable bag, simply because of the status it connotes. So, sure, other bags might carry more stuff, and might objectively be “better” bags. But Gucci is Gucci, and those top departments wouldn’t be caught dead in anything else…"

Yup, I agree. This is prestige bias. It's preferring a bag with Gucci written on it to a better bag with a less prestigious name written on it.

"I've talked to faculty at several top departments about this issue. Many of them feel that they are in a better epistemic position to judge the quality of someone's research program than a random, unidentified journal reviewer."

Sure they think that. But is there any evidence of this? Also, if they really do believe this, then why do they require publications in blind, peer reviewed journals? If they think they're better at identifying good research than the blind peer review system, why don't they start new journals reviewed in-house by them? Then they can make the requirement for tenure publishing with them.

"In fact, publishing a lot of great (but not GB) work might be a good predictor of producing less than GB work. In fact, statistically, we should expect precisely this. After all, GB work is very rare, and most people publishing great (but not GB) work have not published GB work."

But this assumes that the hiring committee can read someone's publications and determine that none of them are GB. I very much doubt the committee can determine this. Many times throughout history GB work has been widely ridiculed until eventually becoming accepted after years and years. Most likely, actual GB work would be read by the committee and at least some members would say, 'this is nonsense, x showed back in xxxx that it was false' or something like that. Real GB work is contentious and offensive; it rocks the boat, and it makes people uncomfortable.

"In France, Einstein was largely ignored until he visited in 1910. In the U.S., a few understood it, but, in general, relativity was ridiculed as “totally impractical and absurd.” In Britain, his theories met with resistance, because relativity was seen as a direct challenge to the widely accepted theory of ether."

https://daily.jstor.org/why-no-one-believed-einstein/

Marcus Arvan

Pendaran:

"He is a confusionist. The Einstein theory is a fallacy. The theory that the 'ether' does not exist, and that gravity is not a force but a property of space, can only be described as a crazy vagary, a disgrace to our age."

- Thomas Jefferson Jackson (University of Chicago, Astronomy)

"[Relativity] is repugnant to commonsense."

- Sir Oliver Lodge (Physics, Birmingham University)

"The supposed astronomical proofs of [Einstein]'s theory, as cited and claimed by Einstein, don't exist."

- Charles Lane Poor (Columbia University, Astronomy)

"Perhaps Einstein has made the greatest achievement in human thought, but no one has yet succeeded in stating in clear language what the theory of Einstein really is."

- Joseph Thomson (Cambridge University, 1906 Nobel Prize Winner in Physics)

"London-born physicist, Oliver Heaviside, another Nobel Prize winner...was one of the few who seemed to understand relativity--and he denounced it as drivel."

-Einstein: A Life (Bryan 1996: p. 101)

"Dr. Arthur Lynch lumped him with history's fakes...Lynch produced an impressive list of European scientists who...turned thumbs down on Einstein's theory: Henry Poincare, mathematician Gaston Darboux, Pail Painleve, Le Roux, Curbastro Ricci, Tullio Levi-Civita, and Emile Picard...For the most part Einstein stood above the battle, declining to cross swords with either critics or crackpots. Nevertheless, he was disappointed that his early mentor, Ernst Mach, did not support him. He also hoped in vain for approval by Albert Michaelson. Still, he was confident he was right."

- Einstein: A Life (Thomas Bryan 1996: p. 104)

Oh, and Kant's Critique of Pure Reason?

"The Critique of Pure Reason has exerted an enduring influence on western philosophy.[79] However, the Critique received little attention when it was first published. There were no reviews in 1781. According to Frederick C. Beiser, who compares the initial lack of response to the book to the way in which Hume's A Treatise of Human Nature (1738) fell "still born from the press", Kant knew that "he could not expect to hear from anyone qualified to appraise" it, and initially heard only complaints about its "unintelligibility and obscurity." Johann Friedrich Schultz wrote that the public saw the Critique as "a sealed book" consisting in nothing but "hieroglyphics". However, in 1782, the first review of the Critique appeared in the Zugaben zu den Göttinger gelehrte Anzeigen. The review, which denied that there is any distinction between Kant's idealism and that of Berkeley, was published anonymously, and became notorious...The review was denounced by Kant, but defended by Kant's empiricist critics...

Kant believed that this anonymous review of the Critique was biased and that it deliberately misunderstood his views. He devoted an appendix of the Prolegomena to refuting the review, accusing its author of failing to understand or even address the main issue addressed in the Critique, the possibility of synthetic a priori judgments, and insisting on the distinction between transcendental idealism and the idealism of Berkeley...Following the controversy...there were no more reviews in 1782 except for a brief notice. The work received greater attention only in 1784." (https://en.wikipedia.org/wiki/Critique_of_Pure_Reason#Reception )

Ha.

Pendaran Roberts

Thanks Marcus. I couldn’t be bothered to do that much research but it is as I expected. GB work is seldom recognized at first and many are hostile towards it. So, why do these committees think they have the magic scope for detecting it?

(I actually think Einstein’s relativity is incoherent nonsense. Haha! It makes no sense to talk of time being relative. So, nice to see many thought so too back in the day. It’s really too bad in my opinion that it caught on. If I was on a committee inspecting Einstein's writing sample i’d certainly fail to recognize that he would end up famous.)

Marcus Arvan

“I actually think Einstein’s relativity is incoherent nonsense. Haha! It makes no sense to talk of time being relative.”

Lol. You do know it’s been experimentally verified, right?

A Non-Mouse

Pendaran and Marcus: I agree that real GB work very often is regarded as nonsense at first. So maybe we should expect that recently published work is not real GB work. In that case, the person who has no publications, but has a project that promises to be GB, is more likely to produce GB work.

Here is an objection: committees are not in a good position to determine real GB work because they would believe the work to be nonsense if it is real GB. Or as Pendaran has already put it: at least some of the members would say "this is nonsense."

Here is the reply: this is not clearly correct. Often departments are partisan. They favor P rather than not P. So, if the project being evaluated for its GB-promise is for P, then they will be amiable. On the other hand, if the work is for not P, they won't. But someone whose work is for P is likely not applying for a job at a department that favors not P. Especially if the work is real GB work since their advisor would know that it is going to be regarded as garbage by such a department.

Pendaran Roberts

‘You do know it’s been experimentally verified, right?‘

This is a totally different debate that would take this thread way off topic. Haha! I mentioned it to show that even I don’t think I can spot what will be famous. But of course no I don’t think it’s been verified. I don’t think it’s even verifiable.

Pendaran Roberts

But to continue my point, Marcus’ reaction to my comment about relativity is the exact kid of reaction most GB work that challenges current orthodoxy is going to get. Many will just laugh at you. So, I don’t see how hiring committees think they are capable of selected based on this, and honestly I don’t think they want GB research anyway. That’s way too contentious and rocks the boat way too much.

Marcus Arvan

Pendaran: touché! ;)

Pendaran Roberts

Thanks Marcus!

Wanted to say that I think you have a fantastic website here! I have fun posting and debating with you and others.

Trust the Process

As someone who published a load, and in very good places, citations through the roof, and who also had lots of other markers of quality on the CV back on the market, and who came from a non-fancy place, and who struggled on the job market for years and years, and (due to friendships and gossip-connections with many people) who had inside info about how searches went down at various fancy places, and who watched top departments systematically bungle hires and overlook people I rated so much higher than whoever was hot at the moment, I've long wondered what exactly was/is going on.

The simplest and cleanest explanation is two-step. First step, explanation of the fact that you have almost zero chance of moving up significantly (to a top R1 or a top SLAC) if you are non-fancy: people at the top want someone inculcated into their bubble-system of social and professional norms, and they do not trust people from outside the bubble (this also explains the snobbery expressed towards successful faculty at non-fancy schools being mentioned at the 'philosophy is flat' discussion going on at daily nous). There is a real bubble, and it comes along with assumptions and expectations and a whole system of invites and workshops and students-of-X and friends-of-Y and the way you talk and with whom you talk and how you assess quality and knowing-what-it's-like to be so fancy and being-on-the-inside of the ego-gossip-competition culture and lots of other things. Fancy places are uncomfortable with non-fancy people, and this invisible barrier is incredibly strong. So the non-fancy usually get cut early, and sometimes late, but very rarely make it all the way through. When they do there is always (of course) a very atypical (i.e., not quality-related) reason they were able to do so, and this reason weighed very heavily on the deciders.

Second step, explanation of what happens when the fancy assess the fancy: they aren't great at it, though they believe they are (witness all the terrible justifications constantly given for loving prestige so much, and also witness how gossip-opinion always coalesces around a couple 'rising stars' every year - remember Leiter's old posts making this explicit?! - which would be hilarious if it weren't so pathetic... okay it's both), and the situation is not too different from other bubbles (speculation about the Oscars, scouts ranking the NBA draft classes). But it is bad for those involved at this stage as well - folks at school #1 worry about whether they are student #1 or #5, folks at school #6 worry that they are not top-5, and so on and so on. The non-fancy complain, but the fancy are just as worried, the vast majority are underemployed if employed, and then have to live with the same kind of non-fancy thing where they feel like the fancy look down on them because they didn't get a fancy-enough job.

Philosophy is actually great, and full of really great people, and many/most fancy-place folks are great, but the culture at the top has flaws.

Pendaran Roberts

"...and many/most fancy-place folks are great, but the culture at the top has flaws."

I wonder if this makes sense. If the culture is so seriously flawed can the people who make up that culture be so great?

Trust the Process

Would you extend this judgment to, e.g., people living in the US, or in Britain, Pendaran? (Maybe your view of how a culture is constituted attributes an implausibly massive role to individual agency.)

Pendaran Roberts

Yes, actually I would.

Maybe my view is implausible, maybe it isn’t. I just wondered out loud whether what you said made sense. I’m not convinced it does.

I don’t have time for a long post right now though.

Trust the Process

haha okay

Taco Tuesday

I'll just say this. I don't have a profile that lends itself to meaningful comparisons (won't say more to preserve anonymity), which is all the more reason to take my non-story with a grain of salt. I finally managed to get ONE offer. For a great job at a SLAC. I could probably give you a story of what worked, but it'd be deceptive. I can't honestly give you a story of why among 9 interviews this year I was lucky to get a flyout (or unlucky to get only one, being told I was on the fence for a couple others). I could tell you why I think I was a good match and what I did well, but it wouldn't tell you what I did better than others or what I think I didn't do well. It wouldn't tell you what imperceptible, uncontrollable factors made me stand out at this particular match but not others. Because I'm not a script, and some days I do well, some days I don't. Because applications and schools are not a script either.

It only takes one, and that's what it took for me. But for the most part I consider myself lucky. Desert is overrated and a fraught concept. Doesn't mean you shouldn't be blessed with your accomplishments or pissed about your failures. But you shouldn't think it has too much to do with desert.

I'll also say this. Ever since I got the offer I've been feeling terrible. For years I thought it'd be a blast. And now I wish I had a good story to tell myself. But I don't believe it. And in some way that's liberating, even though I feel terrible. Getting a job is fantastic news. But I wish I believed it was the predictable outcome of my work. It's fantastic news, but failing a lot doesn't exactly make you confident in your deserving a job, even when you do get a job. And that's terrible even though it's fantastic. It's a very messy process. In many ways it's like sausage.

Trust the Process

Taco Tuesday that makes a lot of sense and I had a similar thing with my one offer and my feelings afterward.

Not Deserving

Taco Tueday and Trust the Process:

Don't feel terrible! If desert is a fraught concept surely so too is its contrary; outcomes like yours (and mine) are if (for that sort of reason) not deserved, then not undeserved either!

Chris

Well, I see I managed to convince virtually no one that there is even a chance that people trying to select for positive, ground breaking projects might select someone with fewer publications over someone with many. At least, virtually no chance that they'll do it well.

Look: I agree there's lots of bias. And like Marcus I can find anecdotes to support my claim - there are people who came out with no publications but fancy pedigree who are now doing great as far as publishing. But of course there's the reverse. As others have noted, many programs are happy to "swing for the fences" and deny tenure if the research program doesn't pan out.

Maybe there problem here is that the term "ground breaking" is too unclear. Let me try again.

One kind of ground breaking is simply having a positive account that addresses some question, rather than criticizing other people's positive accounts.

You can publish - even in top journals - papers that are largely negative or critical. Many philosophers whole careers are full of such papers. Nothing wrong with that.

But: many programs are looking for a positive project - they're not necessarily looking for the next Einstein or Wittgenstein (though maybe some are!). They just want someone who is developing a positive theory of X rather than simply criticizing other's theories of X.

That is part of what I had in mind by "ground breaking". You CAN find such papers - even by junior scholars - in the journals. But many (most?) papers - even in top journals - are not of this sort.

Of course, much of the time, promising young scholars DO publish their positive proposals - sometimes even before finishing the PhD. And fancy schools do hire such people. But of course sometimes it takes a year or two after the PhD (in the old days, it usually did) to publish their positive theories.

There's more to say, too - many philosophers are convinced large parts of the field are not promising research programs. Imagine that you're an atheist and most of the work on say, ethics, approaches the issues from a theological point of view (or imagine the reverse if you're a theist). Should you hire the person who you think is working in a flawed research program, just because they have more publications?
Feel free to substitute your own examples.

of course, you might protest - most philosophers at elite institutions aren't in an epistemic position to know that so many of these other research programs are degenerating, or rest on flawed foundations. Perhaps so. But at least you can see why people might think this.

Some philosophers do think you should hire the person with the most publications, regardless of whether you think their views are on the right track. But I think its actually good that we have a mix of such people who hire (or a mix of hiring departments) - some think you just go for the person with the longest c.v., while other think you go for the person who you think is on a promising line - even if that person isn't the one with the longest c.v.)

At any rate, I think the point about a positive - rather than merely negative - research proposal is independent of whether you think people should be able to hire those who they think are on the right track. (See David Lewis' paper on hiring for a discussion of whether one should instead try to hire those who disagree).

Amanda

The problem I have with your argument Chris, is that the market is so flooded with talent that ANY criteria a search committee prefers, ground breaking positive research or whatever, there is enough options of ppl with publications to hire someone with SOME established record. No reason to do otherwise with the optiobs

Trust the Process

I have some sympathy with Amanda's point here. But I doubt anyone thinks longest CV=most deserving of any particular job. Chris, I don't disagree that it's possible to favor someone with less/no publications because their research project looks so great. Whether that's reasonable in any case is, in my experience, always based on specifics. I'm usually mystified by search committee decisions at elite places, and even by the gossip-consensus about who's cool in any given year. But sometimes, sure, I think: that person looks/is amazing, and I'd sure hire them, in spite of far less pubs than many others.

In a sense, I don't know what we're talking about here. I guess some are trying to shift profession norms in certain ways. But each search is its own thing. I doubt the shifting-norm efforts will be effective, or will come near the intended aim, whatever it is.

Anonymous

Is it that surprising that an audience of disgruntled job market candidates won't easily be convinced that the job search decisions of top departments might be justified? I'm not sure that the fact that this thesis isn't playing well in this context is good evidence in favor of discounting it.

Amanda

Anonymous that is an ad hominem and not relevant. Whether or not the arguments are good ones have nothing to do with how "disgruntled" people are or are not. And you, of course, could say something similar about those who have been successful (it is no surprise those who have jobs at r1s are convinced....)

Another issue: I never said anything about the longest CV. I do think it makes sense to hire someone with less publications over someone with more, if the few published papers show particular promise for the future. What does not make sense is hiring someone with NO or maybe 1 publication, given the deep job market and sheer number of candidates who have proven *something*.

Chris

I also have some sympathy with your point, Amanda. I do think, however, that “No reason to do otherwise” is a bit too strong. I think it should happen less often in this market, and i think it does. I don’t expect we’ll come to agree about this in this forum.

A Non-Mouse

Amanda: you seem to be suggesting that hiring someone without publications is not (or cannot be) justified in the current market. But you haven't offered a principled route toward that conclusion. What you suggest is that it can be justified for a candidate with few (say, 2 or 3) pubs to be hired over a candidate with more pubs if those pubs show particular promise. But it's not clear why pubs are more valuable than unpublished manuscripts. So, you might as well agree (with me) that hiring someone with 1 or fewer pubs over a candidate with many pubs is sometimes justified.

You might say: at least someone with 2 or more pubs has (as you put it above) "SOME established record," while someone with less has no such record. But this is not clearly true, unless you mean "established record *of publishing*." But if you do, then you are simply presupposing that pubs are more valuable than unpublished manuscripts.

If the presupposition seemed true, there would be no problem. But it doesn't seem true. An unpublished manuscript *might be* better than any candidate's published paper. And a set of such manuscripts *might be* better than any set of papers published by a candidate. If so, then this person's manuscripts *might be* more valuable than any candidate's pubs. Thus, the presupposition seems false.

You might say: yes, but the person with pubs has shown that blind reviewers value their work, while the person with unpublished manuscripts has not. And, you might continue, this makes the former more valuable. But this certainly is not true, since even mediocre work that has undergone blind review is sometimes accepted by top journals. So we should not be confident that the value added by blind review is great. This leaves it open whether the unpublished manuscripts might be of such a greater value than another candidates pubs that hiring the candidate with 1 or fewer pubs over the candidate with many pubs is justified.

Philosophy 101

A Non-Mouse, regarding the last paragraph of your comment:

Since you are not willing to identify specific examples non-anonymously and put your credibility on the line, as far as I am concerned the amount of evidence you have provided for you factual premise is nil. Until you provide some evidence of your own expertise, for all I know you have no clue at all about what counts as 'mediocre work' and what doesn't.

But even if your factual premise is true, it is clearly fallacious to infer that anonymously reviewed articles shouldn't count more than unpublished manuscripts for the purposes of hiring and promotion. It's also true that a few people who have taught courses turn out to be extremely bad at it. But obviously, it doesn't follow that hiring and promotion committees shouldn't count actually teaching a course more than merely designing a syllabus for a course you may or may not actually put through the gauntlet.

(Or to put the point more simply: even if some smokers don't get cancer, it still matters a ton that the medical profession has concluded that smoking causes cancer.)

A Non-Mouse

Philosophy 101: About your first point, for the purposes of the point I'm making, it doesn't matter at all whether I know anything about what counts as mediocre work. All that matters is that what I say is true. And it is true.

About the point about fallacious inference, I completely agree. Fortunately, I did not make any such fallacious inference. What I say is that blind review doesn't make pubs more valuable, and I then go on to support what I say with the observation that mediocre papers have undergone blind review and been published. I did not infer that pubs shouldn't count as more than unpublished manuscripts. There is a significant difference between what I said and what you took me to say.

Philosophy 101

A Non-Mouse,

I had charitably interpreted you as making a contribution relevant to the present discussion -- namely, about whether a committee ought to count published vs. unpublished work as more valuable for the purposes of hiring or promotion -- rather than uncharitably interpreting you as meandering into some off-topic discussion about whether one is somehow 'really/intrinsically' more valuable (whatever that would mean).

I stand corrected.

Amanda

The reason why I think publications are more valuable, is that any single philosopher judging a person's work, or a single search committee, is likely to be plagued by all sorts of biases. Peer review shows that at least some other people agree the work is valuable. This protects against letting nothing other than one's own biased (or potentially biased) opinions count for the entire hiring decision.

A Non-Mouse

Philosophy 101: If you understood the relevant part of my comment to be off-topic, then you weren't reading carefully, or you don't understand what it is to respond to objections before they arise. I guess is that it is the former.

Amanda: Your reason is certainly understandable. However, reviewers have biases too. I suspect that this is why work that is less than stellar sometimes gets published at top journals.

Random job market person.

One point about the value of publications: it doesn't just show that some random referee somewhere thought the paper was worth publishing. It shows that the paper was considered worth publishing by one or more experts in the area who were selected by an experienced editor, that the paper was able to withstand the gauntlet of R&R objections, and that the author and referees were able to make a strong enough case to an editor (somebody who reads countless similar referee reports) that the paper is more worthy of being published than 95% (or whatever) of the other papers submitted to the journal. Is this process fallible? Yes, crap papers do get published, and many great papers go a long time without being published. But it is still pretty good evidence of quality.

Philosophy 101

A Non-Mouse: Yes, you are completely correct that I didn't read you carefully enough. I charitably interpreted you as objecting to a position that people were actually defending in this thread. Instead, I should have read you as propping up and then knocking down a straw man. My bad!

Amanda

I agree that bad papers get published and reviewers have their biases. However, the more people that approve of a paper, the greater the chance that it is not just biases but real quality. Hence if ONLY a search committee approves of a paper, that is less evidence that the paper is truly of great quality than if a search committee AND reviewers approve of the paper. So if a committee has a choice between two candidates they like, say, one with publications and one without, there is greater evidence that the former is a better candidate. Given these two choices, it makes sense to choose the candidate with more evidence that they do quality work. The process is still fallible. However it seems one should makes choices that give one the best odds of success.

A Non-Mouse

Amanda: The point about more evidence is a good one, as long as it's qualified. As the number of people who judge a paper as being of a high quality increases, the number of pieces of evidence that a paper is of a high quality increases. So, the evidence of quality is always greater (in one sense) when there are more judgments of quality.

But, as others have mentioned previously, the people on committees at top places likely believe that their judgments of quality are to be given greater weight than those of most reviewers (et. al.) working for most journals. (And this belief is likely justified.) For these reasons, the evidence of quality is not always greater (in another sense) when there are more judgments of quality.

The question is now this: are committees ever justified in believing that the evidence of quality of unpublished manuscripts is greater than the evidence of quality of pubs? Surely they sometimes are. Now consider two candidates, A and B. A has 1 or fewer pubs, but has manuscripts that the committee judges to be of a very high quality. B has many pubs, and the committee judges B's writing sample to be of a lower quality. Can the committee be justified in hiring A over B? It seems to me that the answer is obviously "yes." Apparently, then, hiring a candidate with high quality manuscripts (but no pubs) over a candidate with many pubs is sometimes justified. My guess is that this may happen more frequently than many suppose.

Amanda

A Non-Mouse, your point would be true if it was ever the case that a search committee just had the two candidates you mention. But I doubt this. Hundreds of people apply to any decent research job. So I find it hard to believe there is ever a case where the best two options come down to the situation you mention. And yes, I suspect that given the job market we have, it is hard to imagine a case where a search committee does not have a candidate that they BOTH deem as qualified and has at least a few publications.

A Non-Mouse

Amanda: I don't find it hard to believe that that the two best options might be as I've described. I'm not sure why you do. I suspect that it's because you have a bias toward valuing pubs. But I of course can't be sure, nor wish to accuse you of being biased. I guess we'll have to agree to disagree.

Verify your Comment

Previewing your Comment

This is only a preview. Your comment has not yet been posted.

Working...
Your comment could not be posted. Error type:
Your comment has been saved. Comments are moderated and will not appear until approved by the author. Post another comment

The letters and numbers you entered did not match the image. Please try again.

As a final step before posting your comment, enter the letters and numbers you see in the image below. This prevents automated programs from posting comments.

Having trouble reading this image? View an alternate.

Working...

Post a comment

Comments are moderated, and will not appear until the author has approved them.

Your Information

(Name and email address are required. Email address will not be displayed with the comment.)

Current Job-Market Discussion Thread

Job-market reporting thread

Writing Service


Categories