One of the more common analogies one hears about the academic job market is that it is similar to professional sports. In athletics, the best college teams tend to produce the best athletes. Professional teams, in turn, are looking to hire the best athletes they can. Thus, if you want to end up on a professional team, you should try to get onto the best college team you can and become the best player you can be. By a similar token, the thought is, if you want an academic job, you should go to the best program you can, publish in the best journals you can--and if you do these things, you maximize your chances of getting hired into the "major leagues" (i.e. a tenure track job).
Or maybe not. I've long suspected that this analogy is mistaken, and for these reasons, I collected this data on interviewing and hiring so far this year. Although what we can infer from that data is debatable, the simple fact is that (1) in the hiring data, the vast majority of hires I recorded (40 out of 57, or 70%) had no top-ranked publications, and (2) in the interview data, top-ranked publications did not correlate significantly with interviews, but total publications and low-ranked publications correlated very strongly with interview numbers. How can this be, one commenter, "eugene", asked? Eugene wrote:
These results really are incredible, given that in my experience, everyone really does assume that a publication in a top 5 journal is a golden ticket. (Anyone who has ever seen all the praise on Facebook for those who have managed to secure such publications knows what I'm talking about. Not that I think such praise is a bad thing!).
Anyone have a guess at to what explains this? Do those that land top 5 publications tend to rest too much on such laurels to the neglect of the rest of their dossier?
Although I answered Eugene's query in the comment section there, I'd like to address it again in more detail here--as I believe it makes the same error as the professional sports analogy. Allow me to explain.
I'd like to begin by altering the analogy a bit. In actual professional baseball, there are different "levels" of competition: rookie ball, A-ball, AA-ball, AAA-ball, and the major leagues. The purpose of each level, of course, is to select the best players at the lower-level, and then filter them up to the next level. Thus, the entire system is merit-based: the better one plays baseball, the more likely one is to move up the ladder.
Now, however, suppose that each level had strong incentives to select players that (A) will succeed at their level, but (B) not be so good as to have the desire or means to leave that level once selected. How might this be? Well, suppose that anytime a AA-team gets funding for an extra player, they will lose that spot (and money for it) if the player they hire leaves for a AAA team. In that case, the AA team has a strong incentive not to select a player who's good enough to play AAA or in the major leagues. That is against the AA-team's interest. And suppose this were true of A-level teams too--that if they select someone who would succeed at the AA-level, they would most likely lose that person to a AA team a couple of years later and lose a position on their team.
Finally, let us suppose that there are very few major league teams (10, let's say), a bunch more AAA-teams (100, let's say), a bunch more AA-teams (200, let's say), a bunch more A-teams (1,000) etc. For fun's sake, let's call positions on these teams "tenure-track jobs." If you want to maximize your likelihood of getting a job on some team or other, what will maximize your probability of getting one? Answer: it's not having the best skills or resume--the skills or resume of a major-league baseball player. It's actually having the skills of an A-level baseball player. Indeed, in this story as I told it, the most elite players will actually have the hardest time getting on a team--for the only teams who have incentives to select them, major league teams, are a very small minority of all the teams in existence.
I don't mean to say that we should rank colleges and universities in the same way--as though small liberal-arts colleges are worse than research universities. What I do mean to suggest is that the proportions and incentives are likely to be similar in the academic job market. There are precious few R1 universities, and very few tenure-track jobs at them. There are a whole lot more R2's, SLACs, community colleges, etc. When the latter, larger group of schools look to hire someone for a tenure-track position, they are looking to (A) have a successful search, not (B) a failed search. But, how are they incentivized to define these things? Answer: a "successful search" hires someone who will succeed (get tenure) and not leave. A "failed search", on the other hand, is someone who either doesn't succeed (doesn't get tenure) or who will leave. This is not mere speculation. These incentives absolutely exist. A search for a tenure-track position costs a great deal of money, and a search that hires someone who leaves is just as much as a "failure"--from a cost standpoint, as well as the department's standpoint--as someone who stays but fails to get tenure.
With this in mind, suppose you are someone at at, say, an SLAC deciding who to hire. Do you have incentive to hire someone with publications in Phil Studies, The Philosophical Review, or Mind? The answer, very simply and honestly, is: no. Although that person would almost certainly get tenure should they stay in the position, they are a very real flight risk--and, if they do flee for greener pastures, your department may lose that tenure-stream line: an absolute disaster for your department. The safer thing to do is to select someone who has shown (A) the ability to publish (since you want them to get tenure, should you hire them), but (B) not the ability publish in journals that might give them the ability and incentive to move to a position at another university.
What does all of this suggest? It suggests to me that the academic job market is not at all like professional sports, and that top-ranked publications are probably not a "golden ticket" to an academic job. They may indeed be a golden ticket for a small minority of jobs, but at many other jobs they may well be the exact opposite: a reason to pass over the candidate for someone less likely to be a flight risk.