Our books






Become a Fan

« On What Reasoning Might Be, Part I | Main | Philosophers and intuitions: they go way back »

10/27/2014

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

Olle Blomberg

Hi Elisa!
I think the reviewer's primary responsibility is to provide information about both pros and cons to the selection committee of the funding agency in question so that informed decisions can be made (rather than to the applicant). I'm not sure why the reviewer shouldn't both express enthusiastic approval and provide suggestions about improvements and point out minor weaknesses. After all, I suspect that selection committees have to decide among projects which all get enthusiastic approvals from reviewers (I've never been involved in such a process, so I don't know). In this case, information about minor weaknesses (perhaps conveyed as suggestions) may turn out to be crucial for a committee's final decisions, but why is that a bad thing? At least, this seems better than picking at random from the final pool of projects. But perhaps there are even better alternative procedures that haven't crossed my mind.
Cheers
Olle

Elisa Freschi

Olle, thanks for the comment. The situation is hard and this makes it difficult to be fair. However:

1) my point is that in the new language of funding committees, terms such as "suggestions" now mean "weak points" or "reasons for turning down the project". A referee is very welcome to fill in adequate reasons, she just needs to know what she is doing.

2) I do not agree with the idea that the reviewer's primary responsibility is towards the funding agency and not towards the applicant. Personally, I am convinced that whenever I turn down someone, she has to learn something through the process and to at least gain the opportunity to improve. In this sense, confidential remarks which are only read by the committee are unfair, because they deprive the applicant of the opportunity to grow and learn.

Olle Blomberg

Thanks for the reply Elisa. I didn't mean to disagree with 2, and perhaps you are right about 1. I just meant that that a reviewer should not tailor her/his review to whether or not s/he wants the project to be funded. If s/he really likes a project and thinks it is very worthy of funding, s/he has no responsibility toward the applicant to try to ensure that the project gets funded. The reviewer knows nothing (or not much anyway) about the other projects competing for funding, so it seems to me that s/he should try her/his best to convey the relevant pros and cons to whoever makes the decisions. I also think it is important that the applicant gets information about why the project isn't receiving funding so that the applicant can improve the application (so I don't disagree with 2). It is of course also important that the decisions are as fair as unbiased as possible. I'm sure there is of luck involved, and well as a lot biases. And so, there is a lack of fairness too.

Carlo Ierna

A lot depends on the process used to evaluate the applications. In the Netherlands for the VENI postdocs (success rate between 15-20% over the past few years AFAIK) the candidate is invited to respond to the referee reports. I managed to turn around one negative report (out of three) by succesfully addressing the issues raised by the referee (see: http://blog.ierna.name/2013/01/22/veni-grants/ and http://blog.ierna.name/2012/05/28/from-the-peer-review-to-the-interview/ ). In this situation, the committee gets to evaluate the proposal on the basis of a back-and-forth between the candidate and referees, which mitigates the issues you raise. Instead, to make a rough cut, almost half of the applications are eliminated based on a pre-selection, before the proposal goes to the peer-review phase. Just by looking at CV and a layman's summary the committee chooses which proposals to forward to the referees, but only if there are more than four times as many applications as can be funded within budget.

Elisa Freschi

Hi Olle, and sorry for the late answer. Now, the issue is really thorny, insofar as, as you rightly say, the peer reviewer has no idea about the other competing projects and little idea about projects presented all over the world, but nonetheless s/he in invited to say whether the project is among the top 5% (so in the Austrian FWF) or top 15%--20% (as in Carlo's example) among similar projects *in the world*.
The only point we keep on disagreeing about is whether the peer reviewer is responsible of the final choice or not. I see that in an ideal world explaining pros and cons of a project would be the right thing to do. However, in our biased world, this basically means that the project one is reviewinig will be immediately turned down (see quotation above) or has far less chances (see Carlo's experience as depicted in his posts). I, for one, would not want this to happen if I believe in a certain project and has only minor suggestions about it.

Elisa Freschi

Carlo, thanks a lot for the comment and for the interesting posts about your experience with the NWO. Whenever you have time, it would be interesting to have some more raw data (% of accepted projects among the ones who got less than A in at least one peer-review, whether the number of accepted project is decreasing, number of projects of incoming scholars accepted…).

Verify your Comment

Previewing your Comment

This is only a preview. Your comment has not yet been posted.

Working...
Your comment could not be posted. Error type:
Your comment has been saved. Comments are moderated and will not appear until approved by the author. Post another comment

The letters and numbers you entered did not match the image. Please try again.

As a final step before posting your comment, enter the letters and numbers you see in the image below. This prevents automated programs from posting comments.

Having trouble reading this image? View an alternate.

Working...

Post a comment

Comments are moderated, and will not appear until the author has approved them.

Your Information

(Name and email address are required. Email address will not be displayed with the comment.)

Open job-market threads

Categories