I have already written about the importance of writing project applications for European scholars. For people who do not have a tenure (i.e., the very vast majority of scholars and the almost-totality of scholars under 40), projects are the only way to get their research financed, since it is very difficult or almost impossible to survive out of teaching alone. Recently, a personal experience made me reflect on the ethics of peer-reviewing research projects. At least in part, my reflections harmonise with the ones Marcus, Moti and others have spelt out, on this and other platforms, concerning peer-reviewing articles on journals. Nonetheless, I would be glad to read your opinion on this specific issue (if you think that it lies far away from you, consider the fact that you might be asked to be a peer-reviewer earlier than you think).
Some months ago, departing from Decemeber 2013, I started working on a fascinating project, the formalisation of the deontic logic of some Mīmāṃsā authors (Kumārila, Prabhākara and Maṇḍana). Given that I am not an expert on formal logic, the project has been conceived together with some colleagues working on formal logic and on the IT tools for automating it. After some preliminary work, we submitted a project within the "Mathematics and…" call of the WWTF. The other principal investigator was Agata Ciabattoni and the other collaborators were Björn Lellmann and Ekaterina Lebedeva. Agata and Björn would have been working with me on selecting the logical rules from the relevant Sanskrit texts, translating them in formal logical language and developing automated deduction methods to reason about them.
Ekaterina, as a linguist and an expert of the intersection of language and logic, would have taken care of the fact that our translations of Sanskrit passages into logical rules did not entail logical ambiguities.
Unfortunately, however, the project has not been funded. I read the four reports, and they all sound positive, so that it is difficult to imagine what we could do better in order to improve our chances by the time of a resubmission. This is a pity and made me consider various other failed projects of colleagues and friends. It seems that there are two types of failures:
- justified failure: The peer-reviewers give pertinent reasons for which the project is not worthy of being funded.
- unjustified failure: The reports are entirely positive, or at most contain suggestions for further improvement. Nonetheless, the jury of the funds decides not to fund it.
Strangely enough, it seems that 2) is much more common than 1). In my case, I received a motivated refusal only once, and even then, the refusal was not motivated through scientific reasons, but rather through the fact that I was asking for more money than the university could have wanted to invest (it was a FIRB project in Italy, about the critical edition and study of Nyāyamañjarī 5). In another case (again a FIRB in Italy), the project has not been funded and this was the general comment:
By investigating the work of Jayanta Bhatta from a linguistic point of view, the research project aims to make Indian philosophy available to Western scholars. Topic and methodology are very well thought through. The scientific profile of the research group is outstanding.
The following is an extract of the comments received by a friend on a project which has also not been funded. I have omitted names and titles and further identification marks, but nothing else:
The stand-alone project entitled [om.] submitted by Dr. [om.] appears sound and promising. The applicant proposes to prepare a critical edition with an English translation of the Commentary [om.]. Dr. [om.]'s critical edition of [om.] together with an English transaltion will definitely fill up a long-standing gap in the Indological studies and in philosophy of language, It will be a powerful resource for future research. [om.] The applicant has spelt out the goals and subgoals of his research very clearly. The time line that he has given is pragmatic and the project executable within the proposed time-frame. He has chosen a very competent researcher as his project assistant [om.]. National and international collaborators in the project are all very eminent scholars, in fact stalwarts in the area. I am very hopeful that the project as outlined will make significant contributions to the field. The proposed project lies in the interface area of Indology, Sanskrit Grammar, Philosophy of Language and Metaphysics and has an inter-disciplinary flavour to it.
However, on the last page, under the heading "Suggestions" ---suggested by the jury and not added by the peer-reviewer---, the same peer-reviewer writes:
As the applicant needs to collect material from libraries in India too, he should also try and interact with the established [om.] scholars in India.
This sounds like a genuine suggestion, but has become the reason for rejecting the project.
In the case of deontic logic, the motivation for the refusal was that the project was "very appealing but also very ambitious". The four reports, as I said, are in fact positive or vey positive, although one of them contains indeed the adjective "ambitious" as an answer to the heading "What are the strenghts and weaknesses of the project?" ---the heading is part of the jury's document and had not added by the peer-reviewer:
Strenghts: Even if partly succesful, the project will make a strong contribution to mathematical logic, computational logic and Indology. Truly interdisciplinary, strong feedback between mathematical technology and the application.
Weaknesses: Perhaps a bit too ambitious.
I do not want this to be a rant, but a general reflection. It seems that, given the general economic difficulties, juries are only looking for excuses for letting down projects. Thus, I wonder:
It seems to me that if you care about the project you are peer-reviewing to be funded, then you should just avoid all suggestions. This sounds paradoxical and sad, since one indeed wants to help a project by adding insightful comments. I, as one, cannot remain silent on anything and surely not on something I care for. Nonetheless, it seems that adding comments or suggestions is easily read as a reservation and this is consequently used as the best excuse to let down at least one further project ---given that there are anyway too many good projects to fund them all.
What are your experiences? Which judgements did you receive or write?
Cross-posted, with some changes, on my personal blog
I am not sure how avoiding suggestions would solve the problem of too many projects and not enough money. If there are more projects than there is money to fund, then some deserving projects are not going to be funded even if I avoid suggestions.
I would also find it problematic to accept a request to review a proposal in which I am asked to provide suggestions and then not actually provide them, in the hope that whatever random method they subsequently use to make final funding decisions will benefit a proposal that I like.
Posted by: Robert | 10/26/2014 at 01:23 AM
Robert, you are right, the problem is a general one and there is no plausible solution at hand (one needs to make funding agencies aware of the importance of funding research, but this is a long-term goal).
My only point is: As a reviewer, you need to be aware of what you are doing. You need to know that the vote "good" means "turn down the project without discussing it" and that "very good" means "turn down after discussing". It is like having to learn a new vocabulary and it would be meaningless to insist that one's "good" just meant "good". It no longer means "good" for funding agencies and this is (sadly) a fact. I have added some data in this post: http://philosopherscocoon.typepad.com/blog/2014/10/funding-your-research-projects-some-data.html
Posted by: Elisa Freschi | 10/27/2014 at 05:03 AM