This is a guest post by Finnur Dellsén, Associate Professor at the University of Iceland
As many people have pointed out recently, the peer review system relies on us accepting several referee requests for each paper we publish. So refereeing is a substantial part of our job as researchers, even if we don't get paid for it specifically. But how do we make sure we write helpful, insightful, and constructive referee reports? How do we improve as referees? We get all sorts of feedback on our own work --- from colleagues, editors and referees --- which helps us improve as researchers. We get much less, if any, feedback on our referee reports.
So when I was named Referee of the Year at the British Journal for the Philosophy of Science for 2020, I thought it would be a good opportunity to get the conversation going on how to write a good referee report (see my Facebook post). Below is a list of things that I aim to do in my reports (but I encourage others to add to the list in comments):
- I always summarize the paper first in my own words. This lets the authors and editors know I carefully read and understood the paper. It also forces me to focus on the main contribution of the paper as opposed to minor concerns (which is important because of point 3 below).
- I try to find as many nice things to say about the paper as I possibly can, especially when I recommend rejection. I highlight ideas and arguments that are especially interesting or compelling. This helps the author know what not to change when revising their paper (e.g. for another journal).
- I try to make the amount of text in the review that I spend on each idea in the paper reflect its importance for the main thread of the paper. Every paper has things in it that I don’t agree with or find slightly unconvincing, but it’s usually not worth getting bogged down with those issues.
- When suggesting changes, I say explicitly which of those changes I consider to be most important, even dealbreakers, and which ones are mere recommendations for the author to consider. This makes it clear to the author which changes to focus on.
- I try to indicate clearly, if not to the author then at least to the editors, how confident I am in my recommendations -- not just about whether to publish, but also about individual points or recommendations for changes. In many cases, I am not an expert on all of the things the paper discusses, and when I’m not I try to say so explicitly.
- Finally, referees can gently point to relevant work by underrepresented groups and/or work that is underrecognized in the field (e.g. because of prestige bias). I admit I haven‘t done this as often as I would have liked in the past, but I think it can do a lot to increase diversity and make philosophy more inclusive.
Those are some points that come to mind. What am I forgetting? What do you aim for in your referee reports?
(Thanks to Helen De Cruz for giving me this opportunity to discuss refereeing practices, and to the BJPS for the award!)
I work in philosophy of science, but recently I have started to submit papers in another subfield of philosophy. These are all journals with a good reputation. However, to my surprise, it seems that a pattern is being established: editors ask me to provide suggestions for possible reviewers. This never happened to me in philosophy of science. What's the best practice here? Naturally, I'd propose people I know (assuming they have not read the paper and they do not know I'm working on that specific topic), because I know that those people will take their job seriously. However, even if those people do not know that I'm writing that specific paper, they may recognize who I am. But this, I say, happens even in blind peer-reviewed, especially in highly specialized fields such as philosophy of science. Am I terribly wrong? - Posted by: Don't really know what to do
I have a question related to that of Don't really know what to do. Should a philosophy journal ask the author of a submission to suggest possible reviewers? What are the rationales for and again this? -Posted by: jack
Another reader answered:
This is a common practice in science journals, to ask for (i) possible referees, and (ii) sometimes a list of people who you do not want to review your paper. In science it is common place to have only single blind - so the reviewer sees who the author or authors are. I publish regularly in an empirical field (call it scientific), and the key journal will not send your paper out for review unless you have your name on it.
However, while this may be a common practice in science, I wonder how readers feel about it, and whether anyone has tips for 'best practices.'