On Detective Work
Respecting the double-blind review process.
Academic publication is built on peer review, and many such peer review process are “double blind”, which means that both authors and the reviewers are supposed to be blissfully unaware of each other’s identities. The idea is that scientific work should be reviewed on its own merits rather than the merits of its authors. In fact, except in rare circumstances mostly having to do with prior art or the qualifications of the authors (such as for grant review), there is very little reason to consider who wrote a specific submission.
However, in the age of arXiv and OSF preprints, easily accessible technical reports, and public Github repositories, not to mention overeager authoring tools that automatically add author information to PDF meta-data, there is an unprecedented chance for reviewers to engage in a bit of detective work to determine the identity of a paper’s authors. If you are a reviewer for such a submission, I have one request to ask of you:
Please don’t.
Don’t put paper titles into Google. Don’t trawl Github, OSF, or arXiv looking for likely-looking preprints or source repositories. Don’t search paper references trying to figure out which papers were written by the same authors, and don’t look in the PDF metadata to see if the authors perchance forgot to purge their names from it.
Why? You are hopefully all familiar with the many reasons why anonymization is a good idea. For example, symphonic orchestras routinely engage in blind auditions, and job seekers in the U.S. with English-sounding names have been shown to be more likely to land job interviews than those lacking such names. In academic settings, there are similar reasons why hiding people’s identities leads to better and more fair outcomes. If nothing else, double-blind peer review is an explicit policy for many conferences and journals. Engaging in detective work in this manner circumvents this policy and is a violation of the spirit of the review process.
I can understand the temptation to figure out who wrote a specific paper. Sometimes it’s not even possible to avoid this because you can get a pretty good idea of the author solely based on the style, figures, and topics of the work (be warned that you may well be wrong with your suspicions). But so what if you find out? How will you use this information? If you let your knowledge or suspicion influence your review, positively or negatively, you are no longer reviewing the work but reviewing the authors, and this is not what peer review is about.
In other words, focus on rewarding good science rather than mundane things such as author identities and affiliations.
