Deferring Data Collection Despite Corona Delays
How to complete your conference submissions under social distancing.
Let me preface this by saying that this is my own personal opinion, and not in any way an official stance or policy. Let me also add that continuing research may not actually be practical or even possible for individuals. It depends on your personal situation and your geographical location. However, for some of us, continuing work, at least in some capacity, can be comforting or even liberating. Since I have been thinking and writing about these things lately, I felt it would be useful to post some of these writings for others to potentially benefit.
The challenge I am obviously discussing here is that a lot of researchers around the world are under lockdown, self-isolation, or at least social distancing regulations to mitigate the ongoing COVID-19 pandemic. However, some research projects are fundamentally based on specialized equipment or in-person user studies, making them difficult to both reframe — the work is basically nothing without it — as well as move to a virtual format — the work simply cannot be done online. At the same time, these projects may represent months of hard work, and missing an upcoming conference deadline may seem a grim prospect. In such cases, one plausible strategy may be to simply submit a paper that has everything but the results, but with the plan to fill them in when it is safe to collect them.
In other fields, this approach has become known as a “registered report” (RR), and is basically a method to enable peer review exclusively on research methodology without the pressure to generate impressive findings. A research project, the reasoning goes, should be more about whether the experiment or study is valid and correct rather than whether the findings are significant or not. If the question is an important one, the answer matters little (to the validity of the work). A provisionally accepted RR will be combined with the results and accepted as a single paper if the registered methodology is followed. You can find more information about RRs here: http://cos.io/rr
The benefit in this particular case, of course, is that it would allow you to still submit your work now to meet an upcoming conference deadline, and collect the data at a later time when it is safe. Registered reports are much more than this, of course, but it is a particularly useful aspect under COVID-19 mitigation measures.
Even if registered reports are an accepted practice in many fields and hundreds of journals, it is a virtually unknown mechanism in much of computer science, including visualization. For example, it has never been used at IEEE VIS or TVCG, to my knowledge, which means that it is a submission form of which few reviewers are aware. However, this means that there is no guarantee that all reviewers will read, understand, or even agree with it. In other words, if you do choose to submit a registered report, you may run the risk of being rejected simply due to these reasons. In fact, most conferences would not be properly set up for managing a registered report submission simply due to conferences typically have constrained schedules (although COVID-19 may change some of this). Conferences with journal proceedings, such as IEEE VIS and IEEE TVCG, may be in a better position to handle this.
Practically speaking, if you choose to go this route, I recommend that you clearly signpost this choice in your paper and potentially educate the reviewers to the practice. Don’t submit half a paper (i.e. methods but no results); instead, endeavor to not just write about your methodology, but also your analysis and presentation (e.g. charts, tables, diagrams) methods in exhaustive detail. You may also want to include predictions about the results you expect to see. In case your paper is provisionally accepted, you will be expected to complete the paper with the actual data and analysis when it is safe to collect them. The end result should will still be a single paper. However, when and where it will be published may depend on the conference.
With some luck, if several people adopt this practice — even if it is not officially condoned by specific conferences — we may be able to make long-term changes happen. Good luck!
