Is There Shame in Resubmitting?
In which I explore the surprising resistance against resubmitting work after rejection.
Note: This is a new import of an old blog post from 2019.
A realization has stolen over me in the last few months, something that I have observed for a long time, but which recent events have finally helped crystalize. First there was the lengths some authors would go to disguise that their work had been previously submitted, and the glee with which some reviewers would point out that they had seen the work before. Second, there was the many times when my students would voice similar concerns about their rejected papers as they were preparing them for resubmission. Then I recalled one of my colleagues stating that sometimes they had to “mercy-kill” their rejected papers. And just recently, another colleague told me that he is only ever excited about the first time he submits a paper, and views the paper as more or less a lost cause when it is rejected. This makes me wonder: is there shame in resubmitting a rejected scientific paper?
For the record, I am a serial resubmitter.
Part of the problem is that many of my papers seem to get rejected the first time I submit it, only to get in the second time! Sometimes it takes significantly more than two submissions: Just recently, our paper on data comics, inspired by the “comic strip” genre in Segel and Heer (InfoVis 2010), that we first wrote in the summer of 2014 and submitted to CHI 2015, was accepted to iConference 2019! In other words, we needed more than four years and an embarrassing number of resubmissions to get this paper accepted, and in the course of all this time we saw a lot of other work proposing comics for data-driven storytelling get accepted before ours. Nevertheless, these things happen, and I see them as a natural part of the scientific process. It doesn’t mean our original work should not eventually be published.
This is an extreme case, of course, and I do think that there is some truth to the idea that papers with the best ideas tend to get accepted quickly. Sure, sometimes you need a rejection or two to get the kinks worked out of the paper and to overcome noise in the review process, but as an example, for last year’s InfoVis, it was the new and fresh ideas that were accepted, whereas my resubmissions were uniformly rejected. But that’s still a far cry from the sentiment outlined above, where resubmitting a paper is almost seen as shameful.
On the other side of the spectrum are the rare cases where papers are rejected multiple times, only to finally be accepted with honors. For example, I have a distinct memory of Anind Dey relating such a story in his CHI 2009 (?) talk about a paper he had submitted multiple times to CHI, only to be rejected over and over again, until the paper was finally accepted and given an honorable mention. I am sure there are similar stories.
Now, there are obviously several caveats to this. First of all, there is the practice where people submit half-finished work to a conference just “to get the reviews”. While perhaps understandable, this is a wasteful and inefficient practice as it ties up valuable reviewing effort on work that even the authors themselves know is not ready. Obviously there is a little bit of a thrill involved for the authors: perhaps that paper will happen to get just the right reviewers, and you win the lottery and get in? Still, submitting work before it is ready (bar some minor lack of polish) is not the way, and is borderline abuse of the peer review system.
Second, I have also seen situations where authors turn around a rejected paper to immediately resubmit to a new venue with no revisions. Perhaps all the authors do is to adapt the paper to a new template, and then just send the paper again without as much as fixing grammatical errors or typos pinpointed by reviewers from the first rejection. Basically, such authors are trying to game the system by submitting the same paper and hope that they get a fresh set of reviewers at the new venue. This I think is an actual abuse of peer review, as it is disrespectful and immensely wasteful of reviewer efforts and plays only on the lottery aspect. Granted, sometimes a rejection from a previous submission is a close one, and the reasoning is that it was just bad luck who caused the paper be turned back. Sometimes the reasoning is that the paper wasn’t sufficient for a top venue, but should be “good enough” without revision for a lower-ranked one. It is still a blatant disregard of reviewer effort. Don’t do it.
And third, how many times should a paper be allowed to be resubmitted? Similar to many academics, I have my fair share of dead manuscripts sitting in my desk drawer. Most of these we eventually failed to resubmit due to a combination of reasons such as (a) other work eclipsed and/or subsumed it, (b) the person working on it moved on, or © the reviews were too discouraging. Meanwhile, we persisted in resubmitting the data comics paper a total of 10+ (!) times, mainly because it is a central part of my student’s dissertation work, and the fact that other data comics papers were getting published was a good indication of the basic validity of the idea. In other words, I think the answer is “it depends.”
These are all reasons from the author side whether or not to resubmit work. However, I find it most troubling when these same discussions arise on the reviewing side. As an associate editor, I often come across papers that have been previously rejected at a conference. In particular, IEEE TVCG routinely receives papers that did not get accepted at IEEE InfoVis, VAST, or SciVis. In these situations, associate editors are encouraged to invite a subset of the original conference reviewers for the journal revision so that they can see how the paper has been improved (or not). My view is that I am inviting past reviewers to inject some memory into the system so that the authors get some benefit from their past submission other than the reviews themselves. This avoid situations where one set of reviewers say one thing, the authors revise the work, and then the next set of reviewers reject the paper for a diametrically opposite reason. This was also the reason I was part of the push to allow IEEE VIS authors to include reviews from past rejections in their submission, a practice that conferences such as SIGGRAPH have long allowed. (And even some funding agencies; the NIH explicitly encourages resubmission of declined applications along with a specific introduction detailing the revisions.)
However, when inviting past reviewers, I have sometimes seen reviewers who seem to think that I am reinviting them for the purpose of yet again killing the paper. Perhaps I should not be too surprised about this: after all, peer reviewers are authors, too. It is likely a vicious cycle arising from the fact that many authors seem to be deathly afraid of getting the same reviewers for their next resubmission, and then apply the same reasoning when the tables are turned and they are reviewers themselves. I have even seen how a reviewer will quote a past rejection as an argument to reject the new revision during the discussion phase for a paper (to be fair, this was only one of several arguments in favor of rejecting it). It’s as if a past rejection is a kiss of death, that resubmission is indeed shameful, and the fact that a paper was found wanting in a past iteration is proof enough that it is wanting still.
I would rather see that we erase the stigma of rejection and embrace resubmission. Work should not have to be perfect right out of the gate to have a chance at contributing to the state of the art. It is better to recognize that science, like everything, is a process that can be improved over time and with feedback.
Originally published at https://sites.umiacs.umd.edu.
