Workshops, Conferences, Journals — Oh My!

Understanding different types of scholarly output.

Photo by Thomas Lefebvre on Unsplash.

There are many ways to communicate your work as a scientist: invited talks, mainstream media, consulting, software, datasets, or on social media or online websites. However, the academic publication is the main type of communication. But papers can be produced at several stages of a project — through inception, design, development, evaluation, and finally dissemination — and not just at its end. In this brief post, I want to look at these different types of publications.

Refereed vs. Non-refereed Work

All publications can be classified depending on whether or not they have undergone formal peer review, i.e., whether a collection of the author’s colleagues (peers) have read and verified that the contents of the paper is correct. The difference is mainly that peer-reviewed work has undergone a certain level of scrutiny, giving the reader some confidence that the work is valid and correct.

Peer-reviewed papers are generally what people refer to when they talk about publications. However, in some rapidly evolving fields, such as machine learning, researchers are increasingly posting so-called “preprints” of their papers to preprint services such as arXiv, allowing others to read them even before the peer review process has concluded. The use of preprint servers in data visualization has been growing steadily, and I encourage its use.

There are also disciplines where no peer-review is the norm for all conference papers. In these cases, conference papers are not counted as “archival”, and journal papers are only what counts. In these fields, people submit conference papers mainly to go to the conference and get feedback on their work.

Types of Papers

Papers are often classified by the type of venue that it is published at. Depending on the type of the venue and the status of that particular venue, a paper will be valued more or less. Please remember that when people read your list of publications, such as for a job or for a grant, they will be implicitly valuing your papers depending on their individual values. The type of venue and the venue’s significance are important factors in this. You will always want to have a paper published in as prestigious a venue as possible. In fact, some venues can even be “bad” in that they are predatory (i.e., they cost money to publish in, which is always a red flag) or have very low quality standards. Such “bad” venues can actually have a negative impact on your whole publication list.

Here is a list of the main types of papers ordered in ascending degree of maturity and value.

Workshops

Workshops are more or less informal collections of researchers working on a common topic who meet (often at the same time as a conference) to touch base and discuss the topic. Many conferences started out as workshops and were upgraded to conference status as their popularity grew.

• Workshop position paper: an informal, often very short, statement of an individual researcher’s opinion (position) about a specific issue (often the issue that the workshop deals with). This is the canonical workshop paper, and it exists so that workshop participants can learn of the other participants’ opinion before the actual workshop or during a short session at its beginning.
• Workshop research paper: some workshops have grown from mere discussion venues into having its own technical program. Research papers submitted to a workshop are often similar to conference papers, although the level of novelty required is generally smaller than for a conference. Most workshops will include presentation slots for research papers.

Conferences

Conferences are recurring scientific events, often annual, centered around a specific research field. Symposia are generally the same thing, except at a smaller scale. In some fields, conferences can be informal and not peer-reviewed. In computer and information science, conferences generally have a full peer review process with three or more external reviewers and a low acceptance rate.

The acceptance rate is a measure of how many papers out of all of the submitted papers are actually accepted for publication. In other words, it is a measure of how selective the conference is, and people therefore use it to give an objective measure of a paper’s worth. For a conference to be regarded as selective, a small or new conference should generally have an acceptance rates below 35–40%, and a bigger and established one less than 25–30%.

• Poster: for very early work, it may be appropriate to submit a poster to a conference. Posters are exactly what they sound like: a billboard size poster with text and images that describe your work. Even though posters may be peer-reviewed at some conferences (often by a single or a pair of conference committee members), you generally submit a 1- or 2-page “poster paper” for review, and not the poster itself. The poster paper goes into a compendium, but it is not archival and thus not counted as a publication. Also, there will generally not be a presentation associated with a poster, although there is often a poster viewing session when authors are asked to stand next to their poster to present the work to viewers and answer any questions they might have.
• Short Paper: a shorter version of a full conference paper, generally around 2–4 pages, discussing a small or limited concept that would not warrant a full paper to present. Note that short papers are generally held to the same standards as full papers, i.e. the short paper should present work that is complete, finished, and mature. However, leeway is often given in things like the related works section, as well as the presentation of all details of the work. Most conferences will allocate a time slot for the author to give a talk about the paper, but it is often shorter than for long papers. Note that short papers are generally counted as archival, so publishing a short paper may restrict you from publishing a long paper on the same subject in the future.
• Full Paper: a full-length (8–10+ pages) conference paper at the technical program of the conference. This is the standard and most prestigious of all conference submissions. All full papers will be allocated a time slot during the technical program for the authors to present the work and answer questions about it.

Journals

Journals are periodical scientific publications that contain strictly peer-reviewed and consolidated articles describing mature and validated research. As with other scientific venues, however, journals come in many different flavors and levels of prestige. Because most of the scientific community aims at publishing in journals, there is a standardized system for measuring the impact factor (IF) for a journal as the annual average number of citations that articles published in the last two years in a given journal received.

Because of the “strict peer-review” part, journals generally follow a revision cycle where authors submit a paper, reviewers give feedback, and authors are then asked to prepare changes and resubmit the new revision. This will continue until the reviewers are satisfied, the authors give up, or the editor of the journal stops the process by rejecting or accepting the paper for publication.

• Journal papers: a full-length paper in a peer-reviewed journal. Journal papers also generally have no page restriction, or if they do, the page restriction is often more liberal than for conferences. Because the journal is not an actual physical event, there is no presentation associated with the publication. However, some conferences, such as ACM CHI and IEEE VIS, have partnered up with some journals to bring in authors to present their work at the technical program of the conference.
• Specialized journal papers: these include literature surveys, book reviews, conference reports, invited papers, or editorials. With the exception of the survey, most of these are rare.

Avoid Multiple Submissions!

One of the most important things to remember for all scientific publication is that most (if not all) scientific venues will not accept papers that have been published elsewhere. They will also not allow papers to be submitted for review to multiple venues at the same time. If anyone finds out that you have broken these rules — such as hedging your chances by submitting the same paper to two conferences at once — your paper will most definitely be disqualified. You may also end up on a black list and not be able to get a paper accepted to the same venue again. Don’t do it.

Naturally, there are many times when you do work that follows up on previous work, or which adds extra results about work that you have already published or submitted for review. There are “correct” ways to handle this. Read on to find out more.

What Can I Submit Where?

The above list of events and publication types are organized in increasing archival maturity. As a rule of thumb, for all of these types of papers, you are allowed to submit an improved version of published work to a higher archival tier. In other words, you are allowed to submit work that was published in a workshop to a conference, and work that was published in a conference to a journal. However, you cannot submit another journal paper for work that you already published in a journal.

Of course, once you submit work that was previously presented in a workshop to a conference, you better make sure that there is a significant improvement over the previous work or the reviewers will most likely reject your paper as not adding anything novel to the problem. Again, as a rule of thumb, you will need at least 25% extra material added to a previous publication for a higher venue to accept it — and this is only a rule of thumb, and is up to the discretion of the individual reviewers. Also make sure that you simply don’t add 25% more words to the paper, but that this extra content adds something of value. Remember to reference your previous work in your new submission and clearly discuss the improvements you have made. It is much better to do this outright, than to risk a reviewer finding your previous paper and not understanding the differences. They may reject your new submission on the basis of self-plagiarism (or, worse, plagiarism).

Adding 25% extra material may sound like a lot, especially for a conference paper that describes work that is long since finished and which you have no intention of bringing back to life. However, there are many solutions to this. One of the easiest ways is to combine several conference papers into a single journal paper. For each of the conference papers, you have clearly added more than 25% more material that was not in the conference version. Another approach may be to write about a new study, survey, or literature review.

Following the Path

This brings us to the main message of this post: following the scholarly path. In practice, this means (1) presenting your initial work at a workshop to get formative feedback, then (2) presenting a conference paper to get even more, and finally (3) summarizing your consolidated research project in a journal paper. Compare this to the alternative, where you labor in solitude at your project for a very long time, not letting anyone see your work until it’s done. In the more open approach, you reduce risk by inviting comments at every step of the process. Everyone wins, especially science.