Skip to yearly menu bar Skip to main content


Guidance for Area Chairs

Area Chairs play a critical role in curating the technical programme for ICLR. Use this as a resource for any questions related to your role as an Area Chair.

Thank you for your critical contribution.

Timelines for ICLR 2019

  • November 2: Review deadline

  • November 5 - November 26: Author rebuttals

  • November 26 - December 9: Discussion between ACs, reviewers, authors

  • December 13: AC recommendations  

  • PC email: iclr2019programchairs@googlegroups.com

 

Reviewing Process and Guidance

  • Facilitate discussion. A key part of your role during the review period is to actively facilitate discussion among reviewers, with the aim of clarifying aspects of the papers’ claims, and other questions that have been raised, in fair and respectful ways.  Encourage the reviewers to consider the other reviews, and see if this can lead them toward consensus (but don’t force consensus if there is true disagreement).  

  • Active participation needed. One of the unique features of the ICLR review process is that authors can directly address reviewer concerns, both through discussion and by revising their papers. However, in order for this process to work, it is critical that the reviewers actually read the author responses and participate in the discussion!

  • Moderate comments. We ask that you take responsibility for public reviews in your set of papers. If you feel that any comments unfairly bias a paper, or are inappropriate, we are able remove them from the review process. Please reach out to us in such cases.

  • The AC is encouraged to take active part in the discussion. Remember that your goal during the rebuttal period (which includes a discussion with the authors) is to gather the information that is necessary to make an informed decision. If you have particular points that you would like the reviewers or authors to address, don't be shy about posting a comment or question on the paper. The content of the meta-review should not come as a surprise to the authors: if an issue that you believe is salient to the decision on the paper has not been raised by the reviewers already, go ahead and raise it! It is better to give the authors the opportunity to respond and reviewers the opportunity to weigh in.

Meta-review Template

We recommend taking a look at this structure before the discussion phase, to ensure that the discussion with the reviewers, as well as between reviewers and authors, allows you to address each of the points in the meta-review template. If the reviews are insufficiently detailed, please make sure there is adequate discussion to allow you to address all of these points with high confidence. Italicized text shows some examples of how each point might be addressed. Not all responses need to be this detailed, but it can be particularly important for borderline papers:

1. Describe the strengths of the paper.  As pointed out by the reviewers and based on your expert opinion. Example:

  • The paper presents a promising new algorithm for training generative adversarial networks. The mathematical foundation for the method is novel and thoroughly motivated, the theoretical results are non-trivial and correct, and the experimental evaluation shows a substantial improvement over the state of the art.

 

2. Describe the weaknesses of the paper. As pointed out by the reviewers and based on your expert opinion. Be sure to indicate which weaknesses are seen as salient for the decision (i.e., potential critical flaws), as opposed to weaknesses that the authors can likely fix in a revision. Example:

  • The reviewers and AC note the following potential weaknesses: (1) The evaluation is missing a comparison to Smith et al. ‘16, which is important in order to understand the benefits of the proposed XYZ-GAN when compared to closely related prior stabilization methods; (2) The notation in the methods section is difficult to understand and non-standard -- see reviewer 1’s suggestions for how to improve; (3) The experiments only compare on the CelebA dataset, comparisons on other datasets would substantially strengthen the evaluation. Among these, (2) and (3) did not have a substantial impact on the decision, but would be helpful to address in a subsequent revision. However, (1) makes it very difficult to assess the benefits of the proposed approach, and was viewed by the reviewers and AC as a critical issue.

 

3. Discuss any major points of contention. As raised by the authors or reviewers in the discussion, and how these might have influenced the decision. If the authors provide a rebuttal to a potential reviewer concern, it’s a good idea to acknowledge this and note whether it influenced the final decision or not. This makes sure that author responses are addressed adequately. Example:

  • The authors discussed in their response to the reviews that a comparison to XYZ-GAN was not suitable to include in this paper, since XYZ-GAN provides a modification to the standard GAN objective that is orthogonal to the main contribution in this paper. However, reviewer 1 disagreed, since the XYZ-GAN technical derivation is quite similar to the proposed technique. In considering the author response and reviewer comments, the AC decided that this comparison was indeed important for understanding the contribution in this work, and it is difficult to assess the scope of the contribution without such a comparison.

 

4. If consensus was reached, say so. Otherwise, explain what the source of reviewer disagreement was and why the decision on the paper aligns with one set of reviewers or another.

  • The reviewers disagreed on the importance of the comparison with the XYZ-GAN. While reviewer 1 stated that this comparison was crucial for understanding the paper’s contribution, reviewer 2 and 3 considered such a comparison to be useful but not essential. In considering this point, the AC decided that the paper could be accepted without such a comparison, but the authors are strongly urged to include the comparison in the final revision if possible.

FAQs

This section summarizes answers to common questions that authors may ask you, and common questions from ACs.

Revising parts of papers

  • The authors may revise their papers during the rebuttal period and after the review period has concluded.

Emergency Reviewers

  • If, for any reason, a reviewer is unresponsive or misses the deadline, you may need to get an emergency review.

  • Please be proactive, and do not wait for too long. It is unfair to authors to not receive timely reviews at the start of the rebuttal period. We recommend requesting emergency reviews if a review is not finished three days after the review deadline.

  • It is easy to add new reviewers to the system. Just let us know their names.

  • If you struggle to find an emergency reviewer, please get in touch with the PCs as soon as possible, and we will assist.