Instructions to ECCV2012 Referees

REVIEWER INSTRUCTIONS

DEADLINE: You have until 11:59 GMT, May 11, 2012, to complete your reviews. Your assignments are on CMT: https://cmt.research.microsoft.com/ECCV2012/Default.aspx

 

IMPORTANT: As soon as possible please log into CMT to check that you have no obvious conflict with a paper assigned to you. If there is a problem, please contact the program chairs immediately at eccv-2012-program-chairs@googlegroups.com.

 

Your task is to select the most interesting submissions for ECCV 2012 and to help the authors improve their work, whether or not it is accepted to ECCV 2012. Please use the following instructions as a guide, and refer to CVPR 2012 instructions for further details regarding important issues such as reviewer ethics.

 

The first question on the review form asks for an “executive summary” of the paper’s content and of your recommendation. This summary should clearly convey to the authors and to the area chair that you have carefully read the paper and understood it. To provide the area chair with the most balanced view, we suggest that you also give a concise list of the main strengths and weaknesses of the paper.

 

The subsequent questions ask you to rate and elaborate on the three major components of your recommendation: novelty, technical correctness/soundness, and experimental validation. Question 8 gives you space to comment on additional issues such as clarity (or lack thereof), the breadth of the paper’s appeal, as well as any other detailed remarks about the paper or the supplementary material.

 

Above all, please be careful to avoid unfairness in your responses.

  • When you make a judgment about a paper you ALWAYS need to explain it; for example, rather than saying “The experiment is bad” say “The experiment is bad, because…”
  • Do not say that something has been done before without precise and correct specifics (i.e., who did it and where, meaning paper reference, page number, equation number).
  • Do not claim that a proof is wrong without showing what the error is (e.g., by identifying a counterexample or a logically false step).
  • Avoid any language that can be perceived as unprofessional or abusive.
  • Do not downgrade a paper simply because it doesn’t cite your work, because the authors seem to be direct competitors, or because you wish you had thought of the idea first.
  • Conversely, do not advance a paper simply because it cites you.

 

Sometimes the most interesting papers have flaws. Are the flaws fixable using your feedback? If so, give the authors the benefit of the doubt. For example, a theorem may still be correct even if its proof contains a minor error; the central idea of the paper may be groundbreaking even if the design of the experiments needs work, etc.

 

Question 9 includes an expanded rating scale which explicitly recognizes that oral presentations are considered to be awarded to higher quality papers. This was ambiguous in previous conferences, where some reviewers used “definite accept” only for oral-quality papers, and others used it as instructed: for both posters and orals. We believe that making this explicit will remove some intrinsic variance in the review scores. Remember to dissociate your rating from your confidence in the rating, which is handled in question 10.

 

Question 11 gives you space to add any confidential comments to the committee. Fairness requires that such comments appear only in exceptional situations, because authors won’t see them and so can’t rebut them or understand their influence on the final decision. If this review was done for you by another person, please enter their name here. In all such cases, you should make sure that you can stand by the review’s contents and serve as the point of contact for any subsequent discussion.

 

Finally, question 12 deals with adherence to submission guidelines (/submissions/call-for-paper/paper-submission/), including double submission, anonymity, and supplemental material. When judging novelty, it is certainly acceptable, and indeed expected, to use a search engine to identify related papers. If you find a very closely related paper which has been published in full, you may be inclined to immediately assume it is by the same authors, or that it is a case of plagiarism. However, in each case, the rule is the same: the submission should cite that work, and discuss why the new submission is different. If you suspect dual submission or plagiarism, contact the program chairs, but continue to review the submission as if guidelines were followed until the PCs say otherwise.