A review has three goals, and a good review serves all three at once:

• Assess the paper against the criteria of its track, carefully and fairly.
• Help the authors. Give constructive, polite and actionable feedback that improves the work whether or not it is accepted this time.
• Inform the area chair. Give the area chair (AC, the meta-reviewer for the paper) a concise and well-argued basis for a decision.

The leading principle is the golden rule of reviewing: review for others as you would have others review for you (McPeek et al., 2009).

Why this year is different

The Web Conference 2027 received 8168 valid submissions across twelve research tracks, much more than in 2026. Every paper receives at least three reviews. At this scale the process has very little slack. One thin or late review sends a paper into the rebuttal without a complete picture, and the AC cannot compensate for hundreds of such cases at once. Most of what follows is ordinary good practice. The emphasis on early problem reports, on timeliness, and on explicit participation in the discussion follows directly from the volume.

2.1 Your commitment

• You will be assigned at most 6 papers. The review due is on 15 December 2026.
• A careful review of a research paper takes an experienced reviewer three to five hours. Plan for it.
• If you cannot meet the deadline, tell your AC as soon as you know. Reassigning a paper in the first week is easy. Reassigning it in the last 48 hours is not, and the authors pay for the delay.
• You may be asked to serve as an emergency reviewer for a paper that lost a reviewer. Please answer such requests within 24 hours, even if the answer is no.

2.2 Check expertise and conflicts in the first 48 hours

Open each assigned paper and read the abstract and introduction. Ask yourself whether you can judge the central technical claim. If you cannot, tell your AC now. You may still be asked to review a paper adjacent to your expertise; in that case say so in your confidence score and in the confidential comments.

You have a conflict of interest with a paper if any of the following holds:

• you are an author, or you contributed to the work;
• you have co-authored with any author in the past three years, or have an active collaboration or a joint grant;
• you were the advisor or advisee of an author at any time;
• you are at the same institution or company as an author, or were in the past three years, including different campuses, labs or subsidiaries.
• you have a close personal or financial relationship with an author;
• for any other reason you could not be impartial, or a reasonable observer would doubt that you could.

OpenReview detects many conflicts automatically, but not all. If you discover a conflict, declare it in the system and decline the paper. Do not try to work out who the authors are. If you recognise the work anyway, from a preprint or a talk, you may still review it unless one of the conflicts above applies, but say what you know in the confidential comments to the AC.

2.3 Confidentiality

Submissions are confidential. This is the basis on which authors send us unpublished work.

• Do not share a submission, or discuss it, with anyone outside the review system other than your AC.
• If you want to involve a student or colleague as a secondary reviewer, ask your AC first, name the person in the confidential comments, and make sure they have read this document. You remain responsible for the review.
• Do not use ideas, data, code or results from a submission in your own work until the paper is public.
• Do not upload a submission, or any part of it, to any external service. Section 3 says more about this.
• Delete local copies of the papers after the decisions are announced.

2.4 Double-blind review

Reviewing is double-blind. Do not search for the authors. If a paper reveals the authors’ identity, through names, acknowledgements, first-person references to identifiable prior work, or links to non-anonymised repositories, report it to your AC straight away and continue with the review of the science. The chairs decide whether a violation leads to a desk rejection. Preprints on servers such as arXiv are basically avoided under the 2027 submission policy; do not go looking for them.

Reviewing for The Web Conference 2027 must be done by you, without generative AI. This is the single most important rule in this document.

Do not put any part of a submission into any AI tool. This includes the abstract, the text, figures, tables, equations, code, data, supplementary material, and your own notes if they quote or closely paraphrase the paper. It includes chat assistants such as ChatGPT, Claude, Gemini and Copilot; AI search engines; AI translators; the “summarise” and “ask about this document” features built into PDF readers, browsers, email clients, note-taking apps and operating systems; and any API or local model that sends data to a third party. “Enterprise”, “private” or “no training” settings do not change the rule. The breach is disclosure to a party outside the review process, not training.

Why we are strict about this

• Confidentiality. The authors gave us their unpublished work on the understanding that only the reviewers, the AC and the chairs would see it. AI services log inputs, retain them, may have them read by staff, and can be breached. Many authors’ institutions and funders forbid this disclosure, and so does the confidentiality that ACM peer review requires.
• Quality. Language models produce fluent, generic critiques. They misread specifics, invent references, cannot judge novelty against the literature you know, and tend to flatter. The weaknesses they list usually apply to any paper and so help no author and no AC.
• Accountability. You were invited because of your expertise, and a review is an expert opinion that you may be asked to defend in the discussion. You cannot defend a judgment you did not make.
• Fairness. A paper reviewed by a model gets a lottery, not a judgment. The authors can tell, and increasingly they say so in the rebuttal.

How this is enforced

• The review form asks you to confirm that you read the full paper and that no AI tool was used on the submission or on the review.
• ACs read every review and screen for the signatures of generated text: generic content, errors about what the paper contains, invented references, template structure, questions the paper already answers. Authors may also flag a review in the rebuttal. Where there is a suspicion, you will be asked to explain specific points of your review.
• A review that the chairs conclude was AI-generated, or that was produced by uploading the paper to an AI tool, is discarded and replaced. The reviewer is removed from the 2027 committee, excluded from future Web Conference committees, and in serious cases reported to ACM.

If you are tempted because you are out of time, tell your AC instead.

AI-generated submissions

Some submissions will themselves be largely machine-written. Signs include references that do not exist, results that cannot be traced to any described experiment, inconsistent notation, and prose that says little. Judge such a paper on its merits like any other, check doubtful references yourself in a bibliographic database, describe concrete problems in the review (for example, “reference [23] does not appear to exist”), and state the suspicion in the confidential comments. The 2027 author policy permits AI writing aids. Authors remain fully responsible for all content and must follow ACM’s authorship policy on disclosure of AI use in research.

Step 1: Know the form and the criteria

The review form in OpenReview has these fields:

Field	What it is for
Strengths	What the paper does well, specifically
Weaknesses	The problems, in order of importance, each located in the paper
Questions for the authors	Numbered, answerable in the rebuttal, tied to the weaknesses
Overall recommendation	Your recommendation on the scale in the form
Reviewer confidence	How well placed you are to judge this paper
Confidential comments	For the AC and the chairs only

Step 2: First pass (30 to 45 minutes)

Skim the paper with four questions in mind: Is it in scope for the track? What is the first impression? Is there a potential fatal flaw? Is there anything the AC must know now?

From the abstract alone you should understand the aims, the key data or setting, and the conclusions. If you do not, note that the abstract needs work. Take notes as you go.

Report to your AC immediately, without waiting for the review deadline, if you find:

• an anonymity violation;
• a paper that is out of scope for every track (a paper in the wrong track can often be transferred);
• a page limit or template violation;
• suspected plagiarism, or a paper substantially identical to one under review or published elsewhere;
• a paper you have reviewed before at another venue (you may review it again, but tell the AC and judge the current version on its own);
• a conflict you did not notice at assignment.

Do not go hunting for a minor issue to inflate into a major one. Most papers have no fatal flaw. If one does, specific suggestions for how to fix it are very welcome.

Step 3: Thorough read (two to four hours)

Read with the criteria in front of you. Mark every place where the meaning is unclear, where there is a factual error, where an argument is invalid, and where a claim is unsupported. Then check:

• Is each contribution claimed in the introduction actually delivered?
• Do the experiments test the claims made, or something easier?
• Are the baselines the ones an expert would expect, and were they given a fair chance?
• Could the result be explained by leakage, by tuning on the test set, or by an unfair budget?
• Are limitations acknowledged?
• Does the related work treat prior work fairly? (Whether you are cited is not the question.)
• Are the title and abstract accurate descriptions of what was done?
• Reading the appendices and supplementary material where they bear on the claims is strongly encouraged. Many “missing” details are there.

Step 4: Write the review

Strengths. At least two or three, specific. Every paper has strengths. Say what is good enough to build on.

Weaknesses. In order of importance. For each: what the problem is, where it is (section, table, equation), why it matters for the claims, and how serious it is: does it undermine the main claim, limit the scope, or is it fixable in a revision? Say what would resolve it.

Questions for the authors. Numbered and answerable within a one-page rebuttal. Ask the questions whose answers could change your score, and say which weaknesses they correspond to. Do not ask for new experiments that cannot be done in a week unless they are essential; if they are essential, say so.

Scores. The text is primary and the scores must agree with it. The overall recommendation uses a scale with no middle value, so that every reviewer takes a position.

Confidence: This helps to further support the scores.

Confidential comments to the AC. Use this for what the authors should not see: a conflict you noticed, a suspicion of misconduct, your own uncertainty, the name of a secondary reviewer, an ethics escalation. Do not put the real reasons for your score only here. The authors are entitled to see them.

Writing well

Write plainly, politely and concretely, in a way that a reader whose first language is not English can follow. Critique the work, not the people. Avoid sarcasm, “obviously”, “clearly the authors did not”, and any speculation about who the authors are or why they did something. Write as if you were going to sign it. A good review of a research paper is typically 500 to 1,000 words. Under about 300 words is rarely adequate; over 1,500 rarely helps.

Mistakes that force the AC to discount a review

• “Not novel” without naming the prior work that does the same thing.
• “More experiments are needed” without saying which experiments and what they would show.
• A score driven by writing or language problems that can be fixed before the camera-ready.
• Reviewing the paper you would have written instead of the paper that was submitted.
• Demanding comparison with work that appeared less than three months before the deadline, or concurrently.
• Penalising the authors for not citing you. Mention the work if it is relevant, neutrally.
• Deciding first and then hunting for reasons, or defaulting to reject because it feels safe. The safe choice is the correct choice, argued.
• One-paragraph reviews, and blocks of text reused across papers.
• Scores that do not match the text.
• Factual errors about the paper. Before you criticise something as missing, check the appendix.
• Raising an ethics issue only as a weakness instead of also raising the flag.

Step 5: Re-read before you submit

• Is the review clear and easy to understand?
• Is the tone the one you expect in reviews you receive?
• Is it constructive? Does every major weakness come with a concrete way to address it?
• Is the structure easy to follow? Will the AC grasp your assessment and your reasoning in a minute?
• Could you defend every claim in the review in the discussion?
• Have you distinguished what must change from what would be nice? Prefer the fix that brings clarity with the least disruption to the core content. If the first paragraph does not match the rest of the paper, suggest rewriting the paragraph, not the paper.
• After reading your review, will the authors have what they need to improve the paper, even if not in time for this conference? If the answer is no, your job is not finished.

Step 6: Rebuttal and discussion

Decisions are made in the discussion. The AC’s meta-review must reflect each reviewer’s considered position, and that position must be stated, not inferred.

• Write your review with the rebuttal in mind: it is your chance to get clarifications from the authors.
• Read the rebuttal within 48 hours of its release, with your review next to it. Read the other reviews as well.
• Post at least one message in the discussion saying whether your concerns were addressed, what remains, and whether your score changes. If it changes, update it, and add a short “Post-rebuttal” paragraph to your review explaining why. If it does not change although the authors answered, say why not.
• Engage with disagreement respectfully. If you and another reviewer differ, explain your reasoning and ask them to be specific (“R2, you mention missing baselines; which ones would change your view?”). Be open to changing your mind in either direction. Do not change a score just to converge; the AC does not need unanimity, they need reasons.
• Answer questions from your AC within 48 hours.
• Do not hold the rebuttal against the authors for failing to address things you did not ask.
• If you do not respond, the AC will treat your review as not updated and may give it less weight, and repeated non-response is recorded for future committee invitations.

Raise the ethics flag, with a description, when you see:

• research with people or with personal data without an ethics review, consent, or equivalent safeguards;
• sensitive data handled without protection, or collected in ways that cause real harm;
• potential for misuse that the paper does not discuss;
• plagiarism, including of the authors’ own published work beyond what the policy allows;
• dual submission or prior publication;
• signs of fabricated data or results, including references that do not exist.

Flagging does not decide the paper; it triggers a check by the AC and, if needed, the program chairs. Describe facts, not accusations, and keep the detail in the confidential comments.

Confidentiality does not end with the notification. Do not contact the authors about the review, do not post about the papers, and delete your copies. Once a paper is published you may of course cite it.