Paper Review Rubric

Use this rubric when reviewing paper drafts and final papers. This follows conference-style peer review.

Total Points:

Draft Paper (Week 11): 100 points
Final Paper (Week 13): 400 points

Point Conversion

Score	Points (per criterion)	Description
5	100% of weight	Excellent
4	80% of weight	Good
3	60% of weight	Satisfactory
2	40% of weight	Needs Work
1	20% of weight	Incomplete

Scoring Scale

Score	Meaning	Conference Equivalent
5	Excellent	Strong Accept
4	Good	Accept
3	Satisfactory	Weak Accept / Borderline
2	Below Average	Weak Reject
1	Poor	Reject

Criteria

1. Originality (20% = 20/80 points)

Does the paper make a novel contribution?

Score	Description
5	Highly original; significant new insights or methods
4	Good novelty; clear contribution beyond prior work
3	Some novelty; incremental contribution
2	Limited novelty; mostly replicates existing work
1	No apparent novelty

Guiding Questions:

What is the main contribution of this paper?
How does it differ from prior work?
Is the contribution significant enough to warrant publication?

2. Technical Quality (25% = 25/100 points)

Is the methodology sound and the evaluation rigorous?

Score	Description
5	Rigorous methodology; comprehensive evaluation; solid results
4	Sound methodology; good evaluation
3	Reasonable approach; evaluation has some gaps
2	Methodology has flaws; evaluation insufficient
1	Fundamentally flawed approach

Guiding Questions:

Is the methodology clearly described and justified?
Are the experiments well-designed?
Are the baselines appropriate?
Are the results statistically valid?
Are limitations acknowledged?

3. Clarity (20% = 20/80 points)

Is the paper well-written and easy to understand?

Score	Description
5	Exceptionally clear; well-organized; engaging
4	Clear writing; good organization
3	Understandable but could be clearer
2	Difficult to follow; organizational issues
1	Incomprehensible

Guiding Questions:

Is the paper easy to read?
Is it well-organized with a logical flow?
Are figures and tables clear and informative?
Are technical concepts explained adequately?

4. Significance (20% = 20/80 points)

Does this work address an important problem?

Score	Description
5	Addresses critical problem; high potential impact
4	Important problem; good potential impact
3	Moderately important; some practical value
2	Limited significance; narrow scope
1	Trivial problem or no clear value

Guiding Questions:

Why does this problem matter?
Who would benefit from this work?
What is the potential real-world impact?

5. Reproducibility (15% = 15/60 points)

Can the results be reproduced by others?

Score	Description
5	Fully reproducible; code/data available; detailed methods
4	Mostly reproducible; minor details missing
3	Partially reproducible; some gaps
2	Difficult to reproduce; key details missing
1	Not reproducible

Guiding Questions:

Is the code available?
Is the data available or described sufficiently?
Are all experimental details provided?
Could you replicate the main results?

Review Template

## Paper Review: [Paper Title]

**Reviewer:** [Your Name]
**Confidence:** [High / Medium / Low] (How well do you know this area?)

### Overall Recommendation
[Strong Accept / Accept / Weak Accept / Weak Reject / Reject]

### Scores

| Criterion | Score (1-5) | Weight | Draft Points | Final Points |
|-----------|-------------|--------|--------------|--------------|
| Originality | | 20% | /20 | /80 |
| Technical Quality | | 25% | /25 | /100 |
| Clarity | | 20% | /20 | /80 |
| Significance | | 20% | /20 | /80 |
| Reproducibility | | 15% | /15 | /60 |
| **Total** | | 100% | **/100** | **/400** |

### Summary
[3-4 sentence summary of the paper in your own words]

### Strengths (at least 3)
1.
2.
3.

### Weaknesses (at least 3)
1.
2.
3.

### Questions for Authors
1.
2.
3.

### Detailed Comments

#### Technical Issues
-

#### Presentation Issues
-

#### Missing References
-

### Minor Comments
[Typos, formatting issues, small suggestions]

### Recommendation Summary
[1-2 paragraphs explaining your overall assessment and what would improve the paper]

Conference Review Standards

When writing reviews, follow these principles:

Be Constructive

Focus on how to improve the work, not just what’s wrong
Suggest specific fixes for identified problems
Acknowledge what the authors did well

Be Specific

Reference specific sections, figures, or equations
Provide concrete examples of issues
Avoid vague criticisms like “needs more work”

Be Professional

Use respectful language
Critique the work, not the authors
Assume the authors made their best effort

Be Thorough

Read the entire paper carefully
Check the related work section
Verify claims against the data/results
Consider the supplementary materials if available

Common Review Mistakes to Avoid

Superficial reading - Skimming and missing key details
Unrealistic expectations - Expecting perfection from a course project
Scope creep - Suggesting the paper should solve different problems
Missing the contribution - Criticizing for not doing something it wasn’t trying to do
Ignoring context - Not considering the target venue and audience

back