CIS 6930 Spring 26

Logo

Data Engineering at the University of Florida

Assignment 0: Peer Review Rubric

Total Points: 50

Before You Start Reviewing

  1. Clone the repository: git clone <repo-url>
  2. Install dependencies: uv sync
  3. Copy .env.example to .env and add your NavigatorAI API key
  4. Run the application with the documented commands
  5. Run the tests: uv run pytest -v
  6. Check GitHub Actions status (look for green checkmark)

Important: If you cannot clone or run the repository, document the error and score accordingly.

Scoring Scale

Score Meaning
5 Excellent - Exceeds expectations
4 Good - Meets all expectations
3 Satisfactory - Meets most expectations with minor issues
2 Needs Work - Missing key elements or significant issues
1 Incomplete - Major problems or missing components
0 Missing - Component not present

Criterion 1: API Data Collection (15 points)

Does the application successfully fetch data from an external API?

Scoring Guide

Score Points Description
5 15 Fetches data reliably; handles all errors gracefully; parses JSON correctly; extracts meaningful fields
4 12 Fetches data successfully; handles most errors; good data parsing
3 9 Fetches data but error handling is incomplete; some parsing issues
2 6 Fetches data but crashes on errors; poor parsing
1 3 API calls fail or return unusable data
0 0 No API integration present

What to Check

Examples

Excellent (5 points = 15):

def fetch_earthquake_data(days: int = 7) -> dict:
    """Fetch earthquake data from USGS API."""
    try:
        response = requests.get(
            "https://earthquake.usgs.gov/fdsnws/event/1/query",
            params={"format": "geojson", "starttime": start_date},
            timeout=30
        )
        response.raise_for_status()
        return response.json()
    except requests.exceptions.Timeout:
        logger.error("API request timed out")
        raise
    except requests.exceptions.HTTPError as e:
        logger.error(f"HTTP error: {e.response.status_code}")
        raise

Needs Work (2 points = 6):

def fetch_data():
    # No timeout, no error handling, hardcoded values
    response = requests.get("https://api.example.com/data")
    return response.json()

Red Flags (Deduct Points)

Criterion 2: LLM Processing (15 points)

Does the application use NavigatorAI to process and summarize the API data meaningfully?

Scoring Guide

Score Points Description
5 15 LLM produces insightful analysis; well-crafted prompts; clear output formatting
4 12 LLM processing works well; reasonable prompts; readable output
3 9 LLM processing works but prompts are basic; output formatting could improve
2 6 LLM is called but results are not meaningful or useful
1 3 LLM integration is broken or produces errors
0 0 No LLM integration present

What to Check

Examples

Excellent (5 points = 15):

def summarize_earthquakes(data: dict) -> str:
    """Use LLM to analyze earthquake patterns."""
    prompt = f"""Analyze this earthquake data and provide:
    1. The most significant earthquake (highest magnitude)
    2. Geographic patterns (are earthquakes clustered?)
    3. Any trends in frequency over the time period

    Data: {json.dumps(data[:10])}  # Limit data size

    Provide a concise summary suitable for a news brief."""

    response = call_navigator_api(prompt)
    return response

Needs Work (2 points = 6):

def summarize(data):
    # Vague prompt, no structure, no insight requested
    prompt = f"Summarize this: {data}"
    return call_api(prompt)

Red Flags (Deduct Points)

Criterion 3: Command-Line Interface (10 points)

Is the CLI well-designed and user-friendly?

Scoring Guide

Score Points Description
5 10 Clear arguments; helpful --help; sensible defaults; input validation
4 8 Good CLI with clear arguments; --help works; minor issues
3 6 Basic CLI works; --help is minimal; some confusing options
2 4 CLI is confusing; missing arguments; poor help text
1 2 CLI barely works or is very confusing
0 0 No CLI or completely broken

What to Check

Examples

Excellent (5 points = 10):

$ uv run python -m assignment0 --help
usage: assignment0 [-h] --source {usgs,news,books} [--days DAYS] [--query QUERY]

Fetch API data and summarize with LLM.

options:
  -h, --help            show this help message and exit
  --source {usgs,news,books}
                        Data source to fetch from
  --days DAYS           Number of days of data (default: 7)
  --query QUERY         Search query (for news source)

Examples:
  uv run python -m assignment0 --source usgs --days 7
  uv run python -m assignment0 --source news --query "AI"

Needs Work (2 points = 4):

$ uv run python -m assignment0 --help
usage: assignment0 [-h] s d
# No descriptions, cryptic argument names

Red Flags (Deduct Points)

Criterion 4: Testing (5 points)

Are there adequate tests that verify functionality?

Scoring Guide

Score Points Description
5 5 Comprehensive tests; mocks external APIs; tests parsing, CLI; all pass
4 4 Good coverage; main functionality tested; tests pass
3 3 Basic tests present; some gaps; tests pass
2 2 Few tests; major functionality untested
1 1 Tests present but fail or are trivial
0 0 No tests

What to Check

Examples

Excellent (5 points):

from unittest.mock import patch, MagicMock
from assignment0.api import fetch_earthquake_data

@patch("assignment0.api.requests.get")
def test_fetch_earthquake_data(mock_get):
    """Test that earthquake data is fetched and parsed correctly."""
    mock_response = MagicMock()
    mock_response.json.return_value = {
        "features": [{"properties": {"mag": 5.2}}]
    }
    mock_get.return_value = mock_response

    result = fetch_earthquake_data(days=7)

    assert len(result["features"]) == 1
    mock_get.assert_called_once()

@patch("assignment0.api.requests.get")
def test_fetch_handles_timeout(mock_get):
    """Test that timeout errors are handled gracefully."""
    mock_get.side_effect = requests.exceptions.Timeout()

    with pytest.raises(requests.exceptions.Timeout):
        fetch_earthquake_data(days=7)

Needs Work (2 points):

def test_something():
    # No mocking - calls real API
    result = fetch_data()
    assert result is not None  # Trivial assertion

Red Flags (Deduct Points)

Criterion 5: CI/CD and Project Structure (5 points)

Is the project properly structured with working CI?

Scoring Guide

Score Points Description
5 5 Clean package structure; CI passes; secrets configured; .gitignore correct
4 4 Good structure; CI works; minor issues
3 3 Acceptable structure; CI has issues but runs
2 2 Poor structure; CI broken
1 1 Minimal structure; no working CI
0 0 No CI; completely disorganized

What to Check

Examples

Excellent (5 points) - Project Structure:

cis6930sp26-assignment0/
├── .github/workflows/pytest.yml  ✓ CI configured
├── assignment0/
│   ├── __main__.py               ✓ Entry point
│   ├── api.py
│   ├── llm.py
│   └── cli.py
├── tests/
│   ├── test_api.py
│   └── test_llm.py
├── .env.example                  ✓ Documents required vars
├── .gitignore                    ✓ Includes .env
├── pyproject.toml                ✓ Proper packaging
└── README.md                     ✓ Documents usage

Needs Work (2 points):

assignment0/
├── main.py           ✗ No package structure
├── test.py           ✗ Tests not in tests/ folder
└── requirements.txt  ✗ Should use pyproject.toml

Red Flags (Deduct Points)

Criterion 6: Completing Peer Reviews (5 points)

Did you complete your assigned peer reviews?

Score Points Description
5 5 Completed both reviews thoroughly with constructive feedback
3 3 Completed both reviews but feedback was minimal
1 1 Completed only one review
0 0 Did not complete reviews

This criterion is graded by the instructor based on your submitted reviews.

Review Submission Template

Copy this template for each review you submit:

# Assignment 0 Peer Review

**Repository:** [GitHub URL]
**Reviewer:** [Your Name]
**Date:** [Date]

## Setup Verification

- [ ] Successfully cloned repository
- [ ] `uv sync` completed without errors
- [ ] `uv run pytest -v` executed (note: pass/fail)
- [ ] Checked GitHub Actions status

## Scores

| Criterion | Score (0-5) | Points |
|-----------|-------------|--------|
| 1. API Data Collection | /5 | /15 |
| 2. LLM Processing | /5 | /15 |
| 3. Command-Line Interface | /5 | /10 |
| 4. Testing | /5 | /5 |
| 5. CI/CD & Structure | /5 | /5 |
| **Total** | | **/50** |

## Strengths

List 2-3 things the submission does well:

1.
2.
3.

## Areas for Improvement

List 2-3 specific, actionable suggestions:

1.
2.
3.

## Issues Found

Describe any bugs, errors, or problems:

-

## Code Quality Notes

Comments on readability, organization, or style:

-

## Additional Comments

Any other feedback for the author:

Grading Summary

Component Points
API Data Collection 15
LLM Processing 15
Command-Line Interface 10
Testing 5
CI/CD & Structure 5
Subtotal (Implementation) 45
Completing Peer Reviews 5
Total 50

Tips for Reviewers

  1. Be constructive - Focus on helping the author improve, not just finding faults
  2. Be specific - “Error handling could be improved” is less helpful than “Add try/except around the API call in api.py:25”
  3. Test thoroughly - Don’t just skim the code; actually run the application
  4. Check the basics - Clone, install, run tests, check CI status
  5. Note security issues - Always flag hardcoded secrets or committed .env files

back