claude-gen/fission-python/template/docs/TESTING.md

# Testing Guide

This document covers testing strategies and best practices for Fission Python functions.

## Table of Contents

1. [Test Types](#test-types)
2. [Dependencies](#dependencies)
3. [Unit Testing](#unit-testing)
4. [Integration Testing](#integration-testing)
5. [Test Database](#test-database)
6. [Mocking](#mocking)
7. [Fixtures](#fixtures)
8. [Coverage](#coverage)
9. [Running Tests](#running-tests)
10. [CI/CD Integration](#cicd-integration)

## Test Types

### Unit Tests

Test individual functions in isolation, mocking external dependencies:
- Database calls
- HTTP requests
- File I/O
- External services

**Goal**: Verify business logic correctness without infrastructure.

### Integration Tests

Test the function with real (or test) dependencies:
- Actual database queries
- End-to-end request/response flow
- Real configuration loading

**Goal**: Verify integration points work correctly.

## Dependencies

Install test dependencies:

```bash
pip install -r test/requirements.txt
# Or for dev (includes both runtime and test deps):
pip install -r dev-requirements.txt
```

Required packages:
- `pytest` - Test framework
- `pytest-mock` - Mocking utilities (provides `mocker` fixture)
- `requests` - For integration tests making HTTP calls

## Unit Testing

### Example Test Structure

```python
# test/test_my_function.py
import pytest
from unittest.mock import patch, MagicMock
from src.my_function import create_item
from exceptions import ValidationError

def test_create_item_success():
    """Test successful item creation."""
    # Arrange
    mock_conn = MagicMock()
    mock_cursor = MagicMock()
    mock_conn.cursor.return_value = mock_cursor
    mock_cursor.fetchone.return_value = ("item-id", "Item Name", "active")

    # Mock init_db_connection to return our mock
    with patch("src.my_function.init_db_connection", return_value=mock_conn):
        # Create a mock Flask request
        with patch("src.my_function.request") as mock_request:
            mock_request.get_json.return_value = {
                "name": "Test Item",
                "status": "active"
            }
            mock_request.view_args = {}

            # Act
            result = create_item({}, {})

            # Assert
            assert result["id"] == "item-id"
            assert result["name"] == "Test Item"
            mock_cursor.execute.assert_called_once()
            mock_conn.commit.assert_called_once()

def test_create_item_validation_error():
    """Test validation of missing required fields."""
    with patch("src.my_function.request") as mock_request:
        mock_request.get_json.return_value = {"name": ""}  # Empty name

        with pytest.raises(ValidationError) as exc_info:
            create_item({}, {})

        assert "validation" in str(exc_info.value.error_msg).lower()
```

### Mocking Helpers

Use `patch` to replace dependencies:

```python
# Mock helpers.get_secret
@patch("src.my_function.helpers.get_secret")
def test_with_mocked_secret(mock_get_secret):
    mock_get_secret.return_value = "localhost"
    # Test code...

# Mock entire module
@patch("src.my_function.helpers.init_db_connection")
def test_with_mocked_db(mock_init_db):
    mock_conn = MagicMock()
    mock_init_db.return_value = mock_conn
    # Test code...
```

### Mocking Flask Request

```python
from flask import Request

def test_with_flask_request():
    with patch("src.my_function.request") as mock_request:
        mock_request.get_json.return_value = {"key": "value"}
        mock_request.args.getlist.return_value = []
        mock_request.headers.get.return_value = "user-123"
        # Test code...
```

## Integration Testing

### Test Database Setup

Use a separate test database:

```bash
# Create test database
createdb fission_test

# Or with Docker:
docker run -d -p 5433:5432 -e POSTGRES_PASSWORD=test postgres:15
```

Set environment variables for test database:
```bash
export PG_HOST=localhost
export PG_PORT=5433
export PG_DB=fission_test
export PG_USER=postgres
export PG_PASS=test
```

### pytest Fixtures for Database

```python
# conftest.py (placed in test/ directory)
import pytest
import psycopg2
from helpers import init_db_connection

@pytest.fixture(scope="session")
def db_connection():
    """Create a database connection for the entire test session."""
    conn = init_db_connection()
    yield conn
    conn.close()

@pytest.fixture(scope="function")
def db_cursor(db_connection):
    """Create a cursor for each test, with transaction rollback."""
    conn = db_connection
    cursor = conn.cursor()
    # Start a transaction that will be rolled back
    conn.rollback()
    yield cursor
    # Rollback after each test to keep DB clean
    conn.rollback()
```

### Example Integration Test

```python
# test/test_integration.py
def test_create_and_retrieve_item_integration(db_connection):
    """Test full CRUD cycle with real database."""
    from src.models import ItemCreateRequest
    from src.functions import create_item, get_item

    # Insert test data
    cursor = db_connection.cursor()
    cursor.execute("DELETE FROM items WHERE name = 'Integration Test'")
    db_connection.commit()

    # Create item via function
    with patch("src.functions.request") as mock_request:
        mock_request.get_json.return_value = {
            "name": "Integration Test",
            "description": "Test item"
        }
        mock_request.view_args = {}
        result = create_item({}, {})

    item_id = result["id"]
    assert result["name"] == "Integration Test"

    # Retrieve same item
    with patch("src.functions.request") as mock_request:
        with patch("src.functions.request.view_args", {"id": item_id}):
            result = get_item({"path": f"/items/{item_id}"}, {})
    assert result["id"] == item_id

    # Cleanup
    cursor.execute("DELETE FROM items WHERE id = %s", (item_id,))
    db_connection.commit()
```

## Test Database Migrations

Apply migrations before integration tests:

```python
# conftest.py
import subprocess

def apply_migrations():
    """Apply all SQL migrations to test database."""
    import os
    migrates_dir = os.path.join(os.path.dirname(__file__), "..", "migrates")
    for file in sorted(os.listdir(migrates_dir)):
        if file.endswith(".sql"):
            path = os.path.join(migrates_dir, file)
            subprocess.run(
                ["psql", "-d", "fission_test", "-f", path],
                check=True
            )

@pytest.fixture(scope="session", autouse=True)
def setup_database():
    """Run migrations before any tests."""
    apply_migrations()
    yield
    # Optionally drop and recreate after tests
```

## Mocking

### Built-in unittest.mock

```python
from unittest.mock import patch, MagicMock, mock_open

# Simple patch
with patch("module.function") as mock_func:
    mock_func.return_value = "mocked"
    # call code that uses module.function

# Assert called with specific args
mock_func.assert_called_once_with("arg1", "arg2")

# Mock context manager
with patch("builtins.open", mock_open(read_data="file content")) as mock_file:
    # code that opens file
    mock_file.assert_called_with("path/to/file", "r")
```

### pytest-mock Fixture

Simpler syntax using `mocker` fixture:

```python
def test_with_mocker(mocker):
    mock_func = mocker.patch("src.function.helper")
    mock_func.return_value = {"key": "value"}
    # test code...
```

## Fixtures

Create reusable fixtures in `conftest.py`:

```python
# test/conftest.py
import pytest

@pytest.fixture
def sample_item_data():
    """Provide sample item data for tests."""
    return {
        "name": "Test Item",
        "description": "A test item",
        "status": "active"
    }

@pytest.fixture
def mock_db_connection(mocker):
    """Provide a mocked database connection."""
    mock_conn = mocker.MagicMock()
    mock_cursor = mocker.MagicMock()
    mock_conn.cursor.return_value = mock_cursor
    mock_cursor.fetchone.return_value = None
    return mock_conn
```

Fixtures are automatically available to all tests in the directory.

## Coverage

Measure test coverage with pytest-cov:

```bash
# Install
pip install pytest-cov

# Run with coverage
pytest --cov=src

# HTML report
pytest --cov=src --cov-report=html
open htmlcov/index.html

# Show missing lines
pytest --cov=src --cov-report=term-missing
```

 Aim for high coverage of business logic (80%+). Don't worry about 100% coverage of trivial getters/setters.

### Excluding Files

Add to `pytest.ini`:
```ini
[pytest]
addopts = --cov=src --cov-exclude=src/vault.py
```

Or use `.coveragerc`:
```ini
[run]
omit = src/vault.py
```

## Running Tests

### Basic Commands

```bash
# Run all tests
pytest

# Verbose
pytest -v

# Run specific test file
pytest test/test_my_function.py

# Run specific test function
pytest test/test_my_function.py::test_create_item_success

# Run with markers
pytest -m "integration"  # if using @pytest.mark.integration

# Stop on first failure
pytest -x

# Show print statements
pytest -s
```

### Environment Setup

Create `test/.env` or set environment variables before tests:

```bash
# For integration tests
export PG_HOST=localhost
export PG_PORT=5432
export PG_DB=fission_test
```

Or use a pytest fixture to load from `.env`:

```python
# conftest.py
import os
from dotenv import load_dotenv

@pytest.fixture(scope="session", autouse=True)
def load_env():
    env_path = os.path.join(os.path.dirname(__file__), ".env")
    load_dotenv(env_path)
```

### Markers

Mark tests as unit/integration/slow:

```python
import pytest

@pytest.mark.unit
def test_quick_unit():
    pass

@pytest.mark.integration
def test_full_workflow():
    pass

@pytest.mark.slow
def test_long_running():
    pass
```

Run only unit tests:
```bash
pytest -m "unit"
```

Skip tests:
```bash
pytest -m "not slow"
```

## CI/CD Integration

### GitHub Actions Example

```yaml
# .github/workflows/test.yaml
name: Tests

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    services:
      postgres:
        image: postgres:15
        env:
          POSTGRES_PASSWORD: test
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5
        ports:
          - 5432:5432

    steps:
      - uses: actions/checkout@v3
      - name: Set up Python
        uses: actions/setup-python@v4
        with:
          python-version: '3.11'
      - name: Install dependencies
        run: |
          pip install -r dev-requirements.txt
      - name: Setup database
        run: |
          createdb -h localhost -U postgres fission_test
          psql -h localhost -U postgres fission_test -f migrates/001_schema.sql
        env:
          PGPASSWORD: test
      - name: Run tests
        run: |
          pytest --cov=src --cov-report=xml
        env:
          PG_HOST: localhost
          PG_PORT: 5432
          PG_DB: fission_test
          PG_USER: postgres
          PG_PASS: test
      - name: Upload coverage
        uses: codecov/codecov-action@v3
```

## Best Practices

1. **One assertion per test** - Keep tests focused
2. **Use descriptive names** - `test_create_item_validation_error_for_missing_name`
3. **Arrange-Act-Assert** - Structure tests clearly
4. **Mock external dependencies** - Don't rely on network or external services
5. **Test error cases** - Don't just test happy paths
6. **Use fixtures** - Reuse setup/teardown code
7. **Keep tests independent** - No shared state between tests
8. **Test edge cases** - Empty inputs, null values, boundary conditions
9. **Don't test libraries** - Don't write tests for Flask/Pydantic themselves
10. **Clean up resources** - Use fixtures to ensure cleanup

## Common Patterns

### Testing Exceptions

```python
def test_raises_not_found():
    with pytest.raises(NotFoundError) as exc:
        get_item("nonexistent-id")
    assert exc.value.http_status == 404
```

### Parametrized Tests

```python
import pytest

@pytest.mark.parametrize("input,expected", [
    ("true", True),
    ("false", False),
    ("", None),
    (None, None),
])
def test_str_to_bool(input, expected):
    from helpers import str_to_bool
    assert str_to_bool(input) == expected
```

### Temporary Files/Directories

```python
def test_with_temp_file(tmp_path):
    # tmp_path is a pathlib.Path to a temporary directory
    file = tmp_path / "test.txt"
    file.write_text("content")
    assert file.read_text() == "content"
```

## Troubleshooting

### Tests Fail with Database Errors

- Check test database is running: `pg_isready -h localhost -p 5432`
- Verify migrations applied: `psql -l | grep fission_test`
- Check environment variables: `echo $PG_HOST`

### Mock Not Working

- Ensure you're patching the **correct import location** (where it's used, not where it's defined)
  ```python
  # Wrong: patching where it's defined
  @patch("helpers.get_secret")
  # Right: patching where it's used in your function module
  @patch("src.my_function.helpers.get_secret")
  ```

### Import Errors

Ensure PYTHONPATH includes project root:
```bash
export PYTHONPATH=/path/to/project:$PYTHONPATH
```

Or use pytest's `pythonpath` option in pytest.ini:
```ini
[pytest]
pythonpath = .
```

## Further Reading

- [pytest documentation](https://docs.pytest.org/)
- [pytest-mock documentation](https://github.com/pytest-dev/pytest-mock)
- [Python unittest.mock](https://docs.python.org/3/library/unittest.mock.html)
- [Testing Flask Applications](https://flask.palletsprojects.com/en/2.1.x/testing/)