claude-gen/fission-python/template/docs/STRUCTURE.md

# Project Structure

This document explains the purpose and contents of each directory and file in a Fission Python project.

## Directory Layout

```
project/
├── .fission/              # Fission configuration
│   ├── deployment.json    # Main deployment configuration
│   ├── dev-deployment.json   # Development environment overrides
│   └── local-deployment.json # Local development overrides
├── src/                   # Source code
│   ├── __init__.py       # Package initialization
│   ├── vault.py          # Vault encryption utilities
│   ├── helpers.py        # Shared utility functions
│   ├── exceptions.py     # Custom exception classes
│   ├── models.py         # Pydantic models for validation
│   ├── build.sh          # Build script (executable)
│   └── *.py              # Your function implementations
├── test/                  # Unit and integration tests
│   ├── __init__.py
│   ├── test_*.py         # Test files
│   └── requirements.txt  # Test dependencies
├── migrates/              # Database migration scripts
│   └── *.sql             # SQL migration files
├── manifests/             # Kubernetes manifests (optional)
│   └── *.yaml            # K8s resources
├── specs/                 # Generated Fission specs
│   ├── fission-deployment-config.yaml
│   └── ...
├── requirements.txt       # Runtime dependencies
├── dev-requirements.txt   # Development dependencies
├── .env.example          # Environment variable template
├── pytest.ini            # Pytest configuration
├── README.md             # Project documentation
└── (other project files)
```

## File Purposes

### .fission/deployment.json

This is **the most important configuration file** for Fission deployment. It defines:

- **environments**: Build environment configuration (image, builder, resources)
- **archives**: Source code packaging (typically "package.zip" from src/)
- **packages**: Package definitions linking source to environment
- **function_common**: Default settings applied to all functions
- **secrets**: Secret definitions (literal values are placeholders - actual secrets go in K8s)
- **configmaps**: ConfigMap definitions (non-sensitive configuration)

**Important**: The secret and configmap literals are **placeholders only**. In production, you create actual K8s secrets/configmaps with the same names containing real values.

**Placeholders**:
- `${PROJECT_NAME}` - Replaced with your project name by `create-project.sh`
- Secret name pattern: `fission-${PROJECT_NAME}-env`
- ConfigMap name pattern: `fission-${PROJECT_NAME}-config`

### src/vault.py

Provides encryption/decryption utilities using PyNaCl (SecretBox). This is used when you want to store encrypted values in K8s secrets rather than plaintext.

**Key functions**:
- `encrypt_vault(plaintext, key)` - Encrypt and return vault format string
- `decrypt_vault(vault, key)` - Decrypt vault format string
- `is_valid_vault_format(vault)` - Check if string is vault-encrypted

**Usage in helpers.py**: The `get_secret()` and `get_config()` functions automatically detect vault format (`vault:v1:...`) and decrypt if a valid `CRYPTO_KEY` is set.

### src/helpers.py

Shared utilities used across functions:

**Database**:
- `init_db_connection()` - Creates PostgreSQL connection from secrets
- `db_row_to_dict(cursor, row)` - Convert row tuple to dict
- `db_rows_to_array(cursor, rows)` - Convert multiple rows to list of dicts

**Configuration**:
- `get_secret(key, default=None)` - Read from K8s secret volume
- `get_config(key, default=None)` - Read from K8s config volume
- `get_current_namespace()` - Get current K8s namespace

**Utilities**:
- `str_to_bool(input)` - Convert string to boolean
- `check_port_open(ip, port, timeout)` - TCP port connectivity check
- `get_user_from_headers()` - Extract user ID from request headers
- `format_error_response(...)` - Build standardized error dict

**Logging**:
- Helper uses `current_app.logger` (Flask) for error logging

### src/exceptions.py

Custom exception hierarchy:

```
ServiceException (base)
├── ValidationError (400)    - Invalid input
├── NotFoundError (404)      - Resource not found
├── ConflictError (409)      - Duplicate/conflict
└── DatabaseError (500)      - Database failure
```

All exceptions include:
- `error_code` - Machine-readable code
- `http_status` - HTTP status
- `error_msg` - Human-readable message
- `x_user` (optional) - User identifier
- `details` (optional) - Additional context dict

When raised in a Fission function, these automatically return proper JSON error responses.

### src/models.py

Pydantic models for request/response validation:

**Patterns included**:
- Enums (e.g., `Status`, `DataType`)
- Dataclass filters (e.g., `ItemFilter`, `Pagination`)
- Request models (`ItemCreateRequest`, `ItemUpdateRequest`)
- Response models (`ItemResponse`, `PaginatedResponse`)
- ErrorResponse model (used by exceptions)

**Key concepts**:
- Use `Field(...)` with constraints (min_length, max_length, ge, le)
- Provide `description` for API documentation
- Use `json_schema_extra` for example values
- Set `from_attributes = True` for ORM compatibility

### src/build.sh

Bash script that builds the dependency package. It:
1. Detects OS (Debian vs Alpine)
2. Installs build dependencies (gcc, libpq-dev/python3-dev/postgresql-dev)
3. Installs Python requirements into `src/` directory
4. Copies `src/` to package destination

**Important**: Must be executable (`chmod +x src/build.sh`)

The script expects environment variables:
- `SRC_PKG` - Source package directory (e.g., `src`)
- `DEPLOY_PKG` - Destination package (e.g., `specs/package`)

Fission builder sets these automatically.

### test/

Contains unit and integration tests.

**Structure**:
- `test_*.py` - Test files following pytest conventions
- `requirements.txt` - Test dependencies (pytest, pytest-mock, requests)

**Running tests**:
```bash
pip install -r dev-requirements.txt
pytest
```

## Fission Configuration in Docstrings

Each Python function that should be exposed as a Fission function **must** include a ````fission` block in its docstring:

```python
def my_function(event, context):
    """
    ```fission
    {
        "name": "my-function",
        "http_triggers": {
            "my-trigger": {
                "url": "/api/endpoint",
                "methods": ["GET", "POST"]
            }
        }
    }
    ```
    Human-readable description here.
    """
    # Implementation
```

The Fission Python builder parses these docstrings and generates the `specs/fission-deployment-config.yaml` and other spec files.

**Supported trigger types**:
- `http_triggers` - HTTP endpoints
- `kafka_triggers` - Kafka topics
- `timer_triggers` - Scheduled execution
- `message_queue_triggers` - MQTT, NATS, etc.

## Configuration Precedence

1. **deployment.json** - Base configuration (committed to repo)
2. **dev-deployment.json** - Overrides for dev environment (not always committed)
3. **local-deployment.json** - Local overrides (typically .gitignored)

When deploying:
- `fission deploy` uses deployment.json
- `fission deploy --dev` uses dev-deployment.json if present

## Secrets and Configuration Flow

1. **Define placeholders** in `deployment.json`:
   ```json
   "secrets": {
     "fission-myproject-env": {
       "literals": ["PG_HOST=localhost", "PG_PORT=5432"]
     }
   }
   ```

2. **Create actual K8s secret**:
   ```bash
   kubectl create secret generic fission-myproject-env \
     --from-literal=PG_HOST=prod-db.example.com \
     --from-literal=PG_PORT=5432
   ```

3. **Read in code** via `get_secret()`:
   ```python
   host = get_secret("PG_HOST")
   ```

4. **For vault encryption**:
   - Set `CRYPTO_KEY` in helpers.py or as env override
   - Store encrypted: `vault:v1:base64data` in K8s secret
   - `get_secret()` auto-decrypts

## Summary

- Keep function code in `src/`
- Define Fission metadata in docstring blocks
- Use helpers for common operations
- Define custom exceptions for error handling
- Validate inputs with Pydantic models
- Store tests in `test/` with pytest
- Manage database migrations in `migrates/`
- Do not commit actual secrets to repository