claude-gen/fission-python/template/docs/SECRETS.md

# Secrets and Configuration Management

This guide covers best practices for managing secrets and configuration in Fission Python functions.

## Table of Contents

1. [Overview](#overview)
2. [Kubernetes Secrets vs ConfigMaps](#kubernetes-secrets-vs-configmaps)
3. [Secrets in Fission](#secrets-in-fission)
4. [Vault Encryption](#vault-encryption)
5. [Secret Rotation](#secret-rotation)
6. [Configuration Precedence](#configuration-precedence)
7. [Best Practices](#best-practices)

## Overview

Sensitive data (passwords, API keys) should **never** be:
- Committed to Git
- Hardcoded in source code
- Passed as plaintext in deployment files

Instead, use:
- **Kubernetes Secrets** - For sensitive values
- **Kubernetes ConfigMaps** - For non-sensitive configuration
- **Vault encryption** - For encrypting secrets at rest in K8s

## Kubernetes Secrets vs ConfigMaps

| Feature | Secrets | ConfigMaps |
|---------|---------|------------|
| Purpose | Sensitive data (passwords, tokens, keys) | Non-sensitive config (endpoints, feature flags) |
| Storage | Base64 encoded (not encrypted by default) | Plain text |
| Mount as | Files in `/secrets/` | Files in `/configs/` |
| Access in code | `get_secret(key)` | `get_config(key)` |
| Max size | 1MB total | 1MB total |
| Can be encrypted | Yes, with K8s encryption at rest | Yes |

**Rule of thumb**:
- Use Secrets for: database passwords, API tokens, encryption keys
- Use ConfigMaps for: service URLs, feature flags, log levels, non-sensitive constants

## Secrets in Fission

### Defining Secret References in deployment.json

In `.fission/deployment.json`, declare the secret names your functions expect:

```json
{
  "function_common": {
    "secrets": ["fission-myproject-env"],
    "configmaps": ["fission-myproject-config"]
  },
  "secrets": {
    "fission-myproject-env": {
      "literals": [
        "PG_HOST=localhost",
        "PG_PORT=5432"
      ]
    }
  }
}
```

**Important**: The `literals` array here is **only documentation**. The actual secret values must be created separately in Kubernetes.

### Creating Actual Kubernetes Secrets

```bash
# Create secret with multiple keys
kubectl create secret generic fission-myproject-env \
  --from-literal=PG_HOST=postgres.example.com \
  --from-literal=PG_PORT=5432 \
  --from-literal=PG_DB=mydb \
  --from-literal=PG_USER=myuser \
  --from-literal=PG_PASS='my-password'

# In a specific namespace (Fission namespace)
kubectl create secret generic fission-myproject-env \
  --namespace fission \
  --from-literal=...

# From environment file
kubectl create secret generic fission-myproject-env \
  --namespace fission \
  --from-env-file=.env
```

### How Secrets Are Mounted

Fission mounts secrets as files in the function pod:

```
/secrets/{namespace}/{secret-name}/{key}
```

Example path: `/secrets/default/fission-myproject-env/PG_HOST`

The `helpers.py` `get_secret()` function reads from this path:

```python
def get_secret(key: str, default=None):
    namespace = get_current_namespace()
    path = f"/secrets/{namespace}/{SECRET_NAME}/{key}"
    with open(path, "r") as f:
        return f.read()
```

**Note**: `SECRET_NAME` must match the K8s secret name (`fission-myproject-env`).

### Reading Secrets in Code

```python
from helpers import get_secret

# With default fallback
db_host = get_secret("PG_HOST", "localhost")
db_port = int(get_secret("PG_PORT", "5432"))
db_user = get_secret("PG_USER")
db_pass = get_secret("PG_PASS")

# If key missing and no default, returns None
maybe_value = get_secret("OPTIONAL_KEY")
```

**Always provide a default** for non-critical configuration to avoid crashes if secret is missing.

### ConfigMaps

Same pattern, different mount path: `/configs/{namespace}/{configmap-name}/{key}`

```python
from helpers import get_config

api_endpoint = get_config("API_ENDPOINT", "http://default.api")
feature_flag = get_config("FEATURE_X_ENABLED", "false")
```

Create ConfigMap:

```bash
kubectl create configmap fission-myproject-config \
  --namespace fission \
  --from-literal=API_ENDPOINT=https://api.example.com \
  --from-literal=FEATURE_X_ENABLED=true
```

## Vault Encryption

To encrypt secrets before storing in K8s:

### Generate Encryption Key

```bash
# Generate 32-byte (64 hex char) random key
openssl rand -hex 32
# Example output: e24ad6ceed96115520f6e6dc8a0da506ae9a706823d54f30a5b75447ecf477b6
```

### Encrypt a Value

```python
# Encrypt locally
from vault import encrypt_vault

key = "e24ad6ceed96115520f6e6dc8a0da506ae9a706823d54f30a5b75447ecf477b6"
encrypted = encrypt_vault("my-secret-password", key)
print(encrypted)
# Output: vault:v1:base64-encrypted-data
```

### Store Encrypted Value

Create K8s secret with encrypted value:

```bash
kubectl create secret generic fission-myproject-env \
  --from-literal=PG_PASS='vault:v1:base64...'
```

### Configure decryption in helpers.py

```python
CRYPTO_KEY = "e24ad6ceed96115520f6e6dc8a0da506ae9a706823d54f30a5b75447ecf477b6"
```

### Automatic Decryption

`get_secret()` and `get_config()` automatically:
1. Read the file content
2. Detect if it starts with `vault:v1:` (using `is_valid_vault_format()`)
3. Decrypt using `CRYPTO_KEY` if encrypted
4. Return plaintext

**No code changes needed** - it "just works".

### Verification

```bash
# Test decryption
kubectl get secret fission-myproject-env -o jsonpath='{.data.PG_PASS}' | base64 -d
# Should show: vault:v1:...

# Exec into pod and manually check
kubectl exec -it <pod-name> -- python3 -c "from helpers import get_secret; print(get_secret('PG_PASS'))"
# Should print decrypted value
```

## Secret Rotation

### Rotating a Secret

1. **Generate new value** (new password, new API key)
2. **Encrypt** (if using vault)
3. **Update K8s secret**:
   ```bash
   kubectl create secret generic fission-myproject-env \
     --dry-run=client \
     --from-literal=PG_PASS='new-password' \
     -o yaml | kubectl apply -f -
   ```
4. **Update actual external system** (database, API provider) with new value
5. **Verify applications work** (check logs)
6. **Remove old value** (if rotating from old to new, both may need to coexist temporarily)

### Rotating Vault Encryption Key

**Warning**: Changing `CRYPTO_KEY` requires re-encrypting all secrets!

1. Deploy new code with updated `CRYPTO_KEY` **temporarily** pointing to new key
2. Create new K8s secrets with values encrypted under new key (or re-encrypt via script)
3. Switch `CRYPTO_KEY` back to original (or both keys during transition) - actually this is complex

**Recommended**: Have two keys during rotation:
```python
CRYPTO_KEYS = [
    "old-key-hex...",  # Keep for decrypting old secrets
    "new-key-hex..."   # Use for encrypting new/updated secrets
]
```

Then update `decrypt_vault()` to try each key until one works. After all secrets migrated, remove old key.

## Configuration Precedence

Fission supports multiple deployment configuration files:

1. **deployment.json** - Base configuration (committed to repo)
2. **dev-deployment.json** - Development overrides (usually not committed)
3. **local-deployment.json** - Local overrides (gitignored)

### Override Priority

When using `fission deploy --dev`, Fission loads:
- Base configuration from `deployment.json`
- Overlay from `dev-deployment.json`

Values in the overlay file replace or extend base values.

**Example**: Override secret name for dev:

**deployment.json**:
```json
{
  "function_common": {
    "secrets": ["fission-myproject-env"]
  }
}
```

**dev-deployment.json**:
```json
{
  "function_common": {
    "secrets": ["fission-myproject-dev-env"]
  }
}
```

Now `fission deploy --dev` uses the dev secret, while `fission deploy` uses prod secret.

### Local Overrides

Create `.fission/local-deployment.json` for your workstation:

```json
{
  "function_common": {
    "secrets": ["fission-myproject-local-env"]
  }
}
```

Fission automatically uses this if present (no flag needed). `.gitignore` typically excludes it.

## Best Practices

### Do's ✅

1. **Do use Kubernetes Secrets** - Never hardcode credentials
2. **Do encrypt with vault** - Prevents plaintext secrets in K8s
3. **Do store vault key securely** - In K8s sealed secret, external vault (HashiCorp Vault, AWS Secrets Manager), or as a separate K8s secret in restricted namespace
4. **Do namespace secrets** - Use different secrets for dev/staging/prod
5. **Do rotate secrets regularly** - Especially database passwords, API tokens
6. **Do use ConfigMaps for non-sensitive config** - Cleaner separation
7. **Do provide sensible defaults** - In `get_secret()` calls
8. **Do validate required secrets** - Fail fast at startup:
   ```python
   def init():
       pg_host = get_secret("PG_HOST")
       if not pg_host:
           raise ValueError("PG_HOST secret is required")
   ```

### Don'ts ❌

1. **Don't commit secrets** - Even in `deployment.json` literals
2. **Don't put plaintext in Git** - Use placeholders or remove before commit
3. **Don't embed vault key in code for production** - Use environment-specific override or external secret management
4. **Don't share vault key publicly** - It's a symmetric key - anyone with it can decrypt all secrets
5. **Don't use same secret across namespaces** - Separate environments should have separate credentials
6. **Don't rely on obscurity** - Security through obscurity is not security

### Supply Chain Security

For production deployments:

1. **Store vault key in sealed secrets** (if on K8s):
   ```bash
   kubectl create secret generic crypto-key \
     --from-literal=key='your-hex-key'
   # Then use SealedSecrets controller to encrypt in Git
   ```

2. **Use external secrets operator**:
   ```yaml
   apiVersion: external-secrets.io/v1beta1
   kind: ExternalSecret
   metadata:
     name: db-creds
   spec:
     refreshInterval: "1h"
     secretStoreRef:
       name: vault-backend
       kind: SecretStore
     target:
       name: fission-myproject-env
       creationPolicy: Owner
     data:
       - secretKey: PG_PASS
         remoteRef:
           key: /prod/db/password
   ```

3. **Rotate automatically** with cronjobs or external secret manager

## Environment Variable Alternative

While the template uses secret files mounted by Fission, you can also use environment variables:

```json
"function_common": {
  "environment": {
    "LOG_LEVEL": "INFO",
    "FEATURE_FLAG": "true"
  }
}
```

Access with `os.getenv()`:

```python
import os
log_level = os.getenv("LOG_LEVEL", "INFO")
```

**However**: Environment is less flexible than secrets/configmaps for dynamic updates (requires function restart). Prefer secrets/configmaps for values that may change independently of code deployments.

## Troubleshooting

### Secret Not Available

```bash
# Check secret exists in correct namespace
kubectl get secret fission-myproject-env -n fission

# Check secret keys
kubectl get secret fission-myproject-env -n fission -o jsonpath='{.data}'

# Check pod mount
kubectl exec -it <pod-name> -n fission -- ls -la /secrets/default/
```

Common issues:
- Secret in wrong namespace (use Fission namespace, usually `fission` or as configured)
- Secret name typo in helpers.py `SECRET_NAME` variable
- Secret not mounted due to missing permission (service account restriction)

### Vault Decryption Failing

```python
from vault import is_valid_vault_format, decrypt_vault

vault_str = get_secret("PG_PASS")
print(is_valid_vault_format(vault_str))  # Should be True
print(decrypt_vault(vault_str, "wrong-key"))  # Raises CryptoError
```

Check:
- `CRYPTO_KEY` is set correctly in `helpers.py`
- Key is 64 hex characters (32 bytes)
- Encrypted value format is exactly `vault:v1:base64...`

### Permission Denied Reading Secret

Pod may lack permission to read secret. Check service account:

```bash
# Get function pod's service account
kubectl get pod <pod-name> -n fission -o jsonpath='{.spec.serviceAccountName}'

# Check role bindings
kubectl get rolebinding -n fission
kubectl get clusterrolebinding -n fission

# Add permission if needed (requires cluster admin)
kubectl create clusterrolebinding fission-secret-reader \
  --clusterrole=view \
  --serviceaccount=fission:default
```

## Further Reading

- [Kubernetes Secrets](https://kubernetes.io/docs/concepts/configuration/secret/)
- [Kubernetes ConfigMaps](https://kubernetes.io/docs/concepts/configuration/configmap/)
- [Fission Environment and Config](https://fission.io/docs/usage/env/)
- [PyNaCl Documentation](https://pynacl.readthedocs.io/)
- [SealedSecrets](https://github.com/bitnami-labs/sealed-secrets) - Store encrypted secrets in Git