ducnv/claude-gen
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

ref: up

Browse Source
This commit is contained in:
Duc Nguyen
2026-03-18 20:21:56 +07:00
commit 29667cd92f
58 changed files with 8459 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

438
fission-python/template/docs/SECRETS.md Normal file
View File

@@ -0,0 +1,438 @@
# Secrets and Configuration Management
This guide covers best practices for managing secrets and configuration in Fission Python functions.
## Table of Contents
1. [Overview](#overview)
2. [Kubernetes Secrets vs ConfigMaps](#kubernetes-secrets-vs-configmaps)
3. [Secrets in Fission](#secrets-in-fission)
4. [Vault Encryption](#vault-encryption)
5. [Secret Rotation](#secret-rotation)
6. [Configuration Precedence](#configuration-precedence)
7. [Best Practices](#best-practices)
## Overview
Sensitive data (passwords, API keys) should **never** be:
- Committed to Git
- Hardcoded in source code
- Passed as plaintext in deployment files
Instead, use:
- **Kubernetes Secrets** - For sensitive values
- **Kubernetes ConfigMaps** - For non-sensitive configuration
- **Vault encryption** - For encrypting secrets at rest in K8s
## Kubernetes Secrets vs ConfigMaps
| Feature | Secrets | ConfigMaps |
|---------|---------|------------|
| Purpose | Sensitive data (passwords, tokens, keys) | Non-sensitive config (endpoints, feature flags) |
| Storage | Base64 encoded (not encrypted by default) | Plain text |
| Mount as | Files in `/secrets/` | Files in `/configs/` |
| Access in code | `get_secret(key)` | `get_config(key)` |
| Max size | 1MB total | 1MB total |
| Can be encrypted | Yes, with K8s encryption at rest | Yes |
**Rule of thumb**:
- Use Secrets for: database passwords, API tokens, encryption keys
- Use ConfigMaps for: service URLs, feature flags, log levels, non-sensitive constants
## Secrets in Fission
### Defining Secret References in deployment.json
In `.fission/deployment.json`, declare the secret names your functions expect:
```json
{
"function_common": {
"secrets": ["fission-myproject-env"],
"configmaps": ["fission-myproject-config"]
},
"secrets": {
"fission-myproject-env": {
"literals": [
"PG_HOST=localhost",
"PG_PORT=5432"
]
}
}
}
```
**Important**: The `literals` array here is **only documentation**. The actual secret values must be created separately in Kubernetes.
### Creating Actual Kubernetes Secrets
```bash
# Create secret with multiple keys
kubectl create secret generic fission-myproject-env \
--from-literal=PG_HOST=postgres.example.com \
--from-literal=PG_PORT=5432 \
--from-literal=PG_DB=mydb \
--from-literal=PG_USER=myuser \
--from-literal=PG_PASS='my-password'
# In a specific namespace (Fission namespace)
kubectl create secret generic fission-myproject-env \
--namespace fission \
--from-literal=...
# From environment file
kubectl create secret generic fission-myproject-env \
--namespace fission \
--from-env-file=.env
```
### How Secrets Are Mounted
Fission mounts secrets as files in the function pod:
```
/secrets/{namespace}/{secret-name}/{key}
```
Example path: `/secrets/default/fission-myproject-env/PG_HOST`
The `helpers.py` `get_secret()` function reads from this path:
```python
def get_secret(key: str, default=None):
namespace = get_current_namespace()
path = f"/secrets/{namespace}/{SECRET_NAME}/{key}"
with open(path, "r") as f:
return f.read()
```
**Note**: `SECRET_NAME` must match the K8s secret name (`fission-myproject-env`).
### Reading Secrets in Code
```python
from helpers import get_secret
# With default fallback
db_host = get_secret("PG_HOST", "localhost")
db_port = int(get_secret("PG_PORT", "5432"))
db_user = get_secret("PG_USER")
db_pass = get_secret("PG_PASS")
# If key missing and no default, returns None
maybe_value = get_secret("OPTIONAL_KEY")
```
**Always provide a default** for non-critical configuration to avoid crashes if secret is missing.
### ConfigMaps
Same pattern, different mount path: `/configs/{namespace}/{configmap-name}/{key}`
```python
from helpers import get_config
api_endpoint = get_config("API_ENDPOINT", "http://default.api")
feature_flag = get_config("FEATURE_X_ENABLED", "false")
```
Create ConfigMap:
```bash
kubectl create configmap fission-myproject-config \
--namespace fission \
--from-literal=API_ENDPOINT=https://api.example.com \
--from-literal=FEATURE_X_ENABLED=true
```
## Vault Encryption
To encrypt secrets before storing in K8s:
### Generate Encryption Key
```bash
# Generate 32-byte (64 hex char) random key
openssl rand -hex 32
# Example output: e24ad6ceed96115520f6e6dc8a0da506ae9a706823d54f30a5b75447ecf477b6
```
### Encrypt a Value
```python
# Encrypt locally
from vault import encrypt_vault
key = "e24ad6ceed96115520f6e6dc8a0da506ae9a706823d54f30a5b75447ecf477b6"
encrypted = encrypt_vault("my-secret-password", key)
print(encrypted)
# Output: vault:v1:base64-encrypted-data
```
### Store Encrypted Value
Create K8s secret with encrypted value:
```bash
kubectl create secret generic fission-myproject-env \
--from-literal=PG_PASS='vault:v1:base64...'
```
### Configure decryption in helpers.py
```python
CRYPTO_KEY = "e24ad6ceed96115520f6e6dc8a0da506ae9a706823d54f30a5b75447ecf477b6"
```
### Automatic Decryption
`get_secret()` and `get_config()` automatically:
1. Read the file content
2. Detect if it starts with `vault:v1:` (using `is_valid_vault_format()`)
3. Decrypt using `CRYPTO_KEY` if encrypted
4. Return plaintext
**No code changes needed** - it "just works".
### Verification
```bash
# Test decryption
kubectl get secret fission-myproject-env -o jsonpath='{.data.PG_PASS}' | base64 -d
# Should show: vault:v1:...
# Exec into pod and manually check
kubectl exec -it <pod-name> -- python3 -c "from helpers import get_secret; print(get_secret('PG_PASS'))"
# Should print decrypted value
```
## Secret Rotation
### Rotating a Secret
1. **Generate new value** (new password, new API key)
2. **Encrypt** (if using vault)
3. **Update K8s secret**:
```bash
kubectl create secret generic fission-myproject-env \
--dry-run=client \
--from-literal=PG_PASS='new-password' \
-o yaml | kubectl apply -f -
```
4. **Update actual external system** (database, API provider) with new value
5. **Verify applications work** (check logs)
6. **Remove old value** (if rotating from old to new, both may need to coexist temporarily)
### Rotating Vault Encryption Key
**Warning**: Changing `CRYPTO_KEY` requires re-encrypting all secrets!
1. Deploy new code with updated `CRYPTO_KEY` **temporarily** pointing to new key
2. Create new K8s secrets with values encrypted under new key (or re-encrypt via script)
3. Switch `CRYPTO_KEY` back to original (or both keys during transition) - actually this is complex
**Recommended**: Have two keys during rotation:
```python
CRYPTO_KEYS = [
"old-key-hex...", # Keep for decrypting old secrets
"new-key-hex..." # Use for encrypting new/updated secrets
]
```
Then update `decrypt_vault()` to try each key until one works. After all secrets migrated, remove old key.
## Configuration Precedence
Fission supports multiple deployment configuration files:
1. **deployment.json** - Base configuration (committed to repo)
2. **dev-deployment.json** - Development overrides (usually not committed)
3. **local-deployment.json** - Local overrides (gitignored)
### Override Priority
When using `fission deploy --dev`, Fission loads:
- Base configuration from `deployment.json`
- Overlay from `dev-deployment.json`
Values in the overlay file replace or extend base values.
**Example**: Override secret name for dev:
**deployment.json**:
```json
{
"function_common": {
"secrets": ["fission-myproject-env"]
}
}
```
**dev-deployment.json**:
```json
{
"function_common": {
"secrets": ["fission-myproject-dev-env"]
}
}
```
Now `fission deploy --dev` uses the dev secret, while `fission deploy` uses prod secret.
### Local Overrides
Create `.fission/local-deployment.json` for your workstation:
```json
{
"function_common": {
"secrets": ["fission-myproject-local-env"]
}
}
```
Fission automatically uses this if present (no flag needed). `.gitignore` typically excludes it.
## Best Practices
### Do's ✅
1. **Do use Kubernetes Secrets** - Never hardcode credentials
2. **Do encrypt with vault** - Prevents plaintext secrets in K8s
3. **Do store vault key securely** - In K8s sealed secret, external vault (HashiCorp Vault, AWS Secrets Manager), or as a separate K8s secret in restricted namespace
4. **Do namespace secrets** - Use different secrets for dev/staging/prod
5. **Do rotate secrets regularly** - Especially database passwords, API tokens
6. **Do use ConfigMaps for non-sensitive config** - Cleaner separation
7. **Do provide sensible defaults** - In `get_secret()` calls
8. **Do validate required secrets** - Fail fast at startup:
```python
def init():
pg_host = get_secret("PG_HOST")
if not pg_host:
raise ValueError("PG_HOST secret is required")
```
### Don'ts ❌
1. **Don't commit secrets** - Even in `deployment.json` literals
2. **Don't put plaintext in Git** - Use placeholders or remove before commit
3. **Don't embed vault key in code for production** - Use environment-specific override or external secret management
4. **Don't share vault key publicly** - It's a symmetric key - anyone with it can decrypt all secrets
5. **Don't use same secret across namespaces** - Separate environments should have separate credentials
6. **Don't rely on obscurity** - Security through obscurity is not security
### Supply Chain Security
For production deployments:
1. **Store vault key in sealed secrets** (if on K8s):
```bash
kubectl create secret generic crypto-key \
--from-literal=key='your-hex-key'
# Then use SealedSecrets controller to encrypt in Git
```
2. **Use external secrets operator**:
```yaml
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: db-creds
spec:
refreshInterval: "1h"
secretStoreRef:
name: vault-backend
kind: SecretStore
target:
name: fission-myproject-env
creationPolicy: Owner
data:
- secretKey: PG_PASS
remoteRef:
key: /prod/db/password
```
3. **Rotate automatically** with cronjobs or external secret manager
## Environment Variable Alternative
While the template uses secret files mounted by Fission, you can also use environment variables:
```json
"function_common": {
"environment": {
"LOG_LEVEL": "INFO",
"FEATURE_FLAG": "true"
}
}
```
Access with `os.getenv()`:
```python
import os
log_level = os.getenv("LOG_LEVEL", "INFO")
```
**However**: Environment is less flexible than secrets/configmaps for dynamic updates (requires function restart). Prefer secrets/configmaps for values that may change independently of code deployments.
## Troubleshooting
### Secret Not Available
```bash
# Check secret exists in correct namespace
kubectl get secret fission-myproject-env -n fission
# Check secret keys
kubectl get secret fission-myproject-env -n fission -o jsonpath='{.data}'
# Check pod mount
kubectl exec -it <pod-name> -n fission -- ls -la /secrets/default/
```
Common issues:
- Secret in wrong namespace (use Fission namespace, usually `fission` or as configured)
- Secret name typo in helpers.py `SECRET_NAME` variable
- Secret not mounted due to missing permission (service account restriction)
### Vault Decryption Failing
```python
from vault import is_valid_vault_format, decrypt_vault
vault_str = get_secret("PG_PASS")
print(is_valid_vault_format(vault_str)) # Should be True
print(decrypt_vault(vault_str, "wrong-key")) # Raises CryptoError
```
Check:
- `CRYPTO_KEY` is set correctly in `helpers.py`
- Key is 64 hex characters (32 bytes)
- Encrypted value format is exactly `vault:v1:base64...`
### Permission Denied Reading Secret
Pod may lack permission to read secret. Check service account:
```bash
# Get function pod's service account
kubectl get pod <pod-name> -n fission -o jsonpath='{.spec.serviceAccountName}'
# Check role bindings
kubectl get rolebinding -n fission
kubectl get clusterrolebinding -n fission
# Add permission if needed (requires cluster admin)
kubectl create clusterrolebinding fission-secret-reader \
--clusterrole=view \
--serviceaccount=fission:default
```
## Further Reading
- [Kubernetes Secrets](https://kubernetes.io/docs/concepts/configuration/secret/)
- [Kubernetes ConfigMaps](https://kubernetes.io/docs/concepts/configuration/configmap/)
- [Fission Environment and Config](https://fission.io/docs/usage/env/)
- [PyNaCl Documentation](https://pynacl.readthedocs.io/)
- [SealedSecrets](https://github.com/bitnami-labs/sealed-secrets) - Store encrypted secrets in Git