claude-gen/fission-python/template/docs/DEPLOYMENT.md

Deployment Guide

This guide covers deploying Fission Python functions to Kubernetes, including configuration tuning, troubleshooting, and best practices.

Table of Contents

1. Prerequisites
2. Quick Start
3. Deployment Configuration
4. Executors
5. Resource Tuning
6. Environments
7. Secrets Management
8. Rolling Updates
9. Monitoring & Logging
10. Troubleshooting

Prerequisites

• Kubernetes cluster (v1.19+)
• Fission installed (kubectl apply -f https://github.com/fission/fission/releases/latest/download/fission-all.yaml)
• fission CLI installed and configured
• kubectl configured to access cluster
• Docker registry access (for custom images if needed)

Quick Start

Assuming you have a project set up:

# 1. Build the package (creates specs/ directory)
cd /path/to/project
./src/build.sh

# 2. Verify deployment configuration
fission spec verify --file=.fission/deployment.json

# 3. Deploy to Fission
fission deploy

# 4. Test deployed function
curl http://$FISSION_ROUTER/api/items

That's it! Fission will:

• Build package.zip from src/
• Create environment (if not exists)
• Create package
• Create functions from docstring metadata
• Set up HTTP triggers

Deployment Configuration

deployment.json vs fission.yaml

This template uses deployment.json, not fission.yaml or fission.json. The Fission Python builder extracts function metadata from Python docstrings directly.

Key Sections

environments

Define build environment:

{
  "environments": {
    "myproject-py": {
      "image": "ghcr.io/fission/python-env",
      "builder": "ghcr.io/fission/python-builder",
      "mincpu": 50,
      "maxcpu": 100,
      "minmemory": 50,
      "maxmemory": 500,
      "poolsize": 1
    }
  }
}

• image - Runtime image (Python + libraries)
• builder - Builder image (compiles dependencies)
• Resource limits in millicores (50 = 0.05 CPU) and MB

packages

Define how to build your code:

{
  "packages": {
    "myproject": {
      "buildcmd": "./build.sh",
      "sourcearchive": "package.zip",
      "env": "myproject-py"
    }
  }
}

• buildcmd - Build script inside builder container
• sourcearchive - Generated by builder from sourcepath
• env - Links to environment definition

function_common

Default configuration for all functions:

{
  "function_common": {
    "pkg": "myproject",
    "secrets": ["fission-myproject-env"],
    "configmaps": ["fission-myproject-config"],
    "executor": { ... },
    "mincpu": 50,
    "maxcpu": 100,
    "minmemory": 50,
    "maxmemory": 500
  }
}

• pkg - Package name to use
• secrets / configmaps - K8s resources to mount into functions
• executor - Execution strategy (poolmgr or newdeploy)

secrets / configmaps

Placeholder definitions only. These inform Fission what secret names to expect, but the actual values go in real K8s secrets:

{
  "secrets": {
    "fission-myproject-env": {
      "literals": [
        "PG_HOST=localhost",
        "PG_PORT=5432"
      ]
    }
  }
}

Create the actual secret:

kubectl create secret generic fission-myproject-env \
  --from-literal=PG_HOST=prod-db.example.com \
  --from-literal=PG_PORT=5432 \
  --from-literal=PG_USER=myuser \
  --from-literal=PG_PASS=mypassword

Executors

Fission supports two executor types:

poolmgr (default)

Good for:

• High-concurrency HTTP functions
• Functions that should scale to zero
• Stateless request/response patterns

Configuration:

"executor": {
  "select": "poolmgr",
  "poolmgr": {
    "concurrency": 1,        // Requests per pod
    "requestsperpod": 1,
    "onceonly": false
  }
}

• concurrency - How many concurrent requests each pod handles (usually 1 for Python due to GIL)
• poolsize from environment controls number of pods in pool

newdeploy

Good for:

• Dedicated function instances
• Long-running or background jobs
• Functions needing stable network identity

Configuration:

"executor": {
  "select": "newdeploy",
  "newdeploy": {
    "minscale": 1,       // Minimum pods (set to 0 for scale-to-zero)
    "maxscale": 5,       // Maximum pods
    "targetcpu": 80      // Scale up when CPU > 80%
  }
}

• minscale - Keep at least N pods running (0 = scale to zero)
• maxscale - Maximum pods for auto-scaling
• targetcpu - CPU threshold for scaling

Resource Tuning

Resources are defined in millicores (m) and MB:

• mincpu / maxcpu: 1000 = 1 CPU core
• minmemory / maxmemory: in MB

Example settings:

Function Type	mincpu	maxcpu	minmemory	maxmemory
Simple API	50	100	128	256
DB-intensive	200	500	256	512
ML inference	1000	2000	1024	2048

Tips:

• Start conservatively, monitor, then adjust
• Function pods are killed if they exceed maxmemory
• CPU limits are enforced by Kubernetes scheduler
• Use minmemory >= 128 to avoid OOM kills

Checking Current Usage

# Get function pods
kubectl get pods -n fission

# Describe pod for resource usage
kubectl describe pod <pod-name> -n fission

# See metrics (if metrics-server installed)
kubectl top pod <pod-name> -n fission

Environments

You can have multiple deployment environments (dev, staging, prod):

Using deployment.json variants

• deployment.json - Production (default)
• dev-deployment.json - Development (used with fission deploy --dev)

Example dev-deployment.json:

{
  "namespace": "fission-dev",
  "function_common": {
    "secrets": ["fission-myproject-dev-env"],
    "configmaps": ["fission-myproject-dev-config"]
  }
}

Switching Environments

# Deploy to dev
fission deploy --dev

# Deploy to prod (default)
fission deploy

# Specify namespace
fission deploy --namespace fission-staging

Secrets Management

Creating Secrets

# Basic secret from literals
kubectl create secret generic fission-myproject-env \
  --from-literal=PG_HOST=localhost \
  --from-literal=PG_PORT=5432

# From file
kubectl create secret generic fission-myproject-env \
  --from-file=secrets.properties

# With multiple namespaces
kubectl create secret generic fission-myproject-env \
  --namespace fission-dev \
  --from-literal=PG_HOST=dev-db.example.com

Encrypted Secrets (Vault)

To encrypt sensitive values:

# On your local machine (with PyNaCl installed)
from vault import encrypt_vault

key = "your-32-byte-hex-key-here..."  # 64 hex chars
encrypted = encrypt_vault("super-secret-password", key)
print(encrypted)  # vault:v1:base64...

Store the encrypted string in K8s secret:

kubectl create secret generic fission-myproject-env \
  --from-literal=PG_PASS='vault:v1:base64...'

Set CRYPTO_KEY in helpers.py to the hex key:

CRYPTO_KEY = "e24ad6ceed96115520f6e6dc8a0da506ae9a706823d54f30a5b75447ecf477b6"

Important: Rotate keys periodically. When changing key, re-encrypt all secrets.

Updating Secrets

# Edit secret
kubectl edit secret fission-myproject-env

# Update single key
kubectl set secret secret fission-myproject-env \
  --from-literal=PG_PASS='new-password'

# Roll function to pick up new secret
fission function update --name my-function

Rolling Updates

Deploy Changes

# Build and deploy
./src/build.sh
fission deploy

# Or deploy single function
fission function update --name my-function

Zero-Downtime Deployments

Fission handles rolling updates automatically:

1. New package is built
2. New function pods are created with new code
3. Old pods continue serving traffic until new pods are ready
4. Old pods are terminated

No downtime by default for HTTP triggers.

Canary Deployments

For canary deployments:

1. Deploy new version with different function name: my-function-v2
2. Route some traffic using ingress annotations or service mesh
3. Gradually shift traffic
4. Delete old function

Monitoring & Logging

Viewing Logs

# All function logs in namespace
kubectl logs -n fission -l fission-function=true --tail=100

# Specific function
kubectl logs -n fission -l fission-function/name=my-function --tail=100

# Follow logs
kubectl logs -n fission -l fission-function/name=my-function -f

# Container logs (if multiple containers)
kubectl logs -n fission -l fission-function/name=my-function -c builder

Structured Logging

Use logger from helpers.py (already configured):

logger.info("Processing request", extra={"user_id": user_id})
logger.error("Database error", exc_info=True, extra={"query": sql})

Logs are collected by the container runtime and available via kubectl logs.

Metrics

Fission exposes Prometheus metrics:

# Get metrics endpoint
kubectl port-forward -n fission svc/fission-prometheus-server 9090:9090

# Or query via kubectl
kubectl get --raw "/apis/metrics.k8s.io/v1beta1/namespaces/fission/pods/*" | jq .

Metrics include:

• Request rate
• Error rate
• Response latency
• Pod counts

Troubleshooting

Deployment Fails

Error: Error building package

Check:

• build.sh is executable: chmod +x src/build.sh
• All dependencies in requirements.txt are valid
• Python syntax is correct: python -m py_compile src/*.py

Error: Function not found after deploy

Check:

• Fission docstring block is properly formatted (must be ````fission` with backticks)
• No YAML/JSON syntax errors in docstring
• Function file is in src/ directory

Function Not Responding

Check pod status:

kubectl get pods -n fission -l fission-function/name=my-function

Pod stuck in Pending - Insufficient resources or image pull error

Pod stuck in ContainerCreating - Volume mount issue or image pull

Pod CrashLoopBackOff - Application error. Check logs:

kubectl logs -n fission <pod-name> --previous

Configuration Not Loading

Secrets not available:

# Check secret exists in correct namespace
kubectl get secret fission-myproject-env -n fission

# Verify secret is mounted
kubectl exec -it <pod-name> -n fission -- ls /secrets/default/

ConfigMaps not available:

kubectl get configmap fission-myproject-config -n fission

Profusion parms not reading:

• Ensure SECRET_NAME in helpers.py matches created secret name
• Path format: /secrets/{namespace}/{secret-name}/{key}

Slow Performance

1. Increase resources: Raise maxmemory and maxcpu
2. Connection pooling: Use connection pooler like PgBouncer for heavy DB load
3. Database queries: Check slow queries, add indexes
4. Cold starts: Set minscale: 1 with newdeploy executor to keep warm

Database Connection Errors

Error: could not connect to server: Connection refused

• Verify database is reachable from cluster
• Check security groups/network policies
• Test connectivity from pod:

  kubectl exec -it <pod-name> -n fission -- nc -zv $PG_HOST $PG_PORT
  

Error: password authentication failed

• Verify credentials in secret
• Check PG_USER format (with plaintext: prefix for vault)

Advanced Topics

Custom Runtime Image

If you need system packages:

FROM ghcr.io/fission/python-env:latest
RUN apk add --no-cache gcc libffi-dev

Build and push:

docker build -t myregistry/python-custom:latest .
docker push myregistry/python-custom:latest

Update deployment.json:

"environments": {
  "myproject-py": {
    "image": "myregistry/python-custom:latest",
    ...
  }
}

Environment Variables from ConfigMap

"configmaps": {
  "fission-myproject-config": {
    "literals": [
      "LOG_LEVEL=DEBUG",
      "FEATURE_FLAG_X=true"
    ]
  }
}

Access in code:

import os
log_level = os.getenv("LOG_LEVEL", "INFO")

Lifecycle Hooks

Use function_pre_remove and function_post_remove in deployment hooks:

"hooks": {
  "function_pre_remove": [
    {
      "type": "http",
      "url": "http://cleanup-service/cleanup",
      "timeout": 30000
    }
  ]
}

Common Commands Reference

# List functions
fission function list

# Test function manually
fission function test --name my-function

# Update single function
fission function update --name my-function

# Delete function
fission function delete --name my-function

# View function pods
kubectl get pods -n fission -l fission-function/name=my-function

# View logs
kubectl logs -n fission -l fission-function/name=my-function -f

# Exec into pod
kubectl exec -it <pod-name> -n fission -- /bin/sh

# Describe function
fission function describe --name my-function

# Get function YAML
fission function get --name my-function -o yaml

# Check Fission version
fission version

# Check Fission status
kubectl get pods -n fission

Further Reading

• Fission Deployment Documentation
• Fission Executors
• Kubernetes Resource Management
• Kubernetes Secrets