Apache Airflow Helm Chart: From One Deployment to One Environment per Customer

Direct answer

The official Apache Airflow Helm chart bootstraps an Airflow deployment on Kubernetes. It supports multiple executors including LocalExecutor, CeleryExecutor, and KubernetesExecutor. A release-per-customer model gives each tenant its own Airflow environment with independent executors, secrets, domains, DAG delivery, and upgrade windows. OpenTenant automates the tenant lifecycle around the upstream chart.

What the Airflow Helm chart provides

The chart supports:

Multiple executors: LocalExecutor, CeleryExecutor, KubernetesExecutor, and hybrid executors
PostgreSQL and MySQL database backends
KEDA-based autoscaling for CeleryExecutor
StatsD/Prometheus metrics
Automatic database migration after deployment
Administrator account creation
One-command deployment for any executor type

The chart can be extended by creating a custom chart that depends on the Airflow chart. This is the recommended approach for adding tenant-specific customizations.

The use case: dedicated workflow orchestration per customer

A release-per-customer model fits products that embed or resell Airflow while promising operational separation. Each customer can receive its own release, namespace, hostname, database, executor configuration, DAG delivery path, and upgrade window.

This pattern is useful when customers need:

Independent DAG stores and execution histories
Customer-specific executors and worker scaling
Separate database and metadata stores
Controlled plugin and provider configurations
Maintenance windows that do not affect every tenant at once

Reference architecture

customer Airflow profile
  → validate executor, database, and DAG requirements
  → resolve tenant-scoped database and broker secrets
  → render Airflow values for one release
  → install into the tenant namespace
  → verify database migration, scheduler, and workers
  → promote or roll back independently

For CeleryExecutor, each tenant gets its own PostgreSQL database, Redis broker, worker set, and optional Flower dashboard. For KubernetesExecutor, each task runs in its own pod, providing strong isolation at the task level.

Illustrative tenant declaration

tenant: acme
userspace:
  namespace: tenant-acme-airflow
  domain: airflow.acme.example
deployment:
  engine: helm
  template: airflow
  release: airflow-acme
  config:
    executor: CeleryExecutor
    workers:
      replicas: 3
      resources:
        requests:
          cpu: 500m
          memory: 1Gi
    dags:
      gitSync:
        repo: https://git.acme.com/airflow/dags-acme
        branch: main
  secretScope: tenants/acme/airflow

This is an architectural example. Production declarations should reference versioned templates and Secret objects.

Production decisions before onboarding

Executor selection. Choose LocalExecutor for single-node, CeleryExecutor for distributed workers with a broker, or KubernetesExecutor for per-task pod isolation. The executor choice affects resource requirements, scaling behavior, and operational complexity.

Database topology. Airflow requires PostgreSQL or MySQL. Decide whether each customer receives a dedicated database instance, a dedicated database on a shared service, or a governed shared topology.

DAG delivery. DAGs must be delivered to each tenant's environment. Options include Git sync, persistent volume, or object storage. The Git sync approach allows version-controlled DAG definitions per tenant.

Worker scaling. For CeleryExecutor, configure worker replicas and resources. KEDA autoscaling is supported for CeleryExecutor. Each tenant's worker scaling should be independent.

Secret management. Airflow connections, variables, and XComs contain sensitive data. Store them in tenant-scoped secret storage and reference them through the Airflow chart's Secret configuration.

Evaluation checklist

Does every tenant have an explicit executor configuration?
Are database and broker credentials isolated per tenant?
Are DAGs delivered securely and version-controlled per tenant?
Can one tenant's DAG execution affect another tenant's resources?
Can you upgrade one tenant, verify it, and stop the rollout?
Does offboarding remove routes, runtime access, and retained metadata?
Are database migrations tested and reversible per tenant?

Sources

OpenTenant makes Airflow tenant operations repeatable—each customer gets a complete environment with its own scheduling, workers, and DAG lifecycle.