Skip to main content
Version: v0.8.x

Single Cluster Production Setup

Deploy all OpenChoreo planes on a single Kubernetes cluster with your custom domain and TLS certificates.

Before You Begin​

Read Deployment Planning to understand:

  • Domain requirements per plane
  • TLS certificate options
  • HA considerations

Prerequisites​

  • Kubernetes 1.32+ cluster (3+ nodes, 4 CPU / 8GB RAM each recommended)
  • kubectl and Helm configured
  • Your base domain (e.g., example.com)
  • DNS access to create records
  • LoadBalancer support
  • cert-manager installed in your cluster

Set your base domain:

export DOMAIN="example.com"

Verify cluster access:

kubectl version
helm version --short
kubectl get nodes
Install cert-manager
helm upgrade --install cert-manager oci://quay.io/jetstack/charts/cert-manager \
    --namespace cert-manager \
    --create-namespace \
    --set crds.enabled=true

Wait for cert-manager to be ready:

kubectl wait --for=condition=available deployment/cert-manager -n cert-manager --timeout=120s

Step 1: Setup Control Plane​

Deploy the Control Plane Chart

helm upgrade --install openchoreo-control-plane oci://ghcr.io/openchoreo/helm-charts/openchoreo-control-plane \
  --version 0.8.0 \
  --namespace openchoreo-control-plane \
  --create-namespace \
  --set global.baseDomain=openchoreo.${DOMAIN} \
  --set global.tls.enabled=true \
  --set "backstage.ingress.tls[0].secretName=control-plane-tls" \
  --set "backstage.ingress.tls[0].hosts[0]=openchoreo.${DOMAIN}" \
  --set "openchoreoApi.ingress.tls[0].secretName=control-plane-tls" \
  --set "openchoreoApi.ingress.tls[0].hosts[0]=api.openchoreo.${DOMAIN}" \
  --set "thunder.ocIngress.tls[0].secretName=control-plane-tls" \
  --set "thunder.ocIngress.tls[0].hosts[0]=thunder.openchoreo.${DOMAIN}" \
  --set thunder.configuration.server.publicUrl=https://thunder.openchoreo.${DOMAIN} \
  --set thunder.configuration.gateClient.hostname=thunder.openchoreo.${DOMAIN} \
  --set thunder.configuration.gateClient.port=443 \
  --set thunder.configuration.gateClient.scheme="https"

Wait for pods to start:

kubectl get pods -n openchoreo-control-plane -w

Configure DNS

Wait for LoadBalancer to get an external IP (press Ctrl+C once EXTERNAL-IP appears):

kubectl get svc openchoreo-traefik -n openchoreo-control-plane -w

Create DNS records pointing to the LoadBalancer IP (or hostname for EKS):

RecordValue
openchoreo.${DOMAIN}LoadBalancer IP
api.openchoreo.${DOMAIN}LoadBalancer IP
thunder.openchoreo.${DOMAIN}LoadBalancer IP

Configure TLS

HTTP-01 validation requires your LoadBalancer to be publicly accessible on port 80.

kubectl apply -f - <<EOF
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  name: letsencrypt
  namespace: openchoreo-control-plane
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: admin@${DOMAIN}
    privateKeySecretRef:
      name: letsencrypt-account-key
    solvers:
      - http01:
          ingress:
            ingressClassName: openchoreo-traefik
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: control-plane-tls
  namespace: openchoreo-control-plane
spec:
  secretName: control-plane-tls
  issuerRef:
    name: letsencrypt
    kind: Issuer
  dnsNames:
    - "openchoreo.${DOMAIN}"
    - "api.openchoreo.${DOMAIN}"
    - "thunder.openchoreo.${DOMAIN}"
EOF

Wait for certificate (until READY shows True):

kubectl get certificate control-plane-tls -n openchoreo-control-plane -w

Step 2: Setup Data Plane​

Deploy the Data Plane Chart

helm upgrade --install openchoreo-data-plane oci://ghcr.io/openchoreo/helm-charts/openchoreo-data-plane \
  --version 0.8.0 \
  --namespace openchoreo-data-plane \
  --set gateway.httpPort=19080 \
  --set gateway.httpsPort=19443 \
  --create-namespace

Configure DNS

Wait for LoadBalancer to get an external IP (press Ctrl+C once EXTERNAL-IP appears):

kubectl get svc gateway-default -n openchoreo-data-plane -w

Create DNS records:

RecordValue
apps.openchoreo.${DOMAIN}Gateway LoadBalancer IP
*.apps.openchoreo.${DOMAIN}Gateway LoadBalancer IP

Configure TLS

Wildcard Certificates Require DNS-01

The data plane gateway uses wildcard hostnames (*.apps.openchoreo.${DOMAIN}). HTTP-01 validation cannot issue wildcard certificates - only DNS-01 validation or bring-your-own certificates work.

DNS-01 validation requires configuring your DNS provider. See cert-manager DNS01 docs for all supported providers.

Example for Cloudflare:

Create secret with API token:

kubectl create secret generic cloudflare-api-token \
  --from-literal=api-token=YOUR_CLOUDFLARE_API_TOKEN \
  -n openchoreo-data-plane

Create the Issuer and Certificate:

kubectl apply -f - <<EOF
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  name: letsencrypt
  namespace: openchoreo-data-plane
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: admin@${DOMAIN}
    privateKeySecretRef:
      name: letsencrypt-account-key
    solvers:
      - dns01:
          cloudflare:
            apiTokenSecretRef:
              name: cloudflare-api-token
              key: api-token
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: openchoreo-gateway-tls
  namespace: openchoreo-data-plane
spec:
  secretName: openchoreo-gateway-tls
  issuerRef:
    name: letsencrypt
    kind: Issuer
  dnsNames:
    - "apps.openchoreo.${DOMAIN}"
    - "*.apps.openchoreo.${DOMAIN}"
EOF

Wait for certificate:

kubectl get certificate openchoreo-gateway-tls -n openchoreo-data-plane -w

Register

CA_CERT=$(kubectl get secret cluster-agent-tls -n openchoreo-data-plane -o jsonpath='{.data.ca\.crt}' | base64 -d)

kubectl apply -f - <<EOF
apiVersion: openchoreo.dev/v1alpha1
kind: DataPlane
metadata:
  name: default
  namespace: default
spec:
  agent:
    enabled: true
    clientCA:
      value: |
$(echo "$CA_CERT" | sed 's/^/        /')
  gateway:
    organizationVirtualHost: "openchoreoapis.internal"
    publicVirtualHost: "apps.openchoreo.${DOMAIN}"
  secretStoreRef:
    name: default
EOF

Verify:

kubectl get dataplane -n default
kubectl get pods -n openchoreo-data-plane

Step 3: Setup Build Plane (Optional)​

Deploy the Build Plane Chart

helm upgrade --install openchoreo-build-plane oci://ghcr.io/openchoreo/helm-charts/openchoreo-build-plane \
  --version 0.8.0 \
  --namespace openchoreo-build-plane \
  --create-namespace \
  --set clusterAgent.enabled=true \
  --set external-secrets.enabled=false \
  --set global.baseDomain=openchoreo.${DOMAIN}

Configure DNS - Create record pointing to the Control Plane LoadBalancer:

RecordValue
registry.openchoreo.${DOMAIN}Control Plane LoadBalancer IP

Configure TLS

Registry TLS Required

The container registry must have a valid, trusted TLS certificate. Kubernetes nodes need to pull images from it.

kubectl apply -f - <<EOF
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  name: letsencrypt
  namespace: openchoreo-build-plane
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: admin@${DOMAIN}
    privateKeySecretRef:
      name: letsencrypt-account-key
    solvers:
      - http01:
          ingress:
            ingressClassName: openchoreo-traefik
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: build-plane-tls
  namespace: openchoreo-build-plane
spec:
  secretName: build-plane-tls
  issuerRef:
    name: letsencrypt
    kind: Issuer
  dnsNames:
    - "registry.openchoreo.${DOMAIN}"
EOF

Wait for certificate:

kubectl get certificate build-plane-tls -n openchoreo-build-plane -w

Upgrade with TLS - After the certificate is ready:

helm upgrade --install openchoreo-build-plane oci://ghcr.io/openchoreo/helm-charts/openchoreo-build-plane \
  --version 0.8.0 \
  --namespace openchoreo-build-plane \
  --reuse-values \
  --set global.tls.enabled=true \
  --set global.tls.secretName=build-plane-tls

Register

BP_CA_CERT=$(kubectl get secret cluster-agent-tls -n openchoreo-build-plane -o jsonpath='{.data.ca\.crt}' | base64 -d)

kubectl apply -f - <<EOF
apiVersion: openchoreo.dev/v1alpha1
kind: BuildPlane
metadata:
  name: default
  namespace: default
spec:
  agent:
    enabled: true
    clientCA:
      value: |
$(echo "$BP_CA_CERT" | sed 's/^/        /')
EOF

Verify:

kubectl get buildplane -n default
kubectl get pods -n openchoreo-build-plane

Step 4: Setup Observability Plane (Optional)​

Single-node OpenSearch for development or small deployments.

helm upgrade --install openchoreo-observability-plane oci://ghcr.io/openchoreo/helm-charts/openchoreo-observability-plane \
  --version 0.8.0 \
  --namespace openchoreo-observability-plane \
  --create-namespace \
  --set openSearch.enabled=true \
  --set openSearchCluster.enabled=false \
  --set external-secrets.enabled=false \
  --set clusterAgent.enabled=true \
  --timeout 10m

Register with the control plane:

OP_CA_CERT=$(kubectl get secret cluster-agent-tls -n openchoreo-observability-plane -o jsonpath='{.data.ca\.crt}' | base64 -d)

kubectl apply -f - <<EOF
apiVersion: openchoreo.dev/v1alpha1
kind: ObservabilityPlane
metadata:
  name: default
  namespace: default
spec:
  agent:
    enabled: true
    clientCA:
      value: |
$(echo "$OP_CA_CERT" | sed 's/^/        /')
  observerURL: http://observer.openchoreo-observability-plane.svc.cluster.local:8080
EOF

Link the Data Plane (and Build Plane if installed) to use observability:

kubectl patch dataplane default -n default --type merge -p '{"spec":{"observabilityPlaneRef":"default"}}'
kubectl patch buildplane default -n default --type merge -p '{"spec":{"observabilityPlaneRef":"default"}}'

Verify:

kubectl get observabilityplane -n default
kubectl logs -n openchoreo-observability-plane -l app=cluster-agent --tail=10

DNS Records Summary​

Here's a complete list of all DNS records required for single-cluster setup:

RecordPoints ToPlane
openchoreo.${DOMAIN}Control Plane LoadBalancer IPControl
api.openchoreo.${DOMAIN}Control Plane LoadBalancer IPControl
thunder.openchoreo.${DOMAIN}Control Plane LoadBalancer IPControl
apps.openchoreo.${DOMAIN}Data Plane Gateway LoadBalancer IPData
*.apps.openchoreo.${DOMAIN}Data Plane Gateway LoadBalancer IPData
registry.openchoreo.${DOMAIN}Control Plane LoadBalancer IPBuild (optional)

Access URLs​

ServiceURL
Consolehttps://openchoreo.${DOMAIN}
APIhttps://api.openchoreo.${DOMAIN}
Deployed Appshttps://<component>-<env>.apps.openchoreo.${DOMAIN}
Registryhttps://registry.openchoreo.${DOMAIN} (if Build Plane)

Default credentials: admin@openchoreo.dev / Admin@123

Next Steps​

  1. Deploy your first component
  2. Review Operations Guide for maintenance procedures

Cleanup​

Delete plane registrations:

kubectl delete dataplane default -n default 2>/dev/null
kubectl delete buildplane default -n default 2>/dev/null
kubectl delete observabilityplane default -n default 2>/dev/null

Uninstall OpenChoreo components:

helm uninstall openchoreo-observability-plane -n openchoreo-observability-plane 2>/dev/null
helm uninstall opensearch-operator -n openchoreo-observability-plane 2>/dev/null
helm uninstall openchoreo-build-plane -n openchoreo-build-plane 2>/dev/null
helm uninstall openchoreo-data-plane -n openchoreo-data-plane
helm uninstall openchoreo-control-plane -n openchoreo-control-plane
helm uninstall cert-manager -n cert-manager

Delete namespaces:

kubectl delete namespace openchoreo-control-plane openchoreo-data-plane \
  openchoreo-build-plane openchoreo-observability-plane cert-manager 2>/dev/null

Delete CRDs:

kubectl get crd -o name | grep -E '\.openchoreo\.dev$' | xargs -r kubectl delete

Troubleshooting​

Certificate not issuing​

kubectl describe certificate <name> -n <namespace>
kubectl get challenges -A
kubectl describe issuer <name> -n <namespace>

Common issues:

  • DNS records not propagated yet
  • Firewall blocking port 80 (for HTTP-01)
  • DNS provider credentials incorrect (for DNS-01)

Agent not connecting​

kubectl logs -n openchoreo-data-plane -l app=cluster-agent --tail=20
kubectl logs -n openchoreo-control-plane -l app=cluster-gateway --tail=20

Pods stuck in Pending​

kubectl describe pod <pod-name> -n <namespace>

Common causes: insufficient resources, PVC issues, node taints.