Version: v0.9.x

Deployment Topology

OpenChoreo uses a multi-plane architecture that separates concerns between control, runtime, build, and observability. This guide covers the prerequisites, topology options, and how planes connect.

Prerequisites

Before deploying OpenChoreo to production, ensure your environment meets the following requirements.

Kubernetes Cluster

Requirement	Specification
Kubernetes Version	1.32 or later
Node Count	Minimum 3 nodes (for high availability)
Node Resources	4 CPU cores, 8 GB RAM per node (minimum)
RBAC	Enabled
LoadBalancer	Required for external access
Storage Classes	At least one default StorageClass for PersistentVolumeClaims

note

For development or testing, a single-node cluster with 8 GB RAM and 4 CPU cores is sufficient. Production deployments should use a multi-node cluster with appropriate resource allocation.

Required Tools

Install the following tools on your workstation:

Tool	Version	Notes
kubectl	1.32+	Kubernetes CLI
Helm	3.12+	Package manager
cert-manager	1.12+	Required on all clusters where OpenChoreo planes are deployed

LoadBalancer Requirements

OpenChoreo services require LoadBalancer with the ability to dynamically assign IPs. In production, each plane exposing services on port 443 needs a separate IP address (since multiple services cannot share the same IP:port combination).

If only a single IP is available, ports can be customized per service. Refer to the Helm Charts Reference for port configuration options.

IPs and Ports by Plane

Control Plane:

Port	Purpose
443	Backstage UI, OpenChoreo API, Identity Provider
80	HTTP to HTTPS redirect (optional)

Data Plane:

Port	Purpose
443	Application endpoints, deployment invocations
80	HTTP to HTTPS redirect (optional)

Build Plane (Optional):

No inbound ports required
Only outbound connectivity needed (to Control Plane, container registry)

Observability Plane (Optional):

Port	Purpose
443	OpenSearch ingestion endpoint

Domain Requirements

Each plane requires its own domain(s). These do not need to share a common base domain - configure each independently as needed.

Plane	Domains Required
Control Plane	Console UI domain, API domain, Identity Provider domain
Data Plane	Application endpoints domain (wildcard for dynamic apps)
Build Plane (Optional)	No external domain required
Observability Plane (Optional)	OpenSearch/ingestion endpoint domain

Refer to the Helm Charts Reference for domain configuration options.

TLS Certificate Requirements

Plane	Certificate Type	Domains
Control Plane	Standard (SAN)	Console, API, Identity Provider
Data Plane	Wildcard	`*.{apps-domain}` for application endpoints
Build Plane (Optional)	None required (internal only)
Observability Plane (Optional)	Standard	Ingestion endpoint

Use cert-manager to automatically provision and renew certificates, or bring your own certificates.

Resource Requirements

Control Plane

Component	CPU Request	CPU Limit	Memory Request	Memory Limit
Backstage	200m	2000m	256Mi	2Gi
OpenChoreo API	100m	500m	256Mi	512Mi
Controller Manager	100m	500m	128Mi	512Mi
Cluster Gateway	100m	500m	128Mi	256Mi

Data Plane

Component	CPU Request	CPU Limit	Memory Request	Memory Limit
KGateway Controller	100m	200m	128Mi	256Mi
Cluster Agent	50m	100m	128Mi	256Mi

Build Plane (Optional)

Component	CPU Request	CPU Limit	Memory Request	Memory Limit
Argo Workflows Controller	25m	50m	32Mi	64Mi
Cluster Agent	50m	100m	128Mi	256Mi

Observability Plane (Optional)

Component	CPU Request	CPU Limit	Memory Request	Memory Limit
OpenSearch (per node)	100m	1000m	1Gi	1Gi
Observer	100m	200m	128Mi	200Mi
Cluster Agent	50m	100m	128Mi	256Mi

Storage Requirements

Plane	Component	Default Size
Control Plane	Database	500Mi
Build Plane (Optional)	Container Registry	10Gi
Build Plane (Optional)	Podman Cache	10Gi
Observability Plane (Optional)	OpenSearch (per data node)	5Gi

note

OpenSearch storage requirements grow with log retention. For production with extended retention, increase openSearchCluster.nodePools.data.diskSize accordingly.

Pre-Installation Checklist

Before proceeding with installation:

Kubernetes cluster meets version and resource requirements
kubectl and helm are installed and configured
LoadBalancer service type is available
Storage class is configured for PersistentVolumeClaims
Domains are registered and DNS is configurable
TLS certificate strategy is determined (cert-manager or bring your own)
Network connectivity between clusters is verified (for multi-cluster)
Required ports are accessible through firewalls

Architecture Overview

Plane	Required	Purpose
Control Plane	Yes	Central management, API, UI
Data Plane	Yes	Application runtime
Build Plane	No	CI/CD workflows
Observability Plane	No	Logging and monitoring

Resource Hierarchy

Organization
├── Environment
│   └── references → DataPlane
├── DataPlane
│   └── references → ObservabilityPlane (optional)
├── BuildPlane
│   └── references → ObservabilityPlane (optional)
└── ObservabilityPlane

Cluster Agent Connectivity

OpenChoreo planes communicate via Cluster Agents using WebSocket connections. The agent runs in each remote plane and establishes an outbound connection to the Control Plane's Cluster Gateway.

Single-Cluster Mode: Communication happens via Kubernetes ClusterIP services - no external networking required between planes.

Multi-Cluster Mode: Each plane's cluster agent connects outbound to the Control Plane. Only the Control Plane needs to be reachable from other clusters.

┌─────────────────────────────────────────────────────────────────────────────┐
│                          Control Plane Cluster                              │
│                        ┌─────────────────────┐                              │
│                        │   Cluster Gateway   │                              │
│                        │     (wss://:8443)   │                              │
│                        └──────────▲──────────┘                              │
└───────────────────────────────────┼─────────────────────────────────────────┘
                                    │
            ┌───────────────────────┼───────────────────────┐
            │                       │                       │
            │ WebSocket             │ WebSocket             │ WebSocket
            │ (outbound)            │ (outbound)            │ (outbound)
            │                       │                       │
┌───────────┴───────────┐ ┌─────────┴────────┐ ┌────────────┴────────────────┐
│  Data Plane Cluster   │ │ Build Plane      │ │ Observability Plane Cluster │
│  ┌─────────────────┐  │ │ Cluster          │ │ (Optional)                  │
│  │  Cluster Agent  │  │ │ (Optional)       │ │  ┌─────────────────┐        │
│  └─────────────────┘  │ │ ┌─────────────┐  │ │  │  Cluster Agent  │        │
└───────────────────────┘ │ │Cluster Agent│  │ │  └─────────────────┘        │
                          │ └─────────────┘  │ └─────────────────────────────┘
                          └──────────────────┘

Deployment Workflow (Order of Operations)

When building your production environment, follow this general sequence to ensure dependencies are met:

Control Plane Setup:
- Deploy the openchoreo-control-plane chart
- Configure TLS certificates for the Console, API, and IdP
- Crucial: Configure the Cluster Gateway ingress and certificate - this is the entry point for all other planes
Establish Trust (Multi-Cluster Only):
- Extract the Cluster Gateway CA certificate from the Control Plane
- This CA is required by all other planes to verify the Control Plane's identity
- See Multi-Cluster Connectivity for detailed steps
Observability Plane (Recommended First):
- Deploy the openchoreo-observability-plane chart
- Register it with the Control Plane using the ObservabilityPlane CRD
- This allows subsequent planes to immediately start sending logs and metrics
Data Plane(s):
- Deploy the openchoreo-data-plane chart
- Configure the ingress/gateway for your application workloads (*.apps.example.com)
- Register with the Control Plane using the DataPlane CRD
- Link to Observability Plane (if deployed)
Build Plane (Optional):
- Deploy the openchoreo-build-plane chart
- Configure the Container Registry ingress - this is critical for the Control Plane to read image metadata
- Register with the Control Plane using the BuildPlane CRD

Common Topologies

Single-Cluster

All planes run in the same Kubernetes cluster with namespace isolation. Communication happens via Kubernetes ClusterIP services.

┌─────────────────────────────────────────────────────────────┐
│                     Single Kubernetes Cluster               │
├─────────────────┬─────────────────┬─────────────────────────┤
│ Control Plane   │ Data Plane      │ Build & Observability   │
│ Namespace       │ Namespace       │ Namespaces              │
└─────────────────┴─────────────────┴─────────────────────────┘

Multi-Cluster

Separate clusters provide isolation and independent scaling. Cluster agents establish outbound WebSocket connections to the Control Plane.

┌──────────────────┐     ┌──────────────────┐
│  Control Plane   │◀────│    Data Plane    │
│     Cluster      │     │     Cluster      │
└────────┬─────────┘     └──────────────────┘
         │
         │               ┌──────────────────┐
         │◀──────────────│   Build Plane    │
         │               │     Cluster      │
         │               └──────────────────┘
         │
         │               ┌──────────────────┐
         └───────────────│ Observability    │
                         │     Cluster      │
                         └──────────────────┘

For multi-cluster setup instructions, see Multi-Cluster Connectivity.

Multi-Region

Central Control Plane with Data Planes across regions. Each Data Plane has a unique gateway.publicVirtualHost (e.g., us.apps.example.com, eu.apps.example.com).

                    ┌──────────────────┐
                    │  Control Plane   │
                    │    (Central)     │
                    └────────┬─────────┘
                             │
         ┌───────────────────┼───────────────────┐
         ▼                   ▼                   ▼
┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐
│  Data Plane US   │ │  Data Plane EU   │ │  Data Plane APAC │
└──────────────────┘ └──────────────────┘ └──────────────────┘

Plane Resources

Organizations

An Organization is the top-level tenant resource. All other OpenChoreo resources (Environment, DataPlane, BuildPlane, ObservabilityPlane) are namespaced within an organization.

A default organization (default-organization) is created during installation.

For complete configuration options, see the Organization API Reference.

Environments

An Environment represents a deployment stage (development, staging, production) bound to a specific DataPlane.

Key considerations:

The dataPlaneRef is immutable - to change the DataPlane, delete and recreate the Environment
The gateway.dnsPrefix determines the subdomain for deployed applications

For complete configuration options, see the Environment API Reference.

DataPlane

A DataPlane defines where application workloads run. It requires:

Cluster agent configuration with CA certificate for authentication
Gateway configuration with public virtual host for application endpoints

For complete configuration options and examples, see the DataPlane API Reference.

BuildPlane (Optional)

A BuildPlane provides infrastructure for CI/CD workflows. When enabled:

Requires cluster agent configuration with CA certificate
Can reference an ObservabilityPlane for build logs

For complete configuration options, see the BuildPlane API Reference.

ObservabilityPlane (Optional)

An ObservabilityPlane provides logging and monitoring infrastructure. Key fields:

observerURL: The Observer service endpoint
agent: Cluster agent configuration for communication

Both DataPlane and BuildPlane can reference an ObservabilityPlane via observabilityPlaneRef.

Linking Planes

Planes reference each other in their spec to form a logical topology.

Resource	Reference Field	Target
Environment	`dataPlaneRef`	DataPlane
DataPlane	`observabilityPlaneRef`	ObservabilityPlane
DataPlane	`secretStoreRef`	ClusterSecretStore
BuildPlane	`observabilityPlaneRef`	ObservabilityPlane

For detailed instructions on registering planes and establishing trust between clusters, see Multi-Cluster Connectivity.

Verifying Resources

kubectl get organizations
kubectl get environments -n <organization-name>
kubectl get dataplanes -n <organization-name>
kubectl get buildplanes -n <organization-name>
kubectl get observabilityplanes -n <organization-name>

Troubleshooting

Agent Connection Issues

# Check agent pod status
kubectl get pods -n openchoreo-data-plane -l app.kubernetes.io/name=cluster-agent

# View agent logs
kubectl logs -n openchoreo-data-plane deployment/cluster-agent

Environment Not Ready

# Check environment conditions
kubectl describe environment <name> -n <organization-name>

# Verify referenced DataPlane exists
kubectl get dataplane <dataplane-ref> -n <organization-name>

Plane Status Not Updating

# Check controller-manager logs
kubectl logs -n openchoreo-control-plane deployment/controller-manager

# View plane conditions
kubectl get dataplane <name> -n <organization-name> -o jsonpath='{.status.conditions}'

DataPlane API Reference
BuildPlane API Reference
Environment API Reference
Organization API Reference
Multi-Cluster Connectivity - Step-by-step multi-cluster setup
Container Registry - Registry configuration for Build Plane
Secret Management - External secret store integration

Prerequisites​

Kubernetes Cluster​

Required Tools​

LoadBalancer Requirements​

IPs and Ports by Plane​

Domain Requirements​

TLS Certificate Requirements​

Resource Requirements​

Control Plane​

Data Plane​

Build Plane (Optional)​

Observability Plane (Optional)​

Storage Requirements​

Pre-Installation Checklist​

Architecture Overview​

Resource Hierarchy​

Cluster Agent Connectivity​

Deployment Workflow (Order of Operations)​

Common Topologies​

Single-Cluster​

Multi-Cluster​

Multi-Region​

Plane Resources​

Organizations​

Environments​

DataPlane​

BuildPlane (Optional)​

ObservabilityPlane (Optional)​

Linking Planes​

Verifying Resources​

Troubleshooting​

Agent Connection Issues​

Environment Not Ready​

Plane Status Not Updating​

Related Documentation​