Version: v0.8.x

DataPlane

A DataPlane represents a Kubernetes cluster where application workloads are deployed. It defines the connection to a target Kubernetes cluster and gateway settings for routing traffic to applications.

OpenChoreo supports two modes of communication with the DataPlane:

Agent-based (Recommended): The control plane communicates with the downstream cluster through a WebSocket agent running in the DataPlane cluster
Direct Kubernetes API access: The control plane connects directly to the Kubernetes API server using client certificates or bearer tokens

API Version

openchoreo.dev/v1alpha1

Resource Definition

Metadata

DataPlanes are namespace-scoped resources that must be created within an Organization's namespace.

apiVersion: openchoreo.dev/v1alpha1
kind: DataPlane
metadata:
  name: <dataplane-name>
  namespace: <org-namespace>  # Organization namespace

Spec Fields

Field	Type	Required	Default	Description
`gateway`	GatewaySpec	Yes	-	API gateway configuration for this DataPlane
`agent`	AgentConfig	No	-	Agent-based communication configuration (recommended)
`kubernetesCluster`	KubernetesClusterSpec	No	-	Target Kubernetes cluster configuration (optional when agent is enabled)
`imagePullSecretRefs`	[]string	No	-	References to SecretReference resources for image pull secrets
`secretStoreRef`	SecretStoreRef	No	-	Reference to External Secrets Operator ClusterSecretStore in the DataPlane
`observer`	ObserverAPI	No	-	Observer API integration for monitoring and logging

AgentConfig

Configuration for agent-based communication with the downstream cluster.

Field	Type	Required	Default	Description
`enabled`	boolean	No	false	Whether agent-based communication is enabled
`clientCA`	ValueFrom	No	-	CA certificate to verify the agent's client certificate (base64-encoded PEM)

KubernetesClusterSpec

Configuration for the target Kubernetes cluster. Optional when agent.enabled is true.

Field	Type	Required	Default	Description
`server`	string	Yes	-	URL of the Kubernetes API server
`tls`	KubernetesTLS	Yes	-	TLS configuration for the connection
`auth`	KubernetesAuth	Yes	-	Authentication configuration

KubernetesTLS

TLS configuration for the Kubernetes connection.

Field	Type	Required	Default	Description
`ca`	ValueFrom	Yes	-	CA certificate

KubernetesAuth

Authentication configuration for the Kubernetes cluster. Either mtls or bearerToken must be specified.

Field	Type	Required	Default	Description
`mtls`	MTLSAuth	No	-	Certificate-based authentication (mTLS)
`bearerToken`	ValueFrom	No	-	Bearer token authentication

MTLSAuth

Certificate-based authentication (mTLS) configuration.

Field	Type	Required	Default	Description
`clientCert`	ValueFrom	Yes	-	Client certificate
`clientKey`	ValueFrom	Yes	-	Client private key

ValueFrom

Common pattern for referencing secrets or providing inline values. Either secretRef or value should be specified.

Field	Type	Required	Default	Description
`secretRef`	SecretKeyReference	No	-	Reference to a secret key
`value`	string	No	-	Inline value (not recommended for sensitive data)

SecretKeyReference

Reference to a specific key in a Kubernetes secret.

Field	Type	Required	Default	Description
`name`	string	Yes	-	Name of the secret
`namespace`	string	No	Same as parent resource	Namespace of the secret
`key`	string	Yes	-	Key within the secret

SecretStoreRef

Reference to an External Secrets Operator ClusterSecretStore.

Field	Type	Required	Default	Description
`name`	string	Yes	-	Name of the ClusterSecretStore in the DataPlane

GatewaySpec

Gateway configuration for the DataPlane.

Field	Type	Required	Default	Description
`publicVirtualHost`	string	Yes	-	Public virtual host for external traffic
`organizationVirtualHost`	string	Yes	-	Organization-specific virtual host for internal traffic

ObserverAPI

Configuration for Observer API integration.

Field	Type	Required	Default	Description
`url`	string	Yes	-	Base URL of the Observer API
`authentication.basicAuth.username`	string	Yes	-	Username for basic authentication
`authentication.basicAuth.password`	string	Yes	-	Password for basic authentication

Status Fields

Field	Type	Default	Description
`observedGeneration`	integer	0	The generation observed by the controller
`conditions`	[]Condition	[]	Standard Kubernetes conditions tracking the DataPlane state

Condition Types

Common condition types for DataPlane resources:

Ready - Indicates if the DataPlane is ready to accept workloads
Connected - Indicates if connection to the target cluster is established
GatewayProvisioned - Indicates if the gateway has been configured

Getting the Agent CA Certificate

When using agent-based communication (agent.enabled: true), you need to provide the cluster agent's CA certificate in the DataPlane CR. This certificate is used by the control plane to verify the identity of the data plane agent during mTLS authentication.

Extracting the CA Certificate

The cluster agent automatically generates its CA certificate when deployed to the data plane cluster. You can extract it using:

# For multi-cluster setups, specify the data plane cluster context
kubectl --context <dataplane-context> get secret cluster-agent-tls \
  -n openchoreo-data-plane \
  -o jsonpath='{.data.ca\.crt}' | base64 -d

# Example for k3d multi-cluster setup:
kubectl --context k3d-openchoreo-dp get secret cluster-agent-tls \
  -n openchoreo-data-plane \
  -o jsonpath='{.data.ca\.crt}' | base64 -d

important

In multi-cluster setups, you must specify the --context flag to target the data plane cluster, not the control plane cluster. The cluster-agent-tls secret exists in the data plane cluster where the agent is deployed.

Adding the Certificate to the DataPlane CR

You can add the CA certificate to the DataPlane CR in two ways:

Option 1: Inline value (for testing/development)

# Extract the CA certificate from the data plane cluster
CA_CERT=$(kubectl --context <dataplane-context> get secret cluster-agent-tls \
  -n openchoreo-data-plane \
  -o jsonpath='{.data.ca\.crt}' | base64 -d)

# Create DataPlane in the control plane with inline CA certificate
kubectl --context <control-plane-context> apply -f - <<EOF
apiVersion: openchoreo.dev/v1alpha1
kind: DataPlane
metadata:
  name: my-dataplane
  namespace: my-org
spec:
  agent:
    enabled: true
    clientCA:
      value: |
$(echo "$CA_CERT" | sed 's/^/        /')
  gateway:
    publicVirtualHost: api.example.com
    organizationVirtualHost: internal.example.com
  secretStoreRef:
    name: default
EOF

Option 2: Secret reference (recommended for production)

# Extract the CA certificate from the data plane cluster and save to file
kubectl --context <dataplane-context> get secret cluster-agent-tls \
  -n openchoreo-data-plane \
  -o jsonpath='{.data.ca\.crt}' | base64 -d > /tmp/dataplane-ca.crt

# Create a secret in the control plane cluster
kubectl --context <control-plane-context> create secret generic dataplane-agent-ca \
  --from-file=ca.crt=/tmp/dataplane-ca.crt \
  -n my-org

# Create DataPlane in the control plane referencing the secret
kubectl --context <control-plane-context> apply -f - <<EOF
apiVersion: openchoreo.dev/v1alpha1
kind: DataPlane
metadata:
  name: my-dataplane
  namespace: my-org
spec:
  agent:
    enabled: true
    clientCA:
      secretRef:
        name: dataplane-agent-ca
        namespace: my-org
        key: ca.crt
  gateway:
    publicVirtualHost: api.example.com
    organizationVirtualHost: internal.example.com
  secretStoreRef:
    name: default
EOF

Examples

Agent-based DataPlane

This example shows a DataPlane using agent-based communication. The control plane communicates with the downstream cluster through a WebSocket agent.

apiVersion: openchoreo.dev/v1alpha1
kind: DataPlane
metadata:
  name: agent-dataplane
  namespace: my-org
spec:
  # Agent configuration
  agent:
    enabled: true
    clientCA:
      secretRef:
        name: cluster-agent-ca
        key: ca.crt

  # Gateway configuration
  gateway:
    publicVirtualHost: api.example.com
    organizationVirtualHost: internal.example.com

  # External Secrets Operator integration
  secretStoreRef:
    name: vault-backend

  # Image pull secret references
  imagePullSecretRefs:
    - docker-registry-credentials

  # Observer API (optional)
  observer:
    url: https://observer.example.com
    authentication:
      basicAuth:
        username: admin
        password: secretpassword

Direct Kubernetes API Access DataPlane

This example shows a DataPlane using direct Kubernetes API access with mTLS authentication.

apiVersion: openchoreo.dev/v1alpha1
kind: DataPlane
metadata:
  name: production-dataplane
  namespace: my-org
spec:
  # Direct Kubernetes cluster access
  kubernetesCluster:
    server: https://k8s-api.example.com:6443
    tls:
      ca:
        secretRef:
          name: k8s-ca-cert
          key: ca.crt
    auth:
      mtls:
        clientCert:
          secretRef:
            name: k8s-client-cert
            key: tls.crt
        clientKey:
          secretRef:
            name: k8s-client-cert
            key: tls.key

  # Gateway configuration
  gateway:
    publicVirtualHost: api.example.com
    organizationVirtualHost: internal.example.com

  # Observer API (optional)
  observer:
    url: https://observer.example.com
    authentication:
      basicAuth:
        username: admin
        password: secretpassword

DataPlane with Bearer Token Authentication

This example shows a DataPlane using bearer token authentication instead of mTLS.

apiVersion: openchoreo.dev/v1alpha1
kind: DataPlane
metadata:
  name: dev-dataplane
  namespace: my-org
spec:
  kubernetesCluster:
    server: https://k8s-dev.example.com:6443
    tls:
      ca:
        secretRef:
          name: k8s-ca-cert
          key: ca.crt
    auth:
      bearerToken:
        secretRef:
          name: k8s-token
          key: token

  gateway:
    publicVirtualHost: dev-api.example.com
    organizationVirtualHost: dev-internal.example.com

DataPlane with External Secrets Integration

This example demonstrates using External Secrets Operator for managing secrets and image pull credentials.

apiVersion: openchoreo.dev/v1alpha1
kind: DataPlane
metadata:
  name: secure-dataplane
  namespace: my-org
spec:
  # Agent-based communication
  agent:
    enabled: true
    clientCA:
      secretRef:
        name: agent-ca-cert
        namespace: openchoreo-system
        key: ca.crt

  # External Secrets Operator ClusterSecretStore reference
  secretStoreRef:
    name: vault-backend

  # References to SecretReference resources
  # These will be converted to ExternalSecrets and added as imagePullSecrets
  imagePullSecretRefs:
    - docker-hub-credentials
    - gcr-credentials
    - private-registry-credentials

  gateway:
    publicVirtualHost: secure-api.example.com
    organizationVirtualHost: secure-internal.example.com

Annotations

DataPlanes support the following annotations:

Annotation	Description
`openchoreo.dev/display-name`	Human-readable name for UI display
`openchoreo.dev/description`	Detailed description of the DataPlane

Environment - Runtime environments deployed on DataPlanes
Organization - Contains DataPlane definitions
Project - Applications deployed to DataPlanes

API Version​

Resource Definition​

Metadata​

Spec Fields​

AgentConfig​

KubernetesClusterSpec​

KubernetesTLS​

KubernetesAuth​

MTLSAuth​

ValueFrom​

SecretKeyReference​

SecretStoreRef​

GatewaySpec​

ObserverAPI​

Status Fields​

Condition Types​

Getting the Agent CA Certificate​

Extracting the CA Certificate​

Adding the Certificate to the DataPlane CR​

Examples​

Agent-based DataPlane​

Direct Kubernetes API Access DataPlane​

DataPlane with Bearer Token Authentication​

DataPlane with External Secrets Integration​

Annotations​

Related Resources​