Skip to main content
Version: Next

Enabling Local Development

occ remote lets a developer run code on their own machine against an environment's real upstreams. occ itself opens the local listener for each dialable address a dependency resolves to and owns the tunnel into the data plane; the control plane only resolves the dependencies and issues the capability that authorizes those tunnels, and carries none of their bytes.

The chart default is off on both planes, because the feature needs an address developers can reach, and that is deployment-specific. This page covers what to turn on, what gets provisioned, and how to constrain it.

note

The k3d values files (install/k3d/single-cluster/values-cp.yaml and values-dp.yaml, used by the k3d quick start) already enable both sides, with the router on NodePort 30443 and entrypointAddress: 127.0.0.1:30443. If you installed that way, the feature is live and this page is background reading.

Architecture​

Two components make up the data path, and the control plane is not on it:

ComponentWhere it runsHow it gets there
remote-agent-routerOne per data planeYou enable it in the openchoreo-data-plane chart
remote-agentOne per project + environment data-plane namespaceProvisioned on demand by openchoreo-api, shared between sessions, and reaped when idle

occ dials the router, the single entrypoint for a data plane. The router reads the TLS ClientHello's SNI, looks up the matching agent Service, and splices raw bytes. TLS terminates at the agent, never at the router, so occ pins each agent's own certificate. The agent authorizes every stream back to openchoreo-api and dials the dependency itself.

This means one exposed address per data plane no matter how many projects, environments, or developers are active, and no dependency on your HTTP gateway or its gateway class.

The router finds agents by watching Services labelled app.kubernetes.io/managed-by=openchoreo-api-remote-connect cluster-wide and reading each one's openchoreo.dev/remote-connect-sni annotation, refreshing every 5 seconds. An SNI it has no backend for is dropped and logged, so a connection arriving in the seconds after an agent is provisioned may be retried. That is why occ retries its dial for 30 seconds.

Enabling It​

1. The router, on each data plane​

In the openchoreo-data-plane chart:

remoteAgentRouter:
  enabled: true
  service:
    type: LoadBalancer # NodePort for k3d or bare metal
ValueDefaultNotes
remoteAgentRouter.enabledfalseEnables the router Deployment, Service, and its RBAC.
remoteAgentRouter.listenPort8443The L4 port the router listens on.
remoteAgentRouter.service.typeLoadBalancerOne internal load balancer for cloud, or NodePort for k3d and bare metal.
remoteAgentRouter.service.nodePort30443Fixed node port when the type is NodePort. Ignored otherwise.
remoteAgentRouter.image.repositoryghcr.io/openchoreo/remote-agent-routerOverride for a private registry or air-gapped install.
remoteAgentRouter.logLevelinfodebug logs SNI-to-backend routing decisions.

The router's only cluster permission is list/watch on Services, which is how it discovers agent Services to route to.

2. The control plane​

In the openchoreo-control-plane chart, under openchoreoApi.config:

openchoreoApi:
  config:
    remoteConnect:
      enabled: true
      entrypointAddress: "remote.example.com:8443"

entrypointAddress is the router's host:port as reachable from a developer's machine, and it is required when the feature is enabled, because openchoreo-api refuses to start without it. Take it from the router Service's external address. On a k3d install it is 127.0.0.1:30443, which requires the router Service to be a NodePort on 30443 and the cluster to publish that port:

# install/k3d/single-cluster/config.yaml
ports:
  - port: 30443:30443
    nodeFilters:
      - loadbalancer
note

With more than one data plane, each needs its own router and its own reachable address.

3. Multi-cluster only: point the agents back at the control plane​

Every provisioned agent calls openchoreo-api once per stream to authorize it, and periodically to refresh its liveness. The default authorizeUrl is the control plane's in-cluster Service DNS, which does not resolve from a data plane in a separate cluster. Set it to an address reachable from that cluster, keeping the standard path:

openchoreoApi:
  config:
    remoteConnect:
      authorizeUrl: "https://api.example.com/api/v1/remote-connect:authorize"

Without this the agents come up but every stream is refused. Single-cluster installs need no change.

4. Grant the actions​

Access is decided per dependency at resolve time. A developer's role needs:

ActionGrants
component:connectTunnelling to an endpoint of that component
resource:connectTunnelling to that Resource, and reading its ConfigMap-backed values
resource:read-secretsReading that Resource's Secret-backed values

The built-in developer and platform-engineer roles carry all three. Bind them at whatever scope suits you: a component, a project, a namespace, or cluster-wide. See Custom Roles.

resource:connect and resource:read-secrets are separate on purpose: a role can be given a tunnel to a database without being given the credential that opens it. The component a developer is running needs no grant, since it need not exist in OpenChoreo at all.

Restricting the actions to one environment​

All three actions accept the resource.environment attribute, so a condition can hold tunnelling to a single environment while leaving the rest of the role alone:

apiVersion: openchoreo.dev/v1alpha1
kind: ClusterAuthzRoleBinding
metadata:
  name: developer-binding
spec:
  effect: allow
  entitlement:
    claim: groups
    value: developers
  roleMappings:
    - roleRef:
        kind: ClusterAuthzRole
        name: developer
      conditions:
        - actions:
            - component:connect
            - resource:connect
            - resource:read-secrets
          expression: resource.environment == "default/development"

resource.environment is namespace-qualified: default/development is the Environment named development in namespace default. Use the bare name (development) for a cluster-scoped Environment.

The condition covers only the three actions it lists, so every other action the developer role grants stays unconditional. Resolving against any other environment then reports each dependency as unavailable, the same message a developer sees for a provider that is not deployed.

warning

developer-binding is one of the bindings the control-plane chart's authz bootstrap creates, and that bootstrap re-runs kubectl apply on every helm upgrade, while the chart template renders no conditions field. A condition added to a bootstrap binding out of band is dropped on the next upgrade. To make the restriction durable, remove the three actions from the developer role in openchoreoApi.config.security.authorization.bootstrap.roles and grant them through a role and binding you manage yourself.

What Gets Provisioned​

On the first resolve for a project and environment, openchoreo-api server-side-applies into that project's data-plane namespace:

  • A remote-agent Deployment and a ClusterIP Service, annotated with the SNI the router matches on
  • A TLS Secret holding the agent's self-signed certificate
  • A ServiceAccount, and (only when a session reads dependency values) a Role and RoleBinding

Subsequent sessions for the same project and environment reuse the same agent. It carries the openchoreo.dev/system-component label so per-component network policies admit it.

The certificate is generated on first use with the agent's SNI as its only SAN, is valid for 90 days, and is reissued once it comes within 30 days of expiry. The agent loads its keypair at start-up, so a reissue rolls the pod. An agent pod restarting roughly every two months is that, not a fault. A roll drops any tunnel attached at the time, and occ does not reconnect, so the developer re-runs occ remote.

The agent's read permissions​

The agent's Role is namespace-scoped and restricted by resourceNames to exactly the Secrets and ConfigMaps that live sessions asked for, not to the namespace. A compromised agent cannot read an object no session requested. Past 128 named objects, provisioning fails instead of widening the grant to the whole namespace.

An agent serving no value reads gets no Role at all.

Reaping​

An idle agent is deleted, and its read Role is reaped on a shorter clock of its own:

ValueDefaultNotes
reaperTtlSeconds1800How long an agent may be idle before its Deployment, Service and cert Secret are deleted.
reaperIntervalSeconds300How often the reaper sweeps every data plane.

An agent's liveness is refreshed by a resolve, by each stream it authorizes, and by a heartbeat the agent itself sends every 60 seconds while it holds at least one attached occ session, even one opening no new streams. That last path is what keeps a developer sitting at a breakpoint from being reaped mid-session. Still, set reaperTtlSeconds no lower than a typical session.

The read Role and RoleBinding are reaped separately, after secretTtlSeconds without a read, while the agent keeps running. A session reads its values once at startup, so this revokes the agent's Kubernetes access for the rest of a long session without disturbing its tunnels. The ServiceAccount stays, because the Deployment names it.

warning

Heartbeats are only configured when authorizeUrl ends in the standard /api/v1/remote-connect:authorize path: the provisioner derives the heartbeat URL from it by swapping that path. Point authorizeUrl at a non-standard path and the agent starts with heartbeats off (it logs a warning), leaving an idle-but-attached session reapable at reaperTtlSeconds. Change the host, not the path.

Constraining the Feature​

Session lifetime​

Authorization is decided once, at resolve. The capability's lifetime is the entire revocation window. Revoking a role takes effect at the developer's next session, not the current one.

ValueDefaultNotes
ttlSeconds1800Capability lifetime for tunnelling.
secretTtlSeconds600Capability lifetime when it also authorizes reading dependency values. Shorter on purpose, since that is the revocation window for a credential read. 0 falls back to ttlSeconds.

Shortening these makes revocation faster at the cost of more re-runs: a developer whose session lapses must exit the subshell and re-run occ remote.

Turning value resolution off entirely​

remoteConnect:
  secretsEnabled: false

With this off, no capability authorizes a value read, whatever your roles grant. It is an operator kill switch independent of role configuration. Developers can still tunnel, and each affected binding is reported to them as value resolution is disabled on this control plane (remote_connect.secrets_enabled).

The capability signing key​

Capabilities are Ed25519-signed by openchoreo-api, which is also the sole verifier. Agents hold no key.

ValueDefaultNotes
generateSigningKeytrueGenerates the key into the secret below, reusing an existing one on upgrade.
signingKeySecretNameremote-connect-signing-keyThe Secret holding signing.pem.
signingKeyPath/etc/remote-connect/signing.pemWhere it is mounted.
keyIdremote-connect-1The JWT kid, for rotation.
warning

Key reuse on upgrade relies on a cluster lookup, which client-side renderers such as helm template, or Argo CD without server-side lookup, do not perform. Those regenerate the key on every render, invalidating every live occ remote session. In such a setup, set generateSigningKey: false and manage the Secret yourself. It must already exist, or openchoreo-api will not start.

Other Values​

ValueDefaultNotes
agentImage.repositoryghcr.io/openchoreo/remote-agentOverride for a private registry or air-gapped install.
agentImage.tag"" (chart appVersion)Pin the agent image. Must be a tag your data planes can pull.
agentListenPort8443The TLS port the provisioned agent listens on.
sniSuffixremote-connectAppended to the data-plane namespace to form each agent's SNI host.
issueropenchoreo-control-planeCapability iss claim.
authorizeUrlin-cluster openchoreo-api URLThe endpoint agents call per stream, as reachable from the data plane. Change it for a multi-cluster topology.
authorizeInsecurefalseSkips TLS verification on that call. Development only.

Verifying​

After enabling both planes, check the router is up and has an address:

kubectl get svc -n openchoreo-data-plane openchoreo-remote-agent-router
kubectl logs -n openchoreo-data-plane deploy/openchoreo-remote-agent-router

Then have a developer run occ remote against a workload. On the first resolve, an agent appears in that project's data-plane namespace:

kubectl get deploy,svc -n dp-<namespace>-<project>-<environment>-<hash> \
  -l app.kubernetes.io/managed-by=openchoreo-api-remote-connect

The router logs the SNI it matched and the backend it chose, which is the quickest way to confirm the path end to end.

SymptomCause
openchoreo-api crash-loops with entrypoint_address is requiredThe feature is enabled with no entrypointAddress.
Developers report resolve failing immediatelyThe feature is not enabled on the control plane.
Resolve succeeds, then occ remote fails with connect to remote-agent ...entrypointAddress is not reachable from the developer's machine. occ dials the agents eagerly and retries for 30 seconds, so this fails at startup rather than on the first connection.
Agent pods stuck in ErrImagePullThe agent image is unavailable to the data plane. Expected on k3d after image garbage collection, and a sign of a missing registry mirror in an air-gapped install.
Streams refused with not authorized while resolve succeedsThe agents cannot reach authorizeUrl from the data plane. Check the agent's logs and the value's reachability from that cluster.
Long idle sessions reaped mid-sessionauthorizeUrl uses a non-standard path, so heartbeats were never configured. See the warning under Reaping.