Onboard Self-Managed or Custom Kubernetes Clusters

Use this page to onboard:

Self-managed clusters built with kops, kubeadm, k3s, Rancher, or similar tools, running in AWS, Azure, or GCP.
Managed clusters (EKS / AKS / GKE) where you prefer to authenticate the Controller with a service-account kubeconfig instead of the cloud-account credentials.

The cluster must reside in a VPC or VNet in a supported cloud. On-premises bare-metal clusters cannot be onboarded.

For the standard managed-cluster paths (EKS, AKS, or GKE with cloud-account credentials), see the per-provider onboarding pages: Onboard EKS Clusters , Onboard AKS Clusters , or Onboard GKE Clusters .

Prerequisites

Read Kubernetes Onboarding Prerequisites first. The cluster API server must be reachable from the Controller, SNAT must be disabled, and Resource Discovery must be enabled. In addition, for this onboarding path:

You must have kubectl access to the cluster with permission to create ServiceAccounts, ClusterRoles, and ClusterRoleBindings.
You must be able to extract the cluster’s CA certificate and API endpoint to assemble a kubeconfig.

Step 1: Create a Service Account and ClusterRoleBinding

The Aviatrix Controller authenticates as a Kubernetes ServiceAccount and presents its bearer token. The single avx-controller ClusterRole below covers both SmartGroup discovery and CRD-based DCF policy — there is no separate manifest for SmartGroup-only deployments. Apply this manifest:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: avx-controller
  namespace: kube-system
---
apiVersion: v1
kind: Secret
metadata:
  name: avx-controller-token
  namespace: kube-system
  annotations:
    kubernetes.io/service-account.name: avx-controller
type: kubernetes.io/service-account-token
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: avx-controller
rules:
  - apiGroups: [""]
    resources: ["nodes", "pods", "services", "namespaces"]
    verbs: ["get", "list", "watch"]
  - apiGroups: ["discovery.k8s.io"]
    resources: ["endpointslices"]
    verbs: ["get", "list", "watch"]
  - apiGroups: ["events.k8s.io"]
    resources: ["events"]
    verbs: ["create", "patch"]
  - apiGroups: ["networking.aviatrix.com"]
    resources: ["*"]
    verbs: ["get", "list", "watch", "update", "patch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: avx-controller
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: avx-controller
subjects:
  - kind: ServiceAccount
    name: avx-controller
    namespace: kube-system

Chart release 10.0.1 and later only: helm install ... --set serviceAccount.create=true creates the ServiceAccount and Secret for you. This flag does not exist on the 8.2.0 or 9.0.0 chart releases — on those releases, apply the manifest above regardless of whether you also install the helm chart. Either way, the chart’s own ClusterRoleBinding stays bound to a Kubernetes Group, not this ServiceAccount, so this page’s ClusterRole and ClusterRoleBinding are what bind the ServiceAccount for this onboarding path.

After applying:

TOKEN=$(kubectl get secret avx-controller-token -n kube-system -o jsonpath='{.data.token}' | base64 -d)

Step 2: Assemble a kubeconfig

You need three values: the API server endpoint, the cluster’s base64-encoded CA certificate, and the bearer token from Step 1. Save as avx-kubeconfig.yaml, replacing the placeholders:

apiVersion: v1
kind: Config
clusters:
  - name: my-cluster
    cluster:
      server: <ENDPOINT>
      certificate-authority-data: <CA_DATA>
contexts:
  - name: my-cluster-context
    context:
      cluster: my-cluster
      user: avx-controller
users:
  - name: avx-controller
    user:
      token: <TOKEN>
current-context: my-cluster-context
preferences: {}

The kubeconfig must use inline certificate-authority-data (not certificate-authority) and inline token (not tokenFile or exec). The Controller does not execute external binaries and does not read from the local filesystem.

Step 3: Register the cluster

Terraform
CoPilot UI

data "local_file" "kubeconfig" {
  filename = "./avx-kubeconfig.yaml"
}

data "aviatrix_account" "this" {
  account_name = var.aviatrix_account_name
}

resource "aviatrix_kubernetes_cluster" "this" {
  cluster_id  = "my-custom-cluster-id"
  kube_config = data.local_file.kubeconfig.content

  cluster_details {
    account_name           = data.aviatrix_account.this.account_name
    account_id             = data.aviatrix_account.this.aws_account_number # or azure_subscription_id, etc.
    name                   = "my-custom-cluster"
    region                 = "us-east-2"
    vpc_id                 = "vpc-abc123" # or full Azure resource ID, etc.
    is_publicly_accessible = true
    platform               = "kops"
    version                = "1.30"
    network_mode           = "FLAT"
    tags = {
      environment = "production"
    }
  }
}

The cluster_details block tells the Controller about the cluster since it cannot discover the metadata from a cloud API (no managed cluster to query).

Navigate to Cloud Resources > Cloud Assets > Kubernetes Clusters.
Click Manually Onboard a Cluster.
Fill in the dialog:

Field	Description
Name	Display name for the cluster
Cloud	AWS, Azure, or GCP
Account	Cloud account where the cluster resides
Region	Cloud region
VPC/VNet	VPC or VNet where the cluster nodes run
Network Mode	Flat (recommended) or Overlay
Kubeconfig File	Upload the `avx-kubeconfig.yaml` from Step 2

Click Onboard.

cluster_details argument reference

Argument	Required	Description
`account_name`	Yes	Aviatrix cloud account name
`account_id`	Yes	Cloud account ID (AWS account number, Azure subscription ID)
`name`	Yes	Display name for the cluster
`region`	Yes	Cloud region
`vpc_id`	Yes	VPC/VNet ID. AWS: `vpc-xxx`. Azure: full resource ID.
`is_publicly_accessible`	Yes	Whether the K8s API server is publicly reachable
`platform`	Yes	Free-form string — for example, `kops`, `kubeadm`, `k3s`, `rancher`
`version`	Yes	Kubernetes version string
`network_mode`	Yes	`FLAT` or `OVERLAY`
`project`	No	GCP project ID
`compartment`	No	OCI compartment ID
`tags`	No	Key-value metadata map

Verifying onboarding

After registering, check the Cloud Resources > Cloud Assets > Kubernetes Clusters tab. The cluster should transition to the green Yes status within roughly 30 seconds. From the controller pod:

kubectl exec -ti deploy/cloudxd -- asset-cli status k8s

Expect to see your cluster with status RUNNING.

Concepts & Architecture

Guides

Reference

Onboard Self-Managed or Custom Kubernetes Clusters

Prerequisites

Step 1: Create a Service Account and ClusterRoleBinding

Step 2: Assemble a kubeconfig

Step 3: Register the cluster

Verifying onboarding

​Prerequisites

​Step 1: Create a Service Account and ClusterRoleBinding

​Step 2: Assemble a kubeconfig

​Step 3: Register the cluster

​Verifying onboarding

​Related Topics

Prerequisites

Step 1: Create a Service Account and ClusterRoleBinding

Step 2: Assemble a kubeconfig

Step 3: Register the cluster

Verifying onboarding

Related Topics