Onboarding Private Kubernetes Clusters

Private Kubernetes clusters have an API server that is not exposed to the public internet. Because the Aviatrix Controller must reach the Kubernetes API server to discover resources, it needs a private network path to the cluster before you can onboard it. This page covers the connectivity options. Once connectivity is in place, follow the per-provider onboarding workflow to register the cluster:

All prerequisites from the standard onboarding workflow apply. Review them before proceeding.

Connectivity options

Any method that gives the Controller private reachability to the cluster API server works. The most common options:

Aviatrix Transit (recommended) — Connect the Controller’s VPC and the cluster’s VPC to the same Aviatrix Transit, and use a DCF policy to permit the Controller-to-API traffic. See the example below.
VPC Peering — Direct peering between the Controller VPC and the cluster VPC.
AWS PrivateLink — Create a VPC endpoint for the EKS API server.
AWS Transit Gateway — Native AWS Transit Gateway connecting both VPCs.
Site-to-site VPN — VPN connecting the Controller’s network to the cluster’s network.

Aviatrix Transit (example)

This pattern uses Aviatrix Spoke and Transit Gateways. The Controller VPC and the cluster VPC each attach a Spoke to a shared Transit; a DCF policy permits Controller-to-API-server traffic.

Step 1: Spoke Gateway in the Controller VPC

Deploy an Aviatrix Spoke Gateway in the VPC that hosts the Aviatrix Controller. If a Spoke already exists in this VPC, skip this step. The Controller runs in a public subnet, but the Spoke enables private-address connectivity through the Aviatrix backbone to spokes in other VPCs — including the VPC that hosts the private cluster.

Step 2: Attach both VPCs to the same Transit

Attach the Controller-VPC Spoke and the cluster-VPC Spoke to the same Aviatrix Transit so the two can communicate over the Aviatrix backbone.

Step 3: DCF policy to permit Controller → cluster API

The DCF resource that matches API-server traffic depends on how the Controller addresses the API server. EKS and AKS expose the API server by hostname (FQDN), so the policy uses an aviatrix_web_group with an SNI filter. GKE exposes the private API server by IP only, so the policy uses an aviatrix_smart_group with a CIDR. If you onboard with a kubeconfig, pick the pattern based on the server: field in the kubeconfig.

EKS or AKS (FQDN)
GKE (IP)

resource "aviatrix_smart_group" "controller" {
  name = "controller"
  selector {
    match_expressions {
      type = "vm"
      name = "controller-instance"
    }
  }
}

resource "aviatrix_web_group" "private_cluster" {
  name = "private-cluster"
  selector {
    match_expressions {
      snifilter = "<CLUSTER_API_SERVER_HOSTNAME>"
    }
  }
}

resource "aviatrix_distributed_firewalling_policy_list" "controller_to_k8s" {
  policies {
    name     = "k8s-controller"
    priority = 3
    action   = "PERMIT"
    protocol = "ANY"

    src_smart_groups = [aviatrix_smart_group.controller.uuid]
    dst_smart_groups = ["def000ad-0000-0000-0000-000000000000"]
    web_groups       = [aviatrix_web_group.private_cluster.uuid]
  }
}

Get the API server hostname:

EKS: aws eks describe-cluster --name <CLUSTER_NAME> --query cluster.endpoint --output text (the host portion of the URL).
AKS: az aks show --resource-group <RG> --name <CLUSTER_NAME> --query privateFqdn -o tsv.

resource "aviatrix_smart_group" "controller" {
  name = "controller"
  selector {
    match_expressions {
      type = "vm"
      name = "controller-instance"
    }
  }
}

resource "aviatrix_smart_group" "private_cluster_api" {
  name = "private-cluster-api"
  selector {
    match_expressions {
      cidr = "<API_SERVER_PRIVATE_IP>/32"
    }
  }
}

resource "aviatrix_distributed_firewalling_policy_list" "controller_to_k8s" {
  policies {
    name     = "k8s-controller"
    priority = 3
    action   = "PERMIT"
    protocol = "ANY"

    src_smart_groups = [aviatrix_smart_group.controller.uuid]
    dst_smart_groups = [aviatrix_smart_group.private_cluster_api.uuid]
  }
}

Get the API server private IP:

gcloud container clusters describe <CLUSTER_NAME> --location <LOCATION> --format json \
  | jq -r .controlPlaneEndpointsConfig.ipEndpointsConfig.privateEndpoint

Cluster security group rules

By default, managed Kubernetes services restrict API server access to worker nodes only. Even with private connectivity at the network layer, you must add an inbound rule allowing the Controller’s private IP to reach the API on port 443. For EKS using terraform-aws-modules/eks/aws:

module "eks" {
  source  = "terraform-aws-modules/eks/aws"
  version = "~> 20.0"

  cluster_security_group_additional_rules = {
    avx_controller = {
      cidr_blocks = ["<CONTROLLER_PRIVATE_IP>/32"]
      description = "Allow traffic from Aviatrix Controller"
      from_port   = 443
      to_port     = 443
      protocol    = "tcp"
      type        = "ingress"
    }
  }
}

Replace <CONTROLLER_PRIVATE_IP> with the private IP of your Aviatrix Controller instance. Equivalent rules apply for AKS and GKE — add the Controller’s IP to the cluster API authorized networks list (AKS) or master-authorized-networks (GKE).

Onboarding the private cluster

Once connectivity and security group rules are in place, register the cluster using the per-provider onboarding workflow:

For private clusters, regardless of provider, you can use either the CSP-credential auth path (where supported by the provider) or the service-account kubeconfig path. The kubeconfig path is documented on the Onboard Self-Managed or Custom Clusters page.

The cluster may display as PUBLIC in CoPilot even if it is a private cluster. This is expected behavior and does not indicate a misconfiguration.

Verification

After onboarding, verify the connection from the Controller pod:

kubectl exec -ti deploy/cloudxd -- asset-cli status k8s

The cluster should show a status of RUNNING within roughly 30 seconds.

Concepts & Architecture

Guides

Reference

Onboarding Private Kubernetes Clusters

Connectivity options

Aviatrix Transit (example)

Step 1: Spoke Gateway in the Controller VPC

Step 2: Attach both VPCs to the same Transit

Step 3: DCF policy to permit Controller → cluster API

Cluster security group rules

Onboarding the private cluster

Verification

​Connectivity options

​Aviatrix Transit (example)

​Step 1: Spoke Gateway in the Controller VPC

​Step 2: Attach both VPCs to the same Transit

​Step 3: DCF policy to permit Controller → cluster API

​Cluster security group rules

​Onboarding the private cluster

​Verification

​Related Topics

Connectivity options

Aviatrix Transit (example)

Step 1: Spoke Gateway in the Controller VPC

Step 2: Attach both VPCs to the same Transit

Step 3: DCF policy to permit Controller → cluster API

Cluster security group rules

Onboarding the private cluster

Verification

Related Topics