Secure Self-Hosted AI Agent Workloads on Kubernetes

When complete, each agent workload will have a pod-scoped containment policy committed alongside its Kubernetes manifests and enforced at the VPC boundary by the Aviatrix controller. Permitted destinations (model provider APIs, tool endpoints, MCP servers) are declared per agent. Everything else is denied and logged. East-West policy prevents a compromised agent from pivoting to adjacent workloads in the cluster or the transit fabric. For the business case and architecture overview, see Securing AI Agent Workloads.

Prerequisites

Aviatrix Controller 8.2 or later
CoPilot 4.31 or later
DCF for Kubernetes enabled with Log Enrichment turned on
A cloud account onboarded in CoPilot for the cloud provider hosting the Kubernetes cluster
AI agent workloads deployed as Kubernetes pods (LangGraph, AutoGen, CrewAI, or a custom runtime)

Procedure

CoPilot UI
Terraform

Deploy an Aviatrix spoke gateway in the cluster VPC

An Aviatrix spoke gateway in the same VPC as your Kubernetes cluster is the enforcement point where the Aviatrix controller applies containment policy to MCP server egress traffic.

In CoPilot, navigate to Cloud Fabric > Gateways > Spoke Gateways and click + Spoke Gateway.

Provide the following details:

Parameter	Value
Name	A name for the gateway, for example `mcp-containment-spoke`
Cloud	The cloud provider hosting the Kubernetes cluster
Account	The cloud account for the cluster
Region	The region where the cluster runs
VPC/VNet	The VPC or VNet where the cluster nodes run — this must match the cluster’s VPC
Instance Size	Select an instance size appropriate for your environment

In the Instances section, select a subnet in the cluster VPC for the gateway instance.
Click Save.

Wait for the gateway status to show Up before continuing. You can monitor progress at CoPilot > Monitor > Notifications > Tasks.

Install the Aviatrix Kubernetes Firewall Helm chart

The Aviatrix Kubernetes Firewall Helm chart deploys the in-cluster components that allow the Aviatrix controller to reconcile and enforce containment policies on the cluster.Run the following command against the target cluster, replacing <chart-version> with the current chart version from the Aviatrix k8s-firewall-charts repository:

helm install --repo https://aviatrixsystems.github.io/k8s-firewall-charts --version <chart-version> k8s-firewall k8s-firewall

Verify Aviatrix CRD installation

The Helm chart installs two Aviatrix CRDs on the cluster. Verify both are present before continuing:

kubectl get crds | grep aviatrix

Expected output:

firewallpolicies.networking.aviatrix.com        <timestamp>
webgrouppolicies.networking.aviatrix.com        <timestamp>

If either CRD is missing, re-run the Helm installation and check the output for errors.

Onboard the Kubernetes cluster to Aviatrix DCF

Navigate to Cloud Resources > Cloud Assets > Kubernetes Clusters.
Click Onboard next to the target cluster.
Choose the appropriate access method for your cloud provider:
- AWS (EKS): Select Terraform or Command Line, apply the generated access entry and RBAC configuration, check the confirmation box, then click Onboard.
- Azure (AKS): Select Permissions on Cloud Account if the Aviatrix service principal has the required permissions, or Kubeconfig File to upload a kubeconfig. Click Onboard.
Wait for the cluster status to show Yes (green) on the Kubernetes Clusters tab.

EKS clusters require a view-nodes ClusterRole so the Controller can discover node metadata. The CoPilot onboarding dialog generates the required YAML automatically — apply it before clicking Onboard.

Label agent pods

Aviatrix containment policies target pods by Kubernetes label selector. Ensure each agent Deployment includes a label that uniquely identifies the agent workload.The following example uses app: research-agent:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: research-agent
  namespace: ai-agents
spec:
  selector:
    matchLabels:
      app: research-agent
  template:
    metadata:
      labels:
        app: research-agent
    spec:
      containers:
        - name: research-agent
          image: your-registry/research-agent:latest

Use a consistent label scheme across your agent fleet, for example agent: <name> or app: <name>. The same label is referenced in the containment policy in the next step.

Author a containment policy for each agent

A FirewallPolicy resource defines the allowed egress destinations for pods matching a label selector. Create one policy per agent workload and commit it to the same repository as the agent’s Deployment manifest.The following example permits pods labelled app: research-agent in the ai-agents namespace to reach a model provider API and a single tool endpoint. All other egress is denied.

kind: FirewallPolicy
apiVersion: networking.aviatrix.com/v1alpha1
metadata:
  name: research-agent
  namespace: ai-agents
spec:
  rules:
    - name: allow-model-provider
      logging: true
      selector:
        matchLabels:
          app: research-agent
      action: permit
      protocol: any
      webGroups:
        - name: model-provider
    - name: allow-tool-endpoints
      logging: true
      selector:
        matchLabels:
          app: research-agent
      action: permit
      protocol: any
      webGroups:
        - name: tool-endpoints
  webGroups:
    - name: model-provider
      domains:
        - "api.anthropic.com"
    - name: tool-endpoints
      domains:
        - "api.github.com"
        - "api.slack.com"

Apply the policy:

kubectl apply -f research-agent-policy.yaml

Repeat for each agent workload, adjusting the name, matchLabels selector, and domains list to match that agent’s identity and permitted destinations.

Commit the FirewallPolicy manifest to the same directory as the agent’s Deployment. This ensures it travels through the same review, CI/CD, and rollback process as the workload definition it protects.

Verify containment

Check policy events to confirm the policy was applied successfully:
kubectl get events -n ai-agents
A successful apply produces an event with Reason: UpdatePolicyListSuccess.
Navigate to Security > Distributed Cloud Firewall and select the cluster. Confirm the policy appears with the correct egress allow-list for each agent.
Invoke the agent and confirm that connections to declared destinations appear as PERMIT events in CoPilot FlowIQ.
To verify egress blocking, attempt a connection from an agent pod to a destination not in its domains list and confirm it is denied. The denied attempt appears in CoPilot per-connection logs alongside the agent’s allowed traffic.

Deploy an Aviatrix spoke gateway in the cluster VPC

An Aviatrix spoke gateway must be deployed in the same VPC as the target Kubernetes cluster. If a spoke gateway is already deployed in the cluster VPC, proceed to the next step.

resource "aviatrix_spoke_gateway" "mcp_containment" {
  cloud_type   = var.cloud_type
  account_name = var.account_name
  gw_name      = "mcp-containment-spoke"
  vpc_id       = var.cluster_vpc_id
  vpc_reg      = var.region
  gw_size      = var.gateway_size
  subnet       = var.subnet_cidr
}

Install the Aviatrix Kubernetes Firewall Helm chart

helm install --repo https://aviatrixsystems.github.io/k8s-firewall-charts --version <chart-version> k8s-firewall k8s-firewall

Verify Aviatrix CRD installation

The Helm chart installs two Aviatrix CRDs on the cluster. Verify both are present before continuing:

kubectl get crds | grep aviatrix

Expected output:

firewallpolicies.networking.aviatrix.com        <timestamp>
webgrouppolicies.networking.aviatrix.com        <timestamp>

If either CRD is missing, re-run the Helm installation and check the output for errors.

Onboard the Kubernetes cluster to Aviatrix DCF

EKS clusters are auto-discovered by the Aviatrix controller via CSP account scan. Do not apply the EKS block below when following the Obot EKS containment guide. Explicit registration conflicts with the auto-discovered entry and returns HTTP 409. Use this block only for self-hosted or non-Obot EKS deployments where auto-discovery is not applicable.

For EKS, the following configuration grants Controller access and registers the cluster in a single apply:

data "aws_eks_cluster" "this" {
  name = var.cluster_name
}

resource "aws_eks_access_entry" "aviatrix" {
  cluster_name      = var.cluster_name
  principal_arn     = var.controller_role_arn
  kubernetes_groups = ["view-nodes"]
  type              = "STANDARD"
}

resource "aws_eks_access_policy_association" "aviatrix" {
  cluster_name  = var.cluster_name
  policy_arn    = "arn:aws:eks::aws:cluster-access-policy/AmazonEKSViewPolicy"
  principal_arn = var.controller_role_arn
  access_scope { type = "cluster" }
  depends_on    = [aws_eks_access_entry.aviatrix]
}

resource "kubernetes_cluster_role" "view_nodes" {
  metadata { name = "view-nodes" }
  rule {
    verbs      = ["get", "list", "watch"]
    api_groups = [""]
    resources  = ["nodes"]
  }
}

resource "kubernetes_cluster_role_binding" "view_nodes" {
  metadata { name = "view-nodes" }
  role_ref {
    api_group = "rbac.authorization.k8s.io"
    kind      = "ClusterRole"
    name      = kubernetes_cluster_role.view_nodes.metadata[0].name
  }
  subject {
    kind      = "Group"
    name      = "view-nodes"
    api_group = "rbac.authorization.k8s.io"
  }
}

resource "aviatrix_kubernetes_cluster" "this" {
  cluster_id          = data.aws_eks_cluster.this.arn
  use_csp_credentials = true
  depends_on = [
    aws_eks_access_policy_association.aviatrix,
    kubernetes_cluster_role_binding.view_nodes,
  ]
}

For AKS:

data "azurerm_kubernetes_cluster" "this" {
  name                = var.cluster_name
  resource_group_name = var.resource_group
}

resource "aviatrix_kubernetes_cluster" "this" {
  cluster_id          = lower(data.azurerm_kubernetes_cluster.this.id)
  use_csp_credentials = true
}

Label agent pods

Ensure each agent Deployment includes a label that uniquely identifies the agent workload. The same label is referenced in the containment policy in the next step.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: research-agent
  namespace: ai-agents
spec:
  selector:
    matchLabels:
      app: research-agent
  template:
    metadata:
      labels:
        app: research-agent
    spec:
      containers:
        - name: research-agent
          image: your-registry/research-agent:latest

Author a containment policy for each agent

Commit a FirewallPolicy manifest alongside each agent’s Deployment in the same repository. The following example permits pods labelled app: research-agent to reach a model provider API and a tool endpoint:

kind: FirewallPolicy
apiVersion: networking.aviatrix.com/v1alpha1
metadata:
  name: research-agent
  namespace: ai-agents
spec:
  rules:
    - name: allow-model-provider
      logging: true
      selector:
        matchLabels:
          app: research-agent
      action: permit
      protocol: any
      webGroups:
        - name: model-provider
    - name: allow-tool-endpoints
      logging: true
      selector:
        matchLabels:
          app: research-agent
      action: permit
      protocol: any
      webGroups:
        - name: tool-endpoints
  webGroups:
    - name: model-provider
      domains:
        - "api.anthropic.com"
    - name: tool-endpoints
      domains:
        - "api.github.com"
        - "api.slack.com"

Apply via your GitOps pipeline or directly:

kubectl apply -f research-agent-policy.yaml

Repeat for each agent workload, adjusting the name, matchLabels selector, and domains list.

Verify containment

Check policy events to confirm the policy was applied successfully:
kubectl get events -n ai-agents
A successful apply produces an event with Reason: UpdatePolicyListSuccess.
In CoPilot, navigate to Security > Distributed Cloud Firewall and select the cluster. Confirm the policy appears with the correct egress allow-list for each agent.
Invoke the agent and confirm that connections to declared destinations appear as PERMIT events in CoPilot FlowIQ.
To verify egress blocking, attempt a connection from an agent pod to a destination not in its domains list and confirm it is denied. The denied attempt appears in CoPilot per-connection logs.

Concepts & Architecture

Guides

Reference

Secure Self-Hosted AI Agent Workloads on Kubernetes

Prerequisites

Procedure

​Prerequisites

​Procedure

Prerequisites

Procedure