# CNSF Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/CNSF/index Introduction to the Aviatrix Cloud Native Security Fabric (CNSF), providing zero trust security, network segmentation, and workload protection across multi-cloud environments. # Cloud Native Security Fabric (CNSF) **The missing security layer for the cloud era** ## What is Cloud Native Security Fabric? Cloud Native Security Fabric (CNSF) is a cloud runtime security control layer built **inside** the cloud network, not at the edge. It enforces Zero Trust between every cloud workload by inspecting, segmenting, and controlling workload-to-workload and workload-to-internet communications across regions, accounts, clouds, and data centers. > [!WARNING] > > **The Crisis**: The cloud has created the largest unguarded attack surface in > enterprise history—the space between every workload. While security teams > focused on the edge, adversaries moved inside, exploiting implicit trust to > move laterally and exfiltrate data undetected. *** ## The Vision CNSF is the missing fabric-level control layer for the cloud era. A standard way to declare, enforce, and prove trust between workloads in real time, everywhere they run. **This is not a new category. It is a return to first principles** in a cloud era that's often prioritized speed over security. By embedding trust directly into the fabric of the cloud, CNSF defines the control layer that Zero Trust has lacked since workloads left the data center. > [!NOTE] > > **The Question**: The question is not whether Cloud Workloads need a security > fabric—the breaches prove they do. The question is whether you'll build it > proactively or reactively. The cloud will not wait, and neither will the > adversaries exploiting the gaps that remain unaddressed. *** ## The Problem CNSF Solves The fortress is dead. For two decades, cybersecurity was defined by walls, perimeters, and the illusion of inside versus outside. But the cloud did not move the perimeter, **it vaporized it**. > **Shattered Perimeter** > > Modern applications are distributed across thousands of micro-perimeters: > VPC/VNets, clusters, functions, and services > **Implicit Trust Gap** > > Most tools watch from the outside while attacks spread inside through > implicitly trusted east-west communication paths > **Internet as Enterprise Network** > > Sensitive inter-service communication now routinely traverses public > infrastructure previously considered hostile territory > **Ephemeral Architecture** > > Applications are transient ecosystems of short-lived components creating > constantly evolving attack surfaces ### The New Reality Three irreversible shifts have fundamentally broken traditional security models: 1. **The Internet Became the Enterprise Network**: What was once your controlled internal network and the external internet have merged 2. **The Perimeter Became Exponentially Distributed**: Attack surface has atomized into thousands of individual points requiring defense 3. **Applications are Ephemeral and Deeply Distributed**: Transient ecosystems of containers, functions, and services with intricate, temporary dependencies > [!NOTE] > > **Critical Fact**: 25% of data breaches involve lateral movement, with hackers > spending weeks or months silently moving between systems. The space between > workloads has become the new primary battleground. *** ## Why CNSF is Different Unlike traditional security approaches that bolt on protection after the fact, CNSF is fundamentally different in four critical ways: ### Inline Enforcement **Real protection happens where packets flow** * Controls applied in the data plane—the actual forwarding path * Policy enforced in real time, not merely declared * Line-rate performance with predictable latency ### Agentless & Embedded **Security woven into the fabric, not bolted on** * No agents to deploy or manage * Lives in the data path by design * Travels with workloads across environments ### MultiCloud Consistency **Express intent once, enforce everywhere** * Same policy produces same results across AWS, Azure, GCP, OCI * Abstracts cloud provider differences * Uniform enforcement across hybrid environments ### Developer-Ready **Fits existing workflows** * Deep [Terraform](https://registry.terraform.io/providers/AviatrixSystems/aviatrix/latest/docs) and IaC integration * No application code changes required * Preserves development velocity *** ## Core Capabilities ### Identity-Aware Segmentation * Define who/what can communicate based on labels, identity, and context * Stop lateral movement with deny-by-default policies * Dynamic trust boundaries that adapt to infrastructure changes ### Egress Governance * Explicit allow-lists for approved SaaS/ERP and external services * Eliminate common data exfiltration paths * Monitor and control outbound communications ### Inline Encryption & High-Performance Transport * Protect data in motion across sites, regions, and clouds * Quantum-resistant encryption for future threats * Deterministic performance across multicloud topologies ### Kubernetes & Cloud-Native Coverage * Consistent fabric-level policy for east-west pod/service traffic * Secure ephemeral container communications * No application rewrites required *** ## How CNSF Works **1. Control Plane** Captures intent (segments, trust zones, egress rules) and programs the fabric consistently across environments **2. Data-Path Enforcement** Applies policy inline where workloads actually communicate—between services, VPC/VNets, clusters, and on hybrid links **3. Context Signals** Identity, posture, and labels refine decisions dynamically, keeping policy accurate as infrastructure changes **4. Real-Time Adaptation** Continuous enforcement that scales with workload creation, movement, and destruction *** ## The Accelerants: K8s and GenAI Two Board-mandated initiatives are exponentially increasing the urgency for CNSF: ### Kubernetes Complexity * **96% of enterprises** use Kubernetes with complex, ephemeral networking * Securing east-west traffic between thousands of pods * Consistent policy across multi-cluster deployments ### Generative AI & "Shadow AI" * Autonomous agents and AI-powered tools proliferating within environments * AI systems designed to pull data from anywhere and communicate extensively * New communication patterns bypassing traditional controls > [!WARNING] > > **Shadow AI Crisis**: As "Shadow IT" emerged in the early cloud era, > autonomous AI deployment creates communication pathways that bypass > conventional security controls. *** ## CNSF in the Security Ecosystem CNSF does not replace your security stack—**it activates it** where existing tools cannot reach: **CNAPP & Posture Tools**\ They find misconfigurations; CNSF enforces cloud runtime communication policies derived from those insights **Endpoint/XDR**\ They protect hosts; CNSF protects the paths between hosts and services **SASE/Zero Trust Access**\ They govern user-to-app; CNSF governs service-to-service and workload-to-internet after access is granted **NGFWs**\ They provide ingress/egress defense; CNSF extends Zero Trust inside the cloud runtime fabric *** ## Measurable Outcomes > **Lateral Movement** > > Deny-by-default segmentation prevents unauthorized movement between > workloads > **Data Exfiltration** > > Governed egress and encrypted paths eliminate common data loss vectors > **Audit Time** > > Uniform policy, evidence, and trust-zone visibility across all clouds > **Operational Drag** > > Standardized controls that move with workloads and align to DevOps/IaC *** ## Key Use Cases 1. **Enforce Zero Trust** for cloud workloads across multiple CSPs and regions 2. **Govern and monitor egress** to external services/SaaS to prevent data loss 3. **Segment Kubernetes** services/pods without application rewrites 4. **Secure hybrid communications** (data center ↔ cloud) with consistent policy 5. **Control GenAI/agentic systems** communications safely *** ## Zero Trust Alignment CNSF directly supports the core pillars of the CISA Zero Trust Maturity Model: * **Network**: Dynamic segmentation and encrypted communications * **Application**: Service-to-service trust enforcement * **Data**: Protection of data in motion between workloads Plus all four cross-cutting capabilities: Visibility, Analytics, Automation, and Governance. *** ## Learn More Ready to explore CNSF further? Continue with the detailed implementation guides and architectural overviews: * CNSF Architecture * Use Cases & Solutions # Aviatrix in the China Regions Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/aviatrix-china-overview This document provides an overview of the Aviatrix features that are supported and the requirements for implementing Aviatrix in the China regions. It also provides various options and design patterns for interconnecting Aviatrix in the China regions and Global regions. This document provides an overview of the Aviatrix features that are supported and the requirements for implementing Aviatrix in the China regions. It also provides various options and design patterns for interconnecting Aviatrix in the China regions and Global regions. > [!NOTE] > > You cannot update an IAM role-based policy using the Aviatrix Controller > interface. If you encounter this issue, update the IAM policy manually using > your AWS China account. > [!WARNING] > > The Datadog Agent is not supported on Aviatrix Controllers deployed in AWS > China or Azure China. Do not attempt to configure Datadog telemetry on China > Controllers. ## Features Supported in AWS China, Azure China, and Alibaba China Regions | **Feature** | **AWS China** | **Azure China** | **Alibaba Cloud China and Global** | | - | - | - | - | | Controller Marketplace Launch | Yes | Yes | No | | CoPilot Marketplace Launch | Yes | Yes | No | | Controller Security Group Management | Yes | No | No | | Multi Accounts | Yes | Yes | Yes | | Launch Controller with CloudFormation | Yes | N/A | N/A | | VPC Tool | Yes | Yes | Yes | | FlightPath | Yes | Yes | Yes | | Transit Network Spoke and Transit Gateways | Yes | Yes | Yes | | Aviatrix Transit Gateway Peering | Yes | Yes | Yes | | Transit to External IPsec Devices | Yes | Yes | Yes | | Site2Cloud VPN for All Gateways | Yes | Yes | Yes | | BGP over LAN | No | No | No | | BGP over GRE | No | No | No | | Native Peering | Yes | Yes | No | | Network Segmentation | Yes | Yes | Yes | | Firewall Network | Yes | No | No | | High Performance Encryption Mode | Yes | Yes | No | | Aviatrix Edge | No | No | No | | FQDN Egress Control | No | No | No | | Stateful Firewall | No | No | No | | Advanced NAT | No | No | No | | ThreatIQ | No | No | No | | Micro-Segmentation | No | No | No | | Remote Access UserVPN (OpenVPN) | No | No | No | | PrivateS3 | No | N/A | N/A | | Transit to AWS VGW | No | N/A | N/A | | AWS Transit Gateway Orchestration | No | N/A | N/A | | Controller Migrate | No | No | No | | Terraform | Yes | Yes | Yes | | Backup and Restore | Yes | Yes | Yes | | Logging Service Integration (Rsyslog, Netflow, and CloudWatch) | Yes | Yes | Yes | | Datadog Agent | No | No | No | ## Requirements to Implement Aviatrix in China Regions The following are the requirements to implement Aviatrix in AWS China, Azure China, and Alibaba China regions. * The Aviatrix Controller must be deployed in the China region, for example, AWS China Ningxia region. Currently, an Aviatrix Controller in the Global region (non-China) does not support Aviatrix Gateways deployment and management in the China region. Similarly, an Aviatrix Controller in the China region does not support Aviatrix Gateways deployment and management in the Global region. See [Unsupported Topologies](#unsupported-topologies). * You must have an Internet Content Provider (ICP) license. An ICP license is required for opening a CSP account in the China region. For more information, see [Acquiring a China ICP License](#acquiring-a-china-icp-license). ## Unsupported Topologies The following topologies are not supported. An Aviatrix Controller launched in the Global region does not support Aviatrix Gateways deployment and management in the China region. ![Unsupported topology showing Global region Controller cannot manage China region Gateways](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/aviatrix_china_unsupported_global_manage_china.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=3b2e9fe4e80e455dd2c8779cfb10f565) An Aviatrix Controller launched in the China region does not support Aviatrix Gateways deployment and management in the Global region. ![Unsupported topology showing China region Controller cannot manage Global region Gateways](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/aviatrix_china_unsupported_china_manage_global.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=11a40d715b0767408f504ad5fb07221c) ## Acquiring a China ICP License Regulations in China require you to acquire an Internet Content Provider (ICP) license from the government and register the license with your CSP to provide Internet services in China. In China, an ICP license is required to establish SSL connections between different regions, ISPs, CSPs, or to cross national borders. Aviatrix supports transit gateways using AWS China, Azure China, and Alibaba multicloud networks in the China region. Obtaining and implementing an ICP is a process, and you should follow the directions of your compliance experts. Here are some general guidelines Aviatrix recommends to implement a multi-cloud network in the China region: * Create or use a Legal Entity in China to apply for the ICP license. * Apply for a Legal Domain Name in the China Registration. * Acquire the ICP Certificate from the China Ministry of Industry and Information Technology (MIIT). * Register the ICP Certificate with your CSP in the China region. * Use dedicated lines from certified telecom carries for connections between China and the rest of the world. > [!TIP] > > Slow connection speeds and high-latency associated with the China region can > be overcome by using a dedicated line to create Aviatrix transit connections > and deploying services close to the China region. * Deploy the Aviatrix Controller and CoPilot. * Enter the certificate domain that was submitted during the ICP application in Aviatrix Controller (see [What is a Certificate Domain?](https://legacy.docs.aviatrix.com/documentation/latest/platform-administration/accounts-and-users/certificate-domain.html)) * Deploy Aviatrix Secure Multicloud Network in China. ## Consequences of Non-Compliance with the Chinese Government Regulations The following consequences can result for non-compliance of the Chinese Government Regulations. * The company is not permitted to open an account with a CSP in China region. * Aviatrix Controller is unable to deploy and manage Aviatrix Gateways. * The connection between Aviatrix Gateways is intermittent or becomes disconnected from time to time. ## Interconnecting Aviatrix in the China region and the Global region Site2Cloud can be established between Aviatrix Transit Gateways in the China region and the Global region. The following options are available for the underlying network of Site2Cloud: 1. Public Internet > [!NOTE] > > Public Internet connections maybe unstable due to additional network traffic > processing by the Chinese government. ![Site2Cloud connectivity over public Internet between China and Global regions](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/aviatrix_china_site2cloud_internet.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=a78e4064cd01b66c8bfeeb196a3aba9e) 2. Private connectivity through certified telecom carriers such as China Telecom, China Unicom, and China Mobile ![Site2Cloud connectivity through certified telecom carriers between China and Global regions](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/aviatrix_china_site2cloud_telecoms.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=0dead18bb0a394a559a685cb155de990) 3. Alibaba Cloud Network using VPC Peering or Alibaba Cloud Enterprise Network (Alibaba CEN) [https://www.alibabacloud.com/product/cen](https://www.alibabacloud.com/product/cen) ![Site2Cloud connectivity through Alibaba Cloud Enterprise Network between China and Global regions](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/aviatrix_china_site2cloud_alicloud.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=5903864822f85942a0a4e49c02ce16a9) To create a global multicloud network with low-latency connectivity between the China region and the global region, we recommend that you use private connectivity provided by certified telecom carriers or through the Alibaba Cloud network. For a description of the design patterns for these underlying networks, see [Design Patterns for China Region](#design-patterns-for-china-region). ## Launching Aviatrix Controller in AWS China To launch Aviatrix Controller in AWS China, do the following: 1. Log in to the AWS China Portal. 2. Navigate to the AWS Marketplace for the Ningxia and Beijing Region. 3. Search for the keyword "Aviatrix." ![Aviatrix listing on AWS China Marketplace](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/aviatrix_aws_china_marketplace.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=49523b798ed1528ec266471872ce808e) Use the following URLs to find the Controller and CoPilot on the AWS China Marketplace: * [Aviatrix Secure Networking Platform - BYOL](https://awsmarketplace.amazonaws.cn/marketplace/pp/prodview-tr55yz2zpuzlo) * [Aviatrix CoPilot - BYOL](https://awsmarketplace.amazonaws.cn/marketplace/pp/prodview-m73cvirso7uu6) Use the following URL to launch the Aviatrix Controller from the AWS CloudFormation in AWS China: * [AWS China Cloudformation Aviatrix Controller and IAM Setup-BYOL](https://cn-northwest-1.console.amazonaws.cn/cloudformation/home?region=cn-northwest-1#/stacks/new?stackName=AviatrixController\&templateURL=https://aviatrix-public-download.s3.cn-north-1.amazonaws.com.cn/aws-china/cloudformation-templates/aviatrix-controller-and-IAM-setup-CFT/aviatrix-controller-and-IAM-setup-cft-BYOL.template) ## Launching Aviatrix Controller in Azure China To launch Aviatrix Controller in Azure China, do the following: 1. Log in to the Azure China Portal. 2. Navigate to the Azure Marketplace for the China North region. 3. Search for the keyword "Aviatrix." ![Aviatrix listing on Azure China Marketplace](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/aviatrix_azure_china_marketplace.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=1278a7c6cf680d58bc8e7e8757ec2864) Use the following URL to find the Controller on the Azure China Marketplace: * [Aviatrix Cloud Network Controller](https://market.azure.cn/) ## Design Patterns for China region ### China region only ![Design pattern for China region only deployment](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/aviatrix_china_design_china_only.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=646fab3e8c2038491a93b933291dae63) ### Cross-border connectivity through certified telecom carriers ![Design pattern for cross-border connectivity through certified telecom carriers](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/aviatrix_china_design_cross_border_telecom.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=f02496689fa438b18e06286cdb54f14f) ### Cross-border connectivity through Alibaba Cloud Enterprise Network (Alibaba CEN) ![Design pattern for cross-border connectivity through Alibaba Cloud Enterprise Network](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/aviatrix_china_design_cross_border_alicloud.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=9ddef24e6758718d0bea91b1e57cb498) # AZ Affinity and Symmetric Routing Architecture Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/az-affinity-symmetric-routing How Aviatrix combines AZ-aware nexthop selection with per-spoke symmetric routing to cut cross-AZ costs and enable east-west inspection in AWS. ## Overview In an Aviatrix ActiveMesh fabric, every healthy gateway instance in a spoke or transit gateway group is an equally eligible next hop, regardless of which Availability Zone (AZ) it runs in. In AWS, that design has two costs: AWS bills for every crossing of an AZ boundary, and each crossing adds latency that compounds across a multi-hop inspection path such as spoke-transit-firewall-spoke. It also blocks east-west Layer 7 inspection features, which require the forward and return leg of a flow to traverse the same gateway pair. AZ affinity and per-spoke-group symmetric routing solve both problems together: AZ affinity biases next-hop selection toward the local AZ to reduce cross-AZ charges and latency, and symmetric routing pins a spoke subnet's forward and return traffic to a single spoke gateway instance so that a stateful inspection function running on that spoke gateway sees a complete view of every flow. Use this architecture when you run Aviatrix spokes and transits in AWS across multiple Availability Zones and need to control cross-AZ cost or latency, or to enable east-west inspection. > [!NOTE] > > In Controller 10.1, AZ affinity and symmetric routing are supported for AWS > only, and are not supported on spokes or transits attached to FireNet. ## Architecture Diagram The diagram below shows the topology this architecture assumes: one spoke gateway instance and one transit gateway instance per Availability Zone, with AZ affinity biasing the spoke-to-transit and transit-to-spoke paths toward the local AZ, and symmetric routing pinning a spoke subnet's traffic to a single owning gateway instance for both directions. ![Spoke and transit gateway instances deployed across two Availability Zones, showing the same-AZ next hop preferred between each spoke and transit instance and the cross-AZ next hop used only as a fallback](https://mintcdn.com/aviatrix-14b37c43/0cJ6mdcM_TcZxo37/images/concepts-architectures/architecture/connectivity/az-affinity-symmetric-routing.svg?fit=max\&auto=format\&n=0cJ6mdcM_TcZxo37\&q=85\&s=d235650db788402cb49d4c4b437440f9) Each spoke subnet is "owned" by exactly one gateway instance for both outbound and inbound traffic. Traffic from a spoke instance to the transit group prefers the transit instance in the same AZ; traffic back from the transit group to the spoke prefers the owning instance's AZ. Either direction falls back to a healthy instance in the other AZ when no same-AZ instance is available. AZ affinity never reduces availability: it only changes which healthy instance is preferred. ## Design Principles ### AZ-Local Preference, Not AZ-Only Enforcement AZ affinity biases next-hop selection toward the same-AZ gateway instance rather than restricting traffic to that AZ. When the local-AZ instance is unavailable, routing falls back to a healthy instance in another AZ, so the cost and latency optimization never comes at the expense of high availability. ### Deterministic Flow Symmetry Over Load Distribution Per-spoke-group symmetric routing assigns each subnet's outbound and inbound traffic to a single, consistent gateway instance instead of distributing it across the whole gateway group. This sacrifices some load-spreading in exchange for the connection-state guarantee that stateful east-west inspection depends on. ### Cloud-Native Enforcement The architecture uses the standard AWS per-subnet route table association and gateway route-metric mechanisms already available in the platform. It does not introduce new infrastructure, so enabling it does not require new components or a different deployment topology. ### Attachment-Scoped, Opt-In Control AZ affinity is enabled independently on each spoke-to-transit or transit-to-transit attachment rather than globally. You can adopt it selectively on the attachments where cost, latency, or inspection consistency matters most, and leave others on default ActiveMesh behavior. ## Components This architecture combines the following Aviatrix components: * [ActiveMesh](/docs/enterprise/10.1/concepts-architectures/components/connectivity/activemesh-about) : the next-hop selection and packet-forwarding engine that AZ affinity extends with an AZ-local preference tier. * [Spoke and Transit Gateways](/docs/enterprise/10.1/reference/ui/cloud-fabric/gateways) : the gateway instances between which AZ affinity and symmetric routing operate. The toggle is configured on the spoke-transit or transit-transit attachment. * [WebGroups](/docs/enterprise/10.1/concepts-architectures/components/dcf/webgroups-about) : a Distributed Cloud Firewall policy construct whose east-west enforcement depends on symmetric routing for a consistent, bidirectional view of each flow. * [East-West IDS/IPS](/docs/enterprise/10.1/guides/security/dcf/ips) : intrusion detection and prevention for east-west traffic. Like WebGroups enforcement, it requires symmetric routing to track connection state correctly. ## Traffic Flows ### Spoke to Transit (Outbound) A workload in a spoke subnet sends traffic toward the transit gateway group. The spoke gateway instance handling that subnet prefers a transit gateway instance in its own Availability Zone. If no healthy instance is available in the same AZ, it falls back to a healthy instance in another AZ. ### Transit to Spoke (Return) Return traffic from the transit group back to the spoke prefers the spoke gateway instance that owns the destination subnet, in the same Availability Zone as that instance. If no same-AZ transit instance is available, traffic is load-balanced across the remaining healthy instances. ### Transit to Transit (Peered Domains) When two transit gateway groups are peered (for example, across regions or Aviatrix domains), AZ affinity applies the same same-AZ preference to traffic crossing that peering. This applies independently of whether AZ affinity is also enabled on the spoke attachments behind either transit. ### Symmetric Routing Pins the Subnet's Owner Independently of AZ affinity, per-spoke-group symmetric routing associates each spoke subnet's route table with a single gateway instance. That instance handles both the outbound leg and the inbound leg of every flow originating from or destined to that subnet, so a stateful inspection function on the spoke gateway always sees both directions of the flow. ## Variants and Trade-offs | Decision | Option A | Option B | When to choose B | | - | - | - | - | | AZ affinity on an attachment | Disabled (default ActiveMesh behavior) | Enabled | Traffic on that attachment crosses AZ boundaries often enough that transfer cost or latency matters, or the attachment carries traffic that requires east-west inspection | | Scope of enablement | Enable on every attachment | Enable per attachment selectively | Only some attachments (for example, the ones carrying inspected traffic) need the cost, latency, or symmetry benefit | | Gateway topology | One gateway instance per AZ per group (supported) | More than one gateway instance per AZ per group | Not applicable: this topology is not supported by AZ affinity or symmetric routing in Controller 10.1 | > [!NOTE] > > Enabling AZ affinity limits Equal-Cost Multi-Path (ECMP) load-sharing across > all gateway instances in the group, because traffic is biased toward a single > same-AZ instance instead of being spread across every healthy instance. This > can reduce aggregate bandwidth across the group. Weigh this against the > cross-AZ cost and latency reduction before enabling AZ affinity on a > high-throughput attachment. ## Sizing and Scale Considerations AZ affinity and symmetric routing do not introduce scale limits beyond the existing per-attachment and per-route-table limits of the underlying platform. The one structural constraint is topological, not numeric: the model supports at most one gateway instance per Availability Zone in a given spoke or transit gateway group. Deployments using one gateway instance per Availability Zone do not need to resize or restructure their gateway groups to adopt AZ affinity or symmetric routing. CoPilot includes built-in capability audits for both AZ affinity and symmetric routing. Each audit checks whether a spoke's VPC route tables and subnets are structured correctly for the corresponding feature, and reports any configuration gaps before you enable it in production. ## Security Considerations Symmetric routing is a prerequisite for any east-west Layer 7 inspection feature that depends on connection state, including east-west IDS/IPS and WebGroups-based policies. It guarantees flow-level symmetry at the spoke gateway only: the same spoke gateway instance always sees both the forward and return leg of a given flow. It does not guarantee that the same transit gateway instance is used in both directions. If symmetric routing is not enabled on a spoke's transit attachment, ActiveMesh can resolve the forward and return legs of the same flow to different spoke gateway instances. When that happens, neither instance sees the complete flow, and a stateful inspection function running on the spoke gateway loses visibility into part of the connection. Traffic that is expected to be inspected can pass with incomplete coverage. AZ affinity itself is a routing cost and latency optimization and has no direct security property. Because it changes which gateway instance a flow's traffic is likely to traverse, review any per-gateway logging, monitoring, or compliance controls to confirm they account for AZ-local traffic patterns once AZ affinity is enabled. ## Limitations * AWS only. AZ affinity and symmetric routing are supported for AWS spokes and transits in Controller 10.1. AZ affinity is primarily an AWS cost optimization; symmetric routing's underlying design is cloud-agnostic, but only AWS is supported in this release. Other cloud providers are not covered by this architecture. * High Performance Encryption required. Both gateways in the attachment or peering must have High Performance Encryption enabled to use AZ affinity. * Not supported with FireNet. AZ affinity and symmetric routing are not supported on spokes or transits attached to FireNet in Controller 10.1. * One gateway instance per AZ per group. The model assumes exactly one gateway instance per Availability Zone in a spoke or transit gateway group. Topologies with more than one instance per AZ per group are not supported. ## Related Resources * [Spoke and Transit Gateways](/docs/enterprise/10.1/reference/ui/cloud-fabric/gateways) : configure the AZ affinity toggle on a spoke-transit or transit-transit attachment. * [East-West IDS/IPS](/docs/enterprise/10.1/guides/security/dcf/ips) : configuration guide for the inspection feature that requires symmetric routing. # Bidirectional Forwarding Detection for BGP Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/bgp-bfd This document describes Bidirectional Forwarding Detection (BFD) for BGP in Aviatrix. This document describes Bidirectional Forwarding Detection (BFD) for BGP in Aviatrix. Aviatrix's Bidirectional Forwarding Detection (BFD) for BGP delivers fast, reliable detection of network link and node failures allowing for quick network recovery. By detecting failures rapidly and initiating prompt recovery, BFD helps maintain the high availability and performance standards required in network environments. This capability is essential for business-critical applications that depend on consistent network operation. ## Supported Gateways BFD is supported on all Aviatrix gateways that support external BGP connections. ## What is Bidirectional Forwarding Detection? Bidirectional Forwarding Detection (BFD) is a network protocol that enables fast detection of network failures between two nodes. In Aviatrix, you can enable BFD to detect link or node failure between an Aviatrix gateway and its remote BGP peer. One of the benefits of BFD is that the BFD timer can be tuned more aggressively than the regular BGP timers to monitor the status of a BGP session. ### How BFD Works in Aviatrix BFD can be enabled at the connection-level for any Aviatrix gateway that supports external BGP connections for rapid failure detection and recovery. When BFD is enabled: 1. The Aviatrix gateway exchanges BFD control packets with the remote peer at the negotiated intervals. 2. If the gateway stops receiving the control packets within the expected interval, the peer is considered down, indicating a network failure. 3. The gateway terminates the BGP session. 4. The gateway notifies the Aviatrix Controller, triggering network traffic to be rerouted to the backup link for network recovery and stability. ## About BFD Configuration Enabling BFD for an external BGP connection involves configuring these BGP options used in the network failure detection and recovery process: * [BFD Timers](#bfd-timers) * [BGP Neighbor Status Polling Time](#bgp-neighbor-status-polling-time) * [BGP Multihop](#bgp-multihop) ### BFD Timers BFD timer defines how often BFD sends and receives control packets from the BGP peer to detect connection failure. The following parameter are used to configure a BFD timer: * **Transmit Interval** is the minimum interval you want to send BFD control packets to the remote peer. The Aviatrix gateway sends BFD control packets to the remote peer based on the higher of this transmit interval and the peer's receive interval. The default is 300 milliseconds (ms). You can change the default interval. The supported range is 10 to 60000 (ms). - **Receive Interval** is the minimum interval you want to receive BFD control packets from the remote peer. The remote peer sends BFD control packets to the Aviatrix gateway based on the higher of this receive interval and the peer's transmit interval. The default is 300ms. You can change the default interval. The supported range is 10 to 60000 (ms). * **Multiplier** is used to determine the total time for failure detection. The negotiated transmit interval is multiplied by the multiplier. If a control packet is not received within that time, the peer is declared down. This correlates to the number of packets that must be lost for the peer to be considered down. The default is 3. You can change the default. The supported range is 2 to 255. For example, if the Aviatrix gateway's **Transit Interval** is 150ms and the remote peer's **Receive Interval** is 300ms, the remote peer will send to the Aviatrix gateway control packets every 300ms. If the Aviatrix gateway's **Multiplier** is 3, then the total time for failure detection is 900ms. If a control packet is not received within 900ms, the BGP session is terminated. See [Enable Bidirectional Forwarding Detection](#enable-bidirectional-forwarding-detection). ### BGP Neighbor Status Polling Time BGP Neighbor Status Polling Time sets how often the gateway checks the BGP peer's status. If the gateway determines a peer is down, the system re-routes network traffic to the backup path. BGP Neighbor Status Polling Time interval can be adjusted to change how quickly the system responds to BGP down events to reduce BGP convergence time after failures. By default, the gateway polls the BGP peer status every 5 seconds. See [Set BGP Neighbor Status Polling Time](#set-bgp-neighbor-status-polling-time). ### BGP Multihop When the external BGP connection remote peer is not directly connected (separated by multiple hops) to the Aviatrix gateway, **BGP Multihop** mode must be enabled for the connection for the gateway to establish a BFD session. > [!NOTE] > > The multihop setting must match the multihop setting on the remote peer for > the gateway to establish the BFD session. See [Enable Bidirectional Forwarding Detection](#enable-bidirectional-forwarding-detection). ## Enable Bidirectional Forwarding Detection You can enable [Bidirectional Forwarding Detection (BFD)](#what-is-bidirectional-forwarding-detection) for a BGP over IPsec, BGP over GRE, or BGP over LAN external connection to a remote peer. BFD is supported for Aviatrix gateways that support external BGP connections. > [!TIP] > > When enabling BFD, you must ensure that a backup path exists to reroute > network traffic when a network failure is detected. > [!NOTE] > > For existing BGP connections that were created prior to the availability of > the BFD feature in Aviatrix, after you > [upgrade your Aviatrix Controller and Gateways](/docs/enterprise/10.1/guides/maintenance/controller-gateway-upgrade-steps) > , you will need to [manually enable BFD on the BGP > connection](#enable-bfd-for-an-existing-bgp-connection). To enable BFD: * For new BGP connections, see [Setting Up External Device Connection.](/docs/enterprise/10.1/guides/connectivity/vpn/transit-externaldevice-workflow#setting-up-external-device-connection) * For existing BGP connections, see [Enable BFD for an Existing BGP Connection](#enable-bfd-for-an-existing-bgp-connection). ### Enable BFD for an Existing BGP Connection To enable BFD for an existing BGP connection: 1. In Aviatrix CoPilot, go to **Networking > Connectivity > External Connections (S2C)** tab. 2. From the list of BGP connections, select the connection for which you want to enable BFD. > [!TIP] > > For quick access, you can use the Filter or Search option from the toolbar to > find by name or matching condition. 3. In the connection's **Settings** page, expand the **Border Gateway Protocol** section. 4. If the remote peer is directly connected to the gateway, locate **BGP Multihop** card and set **BGP Multihop** toggle to **Off**. 5. Click **Save**. 6. Locate the **BFD** card and set **BFD** toggle to **On**. 7. Configure the BFD timer. Specify the following values: 1. [**Transmit Interval**](#transmit-interval): The minimum interval for sending the BFD control packets to the BGP peer. The default is 300 milliseconds(ms). The supported range is 10 to 60000 (ms). 2. [**Receive Interval**](#receive-interval): The minimum interval for receiving the BFD control packets from the BGP peer. The default is 300 milliseconds(ms). The supported range is 10 to 60000 (ms). 3. [**Detect Multiplier**](#multiplier): The multiplier to calculate the total time for failure detection. The default is 3. The supported range is 2 to 255. 8. Click **Save**. > [!TIP] > > After enabling BFD, if you notice an increase in BGP session instability, > consider raising the BFD timers and/or the detect multiplier to reduce BGP > session instability. See [Troubleshooting BGP Session Instability After > Enabling BFD](#troubleshooting-bgp-session-instability-after-enabling-bfd). ## Set BGP Neighbor Status Polling Time For all external BGP connections, the system checks for BGP connection failures every five seconds, by default. The polling time can be adjusted to change how quickly Aviatrix responds to BGP connection down. > [!TIP] > > Setting BGP Neighbor Status Polling Time to a low number of seconds generates > additional load. Using short polling times on many gateways has the potential > to cause instability, especially if your fabric has many BGP sessions and/or > lots of BGP routes. To adjust the polling time: 1. In Aviatrix, go to the gateway's Settings page. * For Transit or Spoke Gateway, go to **Cloud Fabric > Gateways**, then select **Transit Gateways** or **Spoke Gateways** tab. * For Edge Gateway, go to **Cloud Fabric > Hybrid Cloud > Edge Gateways** tab. 2. From the gateways list, locate and select the gateway. 3. In the gateway's page, click the **Settings** tab. 4. Expand the **Border Gateway Protocol (BGP)** section. 5. Locate the **BGP Neighbor Status Polling Time** card and change the default polling time. 6. Click **Save** to save your changes. ## View BFD Status for a BGP Connection When you enable BFD for an external BGP connection, you can view the BFD status. To view BFD status: 1. In Aviatrix CoPilot, go to **Networking > Connectivity > External Connections (S2C)** tab. 2. From the list of BGP connections, select the connection for which you want to view BFD status. > [!TIP] > > For quick access, you can use the Filter or Search option from the toolbar to > find by name or matching condition. 3. Click the **Details** tab. 4. In the **BGP Neighbor** table, locate **BFD Status** to view the status. BFD status can be one of the following: * **Initializing** when BFD is first enabled for BGP a connection. * **Up** when BFD detects the BGP peer connection is up. * **Down** when BFD detects the BGP peer connection is down. * **Disabled** when BFD is not enabled for the BGP connection. ## Troubleshooting BGP Session Instability After Enabling BFD BFD is designed for rapid detection of network failures. It can cause flapping of BGP sessions or session termination if BFD timers are tuned too aggressively or there are issues with the network link between the two peers. By adjusting timers, increasing detect multipliers, and verifying peer configuration, you can maintain a stable and efficient network environment. ### Recommendations for Resolving Instability To address BGP session instability after enabling BFD, consider doing the following: 1. **Adjust BFD Timers** BFD timers determine how often BFD sends and receives control packets from the BGP peer to detect connection failure. If sessions become unstable, raise the timers to decrease the detection intervals: * Transmit Interval: Increase this value to reduce the frequency of BFD packets sent. * Receive interval: Increase this value to decrease the rate of BFD packets received. 2. **Increase the Detect Multiplier** The detect multiplier determines the number of missed BFD packets before considering the session as down. Increasing this value allows more time for network link issues to resolve without prematurely terminating the BGP session. 3. **Verify Peer Configuration** Ensure that the BFD settings on both BGP peers align to avoid mismatched expectations. Differences in timer or multiplier settings can lead to session negotiation failures. Once adjustments are made, validate the stability by [monitoring the BFD status for the connection to ensure consistent peering](#view-bfd-status-for-a-bgp-connection). # BGP Communities Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/bgp-communities This document outlines BGP communities in Aviatrix and describes how to configure BGP communities for implementing routing policies that direct traffic between on-premises or remote offices and the Aviatrix fabric. This document outlines BGP communities in Aviatrix and describes how to configure BGP communities for implementing routing policies that direct traffic between on-premises or remote offices and the Aviatrix fabric. ## Supported Gateways BGP communities are available on all Aviatrix gateways that support BGP connectivity. ## What is a BGP Community? A BGP community refers to a tag assigned to BGP routes that exhibit common characteristics. These tags are exchanged among BGP peers during route advertisement. BGP communities serve as an effective tool for network administrators, empowering them to implement routing policies efficiently without altering the actual route attributes. BGP communities provide these benefits: * Scalable solution for managing routing policies across large networks * Provides granular route control while maintaining configuration simplicity * Enables consistent policy application throughout multi-homed BGP environments ### Primary Use Cases The primary applications of BGP communities include influencing routing decisions such as route filtering and traffic engineering. **Route Filtering** Route filtering enables control over route advertisement and acceptance between BGP peers: * Network administrators may choose which routes to advertise based on specific BGP community tags. * BGP peers can filter inbound routes by evaluating the associated BGP community tags. * Route filtering minimizes route table updates by explicitly defining which routes should be propagated. **Traffic Engineering** Traffic engineering leverages BGP community tags to guide traffic flow throughout the network: * Network administrators can identify specific routes and apply specific policies using BGP community tags. * BGP community tags support implementation of policies such as AS-Path Prepend. * Traffic engineering creates predictable traffic flows by consistently applying definfed policies to tagged routes. ### Use Case Example Consider a multi-region cloud deployment illustrated in the following network diagram. In this scenario, the SDWAN devices receive BGP community tags and use these tags to influence routing decisions. For example, if the connection between SDWAN POP1 and Edge Site1 fails, traffic is redirected from SDWAN POP1 to Edge Site2, leveraging the BGP community received from Edge Site2 along with the preference configured on the SDWAN POP1 customer router. Similarly, should the link between SDWAN POP2 and Edge Site2 fail, traffic will reroute from SDWAN POP2 to Edge Site3 based on the BGP community received from Edge Site3 and the preference set on the SDWAN POP2 customer router. ![BGP Communities](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/bgp-communities.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=169641b75e54e07d06c0291422a8c140) ## Types of BGP Communities Aviatrix provides support for Standard BGP Communities (RFC 1997), including both well-known and user-defined community implementations. Support for Extended BGP Communities (RFC 4360) and Large BGP Communities (RFC 8092) is not available. ### No-Advertise BGP Community No-Advertise BGP community is one of the well-known BGP communities. When applied to a route from an external BGP peer, prevents the Aviatrix gateway from advertising the route to any other BGP peer, whether internal or external. Common use cases include: **Limited Route Distribution** * Restricting route propagation to specific segments of the network * Preventing specific routes from being advertised beyond a desginated point **Network Segmentation** * Establishing routing boundaries within an autonomous system * Isolating portions of the network for security or design purposes **Preventing Transit Traffic** * Limiting access to destinations to direct connections only * Avoiding unwanted transit traffic through network segments ### No-Export BGP Community No-Export BGP community is one of the well-known BGP communities. When applied to a route from an external BGP peer, prevents the Aviatrix gateway from advertising the route to any external BGP peers; it will only advertise the route to other Aviatrix gateway peers. Common use cases include: **AS Boundary Control** * Preventing the advertisement of internal routes to external networks * Implementing local routes that are not intended for global propagation **Backup Path Management** * Creating backup routes that are not re-advertised to the Internet * Controlling route propagation in multi-homed environments **Route Filtering at Scale** * Simplifying route filtering policies across multiple border routers * Ensuring consistent external routing policies **Security Considerations** * Limiting the visibility of specific network segments * Preventing inadvertent route leaks to external networks ### Numeric BGP Community Numeric BGP community is a user defined community implementation. Numeric BGP community tags are numeric values specified as two 16-bit numbers (0-65535):(0-65535). Numeric BGP tags created by network administrators to categorize and influence BGP routes according to specific organizational needs. Common use cases include: **Geographic Route Tagging** * Enabling location-based routing policies **Customer Classification** * Identifying routes by customer type or service level **Route Preference Control** * Marking routes for specific local-preference settings * Implementing complex traffic engineering without AS-Path manipulation **Advanced Route Filtering** * Creating granular route acceptance policies * Establishing selective route redistribution boundaries ### Auto-Derived BGP Communities Auto-derived BGP communities are automatically generated according to cloud type and cloud region and are attached to routes on the VPC or VNet. These communities provide additional information about the origin of a route, which can be used for routing policies. For example, auto-derived communities can indicate the geographic regions from which workloads originate (such as North America or Europe), allowing identification of route source locations for policy application. Benefits of Auto-Derived Communities: * **Policy Implementation Efficiency**: Policies can be defined using these tags, contributing to scalable and efficient implementation. * **Traffic Engineering Capabilities**: Derived communities enable control over traffic flow based on attributes such as geographic location or network type. * **Route Visibility**: These communities facilitate visibility into route characteristics, which assists with network management and troubleshooting. ## BGP Communities Configuration Aviatrix has three levels of BGP communities configuration: global, gateway, and connection. ### Global BGP Communities Setting Global BGP Communities is a universal setting. This setting is Off, by default. When Global BGP Communities is enabled, Aviatrix gateways accept and send communities from and to external BGP peers and its attachments, unless the [override BGP communities is set at the gateway-level](#gateway-level-bgp-communities-setting). #### Auto-Derivation BGP Communties Setting When Auto-Derivation is enabled, Aviatrix generates BGP community tags using the specified Community Prefix. ![BGP Communities Auto Derivation](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/bgp-communities-auto-derivation.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=3d4775fe1f331bad3a028e4b9bdf8913) The Community Prefix is a 16-bit number representing the first part of the BGP Community tag format (0-65535):(0-65535); the second 16-bit segment is auto-generated based on cloud type and region. For instance, if the Community Prefix is 12345, the system may generate communities such as 12345:1027, 12345:1028, 12345:4100, and 12345:1047. Auto-Derivation requires Global BGP Communities Support to be enabled. ### Gateway-Level BGP Communities Setting By default, a gateway will handle BGP communities according to the [global BGP communities](#global-bgp-communities-setting) setting. Sometimes, you might want individual gateways to behave differently from the global setting, such as when testing before turning the feature on. In that case, you can [override the global setting on individual gateways](#enable-bgp-communities-on-a-specific-gateway). Using a gateway's BGP communities settings, you can choose whether the gateway follows the global BGP setting or ignores it. In this example, when both the gateway's override settings are unchecked, the gateway will follow the global BGP setting. ![BGP Communities Gateway Setting 3](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/bgp-communities-gw-setting3.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=806927712cf4de3a8772828b321be3e2) In this example, when the gateway's **Override BGP Communities Advertisement** is checked, the gateway will ignore the global BGP communities setting and not advertise BGP communities to its BGP peers. ![BGP Communities Gateway Setting 1](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/bgp-communities-gw-setting1.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=a15535177455aab5decd0fd92f9a8d9e) In this example, when the gateway's **Override BGP Communities Advertisement** is checked and turned On, the gateway will ignore the global BGP communities setting and advertise BGP communities to its BGP peers. ![BGP Communities Gateway Setting 2](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/bgp-communities-gw-setting2.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=bb327264bf294394a5c11506b38a1fef) ### Connection-Level BGP Communities Setting Connection-level BGP communities allows you to [set BGP communities for a specific external BGP peer](#enable-bgp-communities-for-a-connection). With connection-level BGP communities, you can choose to: * **Block BGP communities**. The gateway will not advertise BGP communities to the external BGP peer. * **Advertise BGP communities additively**. The gateway will advertise all BGP communities that were received from peers. If any additional communities are specified, they will be added to all advertisements over this connection. * **Advertise BGP communities as a replacement**. The gateway will advertise only the additional BGP communities specified for the BGP connection and ignore the BGP communities tagged on a route to the external BGP peer. You can [enable connection-level BGP communities](#enable-bgp-communities-for-a-connection) when you create a BGP over IPsec, BGP over GRE, or BGP over LAN external connection in Aviatrix. Connection-level BGP communities are *only* advertised when the gateway advertise to BGP communities to peers setting is enabled either from the global BGP communities setting or overridden at the gateway-level. In this example, when Global BGP Communities setting is enabled with gateway override options unchecked, then connection-level BGP communities are advertised. ![BGP Communities Connection Setting 3](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/bgp-communities-conn-setting3.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=468cd6e66c7ff5d575053e308e8646db) In this example, when Global BGP communities is disabled and gateway Advertise BGP Communities to Peers override is checked and On, then connection-level BGP communities are advertised. ![BGP Communities Connection Setting 4](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/bgp-communities-conn-setting4.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=49f6b4e3e0f33793279d5dfa78dc18b5) ## Enable BGP Communities Globally To enable global BGP communities: 1. In Aviatrix CoPilot, go to **Cloud Fabric > Gateways > Settings** tab. 2. Click **Global BGP Communities Support** toggle to **On**. 3. Click **Save** to save your changes. You can [override this setting for a specific gateway](#enable-bgp-communities-on-a-specific-gateway) and choose not to accept or advertise communities to BGP peers. ## Enable Auto-Derivation of BGP Communities To enable auto-derivation of BGP communities: 1. In Aviatrix CoPilot, go to **Cloud Fabric > Gateways > Settings** tab. 2. In Global BGP Communities Support card, set the **Global BGP Communities Support** toggle to **On**. 3. Set the **Auto-Derivation** toggle to **On**. 4. In **Community Prefix**, enter the first 16-bit prefix (required to auto-generate tag) of the BGP Community tag format (0-65535):(0-65535). 5. Click **Save** to apply the changes. ## Enable BGP Communities on a Specific Gateway If you prefer to override the global setting for an individual gateway, follow these steps: 1. In Aviatrix, go to the gateway's Settings page. * For Transit or Spoke gateway, go to **Cloud Fabric > Gateways**, then select the gateway's tab. * For Edge Gateway, go to **Cloud Fabric > Hybrid Cloud > Edge Gateways** tab. 2. In the table, locate and select the gateway. 3. In the gateway's page, click the **Settings** tab. 4. Expand the **Border Gateway Protocol (BGP)** section. 5. On the **BGP Communities Support** card: * To choose whether the gateway accepts BGP communities from its BGP peers. 1. Check **Override BGP Communities Acceptance**. 2. Click **Accept BGP Communities from Peers** toggle to **On** or **Off**. * To choose whether the gateway advertises BGP communities to its BGP peers. 1. Check **Override BGP Communities Advertisement**. 2. Click **Advertise BGP Communities from Peers** toggle to **On** or **Off**. 6. Click **Save** to save your changes. ## Enable BGP Communities for a Connection > [!NOTE] > > Connection-level BGP communities are advertised only when the gateway > advertise to BGP communities to peers setting is enabled either from the > global BGP communities setting or at the gateway-level (see [Connection-Level > BGP Communities Setting](#connection-level-bgp-communities-setting)). To configure BGP Communities for *a new BGP connection*, see [Setting Up External Device Connection](/docs/enterprise/10.1/guides/connectivity/vpn/transit-externaldevice-workflow) . To configure BGP communities for an existing BGP connection: 1. In Aviatrix CoPilot, go to **Networking > Connectivity > External Connections (S2C)** tab. 2. From the list of BGP connections, select the connection for which you want to configure BGP communities. For quick access, you can use the Filter or Search option from the toolbar to find by name or matching condition. 3. In the connection's **Settings** tab, expand the **Border Gateway Protocol** section. 4. Locate **Advertise BGP Communities to Peer** card. 5. From the **Advertised BGP Communities** dropdown, select one of the following: * **Same as Gateway**: This is the default setting for the connection. The gateway advertises the BGP communities that were received or tagged on the route. * **Block Advertisement**: The gateway will not to advertise BGP Communities received from any BGP peer or attachment. * **Add Additional Communities**: The gateway will advertise the additional BGP communities specified to the external BGP peer in addition to the BGP communities that were received or tagged on the route. You can choose to enter a numeric BGP community tag or select from the dropdown menu. Aviatrix supports: * **Numeric** - You can specify numeric BGP community tags specified as two 16-bit numbers (0-65535):(0-65535). * **No-Advertise** - When a No-Advertise BGP community is attached to a route from an external BGP peer, the Aviatrix gateway will not advertise the route to any gateway attachments or external BGP peers. > [!NOTE] > > Aviatrix gateways only support eBGP to external BGP peers. * **No-Export** - When a No-Export BGP community is attached to a route from an external BGP peer, the Aviatrix gateway will not advertise the route to any external BGP peers and advertise only to gateway attachments. ![BGP Communities Connection Setting 2](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/bgp-communities-conn-setting2.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=e76ef3cda686877a5ba34d4fbe114713) * **Specific Communities**: The gateway will advertise to the external BGP peer only these BGP communities specified and ignore the BGP communities that are already tagged on the route. ![BGP Communities Connection Setting 1](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/bgp-communities-conn-setting1.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=247c2eea3b984a5cae60f656afb26e43) 6. Click **Save**. ## Reset BGP Community Overrides You can reset *all* gateways to inherit the global BGP community setting. This will remove (uncheck) all [gateway-level BGP communities overrides](#gateway-level-bgp-communities-setting), ensuring all gateways follow the global setting. Resetting the override will cause all gateways to stop or start handling communities, according to the global setting. This is especially useful if you need to make sure that communities are disabled across your entire fabric. > [!NOTE] > > Resetting gateway-level BGP communities overrides does not reset the > connection-level BGP communities. However, if gateway advertise BGP > communities is not enabled either from global BGP communities setting or at > the gateway-level, then connection-level BGP communties are not advertised. 1. In Aviatrix CoPilot, go to **Cloud Fabric > Gateways > Settings** tab. 2. Locate **Global BGP Communities Support** card. 3. Click **Reset Gateway Overrides**. This will remove (uncheck) all gateway-level BGP communities overrides, ensuring all gateways follow the global setting. 4. Click **Reset** to confirm. ## Viewing BGP Communities Status for a BGP Connection When you configure BGP communities for Aviatrix gateway, you can view the BGP communities' status and the list of BGP communities received (communities tagged on the route) and additional BGP communities configured for the BGP connection. To view BGP Communities status: 1. In Aviatrix CoPilot, go to **Networking > Connectivity > External Connections (S2C)** tab. 2. From the list of BGP connections, select the connection for which you want to view BGP communities status. For quick access, you can use the Filter or Search option from the toolbar to find by name or matching condition. 3. Click the **Details** tab. 4. In the **BGP** table: * Locate **Advertise BGP Communities to Peer** to view the status. Status is: * **Enabled** when **Advertise BGP Communities to Peer** is turned On. * **Disabled** when **Advertise BGP Communities to Peer** is turned Off or blocked. * Locate **Received Communities** to view the list of BGP communities received (tagged on the route). * Locate **Advertise Communities** to view the list of additional BGP communities configured for the BGP connection. # Edge Spoke Gateway Design Patterns Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/edge-spoke-design-patterns This document describes these common design patterns for deploying Aviatrix Edge Spoke Gateway. This document describes these common design patterns for deploying Aviatrix Edge Spoke Gateway. * [Aviatrix Hybrid Cloud Connectivity over Private Network](#single-transit-with-single-edge) * [Aviatrix Hybrid Cloud Connectivity over Public Network](#attachment-over-public-network) * [Transit Connectivity with Redundant Edge](#single-transit-with-redundant-edge) * [Transit Connectivity with Redundant Edge and Redundant Circuit](#single-transit-with-redundant-edge-and-redundant-circuits) * [Multicloud Connectivity with Aviatrix Hybrid Cloud Connectivity](#multicloud-transit-networking-with-edge) * [Aviatrix Hybrid Cloud Connectivity to VLANs](#aviatrix-edge-spoke-gateway-connectivity-to-vlan) ## Single Transit with Single Edge ### Attachment over Private Network In this design, a single Edge Spoke Gateway is attached to an Aviatrix Transit Gateway over a private network, such as AWS Direct Connect, Azure Express Route, and GCP Interconnect. ![edge\_private\_network](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/edge_private_network.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=07fb6148ed8b5468a271dc494fa7b337) In this scenario: * The WAN router runs a BGP session for underlay CSP to exchange routes with the CSP and the CSP advertises the Transit CIDR, for underlay reachability. You can also use Aviatrix Edge Spoke Gateway to terminate CSP underlay in Equinix Network Edge without a need for a WAN router as shown below. ![edge-terminate-wan-bgp](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/edge-terminate-wan-bgp.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=18ad9094dc9f5cee63415e15f7f7765b) * The Edge Spoke Gateway LAN interface runs a BGP session to the LAN BGP router, where the LAN BGP router and the Edge Gateway exchange their network address range to each other over the BGP connection. * The Edge Spoke Gateway is attached to the Aviatrix Transit Gateway in the Transit VPC, as overlay, where the Aviatrix Transit Gateway and Edge Gateway exchange all their CIDRs to each other. * The Edge Spoke Gateway registration via Management with default route towards the Internet Firewall or router. Registration can be done via the Internet such as per the diagram or via private network. ### Attachment over Public Network In this design, a single Edge Spoke Gateway is attached to an Aviatrix Transit Gateway over the public network. Key ideas are similar to Attachment over Private Network design except the WAN Router provides Internet connectivity to Transit > [!NOTE] > > If you have multiple Edge Spoke Gateways, make sure each gateway has a unique > WAN Public IP. ![edge\_public\_network](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/edge_public_network.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=f531bc6125774fc4141b662ca8df2c1b) ## Single Transit with Redundant Edge In this design, multiple Edge Spoke Gateways are deployed to provide redundancy over a single private network circuit. Multiple Edge Spoke Gateways can be deployed in Active-Active mode with ECMP or Active-Standby. > [!NOTE] > > In the Active-Active deployment model, the network device connected to the > Edge Spoke Gateways needs to be able to handle asymmetric routing. ![edge-single-transit-redundant](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/edge-single-transit-redundant.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=5ea4b0fdf6fef018ddbd208f3a2aeba5) ## Single Transit with Redundant Edge and Redundant Circuits In this design, multiple Edge Spoke Gateways are deployed with redundant private network circuits. Multiple Edge Spoke Gateways can be deployed in Active-Active mode with ECMP or Active-Standby. > [!NOTE] > > In the Active-Active deployment model, the network device connected to the > Edge Spoke Gateways needs to be able to handle asymmetric routing. ![edge-redundant-circuit](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/edge-redundant-circuit.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=35b9f193bba633e2ac9076234b6eefb5) ## Multicloud Transit Networking with Edge In a multicloud setup scenario, Edge Spoke Gateway can function as a transitive router providing high-performance encryption and routing the traffic between cloud service providers. The key ideas for this scenario are: * Edge Spoke Gateway is attached to multiple Transit Gateways (for example, Transit in AWS and Transit in Azure) * Transitive Routing feature is enabled on the Edge Spoke Gateway. * (Optional) Transit Peering over public network between Transit in AWS and Transit in Azure. * By default, Transit Peering will be the preferred path. To make Transit Peering less preferred, use [Connection AS Path Prepend feature](https://legacy.docs.aviatrix.com/documentation/latest/network/bgp-connection-settings.html?expand=true#as-path-prepend). * Edge redundancy can be achieved by deploying multiple Edge Spoke Gateways in Active-Active or Active-Standby configurations. ![edge-multiple-transit-single-edge](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/edge-multiple-transit-single-edge.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=664b1fc8e72b49bc4580d33458a607eb) ## Aviatrix Edge Spoke Gateway Connectivity to VLAN Aviatrix Edge Spoke Gateway can be deployed in on-premises with LAN gateway functionality and VLAN and VRRP support. Edge Spoke Gateways can connect to a single or multiple Transit Gateways in this design. ![edge-vlan-topology](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/edge-vlan-topology.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=ab0b37c3ccb85613626c29bee31ce5a4) # Edge Spoke Gateway High Availability Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/edge-spoke-ha This document provides a brief overview of Edge Spoke Gateway high availability. This document provides a brief overview of Edge Spoke Gateway high availability. Before reading this document, you should be familiar with the following: * [Aviatrix Multicloud Transit Network Architecture](/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/transit-network-faq) * [Aviatrix ActiveMesh](/docs/enterprise/10.1/concepts-architectures/components/connectivity/activemesh-about) ## Topology The Edge Spoke Gateways support active-active and active-standby peering connections for high availability. ## High Availability Modes Aviatrix Edge Gateways are associated with an [edge site](https://legacy.docs.aviatrix.com/documentation/latest/network/edge-gateway-settings.html#site). ### Active-Standby Edge In an active-standby configuration, an *edge site* can consist of only two Edge Gateways with one active peering and one standby peering. Only the primary Edge Gateway actively forwards network traffic. The secondary Edge Gateway takes over when the primary gateway is down. In an active-standby deployment, you can set *Preemptive* mode to automatically switch back to using the primary gateway when the gateway connection is back up. By default, the network continues to use the secondary gateway even after the primary gateway is back up. ### Active-Active Edge In an active-active configuration, both the primary and secondary gateways have active peerings. Network traffic is load balanced and forwarded across both active peerings by using Equal Cost Multi Path (ECMP) routing. In an active-active configuration, an *edge site* can consist of multiple Edge Gateways. You can deploy multiple gateways to scale your Edge Gateways for throughput. For more information, see [Edge Spoke Gateway Horizontal Scaling](/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/edge-spoke-scaling). ## About Edge Spoke Gateway to Transit Gateway High Availability For high availability between Edge Spoke Gateway and Transit Gateway in the cloud, you can deploy Edge Spoke Gateway in *active-active* configuration. This feature is supported on all [Edge platforms](/docs/enterprise/10.1/concepts-architectures/components/edge-gateways/edge-overview#edge-platforms). ## About Edge Spoke Gateway to LAN High Availability Edge Spoke Gateways can connect to a LAN side router using BGP over LAN or to an external device such as a server or firewall using static routes. Edge Spoke Gateways support *active-active* and *active-standby* connections to provide LAN side high availability. This feature is supported on all [Edge platforms](/docs/enterprise/10.1/concepts-architectures/components/edge-gateways/edge-overview#edge-platforms). This diagram shows Edge Spoke Gateways deployed on multiple edge sites with active and standby peering connections to different LAN routers. ![Edge Spoke Gateways deployed on multiple edge sites with active and standby peering connections to different LAN routers](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/edge-no-hs-lan-connection.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=914fa59cb5de60ee4448918bbbb75f39) This diagram shows Edge Spoke Gateways deployed on the same site with active and standby peering connections to different LAN routers. ![Edge Spoke Gateways deployed on the same site with active and standby peering connections to different LAN routers](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/edge-no-horizontal-scale.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=035952899b29b65b6b491059a7371057) ## About Edge Spoke Gateway to VRRP High Availability Edge Spoke Gateways support Virtual Router Redundancy Protocol (VRRP) protocol to provide high availability for Virtual Local Area Network (VLAN) connectivity. You can deploy Edge Spoke Gateways in on-premises locations with the VRRP protocol enabled. This feature is supported on the *Aviatrix Edge Platform* and *Self Managed* platforms. > [!NOTE] > > If you deploy an Edge Spoke Gateway with VRRP enabled, then all Edge Spoke > Gateways in the same *edge site* must be created in active-active high > availability mode. The diagram below illustrates a VRRP configuration with Edge Spoke Gateways as the default LAN router. LAN traffic flows to the VRRP active gateway to the cloud. When the VRRP active gateway is down, traffic flows to the VRRP standby gateway to cloud. ![Edge Spoke Gateways VRRP configuration with multiple VLANs](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/edge-multi-vlans.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=61981743be91d0b7eb7b35633e229e2b) To learn more about VLAN and VRRP support, see [About Extending VLAN Segmentation to the Cloud](/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/edge-spoke-vlan-segmentation). ## About Creating Highly Available Edge Gateways The steps to deploy highly available Edge Spoke Gateways is different depending on the Edge platform. ### Aviatrix Edge Platform To deploy high availability Edge Spoke Gateways in the Aviatrix Edge Platform, you must first onboard the devices where you want to deploy the gateways (see [Installing Edge OS and Onboarding Dell Hardware](/docs/enterprise/10.1/guides/connectivity/edge/edge-device-onboarding)). When you create an Edge Spoke Gateway for an *edge site*, you have the option to enable High Availability (HA) mode as active-active mode or active-standby mode. The primary and secondary HA gateways are then created in the primary and secondary devices that you configure. > [!NOTE] > > * If your LAN network uses VRRP configuration, then High Availability mode must be set to active-active mode. > * High Availability mode cannot be changed after the primary and secondary (HA) Edge Spoke Gateways are created. To change the High Availability mode, you must first delete the HA Edge Gateway. > [!NOTE] > > * If your LAN network uses VRRP configuration, then High Availability mode must be set to active-active mode. > * High Availability mode cannot be changed after the primary and secondary (HA) Edge Spoke Gateways are created. To change the High Availability mode, you must first delete the HA Edge Gateway. For the deployment workflow, see [Deploy an Edge Spoke Gateway on Aviatrix Edge Platform](/docs/enterprise/10.1/guides/connectivity/edge/edge-spoke-workflow-onpremise). ### Network Service Provider and Self Managed Platforms To deploy high availability Edge Spoke Gateways in a Network Service Provider (NSP) platform (Equinix or Megaport) or Self Managed platform, the high-level steps are as follows: 1. Create the primary Edge Spoke Gateway, which generates a ZTP *cloud-init* file. in VM. > [!NOTE] > > High Availability mode is always off when creating the primary gateway. 2. Deploy the cloud-init file in the Edge VM on the NSP platform. 3. After the primary Edge Spoke Gateway is up, edit the primary Edge Spoke Gateway configuration and set the HA mode to active-active mode or active-standby mode. This generates a cloud-init file for the HA Edge Spoke Gateway. > [!NOTE] > > * (Self Managed platform only) If your LAN network uses VRRP configuration, > then High Availability mode must be set to active-active mode. \* High > Availability mode cannot be changed after the primary and secondary (HA) > Edge Spoke Gateways are created. To change the High Availability mode, you > must first delete the HA Edge Spoke Gateway. 4. Deploy the cloud-init file for the HA Edge Spoke Gateway in another Edge VM on the NSP platform. The figure below shows an *active-standby Edge deployment with two gateways* (one primary and one HA Edge Gateway) for the same edge site. The HA gateway name is auto-generated in the format ``. ![Active-standby Edge deployment with two gateways showing primary and HA Edge Gateway](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/edge-deploy-active-standby.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=2f6381a204133559390cef0f81b3c6bb) # Local Internet Breakout at Network Edge Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/edge-spoke-local-internet-breakout Aviatrix Hybrid Cloud Edge solution offers Local Internet Breakout capability. Aviatrix Hybrid Cloud Edge solution offers Local Internet Breakout capability. > [!NOTE] > > Local Internet Breakout capability is supported only on the Aviatrix Edge > Platform. This capability allows the Edge Spoke Gateway to route traffic from on-premises and remote sites to remote destinations via overlay tunnels and route internet-based traffic directly to the local ISP at the edge location. In SD-WAN networks typically, by routing internet-based traffic locally, you avoid the need to send internet traffic to your central datacenter for processing, which reduces network latency and costs. To support Local Internet Breakout, the Edge Spoke Gateway functions as a stateful firewall. Using the Edge Spoke Gateway's SNAT feature, you define the rules that the Edge Spoke Gateway uses to route traffic from the remote sites to the Aviatrix Transit Gateway to cloud and the internet. ![edge-local-internet-breakout](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/edge-local-internet-breakout.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=758d6e3c45deada3e24054e544fceb6d) To define SNAT rules for Local Internet Breakout on the Edge Spoke Gateway: 1. In Aviatrix CoPilot, go to **Cloud Fabric > Hybrid Cloud > Edge Gateways** tab. 2. Click **Spoke Gateways**. 3. Select the Edge Spoke Gateway for which you want to enable local internet breakout. 4. Click the Edge Spoke Gateway's **Settings** tab and expand the **Network Address Translation (NAT)** section. 5. Set the **Source NAT** toggle to **On**. 6. Select **Single IP**, then click **Save**. # Edge Spoke Gateway Horizontal Scaling Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/edge-spoke-scaling This document provides a brief overview of Aviatrix Edge Spoke Gateway horizontal scaling. This document provides a brief overview of Aviatrix Edge Spoke Gateway horizontal scaling. > [!NOTE] > > Edge Spoke Gateway horizontal scaling is supported only with the latest > version of the Aviatrix Controller. Edge Spoke Gateway horizontal scaling provides you the flexibility to add or remove gateways to plan for capacity during peak times to ensure high availability and improve and maintain network stability and performance. Horizontal scaling requires all Edge Spoke Gateways in an [edge site](https://legacy.docs.aviatrix.com/documentation/latest/network/edge-spoke-settings.html) to be deployed in *active-active* mode. A maximum of eight Edge Spoke Gateway instances are supported per site. Edge Spoke Gateways can be scaled horizontally and peered to the Transit Gateway or Edge Transit Gateways across single or multiple clouds and regions. The Edge Spoke Gateways connects to the Transit Gateways and Edge Transit Gateways with active peering connections. Network traffic between the Transit Gateways and Edge Spoke Gateways or between Edge Transit Gateways and Edge Spoke Gateways is load balanced and forwarded across all active peering connections by using Equal Cost Multi Path (ECMP) routing. The following figure shows multiple Edge Spoke Gateways deployed in the same edge site for horizontal scaling with multiple active-active high availability connections for multicloud to edge connectivity. ![Diagram showing Edge Spoke Gateway horizontal scaling with multicloud to edge connectivity](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/edge-horizontal-scale.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=d28496a6a9c4b3af0d1349cf2987d1ba) The following example shows an Edge configuration in CoPilot for horizontal scaling with Edge Spoke Gateway instances for the same edge site in *active-active high availability mode*. ![Edge configuration in CoPilot showing horizontal scaling with Edge Spoke Gateway instances in active-active high availability mode](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/edge-spoke-scaling-ha.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=af0ac788efd6b946f39aa091d111f5d2) When a high availability mode is not selected, Edge Spoke Gateways deployed in the same edge site default to active-active mode. The following example shows an Edge configuration in CoPilot for horizontal scaling with Edge Spoke Gateway instances in *active-active* mode (no HA) for the same edge site. ![Edge configuration in CoPilot showing horizontal scaling with Edge Spoke Gateway instances in active-active mode without HA](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/edge-spoke-scaling-no-ha.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=85d9c37681f06579236a31e98fadbfa4) For more information about creating HA Edge Gateways, see [About Creating Highly Available Edge Gateways](/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/edge-spoke-ha). # Customized SNAT and DNAT on Edge Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/edge-spoke-snat-dnat The Aviatrix Hybrid Cloud Edge solution supports customized SNAT and DNAT for the use case where the cloud service provider (CSP) network CIDR overlaps with the on-premises network CIDR. The customized SNAT and DNAT apply to HPE-enabled and HPE-disabled Edge Spoke gateways. The Aviatrix Hybrid Cloud Edge solution supports customized SNAT and DNAT for the use case where the cloud service provider (CSP) network CIDR overlaps with the on-premises network CIDR. The customized SNAT and DNAT apply to HPE-enabled and HPE-disabled Edge Spoke gateways. The following NAT scenarios are supported: * Single IP and Customized SNAT on Edge Spoke Gateway - For network traffic initiated from the edge location towards the CSP. * DNAT on Edge Spoke Gateway - For network traffic initiated from the edge location towards Transit Gateway in cloud or CSP. > [!NOTE] > > Customized SNAT on Edge Spoke Gateway is not supported when VLAN segmentation > is also configured in the same network domain. This diagram shows overlapping CSP and on-premises network CIDRs. ![Diagram showing overlapping CSP and on-premises network CIDRs with SNAT and DNAT on Edge](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/edge-snat-dnat.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=2d0902413cc8a64e1796a0a622d65bed) In this example, to resolve the overlapping CIDR issue, you would perform these steps: 1. Create a mapping of the Real CIDR to Virtual CIDR for both the cloud instances and on-premises hosts or workloads. For example: | **Network** | **Real CIDR** | **Virtual CIDR** | | - | - | - | | Cloud workload | 10.3.0.86/32 | 10.203.0.86/32 | | On-prem workload | 10.3.0.85/32 | 10.103.0.85/32 | 2. Configure DNAT on Edge Spoke Gateway for traffic initiated from on-premises to cloud. a. In Aviatrix CoPilot, go to **Cloud Fabric > Hybrid Cloud > Edge Gateways** tab. b. Click **Spoke Gateways** c. Select the Edge Spoke Gateway for which you want to enable DNAT. d. In the Edge Spoke Gateway's **Settings** tab, expand the **Network Address Translation (NAT)** section. e. Set the **Destination NAT** toggle to **On**. f. In **Destination NAT**, from the **Instance** dropdown menu, select the Edge Spoke Gateway. g. Click **+ Rule** and provide the following information. | **Field** | **Description** | | - | - | | Instance | From the dropdown list, select the Edge Gateway instance. | | Src CIDR | Enter 10.3.0.85/32. | | Dst CIDR | Enter 10.203.0.86/32 (the virtual IP of the cloud instance). | | Connection | From the dropdown list, select the connection which reflects the connection to the Transit Gateway. | | Mark | (Optional) Enter a unique value. Value should be between 65535 - 99999. | | DNAT IP | Enter 10.3.0.86 (cloud instance). | 3. Configure a Manual BGP Advertised CIDR List to advertise the DNAT virtual IP from the Edge Spoke Gateway to on-premises via BGP. a. In the Edge Spoke Gateway's **Settings** tab, expand the **Border Gateway Protocol (BGP)** section. b. In **Manual BGP Advertised CIDR List**, enter the following information. | **Field** | **Description** | | - | - | | Advertised CIDRs (Per Gateway) | Leave this blank. | | Connection | From the dropdown menu, select the connection to the on-prem BGP peer. | | Advertised CIDRs (Per Connection) | Enter 10.203.0.86/32 (the virtual IP of the cloud instance) | 4. Configure SNAT on the Edge Spoke Gateway for traffic initiated from cloud to on-premises. a. In the Edge Gateway's **Settings** tab, expand the **Network Address Translation (NAT) section**. b. Click the **Source NAT** switch to **On**, then click **Customized SNAT**. c. From the **Instance** dropdown menu, select the Edge Spoke Gateway. d. Click **+ Rule** and provide the following information. | **Field** | **Description** | | - | - | | Connection | Select the output connection where the rule will apply. | | Mark | Enter the value that was defined in the DNAT settings. Specifies a TCP session where rule applies. | | SNAT IP | Enter 10.103.0.85 (virtual IP of the on-prem host). | # Transit Peering Over Public Network with Edge Spoke Gateway Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/edge-spoke-transit-peering If you have a multicloud environment across cloud service roviders (CSPs), for example, AWS and Azure, you can create Transit Gateway Peering over public network and use the Transit Gateway Peering as a secondary or backup path while the Edge Spoke Gateway with Transitive Routing enabled is... If you have a multicloud environment across cloud service roviders (CSPs), for example, AWS and Azure, you can create Transit Gateway Peering over public network and use the Transit Gateway Peering as a secondary or backup path while the Edge Spoke Gateway with Transitive Routing enabled is used as the primary path for forwarding traffic. ![Transit peering over public network with Edge Spoke Gateway topology diagram](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/edge-transit-peering.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=55777fd4069e2df2685703fb9e55334a) ## Configuring Transit Peering over Public Network with Edge Spoke Gateway To create Transit Peering over public network, follow these steps. 1. Ensure that you have AS numbers configured on both Transit Gateways. 1. In Aviatrix CoPilot, go to **Cloud Fabric > Gateways > Transit Gateways** tab. 2. Select the Transit Gateway in one cloud service provider. 3. In the Transit Gateway's **Settings** tab, expand the **Border Gateway Protocol (BGP)** section. 4. Ensure the **Local AS number** is configured, and click **Save**. > [!NOTE] > > Make note of this AS number to use later to set AS Path Prepend. ![Transit Gateway BGP Local AS number configuration](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/edge-transit-peering-1.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=cd222393f8b98f57a8924e26b9637e30) 2. Repeat Step 1 to set the Local AS Number for the other Transit Gateway in another cloud service provider. 3. Peer the Transit Gateways. 1. Go to **Cloud Fabric** > **Gateways** > **Transit Gateways** tab. 2. Locate the Transit Gateway in one cloud service provider and click the Edit icon. 3. In Edit Transit Gateway, from the **Peer To Transit Gateways** dropdown list, select the Transit Gateway in the other cloud service provider and click **Save**. > [!NOTE] > > * Both Transit Backbone VNE/VPCs need to match configuration. \* You cannot > peer non-HA Transit Gateway with another Transit Gateway in HA mode. 4. Set the AS Path Prepend. 1. Go to **Cloud Fabric** > **Gateways** > **Transit Gateways** tab. 2. Select the Transit Gateway where you want to set Prepend. 3. In the Transit Gateway's **Settings** tab, expand **Border Gateway Protocol (BGP)** section. 4. Scroll down to **AS Path Prepend**, and select the Transit Peering connection name. You can configure per connection or per gateway (all connections). 5. In **Prepend AS Path (Per Connection)** field, input the Transit Gateway Local AS Number three times separated by space. ![AS Path Prepend configuration for Transit Peering connection](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/edge-transit-peering-2.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=3815f9499c5ea9ed7f5207f6fdf83c45) 5. To view the AS Prepended routes: 1. Go to **Cloud Fabric** > **Gateways** > **Transit Gateways** tab. 2. Select the Transit Gateway where you set the As Path Prepend. 3. Click the Transit Gateway's **Route DB** tab to view the prepended routes. ![Route DB tab showing AS prepended routes](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/edge-transit-peering-3.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=88e8de828d2a4eadda5a024636e869c9) # Transitive Routing with Edge Spoke Gateway Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/edge-spoke-transitive-routing The Transitive Routing feature allows an Edge Spoke Gateway to forward routes between multiple Transit Gateways that are connected to it. You have the option to enable or disable Transitive Routing for an Edge Spoke Gateway; it is disabled by default. The Transitive Routing feature allows an Edge Spoke Gateway to forward routes between multiple Transit Gateways that are connected to it. You have the option to enable or disable Transitive Routing for an Edge Spoke Gateway; it is disabled by default. ![edge-transitive-routing.png](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/edge-transitive-routing.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=f4c45050b9ee94366876bd2a35cdc8d7) ## Configuring Transitive Routing To configure Transitive Routing for an Edge Spoke Gateway, follow these steps. 1. Attach the Edge Spoke Gateway to the first Transit Gateway. Follow the steps in [Attach Edge Gateway to Transit Gateway](/docs/enterprise/10.1/guides/connectivity/edge/edge-spoke-attachments). 2. Repeat and attach the Edge Spoke Gateway to the second Transit Gateway. 3. Go to *Cloud Fabric > Hybric Cloud > Edge Gateways* tab. 4. Click *Spoke Gateways*. 5. Select the Edge Spoke Gateway. 6. In the Edge Spoke Gateway's *Settings* tab, expand the *Routing* section. 7. Set the *Transitive Routing* toggle to *On*. 8. Verify the routes on each Aviatrix Transit Gateway. # Extending VLAN Segmentation to the Cloud Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/edge-spoke-vlan-segmentation This document describes the Aviatrix Hybrid Cloud Edge solution to extend virtual LAN (VLAN) network segmentation into cloud. This document describes the Aviatrix Hybrid Cloud Edge solution to extend virtual LAN (VLAN) network segmentation into cloud. Before reading this document, you should be familiar with the following: * Aviatrix Multicloud Transit Network architecture * Aviatrix Gateway High-Availability * Virtual LAN networking concepts * Virtual Router Redundancy Protocol (VRRP) concepts Virtual Local Area Networks (VLANs) enables you to create network segmentation to maintain separation between Local Area Networks. Businesses may deploy network segmentation for additional security or to keep workloads in their network separate. Aviatrix Edge Spoke Gateway supports multiple VLANs and VRRP architecture. This capability enables you to segment your on-premises network traffic for your LAN network segments. For example, you can segment the workload in the cloud that supports PoS (Point-of-Sale) application to the PoS application at the Edge in that specific LAN segment only. > [!NOTE] > > VLAN and VRRP support is only available on the Aviatrix Edge Platform and > Self-Managed Platform. It is not available on the Equinix Platform. ## VLAN TOPOLOGY ### Single VLAN Topology The topology below illustrates an Edge site with a single VLAN, Virtual Router Redundancy Protocol (VRRP), and the Edge Spoke Gateway deployed as the default LAN router. ![Single VLAN topology diagram showing an Edge site with VRRP and Edge Spoke Gateway as the default LAN router](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/edge-single-vlan.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=346df7d12ed95bea4e21504bfc99ceba) In this diagram: * The VRRP and VLAN segment is configured on the primary Edge Spoke Gateway. * The VLAN segment is automatically created symmetrically on the secondary Edge Spoke Gateway. * The VRRP Virtual IP is used by the LAN hosts as the Default Gateway IP. > [!NOTE] > > * VRRP support requires that the Edge Spoke Gateway is created with Active-Active high availability peering connection to the Transit Gateway(s). > * The Edge Spoke Gateways advertise the CIDRs associated with the LAN and VLAN sub-interfaces to the Aviatrix Transit Gateway(s). You can disable CIDR propagation, if not needed. See [Disabling LAN and VLAN CIDR Propagation to the Transit Gateway](#disabling-lan-and-vlan-cidr-propagation-to-the-transit-gateway). ### Multiple VLAN Topology The topology below illustrates an edge site with multiple VLANs, Virtual Router Redundancy Protocol (VRRP), and Edge Spoke Gateway deployed as the default LAN router. ![Multiple VLAN topology diagram showing an Edge site with multiple VLAN segments, VRRP, and Edge Spoke Gateway as the default LAN router](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/edge-vlan-topology.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=ab0b37c3ccb85613626c29bee31ce5a4) In this diagram: * The VRRP and VLAN segments are configured on the primary Edge Spoke Gateway. > [!NOTE] > > For multiple Edge Spoke Gateways deployed on an edge site, you configure the > VLAN segments on each primary Edge Spok Gateway. The VLAN segments must be > configured with the same VLAN ID and CIDR across all the Edge Spoke Gateways > on the Edge site. * The VLAN segments are automatically created symmetrically on the secondary Edge Spoke Gateway(s). * The VRRP Virtual IP is used by the LAN hosts as the Default Gateway IP for the VLAN segments. * The Edge Spoke Gateway provides the trunk interface connectivity to the LAN side switch. > [!NOTE] > > * VRRP support requires that the Edge Spoke Gateway is created with Active-Active high availability peering connection to the Transit Gateway. > * The Edge Spoke Gateways advertise the CIDRs associated with the LAN and VLAN sub-interfaces to the Aviatrix Transit Gateway(s). You can disable CIDR propagation, if not needed. See [Disabling LAN and VLAN CIDR Propagation to the Transit Gateway](#disabling-lan-and-vlan-cidr-propagation-to-the-transit-gateway). ## Extending On-Premises VLAN Segmentation into Cloud To extend on-premises VLAN segmentation into cloud involves these two steps. 1. [Configure VLAN Interfaces on the Edge Spoke Gateway](#configuring-vlan-interfaces-on-the-edge-spoke-gateway). 2. [Associate VLANs to Network Domains](#associating-vlans-to-aviatrix-network-domains). ## Configuring VLAN Interfaces on the Edge Spoke Gateway You can configure VLAN interfaces on the Edge Spoke Gateway during or after the gateway is created. To configure VLANs for a new Edge Spoke Gateway, follow the instructions for: * [Create the Primary and Secondary Edge Spoke Gateway (Aviatrix Edge Platform)](/docs/enterprise/10.1/guides/connectivity/edge/edge-spoke-workflow-onpremise) * [Creating the ZTP ISO for the Edge Gateway (Self-Managed Platform)](/docs/enterprise/10.1/guides/connectivity/edge/edge-spoke-workflow-selfmanaged) To configure VLANs for an existing Edge Spoke Gateway, perform the steps below. 1. In Aviatrix CoPilot, go the *Cloud Fabric > Hybrid Cloud > Edge Gateways* tab. 2. Click *Spoke Gateways*. 3. In the table, locate the Edge Spoke Gateway and click the Edit icon on right side of the row. 4. In Edit Edge Spoke Gateway, click *Next* to go to *Interface Configuration* step. 5. Click *LAN* and enable VRRP, if applicable. | **Field** | **Description** | | - | - | | IP Assignment | The default is **Static** for static IP assignment on this LAN interface. **DHCP** for dynamic IP address assignment is not supported. | | VRRP | To enable Virtual Router Redundancy Protocol (VRRP) on the Edge Spoke Gateway, set this toggle to **On**. | 6. Click *+ VLAN Interface* to add one or more VLAN sub-interfaces. | **Field** | **Description** | | - | - | | Interface CIDR | Enter the native VLAN interface IP address. This interface is where untagged packets are sent. | | Default Gateway IP, VRRP Gateway IP | Enter the Default gateway IP address. If VRRP is enabled, enter the VRRP Gateway IP address. If VRRP is disabled, enter the Default gateway IP address for the native VLAN interface. | | Interface Labels | Enter a name to identify this native VLAN interface. | | VLAN ID | Enter the VLAN ID. VLAN ID must be a number between 2 and 4092. | | VLAN Interface CIDR | Enter the VLAN's interface IP address. | | Default Gateway IP | Enter the Default gateway IP address for this VLAN interface. | | Sub-Interface Tag | Enter a name to identify this VLAN interface. | If high availability is enabled, the VLAN configurations are added to the secondary Edge Spoke Gateway. If the properties are shared, the fields are disabled on the secondary gateway and non-editable. 7. If multiple Edge Spoke Gateways are deployed on the edge site, repeat the steps above for each primary Edge Spoke Gateway. > [!TIP] > > * VLAN segments must be configured with the same VLAN ID and CIDR across all the Edge Spoke Gateways for the edge site. > > * You cannot edit the VLAN ID after it is configured. To edit the VLAN > sub-interface attributes, it is highly recommended to delete and recreate the > VLAN sub-interface configuration. ## Associating VLANs to Aviatrix Network Domains Aviatrix Edge Spoke Gateway enables you to segment your on-premises network traffic through network domains and connection policies. The Edge Spoke Gateway routes on-premises and inter-VLAN traffic flow based on the network domain connection policies. To segment the on-premises network traffic, you associate the VLAN segment and its workload in the CSP to the same network domain. For example, the diagram below illustrates on-premises network traffic flow between the yellow and orange domains that are segmented. * VLAN 100 is associated to the same network domain as its workload in the CSP. * VLAN 200 is associated to the same network domain as its workload in the CSP. ![VLAN segmentation diagram showing on-premises VLAN 100 and VLAN 200 associated to their respective network domains and CSP workloads](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/edge-csp-vlan-segmentation.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=fb93f8e00e816450d15841257f251bea) To create network segmentation between VLANs and Spoke VPCs Or VNets, you associate the VLANs and Spoke VPCs or VNets to network domains. When network segmentation is configured with multiple Edge Spoke Gateways using VLAN, the same VLAN ID is associated to the network segment *across all the gateways for the edge site*. > [!TIP] > > * The VLANs for an edge site must be created before the network domain association is configured. > * The VLANs must be configured with the same VLAN identifier (ID) and CIDR across all the Edge Spoke Gateways on the edge site before the network segmentation is configured. > [!NOTE] > > If VLANs are created or added after the network domain association is created, the association must be removed and reconfigured. > > 1. Remove the VLAN association with the network domain. > 2. Ensure the new VLAN is configured with the same VLAN ID and CIDR across all > the Edge Spoke Gateways on the edge site. > 3. Re-create the VLAN association with the network domain. To associate VLANs to Aviatrix Network Domains: 1. In Aviatrix CoPilot, go to *Networking > Network Segmentation > Network Domains* tab. 2. To add a new network domain, click *+ Network Domain*. Provide the following information: | **Field** | **Description** | | - | - | | Name | Enter a unique name for this network domain. A network-domain name can only have letters, digits, a hyphen (-), and an underscore (\_). The name must start with a letter and must have 2-27 characters. For example, Dev\_Domain. After you create and save a network domain, you cannot change its name. | | Association | From the dropdown list, select the Spoke VPC/VNet and the VLAN that are in this network domain. | | Connect to Network Domain | To connect this network domain to another network domain, from the dropdown list, select the other network domain. This enables traffic flow between both network domains. | 3. Click *Save*. ## Disabling LAN and VLAN CIDR Propagation to the Transit Gateway The Aviatrix Edge Spoke Gateway exchanges routes with the Transit Gateways. The Edge Spoke Gateway advertises the CIDRs associated with the LAN and VLAN interfaces to the Transit Gateway, by default. If you do not need LAN CIDR propagation, you can disable this feature. > [!NOTE] > > If there are workloads on the LAN or the VLAN CIDR, then disabling LAN and > VLAN interface CIDR propagation may cause network traffic to be dropped, since > Transit Gateway will not have learned the LAN or VLAN CIDRs. To disable LAN and VLAN CIDR propagation: 1. In Aviatrix CoPilot, go to *Cloud Fabric > Hybrid Cloud > Edge Gateways* tab. 2. Click *Spoke Gateways*. 3. Select the Edge Spoke Gateway for which to disable LAN CIDR propagation. 4. In the Edge Spoke Gateway's *Settings* tab, expand the *Routing* section. 5. Set *LAN/VLAN Interface CIDR Propagation* toggle to *Off*. # Edge Transit Gateway Design Patterns Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/edge-transit-design-patterns Edge Transit Gateway allows you to build high performance encrypted connections across Equinix and Megaport fabric to cloud environments. You can also deploy Edge Spoke Gateways in on-premises for high performance encrypted connections to Edge Transit Gateway in Equinix and Megaport. Edge Transit Gateway allows you to build high performance encrypted connections across Equinix and Megaport fabric to cloud environments. You can also deploy Edge Spoke Gateways in on-premises for high performance encrypted connections to Edge Transit Gateway in Equinix and Megaport. Additionally, you can also deploy Edge Transit Gateway in on-premises to run high performance encryption across datacenter (DC) sites. Combined with Aviatrix management and visibility plane, you get the network insights and complete abstraction of complexity with building hybrid and multiple cloud environments. The following topology shows Edge Transit Gateway running in Equinix or Megaport on in customer DC environment where in Edge Transit Gateway peers with Transit Gateways in the cloud and Edge Spoke Gateways at the remote locations for end-to-end encrypted connections. ![edge transit desgin1](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/edge-transit-desgin1.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=edf74d35bbde64e77dedc00d8b39bc18) Aviatrix Hybrid Cloud solution can integrate into existing cloud connect routers such as Direct Connect gateways, virtual network gateways to seamless integrate cloud to on-premises connectivity. By leveraging Edge Transit Gateway in Equinix and Megaport, you can build virtual connections to cloud environments in a matter of minutes avoiding physical hardware needs. Edge Transit Gateway replaces expensive MACSec implementations and overhead associated with managing MACSec environments. Edge Transit Gateway supports the following features: * High Performance Encryption (HPE) and non-HPE connections * HPE over public and private networks on the Aviatrix Edge Platform (AEP). * HPE over private networks on Equinix and Megaport platforms. * Non-HPE over both public and private networks. * Edge Transit Gateway to Edge Transit Gateway peering. * Edge Transit Gateway to Transit Gateway (in cloud) peering. * Edge Transit Gateway BGP underlay connection to terminate private cloud service provicer (CSP) circuits. * Terminating external BGP over LAN connections on Edge Transit Gateway. * Edge Transit Gateway to Edge Spoke Gateway peering. * Segmentation domains and NAT use cases at Edge Spoke. The following topology shows connecting two regions in AWS via Equinix private connectivity and encrypting connections to cloud and across Equinix Metros. This provides simplified on-demand cloud connectivity via high bandwidth private circuits that Equinix provides. Aviatrix provides an overlay solution for high performance encrypted and highly visible network across cloud and Equinix. ![Two AWS regions connected via Equinix private connectivity with encrypted connections across Equinix Metros](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/edge-transit-design2.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=e66ce2a10fd274b4c280b49004c16b84) The following topology shows datacenter or on-premises traffic encryption using Aviatrix High Performance Encryption (HPE) that is otherwise expensive to implement using technologies like MACSec. ![Datacenter or on-premises traffic encryption using Aviatrix High Performance Encryption](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/edge-transit-design3.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=fd5ead5fa20d40911c3d931ed0647f5d) The following topology shows how Edge Transit Gateway can be leveraged to build encrypted connectivity across customer on-premises and cloud service providers. Additionally, Edge Transit Gateway can be used as a landing zone for partner connectivity via BGP connections and enable segmentation to allow access to required applications across on-premises and clouds. ![Edge Transit Gateway providing encrypted connectivity across on-premises and cloud service providers with BGP and segmentation](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/edge-transit-design4.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=084090d92e84a1cde3e78a0bdca0c331) # Edge Transit Gateway External Connection Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/edge-transit-s2c-overview Many enterprises today use cloud adjacent and on-premises data center locations for connecting various remote sites and partners. The ability to seamlessly build a secure network fabric to cloud and remote sites and deploy segmentation controls to meet agility and business zero trust... Many enterprises today use cloud adjacent and on-premises data center locations for connecting various remote sites and partners. The ability to seamlessly build a secure network fabric to cloud and remote sites and deploy segmentation controls to meet agility and business zero trust architecture is increasingly vital for customers with workloads that span multiple locations. ## What is Edge Transit Gateway External Connection? Aviatrix Edge Transit Gateway external connection site-to-cloud (S2C) allows enterprises to build secure connectivity across non-CSP locations and build networks that address segmentation and IP address overlapping for M\&A deployments. You can leverage Edge Transit Gateway external connection to land VPN connections from on-premises, datacenters (DCs), and network provider locations, such as Equinix and Megaport, in a simple and secure way to protect workload following Zero Trust Network Access (ZTNA) principles. Aviatrix Edge Transit Gateway external connection (S2C) provides these key benefits: * **Simplified Deployment**: Build secure tunnels seamlessly from any on-premises, datacenter, and network provider locations such as Equinix and Megaport. * **Scalability**: Single Edge Transit Gateway can support multiple external connections. * **Security**: Build segmentation for tunnels to protect and secure hybrid and partner connectivity. * **Automated Network Address Translation (NAT)**: Simplifies NAT policy configuration, making it easier to manage overlapping IP addresses improving network integration and communication. * **Overlapping IPs**: Solves IP address overlapping challenges with NAT on Edge Transit Gateway tunnels. This helps with M\&A onboarding and rapid time to value all while securing the networks. * **On-Premises Device Compatibility**: The solution seamlessly interoperates with various on-premises devices, supporting different encryption algorithms and configurations, especialy beneficial for SaaS providers connecting with diverse customer equipment. * **High Availability and Redundancy**: Support for dynamically rerouting traffic to backup gateways if the primary tunnel experiences downtime. * **Visibility and Troubleshooting**: Aviatrix provides detailed insights and diagnostic tools, aiding in effective troubleshooting and network management. ## How does Edge Transit Gateway External Connection work? Aviatrix's Edge Transit Gateway establishes encrypted connections to partner, remote sites, and external networks, facilitating secure data transfer over the internet or over private networks. The Edge Transit Gateway supports BGP over IPsec, BGP over GRE, and BGP over LAN connections. It runs a BGP session to an external device, such as a remote router or firewall to dynamically exchange routes. The remote endpoints can be active-mesh or standalone devices. The gateway uses the tunneling protocol to build a tunnel to the external device for packet forwarding. ## How do you create Edge Transit Gateway External Connection? Aviatrix Edge Transit Gateway external connection (S2C) can be created in: * Appliance form factor on the Aviatrix Edge Platform (AEP) * Equinix * Megaport For more information, see [Edge Transit Gateway External Connection Workflow](/docs/enterprise/10.1/guides/connectivity/edge/edge-s2c-workflow). ## Design Examples **Connect remote sites and on-premises datacenter to cloud** The following architecture shows Edge Transit Gateway external connection (S2C) to connect remote offices and on-premises datacenter to cloud workloads. ![Edge Transit Gateway external connection connecting remote offices and on-premises datacenter to cloud workloads](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/edge-transit-s2c1.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=1434951e35939e5b91636d27fc450b5d) **Edge Transit Gateway External Connection in Megaport** The following diagram shows Edge Transit Gateway external connection (S2C) to connect a customer network at DC or in another MVE to cloud. ![Edge Transit Gateway external connection in Megaport connecting customer network to cloud](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/edge-transit-s2c2.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=50a574b16338c1998d311b287c4fbcf2) The diagram shows: * Edge Transit Gateway external IPsec connection to a remote customer network. * Edge Transit Gateway external IPsec connection to SDWAN. **Edge Transit Gateway External Connection for Peering Between Edges** The following diagram shows two Transit Edge Gateways deployed in two Megaport locations to peer between edges. ![Two Transit Edge Gateways deployed in Megaport locations peering between edges](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/edge-transit-s2c3.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=9a5af6331ebfab7484cbe1c4ce1be41b) The diagram shows: * Edge Transit Gateway external connection to another MVE via SDWAN device. * Edge Transit Gateway could also have an external connection to a remote network over internet. Edge Transit Gateway external connection (S2C) simplifies building end-to-end secure network and enables setting up segmentation for different connections for Zero Trust Network Access (ZTNA). # Enabling GCP Global VPC Routing Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/gcp-global-vpc GCP Global VPC creates regional awareness between the VPC and Aviatrix gateways, letting you restrict spoke gateway traffic to transit gateways in the same region as the spoke gateway and, starting with Controller 10.1, configure a distinct egress mode for each region. ## Overview GCP Global VPC creates regional awareness between the VPC and Aviatrix gateways, allowing you to restrict spoke gateway traffic to transit gateways in the same region as the spoke gateway. Without global VPC, communications between spokes over transit in the same region are routed outside the region. Regional awareness is achieved by appending regional network tags to virtual machines and adding regional routes to the gateways in the routing table using tags. > [!NOTE] > > If you use a **Regional VPC** (Global VPC disabled) with **Single IP SNAT** on > a spoke gateway, Controller **9.0** and later supports tagless default routes, > allowing serverless and untagged workloads egress through the gateway. See > [GCP Spoke Single IP SNAT and Tagless Default Routes (Regional VPC)](/docs/enterprise/10.1/guides/connectivity/routing/gcp-spoke-single-ip-snat-regional-vpc) > . Global VPC enabled spokes can be deployed in a single region, multi-region, or hybrid configuration to segment your workload traffic by region. A multi-region spoke VPC is a spoke VPC that has a pair of spokes in each region where workloads are deployed. **Multi-Region Spoke VPC** ![Multi-Region Spoke VPC](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/gcp-multi-region-vpc.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=dd0df493a2f5f2e98f4168a702acc900) A regional spoke VPC is a spoke VPC with a unique group of spoke gateways, and workloads in one or more individual regions. Workloads outside the spoke gateway region can exit the VPC through the spoke gateways using Global VPC global routing tables. Starting with Controller **10.1**, you can also configure a distinct egress mode independently for each region: FireNet, local egress with SNAT/DNAT/FQDN filtering, Single IP SNAT, or customized SNAT. See [Configuring Regional Spoke Egress Mode](#configuring-regional-spoke-egress-mode). **Regional Spoke VPC** ![Regional Spoke VPC](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/gcp-regional-vpc.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=b3f023b950766f34e08eeab835a0d11d) You can deploy a hybrid configuration where the spoke VPC has more than one pair of spoke gateways, and workloads in a region with no spoke gateways. Workloads can communicate with other workloads in the same VPC, but they cannot communicate outside the VPC using the Aviatrix transit gateway. In this example, the region's gateways need to manually advertise subnets without gateways using the [custom spoke advertise CIDR](https://legacy.docs.aviatrix.com/documentation/latest/network/spoke-gateway-route-settings.html?expand=true#cust-spoke-advertised-cidrs) feature so that if the subnets are advertised, the tagging is performed. **Hybrid Spoke VPC** ![Hybrid Spoke VPC](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/gcp-hybrid-vpc.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=e10af08a29ad041765b19bdcb00d4dc0) ## Enabling and Disabling GCP Global VPC When global VPC is enabled, the spoke only advertises its own region to the transit network. It also makes changes to the routing tables and virtual machine tags to make the RFC 1918 traffic regional to the gateway. > [!NOTE] > > You can enable and disable global VPC on spokes that are already part of a > Transit FireNet deployment. To enable Global VPC tag routing: 1. Go to **Cloud Fabric > Gateways > Spoke Gateways >** and select the gateway to be configured. 2. Select the gateway in the Edit Spoke Gateway window and click the Edit icon. 3. Use the **Global VPC** button Gateway to turn Global VPC on or off for the selected gateway region. 4. Click **Save** to save your spoke gateway configuration changes. ## Tagging GCP Global VPC Spoke Gateways > [!NOTE] > > You can add and manage tags on spokes that are already part of a Transit > FireNet deployment. To add and manage GCP Global VPC tags: 1. Go to **Cloud Fabric** > **Gateways** > **Settings** > **GCP Global VPC Configuration** and click the **Modify** button. 2. Select the method of tagging for your Aviatrix gateways. There are three methods of tagging GCP spoke gateways: * **Tag on Changes**: Aviatrix recommends this method. Any time there is a configuration change to the gateway or connections to the gateway, Aviatrix reevaluates the tags in your environment and verifies all gateways are regionally aware of the changes and that the regions can communicate with each other. * **Auto Tag**: Aviatrix Controller monitors virtual machines launched in the VPCs and automatically adds tags for newly launched virtual machines in the VPC and removes tags from virtual machines that should no longer be tagged. * **Manage Manually**: You do all the tagging through the GCP console and Aviatrix becomes regionally aware of those tags. 3. Optional. Exclude virtual machines connected to the gateway from GCP Global VPC routing by selecting them from the drop-down menu in the Modify GCP Global VPC Configuration window. Any virtual machines excluded from the Aviatrix tags are still accessible through the global routing tables. 4. Optional. Use the notification button in the Modify GCP Global VPC Configuration window to start or stop notifications when instances are discovered and tagged in your global VPC environment. 5. Click **Save** to save your Global VPC configuration changes. ## Reapplying Tags to Sync VPC Subnets Whenever new subnets are added, the Reapply Tags operation must be performed to sync VPC subnets to update the routing tables and add the routes to the newly deployed regions. This operation applies tags to new or existing virtual machines in the new region that have not been tagged. > [!NOTE] > > The Reapply Tags operation can only be performed in Tag on Changes and Auto > Tag configurations. To reapply GCP Global VPC tags: Go to **Cloud Fabric** > **Gateways** > **Settings** > **GCP Global VPC Configuration** and click the **Reapply Tags** button. ## Configuring Regional Spoke Egress Mode Controller **7.2** introduced regional spoke egress for GCP Global VPC: a region's spoke gateway could send egress traffic through Transit FireNet, but Source NAT (SNAT), Destination NAT (DNAT), and FQDN-based egress filtering were not supported on that path. Controller **10.1** removes that restriction. For each region in a multi-region, regional, or hybrid spoke VPC, you can now independently choose one of the following egress modes: * **FireNet egress**: Traffic continues to route through Transit FireNet, unchanged from earlier Controller versions. * **Local egress with filtering**: The region's spoke gateway applies SNAT, DNAT, and FQDN-based egress filtering directly, instead of routing through FireNet. * **Single IP SNAT**: The region's spoke gateway performs Source NAT using a single IP address. * **Customized SNAT**: The region's spoke gateway performs Source NAT using a customized SNAT policy. Regions that keep the FireNet egress mode and regions configured for local egress coexist in the same Global VPC deployment. You do not have to migrate every region at once, and the Controller reports each region's egress mode independently in a mixed deployment. > [!NOTE] > > Configuring a region's egress mode requires CoPilot **4.37** or later. **1. Open the Egress VPC/VNets table** Go to **Security > Egress > Egress VPC/VNets**. **2. Select the region to migrate** Locate the row for the GCP Global VPC, or expand it to select an individual region's row. **3. Enable local egress for the region** Open the **Actions** menu for the row and select **Enable Local Egress**. **4. Confirm the change** Confirm. Enabling **Local Egress** changes the region's default route to point to its spoke gateway and enables SNAT, DNAT, and FQDN-based egress filtering for that region. > [!NOTE] > > When the regions in a Global VPC use different egress modes, the Egress > VPC/VNets table reports the parent VPC row's egress status as **Mixed**. ### How the Controller Programs Egress Mode Changes When you select an egress mode other than FireNet for a region, the Controller programs GCP VPC routes the same way it does for non-global (Regional VPC) SNAT. It tags a `0.0.0.0/0` route with a region-specific network tag in the form `avx-snat-noip-` (for example, `avx-snat-noip-europe-west1`) and points that route at the region's spoke gateway. This is the same tag format documented for Single IP SNAT in Global VPC mode; see [GCP Spoke Single IP SNAT and Tagless Default Routes (Regional VPC)](/docs/enterprise/10.1/guides/connectivity/routing/gcp-spoke-single-ip-snat-regional-vpc) for the equivalent Regional VPC (Global VPC disabled) behavior. > [!NOTE] > > Because this route is network-tag-based, it applies only to tagged virtual > machine instances. Serverless compute (Cloud Run, Cloud Functions, App Engine) > does not receive GCP network tags, so serverless workloads in a region > configured for local egress are not covered by this route. Only the tagless > default route described in the linked Regional VPC document avoids this > limitation, and that route is not available in Global VPC mode. ## Limitations * **Each region needs its own spoke gateway.** You can choose a distinct egress mode only for a region that has its own spoke gateway pair. A region with workloads but no spoke gateway continues to depend on the gateways in a neighboring region, per the Hybrid Spoke VPC behavior described in the Overview. * **Serverless workloads are not covered by tag-based egress routes.** See the Note above. * **Excluding CIDRs from egress redirection.** To keep east-west traffic between spokes in the same VPC from being translated, exclude the relevant CIDR ranges from egress redirection using the same exclude-CIDR configuration that applies to existing SNAT and Single IP SNAT deployments. # Multicloud Transit Network Design Patterns Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/multi-cloud-design-patterns This document describes how to configure your VPC/VNets for a variety of single and multi-region scenarios. This document describes how to configure your VPC/VNets for a variety of single and multi-region scenarios. ## Single Region Transit VPC/VNet Design The use case for this design is if you have one connection from an on-prem network to the cloud (Direct Connect/ExpressRoute/InterConnect/FastConnect) or the Internet to a VPC/VNet. The Aviatrix Multicloud Transit Network solution provides default network segmentation. Since Spoke VPC/VNets have no connectivity to each other via a Transit Gateway, there is no need to create multiple Transit Groups for network isolation purposes. In AWS, you can set up connectivity between the Shared Service VPC/VNet and Spoke VPC/VNets, and between Spoke VPC/VNets, using [AWS peering](/docs/enterprise/10.1/guides/connectivity/peering/aws-vpc-peering). The Transit Gateway in this scenario is only used for traffic between on-prem and cloud (Spoke VPC/VNets). Cloud-to-cloud traffic, such as Shared Service VPC/VNet to Spoke VPC/VNets does not go through the Transit Gateway. Decoupling the different traffic streams reduces the performance bottleneck and removes the single failure point. > [!TIP] > > A Spoke network can be deployed in different regions and different clouds (AWS > and Azure). ![Single Region Transit](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/single-region.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=91da6d2d33747398602f13533a074368) ## Multi-Region Transit VPC Design If you have data centers in multiple regions and its corresponding cloud service provider regions, you build network redundancy to reach the cloud by leveraging VPN Gateway (VGW/VPN Connect) termination. In the diagram below, which uses AWS as an example, there are two Transit Gateways (primary and HA), one in each region. The VPN Gateway or VGW has an on-premises to the cloud (Direct Connect/ExpressRoute/InterConnect/FastConnect) or to one datacenter. The same VGW is also used for backup connectivity over the Internet from the second datacenter. If a data center loses connectivity to the VGW, the backup link can take over and use the alternate route. One Aviatrix Controller manages both Transit Gateways. If you need connectivity between any two Spoke VPC/VNets in each region, you can build an [AWS Peering](/docs/enterprise/10.1/guides/connectivity/peering/aws-vpc-peering). ![Multi Region Transit](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/multi-region.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=df4457e43a5866d909f9e9a411384f2b) ## Connected Transit Design If you want to build a Transit Network where all Spoke VPC/VNets are connected via a Transit Gateway, you can accomplish that by enabling the Connected Transit property for the Transit Gateway. When Connected Transit is enabled, you do not need to build additional tunnels between shared service VPC/VNet to other VPC/VNets. ![Connected Transit](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/connected-transit4.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=82e4075dbc3a2788f8d00b6dde48982a) ## 10Gbps Transit VPC/VNet Design If you have applications that need 10Gbps bandwidth, you can place these applications in a VPC/VNet that terminates on the VPN Gateway/VGW with the 10Gbps VIF DX. Place the Aviatrix Transit Gateway in a separate VPC/VNet and connect it to the VPN Gateway/VGW through the normal [Multi-Cloud Transit Network](/docs/enterprise/10.1/guides/connectivity/transit/transitvpc-workflow). ![10Gbps Transit Pattern](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/ten-gbps-pattern.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=dd0f4350c6024ec4cb26994219154710) Alternatively, you can place the high bandwidth application in a separate VPC/VNet that terminates directly on a VIF or network interface, as shown below. ![10Gbps Transit Pattern Alternative](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/ten-gbps-pattern-two.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=ce1a0ed87c9248b3e34686c7d456f448) ## Distributed Egress Control with Aviatrix If you are using a NAT Gateway as your egress control for Internet access, you can use Aviatrix FQDN Filtering (Legacy) to improve egress control. > [!NOTE] > > As of Controller 7.1, [Distributed Cloud Firewall](/docs/enterprise/10.1/reference/dcf/dcf-configuring) with WebGroups is the recommended method for configuring and implementing Egress Security. > > The content in this section describes Egress functionality available in the > Aviatrix Controller prior to Controller 7.1. Aviatrix provides [L7 FQDN](/docs/enterprise/10.1/guides/security/egress/egress-legacy-overview) to whitelist and blacklist public sites that applications in a Spoke VPC/VNet need to make API calls. The function is embedded in the Aviatrix Gateway. It is transparent to user instances and requires neither agents nor certs. ![Egress Control](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/egress-control-two.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=994cf4043469f6803d547c3728435f86) ## SD-WAN Integration The Aviatrix Multicloud Transit Network integrates with SD-WAN cloud instances with BGP over LAN where both BGP routes and data packets are exchanged between Aviatrix Transit Gateways and SD-WAN gateways deployed in the same Transit VPC/VNet, as shown in the diagram below. ![SD-WAN Integration](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/sd-wan-integ.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=f5d98ce5b6868e00751a9bcb25cff536) # Aviatrix Multicloud Transit Architecture for Azure Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/nextgen-transit-azure The most common design topology within Azure is the Hub and Spoke model. The hub is a virtual network (VNet) in Azure that acts as a central point of connectivity to your on-premises network. ## Azure Native Transit The most common design topology within Azure is the Hub and Spoke model. The hub is a virtual network (VNet) in Azure that acts as a central point of connectivity to your on-premises network. The spokes are VNets that peer with the HUB and can be used to isolate workloads, departments, subscriptions, etc. Traffic flows between the on-premises datacenter and the hub through an ExpressRoute or VPN gateway connection. It is common in these environments for spoke to spoke communication to be desired both within and across regions. To facilitate spoke to spoke communication, Azure natively provides three methods for performing this functionality. Each of these options has advantages and disadvantages. These options can be used simultaneously for customers to apply the right transit method for the desired outcome. ![hub-spoke](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/hub-spoke.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=424d15fe3dd60ffbae495ad9cb1cf80a) ### Intra-Region Transitive Options: 1. **Leveraging ExpressRoute** - the most common transitive method is for customers to leverage their ExpressRoute circuits to provide spoke to spoke communication. This method requires a default (0.0.0.0/0) or summary route to be advertised from on-prem to allow spoke to spoke traffic to hairpin off the Microsoft Edge Routers. The advantage to this method is that this traffic will not incur VNET peering charges and this provides any to any spoke connectivity. The disadvantage to this approach is that bandwidth is limited by the ExpressRoute gateway SKU, traffic takes a longer path from spoke to spoke, a lack of granular control as this method provides any to any communication, and the fact that this is not a recommended approach as there is no dedicated bandwidth allocation on the Microsoft Edge Routers for this configuration. ![S2SER](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/s2ser.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=dbecadf3e00a78bbb3a1c05f608a9fbd) 2. **Leveraging a HUB Network Virtual Appliance (NVA)** - in this option, an NVA is deployed in the HUB VNET and User Defined Routes (UDRs) are created in each spoke to route traffic from spoke to spoke. The advantage to this approach is that traffic takes a more ideal path, does not require any route advertisements from on-premises, and potentially provides additional security functionality depending upon the NVA being leveraged. The disadvantage to this approach comes with the management of UDRs at scale, potential bandwidth limits of the NVA itself, and the configuration of NVA high availability (HA) to ensure redundancy in case of failure. ![S2SNVA](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/s2snva.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=7df65925b0f4be396e973187a143cd32) 3. **VNET Peering** - the recommended approach for spoke to spoke communication is VNET peering as this leverages the MSFT backbone directly and always takes the preferred path. This option provides the lowest latency possible and has no bandwidth restrictions as opposed to the options previously discussed. The disadvantage of this model is this connectivity is a 1 to 1 mapping. Each spoke much be peered directly with another spoke in order to facilitate communication which at scale becomes a web of interconnected fully meshed VNETS. As such, customers often have challenges in managing the scale of VNET peers. ![S2SPeer](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/s2speer.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=49e32c629d616f0b864f646d7e45fa9c) ### Inter-Region Transitive Options: The options for spoke to spoke communication across regions follow the same patterns above with a few notable nuances. 1. **Leveraging ExpressRoute** - this method is similar to what was described in Intra-Region however, as ExpressRoute circuits are terminated across regions the routes are propagated automatically. To facilitate cross region spoke to spoke communication, no summary or default route is required. The same advantages and disadvantages apply. 2. **Leveraging a HUB Network Virtual Appliance (NVA)** - this method is also similar to what was previously described however, the number of UDRs increases as additional routes must be defined in the HUB VNETs to facilitate routing across regions to another HUB. Additionally, a VNET peer must be leveraged between the HUB to facilitate this HUB to HUB transit path. 3. **VNET Peering** - the only change in VNET peering across regions is in naming convention. Microsoft refers to this as Global VNET Peering but still has the same advantages and disadvantages previously discussed. Azure Virtual WAN is another native architectural approach which can also provide transitive functionality. Aviatrix Transit can integrate with Azure Virtual WAN and is not covered in detail here. ## Aviatrix Transit for Azure Aviatrix Transit for Azure is an architecture to interconnect multiple VNets and on-prem leveraging the hub and spoke deployment model while adding additional functionality and features. This deployment is shown in the diagram below. ![nextgentransit\_for\_azure](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/nextgentransit-for-azure.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=87ce6b10d27b8cbd0b5bcc4479a21901) In the above diagram, the Aviatrix Controller is a VM that manages all networking connections from VNETs to on-prem as well as between VNETs themselves. It deploys one Aviatrix gateway (two for redundancy) in each VNet. The Transit gateway is deployed in the transit VNet and connects to on-prem over Express Route or Internet. The Transit Gateway is then peered to each spoke VNET gateway to provide end to end communication. Communication can be granularly controlled to provide any to any communication between the spokes and to/from on-prem however, the transit gateway can also block certain traffic to keep spokes isolated. Additionally, all Spoke UDRs are orchestrated from the controller based on desired traffic flows. For cross region communication, multiple Transit Gateways can also be interconnected. Spoke VNets can communicate to remote Spoke VNets through the two connected Transit Gateways with the same granular controls mentioned previously. Additionally, route advertisements between the two transit gateways can be controlled to provide additional functionality like summarization, route exclusion, etc. This topology is depicted below. ![multiregion\_azure](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/multiregion-azure.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=b6ecae44af335a16ecf887dc85492244) Another important advantage of using Aviatrix Transit is that all communications are encrypted by default providing additional levels of security. Azure does not provide any native encryption across the Microsoft Backbone and depends upon third party NVAs to provide this functionality should customers require it. The Aviatrix Controller also has the ability to orchestrate native VNET peering for Azure VNETs should customers not wish to deploy gateways within spoke VNETs. While customers will lose the encryption and visibility benefits across these links, all appropriate UDRs will be orchestrated to facilitate transitive communication as desired. It is also important to note that certain native limitations may apply as to the number of peerings allowed as well as restrictions to overlapping IP space when native peering is leveraged. ![native\_peering](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/native-peering-updated-resized-2.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=b0965062a3556864615b4717c37e6f0e) ## Why do I need Aviatrix Transit for Azure? Transit architecture is about building connectivity between cloud and on-prem in the most agile manner possible. In the Transit architecture, there is one connection (not including the backup) between on-premises and a Transit Hub VNet. Everything else (the Spoke VNet to on-prem traffic) is routed through the Transit Hub VNet. The alternative to Transit architecture is to leverage the native options already mentioned or is to build one connection (often referred to as "flat" architecture), either IPSEC over Internet or Express Route, each time you spin up a new VNet in Azure. This requires changes at the on-premises edge, which requires a change control process that takes from days to weeks. Additionally, this method often facilitates the default any to any connectivity which may require additional configuration to prevent. ## The Benefits of Aviatrix Transit for Azure * **Simplicity** The Aviatrix Controller provides an abstraction layer and workflow to build the Transit network. You do not need to program any Azure route tables, manage the route entries or understand the significant details about Azure networking. * **Multi Subscriptions Support** The Controller provides a single pane of glass to manage the entire cloud network of multiple Azure subscriptions. * **Logging Service Integration** Out-of-the-box integration with Splunk, Sumo Logic, DataDog, ELK, remote syslog and Netflow. * **Visibility** View connectivity status, network latency and traffic statistics from a central dashboard. * **Granular Routing Control** Route redistribution can be controlled to selectively allow specific route propagation and/or summarization. * **Advanced Networking Features** Support for Network Address Translation, NGFW Insertion, Egress FQDN filtering (Legacy), etc. * **No Route Limits** The Aviatrix solution auto summarizes the on-prem and Spoke VNet routes so that Spoke VNet route entries do not exceed the route limits. * **End-to-End Encryption** All traffic in flight, between Spoke VNets and between Spoke to on-prem, is encrypted. ## How does it work? Aviatrix Transit Network is a Duo Mode architecture. While the Transit Gateway runs BGP protocol, advertising Spoke VNets CIDRs to an on-prem network and learning the on-prem network CIDRs, Spoke VNets do not run dynamic routing protocols. Learned routes by the Transit Gateway are reported to the Controller which in turn propagate to the Spoke VNets. By minimizing dynamic protocol running in the network, operations and troubleshooting become simple. CloudOps engineers without extensive networking background are able to build and manage the network. ## How do I deploy it? The Aviatrix Controller is available in the Azure Marketplace. 1. Follow the [Azure Startup Guide](/docs/enterprise/10.1/guides/controlplane/getting-started-guide-azure) to launch the Controller. 2. Follow the onboarding steps to set up Azure API credentials so that the Controller can launch gateways on behalf of the Azure account. 3. Select the use case Next-Gen Transit Network and follow the [workflow](/docs/enterprise/10.1/guides/connectivity/transit/transitvpc-workflow) to start building the transit network. # Azure Transit Network Design Patterns Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/planning-azure There are many design patterns for networking and networking security deployment in the cloud. This document summarizes these design patterns that apply to Azure networks. There are many design patterns for networking and networking security deployment in the cloud. This document summarizes these design patterns that apply to Azure networks. ## Aviatrix Encrypted Transit Network In this design, all packets in flight between Spoke VNets and Transit to on-prem are encrypted. ![aviatrix\_transit\_azure](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/aviatrix-transit-azure.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=41078e5cfa8215ab9b6a132ac422fc7d) > [!TIP] > > Aviatrix Transit supports high performance (High Performance Encryption Mode) > IPsec performance over ExpressRoute and Azure Peering. ## Aviatrix Transit with Native Spokes Aviatrix Transit also supports Azure native Spoke VNets. ![aviatrix\_transit\_native\_spoke](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/aviatrix-transit-native-spoke.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=bcd91a240a092d0b88a50bc492a7c2d8) ## Transit FireNet with Aviatrix Spokes You can apply firewall inspections for east-west, north-south and ingress/egress traffic. ![transit\_firenet\_aviatrix\_spokes](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/transit-firenet-aviatrix-spokes.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=5a9e4933880e83b6743e7a609f2a132b) ## Transit FireNet with Native Spokes Firewall inspections can be applied to native Spoke VNet, on-prem to transit and north-south traffic. ![transit\_firenet\_native\_spokes](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/transit-firenet-native-spokes.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=c816a413fb8c8ec1a50659a300f1ced2) See [Transit FireNet Workflow for Azure](/docs/enterprise/10.1/guides/security/firenet/transit-firenet-workflow-azure) for more details. ## SD-WAN Integration If you have multiple sites to connect to the cloud, you can use an Aviatrix Transit Gateway to terminate the many Site2Cloud to branch offices. Alternatively, you can use a SD-WAN termination point in the VNets to connect to the branches. Both options can be described in the diagram below. ![transit\_sdwan](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/transit-sdwan.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=bf30bdc1bd5acd6fa38944024cf038a1) ## Aviatrix Transit Gateway for Azure Spoke-to-Spoke Connectivity If you use Azure ExpressRoute gateway to connect Spoke VNets to on-prem, you can use Aviatrix Transit Gateway for Spoke-to-Spoke connectivity, as shown in the diagram below. To connect Spoke VNet, follow the instructions in [Multicloud Transit Network workflow](/docs/enterprise/10.1/guides/connectivity/transit/transitvpc-workflow). ![transit\_azure\_native\_spoke](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/transit-azure-native-spoke.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=786eb4e8f4ddf16fc26b253fae5096ae) ## Multicloud Transit with Native Spokes Use Aviatrix Transit Gateways to interconnect transit network for a multi cloud network deployment, as shown in the diagram below. ![multi\_cloud\_transit\_native](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/multi-cloud-transit-native.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=bb40d88bc3ebaf0aa6257ea09bb8c01f) # TGW Design Patterns Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/tgw-design-patterns Many design patterns exist to deploy your network with the AWS Transit Gateway Orchestrator. Here are some examples. Many design patterns exist to deploy your network with the AWS Transit Gateway Orchestrator. Here are some examples. > [!TIP] > > While the design pattern diagrams use a single symbol to represent the > Aviatrix Gateways, all designs can be implemented with multi-AZ high > availability. ## Dev & Prod Isolated Design If you want to build network segmentation between Dev/QA VPCs and Production VPCs, but require the shared service VPC and on-premise to reach each VPC, consider the diagram below. ![dev\_prod\_design](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/dev-prod-design.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=0d7ea3d637135005f5dc791eb9c8346a) In this network design, you need to create two custom Network Domains, Dev\_Domain and Prod\_Domain. At the Plan page Step 2, select *Create Custom Security Domain* and fill in the information. Make sure you multi-select Shared\_Service\_Domain and Aviatrix\_Edge\_Domain for Connect to Security Domains. Apply this step for both Dev\_Domain and Prod\_Domain. ## Dev & Prod Isolated Design with TGW Direct Connect or VPN Aviatrix integrates native TGW Direct Connect and VPN to connect to on-prem while allowing you to connect to multiple cloud as Spoke VPCs. ![tgw\_hybrid](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/tgw-hybrid.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=5dc2fd23862ea5a7049d15d880afdc6d) ## All-in-Cloud with Multi Network Domains If you are only concerned about VPC-to-VPC segmentation, you can deploy the Aviatrix Controller for an all-in-cloud segmented network, as shown below. ![all-in-cloud](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/all-in-cloud.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=3eef39f4b64e1c0e266d6a1269ed1137) ## Connecting Transit Gateways in Multi-Regions Multicloud You can use Aviatrix Transit GWs to connect AWS Transit Gateways in multi-regions and multicloud deployment, as shown below. ![multi-region](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/multi-region.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=df4457e43a5866d909f9e9a411384f2b) ## TGW Orchestrator for Cross-Region and Multicloud Spoke You can extend the TGW to a different region with transit peering and then spokes in a different cloud. ![multi\_cloud\_transit\_peering](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/multi-cloud-transit-peering.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=477598d7a812ead329ca9b50652f327e) ## Full-Mesh Network Design If you like to build a full-mesh network that allows all VPCs and on-prem to communicate with each other, you do not need to create any custom Network Domains. Simply use the built-in Default\_Domain and Aviatrix\_Edge\_Domain for the deployment, as shown below. ![default\_domain\_design](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/default-domain-design.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=32eb3b57ffe406bf6d67d3d91bb4e24e) At Plan page Step 2, select *Full mesh network*. ## Fully Isolated Network Design - 1 If you would like to build a fully isolated network where no VPCs can communicate with each other except to the shared service VPC and on-prem, you need to create a Network Domain for each VPC and connect each domain to the Shared\_Service\_Domain. ![fully\_isolated\_network\_design](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/fully-isolated-network-design.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=27d4a2b22e8f3c0b344b7e9abcb4cc6c) In this network design, you need to create a custom Network Domain for each VPC. If this design does not scale for you, consider the [Aviatrix Transit Network workflow](/docs/enterprise/10.1/guides/connectivity/transit/transitvpc-workflow) where all VPCs are by default isolated to each other. ## Fully Isolated Network Design - 2 An alternative design for a fully isolated deployment is to have a group of VPCs share one Network Domain but [disabling VPC route propagation](/docs/enterprise/10.1/reference/ui/networking/connectivity#edit-aws-tgw) when attaching a VPC, as shown in the diagram below. ![fully\_isolated\_2](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/fully-isolated-2.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=e367daee07c12efbe89cfb477c74c79e) The advantage of this design is to keep the Network Domains to a minimum. You can specify connection policies for a domain to communicate with another domain, such as Aviatrix Edge Domain or Aviatrix FireNet Domain, without the VPC in the domain being able to talk to each other. ## Fully Isolated Network with Multi-Sites VPN You can use TGW native VPN capability to connect to multi-sites VPN. Since VPN connection is in Default Security Domain, you need to build connection policy for each VPC domain. ![tgw\_multi\_sites](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/tgw-multi-sites.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=e357583c204a17c403da35eeacf548fd) ## Integrating with Distributed Egress Control Design For any of the TGW design patterns, you may deploy Aviatrix distributed Egress FQDN (Legacy) in each VPC. In this example, a full-mesh deployment is expanded to include Egress FQDN support, as shown below. ![default\_egress](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/default-egress.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=770cc41282c31b174c1654daad665dd4) You can configure [Egress Control Filter](/docs/enterprise/10.1/guides/security/egress/egress-legacy-overview) , or configure [a DCF-based egress solution (preferred)](/docs/enterprise/10.1/guides/security/egress/egress-introduction) . ## High Performance Transit - High Performance Encryption (HPE) Mode Deploy an Aviatrix hardware appliance on-prem to achieve 10Gbps Transit Network throughput. Added benefit is that traffic over Direct Connect is encrypted. ![insane-mode](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/insane-mode.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=f7d32578e8a14f7ef6197ccf4e2aae54) ## Firewall Network Simplify and scale your firewall deployment with Aviatrix Firewall Network solution. For more information, see [Firewall Network Overview](/docs/enterprise/10.1/guides/security/firenet/firewall-overview) . ![firewall\_network](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/firewall-network.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=b98437f171d58c2f8bc342d295e868f2) ## TGW Native Hybrid Network Aviatrix supports TGW VPN and TGW Direct Connect for connecting to remote site or on-prem network, as shown in the diagram below. ![firenet](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/firenet.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=a5929f0daf9cb62ae34047f258fee639) ## Connecting to China Regions If the majority of deployment is outside China regions, the best way to connect China region VPC or VNets are to use the cloud native AWS VGW or Azure VPN gateway and connect them to Aviatrix Transit Gateway by IPsec tunnels, as shown in the diagram below. This architecture applies to all other cloud providers that have presence in China regions. On the Aviatrix side, use the option [External Devices](https://legacy.docs.aviatrix.com/documentation/latest/network/transit-externaldevice-connect.html) when making the connection. ![tgw\_china](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/tgw-china.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=b4856b7389e08e778a05583a7f4f5b18) ## Connecting to Other Cloud Providers To connect any network of a cloud provider that is not AWS, Azure, GCP, and Oracle Cloud, use the native VPN gateway of these cloud providers to build VPN tunnels to the Aviatrix Transit Gateway to connect to the rest of the deployment, as shown in the diagram below. On the Aviatrix side, use the option [External Devices](https://legacy.docs.aviatrix.com/documentation/latest/network/transit-externaldevice-connect.html) when making the connection. ![tgw\_other\_cloud](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/tgw-other-cloud.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=2059d3e83527f153ea37203b0faabc1f) ## Extending Network Domains to On-Prem Sites If the Aviatrix Transit Gateway connects to multiple sites over IPsec or GRE tunnels, the Network Domains can be extended to each site as shown below, where Blue Domain in the cloud can only communicate with Site 2, Green Domain can only communicate with Site 1. Routes are only advertised within the domain and data traffic is segmented by the Network Domains. ![edge\_seg](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/edge-seg.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=b21dba09751e594764d9046bea3acbe8) # Overview of Transit Gateway to External Devices Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/transit-externaldevice-usecase This document focuses on Aviatrix Transit Gateway connection to an external device. This document focuses on Aviatrix Transit Gateway connection to an external device. ## What are the use cases for connecting to an external router? * **Overcoming the AWS VGW 100 route limit** AWS has route entry limits of 100 per each routing table. Combining the number of VPC CIDRs and the list of on-prem CIDRs, this route limit is fast approaching or already a problem. Typically, an Aviatrix Transit Gateway connects to VGW over IPsec and runs a BGP session with VGW. VGW then connects to on-prem devices. By connecting directly to an external device, the VGW is bypassed. * **Overcome AWS VGW performance reset** VGW adjusts instance size based on network bandwidth consumption, leading to unexpected outage. * **Azure Transit Network** This feature allows an Aviatrix Transit GW to connect to on-prem over Azure Express Route or Internet. * **High Performance with on-prem** By using GRE tunneling protocol, Aviatrix Multicloud Transit Gateway [builds multiple GRE tunnels to on-prem routers](/docs/enterprise/10.1/guides/connectivity/vpn/transit-externaldevice-aws-gre-tunneling) to achieve 10Gbps throughput. * **Integrate with SD-WAN gateways deployed in the cloud** BGP over LAN as part of the External Device option provides an efficient mechanism to connect to SD-WAN cloud gateways by interoperating with them over LAN in the same VPC/VNet while exchanging routes dynamically via BGP. * **All Other Cloud Providers** Use this feature to connect to network of cloud providers such as Alibaba Cloud, Tencent Cloud, VMware Cloud, IBM Cloud and others. Aviatrix Transit Network solution solves these problems when connecting datacenters to a growing number of VPC/VNets. * **AWS Transit Gateway** AWS released Transit Gateway (TGW), I need to migrate my current CSR-based Transit VPC solution. * **No Route Propagation** AWS Transit Gateway (TGW) does not propagate on-prem learned routes to Spoke VPC route table, it requires manual programming. * **Transit Solution for Azure** We have multiple Azure VNETs now, we need to form a transit network and connect them to on-prem and to AWS Transit network. * **Change Control** Each time a new VPC is stood up, a change control process has to take place to modify the edge router for Direct Connect or IPsec over Internet. This is not agile and the risk of errors in configuration is not acceptable. * **BGP** The CSR-based Global Transit solution runs VGW in each spoke VPC/VNet which runs a BGP session to Transit hub. This is operationally challenging to manage and troubleshoot. The BGP in VGW is a black box and invisible to the outside. * **Not Secure** All spoke VPC/VNets in the CSR-based Global Transit solution have connectivity to each other through BGP route propagation. There is no network segmentation. The blast radius is my entire cloud network and datacenters. This is not acceptable by the security team. * **Extra Charge** In the CSR-based solution, traffic from one spoke VPC/VNet to another spoke VPC/VNet traverses through one transit and sometimes two transit hubs, resulting in 2x or 3x egress charge. * **Too Complex** The CloudOps is a team of 6 engineers managing 34 AWS services, the skill set and resources it takes to manage the CSR-based Transit network is beyond what we want to handle. * **10Gbps Transit** My current Transit network performance is capped at 1.25Gbps, our network requires much higher bandwidth. ## How does it work? The Aviatrix Transit Gateway runs a BGP session to an external router to dynamically exchange routes. It also establishes an IPsec tunnel, GRE tunnel, or direct Ethernet to the router for packet forwarding. For IPsec tunneling, static routing option is also supported. The mechanism works for AWS Direct Connect, Azure ExpressRoute, or the Internet. ### Over Private Network in AWS When the underlying infrastructure is AWS Direct Connect, the diagram is shown as below. ![transitgw\_private\_aws](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/transitgw-private-aws.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=b95a1f797b51f31639ed8c9ae752dbff) Make sure: * The VGW is attached to the Transit VPC for IPsec over Direct Connect and GRE over Direct Connect. * The external device advertises its IP address to VGW. * The external device advertises the on-prem network CIDR list to Aviatrix Transit GW. ### Over a Private Network in Azure When the underlying infrastructure is Azure Express Route, the external Device options are IPsec or LAN, as shown below. ![transitgw\_private\_azure](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/transitgw-private-azure.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=5d985fc17a78514c5856817ca9c53ec3) > [!NOTE] > > GRE is not supported on Azure. ### Over the Internet When connecting over the Internet, the diagram is shown as below. ![transitgw\_internet](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/transitgw-internet.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=0f31bcc64ca51f2e122fb545015b55d7) ## How do I connect to an external device? There are three options to connect a Transit Gateway to an on-prem network. Choose one option that meets your network requirements. 1. AWS VGW (This is the default setting) 2. External Device (over Direct Connect or over Internet) 3. Azure VNG > [!NOTE] > > Currently, only one connection is supported on a specific Transit Gateway/VPC, > regardless of which of the three options above is chosen. ![Transit Gateway connection options to on-premises](https://mintcdn.com/aviatrix-14b37c43/Q1J71JuX1AhUNBwG/images/concepts-architectures/transit-onprem-2.png?fit=max\&auto=format\&n=Q1J71JuX1AhUNBwG\&q=85\&s=abb6e83b25c06544ca3c9d6333a086f1) | Transit Gateway | Connect Type | Performance | HA | Route Limit | Deployment notes | | - | - | - | - | - | - | | AWS VGW | | 1.25Gbps | Active/Active | 100 | VGW should be detached. Use the [instruction here](https://aws.amazon.com/premiumsupport/knowledge-center/create-vpn-direct-connect/) to build encryption between VGW and on-prem router. | | External Device | | Up to 10Gbps | Active/Standby | Unlimited | VGW should be attached. Aviatrix Transit Gateway establishes BGP + IPSEC with on-prem router. | | Azure VNG | | 10Gbps | Active/Active | Unlimited | VNG should be attached. | To connect AWS VGW, see [Transit Gateway Integration with AWS VGW Workflow](/docs/enterprise/10.1/guides/connectivity/vpn/transit-externaldevice-aws-vgw) . To connect to Azure VNG, see [Transit Gateway Integration with Azure VNG Workflow](/docs/enterprise/10.1/guides/connectivity/vpn/transit-externaldevice-azure-vng) . To connect to an external device, see [Transit Gateway to External Devices Workflow](/docs/enterprise/10.1/guides/connectivity/vpn/transit-externaldevice-workflow) . Connect transit gateways to external devices. # Aviatrix Multicloud Transit Network FAQ Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/transit-network-faq Aviatrix Multicloud Transit Network architecture is about building connectivity between the cloud and on-prem in the most agile manner possible. In the Aviatrix Multicloud Transit Network architecture, there is one connection (not including the backup) between on-prem and a Transit VPC or... ## Why should I choose Aviatrix Multicloud Transit Network Architecture? Aviatrix Multicloud Transit Network architecture is about building connectivity between the cloud and on-prem in the most agile manner possible. In the Aviatrix Multicloud Transit Network architecture, there is one connection (not including the backup) between on-prem and a Transit VPC or VNet. Everything else (the Spoke VPC and VNets to on-prem traffic) is routed through the Transit VPC or VNet. The alternative to Transit architecture (often referred to as "flat" architecture) is to build one connection, either IPsec over the Internet or Direct Connect, each time you spin up a new VPC or VNet in the cloud. This requires changes at the on-prem edge, which requires a change control process that may take days to weeks. ## How does the Aviatrix Transit Network solution differ from Cisco's CSR-Based solution? They differ in the following areas: * **Central Control** - With the Aviatrix solution, the Aviatrix Controller is the single pane of glass for all networking in the cloud. * **AWS Transit Gateway Integration** If you have AWS deployment, Aviatrix Transit integrates with an AWS TGW seamlessly for high bandwidth Spoke VPC connection. Customers who do not require end to end encryption can now use the TGW native service to connect the Spoke VPCs. * **Network Segmentation** In the CSR-based solution, all Spoke VPCs have connectivity to each other through the Transit GW, even though these Spoke VPCs belong to different AWS accounts or business teams. In contrast, in the Aviatrix solution the Spoke VPC/VNets have no connectivity to each other, by default. Connectivity is built by design. With the TGW integration, you can customize the [Network Domains](/docs/enterprise/10.1/concepts-architectures/components/tgw/tgw-orchestrator-faq) to meet your segmentation requirements. * **Connectivity Efficiency** In the Aviatrix solution, traffic between any two Spoke VPC/VNets can be routed via TGW or directly, as opposed to going through the instance based Transit GW as required by the CSR-based solution. Decoupling the different traffic streams reduces performance bottlenecks and removes single failure points. * **No unwanted route propagation** Since Spoke VPC/VNets run BGP in CSR solution, if a Spoke VPC/VNet also connects to a partner network via VGW, the partner network routes could be propagated to your own on-prem network. * **Simplicity** In Aviatrix's solution, BGP is only deployed between Transit GW and VGW. No Spoke VPCs run BGP. Simplicity leads to stability. Workflow-based, step-by-step instructions help you build out a Transit VPC/VNet solution in minutes. * **Monitoring** The Aviatrix solution integrates with Splunk, Sumo, remote syslog, ELK and DataDog to forward events from gateways to your favorite central logging service. * **Scalable** AWS has various limits in its infrastructure, such as a route entry limit of 100. This limits how many on-prem CIDRs and VPC CIDRs can be carried on a Transit GW. The Aviatrix solution overcomes that limitation. ## Does the Aviatrix Transit Network support high availability? Yes. Aviatrix Transit Gateways operates in ActiveMesh Mode. ActiveMesh mode is enabled by default. To learn more, see [About ActiveMesh](/docs/enterprise/10.1/concepts-architectures/components/connectivity/activemesh-about). ## How do I deploy high bandwidth applications in the Aviatrix Transit solution? Aviatrix's High Performance Encryption solution provides 10Gbps Transit network throughput. To learn more, see [High Performance Encryption Mode solution](/docs/enterprise/10.1/concepts-architectures/components/security/hpe-about). ## How do I configure a global Transit Network with the Aviatrix solution? To configure Aviatrix Multicloud Transit Network, follow the instructions in [Single-Region Multicloud Transit Network Workflow](/docs/enterprise/10.1/guides/connectivity/transit/transitvpc-workflow). If you plan to deploy an AWS Transit Gateway (TGW) based transit network, follow the instructions in [Aviatrix Transit Gateway Orchestrator Workflow](https://legacy.docs.aviatrix.com/documentation/latest/network/remote-user-to-cloud.html/tgw/tgw-overview). If you plan to deploy an Azure transit solution that does not require to encrypt traffic between the Transit VNet and Spoke VNet, follow the instructions in [Single-Region Multicloud Transit Network Workflow](/docs/enterprise/10.1/guides/connectivity/transit/transitvpc-workflow). ## How do I build a Multi-Region Transit solution? Inter-region transit network (see [TGW Design Patterns](/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/tgw-design-patterns) ) can be connected directly. Follow the instructions in [Multi-Region Multicloud Transit Gateway Peering Workflow](/docs/enterprise/10.1/guides/connectivity/peering/transit-gateway-peering-workflow) . ## How can I fit an Egress Firewall into the Aviatrix Transit Network solution? There are two types of requirements. ### Egress Control Policies If your compliance requires egress policies and you have currently implemented AWS NAT gateways, consider using [Aviatrix FQDN Filtering (Legacy)](/docs/enterprise/10.1/guides/security/egress/egress-legacy-overview) . This is the most efficient way to provide a FQDN filter for all TCP/UDP protocols. ## How do I deploy a UserVPN on Aviatrix Multicloud Transit Network solution? We recommend that you deploy UserVPN in a shared service VPC/VNet. If this shared service VPC/VNet has connectivity to all other VPC/VNets, a user can reach any instances in these VPC/VNets as long as his/her profile policy allows. To learn more, see [UserVPN](/docs/enterprise/10.1/guides/connectivity) . ## Does Aviatrix Multicloud Transit Network support Azure VNet? You can launch a Spoke Gateway in Azure VNet. A best practice is to set up the Azure VNet the same way you usually do with AWS VPC: two types of subnets, public subnets and private subnets with respective routing tables, where the Spoke Gateway is launched in public subnet. > [!TIP] > > In Azure there is no explicit concept of public subnet. The idea here is to > set up separate subnets and respective routing tables for the Aviatrix Gateway > and user VMs. For convenience, we use the term "public subnet" to describe the > subnet where Aviatrix Spoke gateway is launched. Such separation of subnets and routing tables provides you with the flexibility if you plan to use Spoke gateway also for FQDN functions. ## Can I automate the Aviatrix Multicloud Transit Network setup? There are multiple resources to help you automate Aviatrix Multicloud Transit Network setup. Note that if you are building a Transit Network following the workflow, you should [use Terraform](https://registry.terraform.io/providers/AviatrixSystems/aviatrix/latest). ## How do I build encryption over Direct Connect? AWS provides native solutions to add VPN capability between VGW and on-prem over Direct Connect. This improves security as data in motion is encrypted. Follow [the instructions here](https://aws.amazon.com/premiumsupport/knowledge-center/create-vpn-direct-connect/) for this capability. We build an encryption between Aviatrix Transit GW and a VGW and between a Transit GW and a Spoke GW to provide an end-to-end encryption protection. ## How do I build redundancy between VGW and on-prem? AWS provides a few native options for redundancy between VGW and on-prem. You can build redundant active/active VPN connections, redundant active/active DX connections and DX with backup VPN connections. [Read this doc](https://aws.amazon.com/answers/networking/aws-multiple-data-center-ha-network-connectivity/) for implementation details. ## Should I deploy one Transit Group for Dev and one for Prod? If your reason for two Transit hubs is security and a smaller blast radius, you need not worry about these when using the Aviatrix solution. Just create two Network Domains in your deployment. # Aviatrix UserVPN Design for Multi-Accounts and Multi-VPC/VNets Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/user-vpn-designs This reference design helps you build an end-to-end secure cloud network, from users accessing the network (AWS VPC, Azure VNet, or GCP VPC) to routing packets among the VPC/VNets, such that once a user is connected via VPN, they can access any private resources in the cloud no matter where... This reference design helps you build an end-to-end secure cloud network, from users accessing the network (AWS VPC, Azure VNet, or GCP VPC) to routing packets among the VPC/VNets, such that once a user is connected via VPN, they can access any private resources in the cloud no matter where that resource is. There are three use cases covered, from simple to more complex. You can decide which one suits you or combine parts from different ones to create a network that meet your requirements. You can easily build a full-mesh network. ## Multiple VPC/VNets in One Region The network shown below has all VPC/VNets in the same region. The Aviatrix Platform instance can be in the same VPC/VNet or a different VPC/VNet. ![One Region VPC Reference](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/oneregionvpc-reference.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=3b818fe2e4ef22028c1fec15cce01d84) This example assumes that you have created four VPC/VNets in the same region (us-west-2 in this case). You would like to use the VPC/VNet with CIDR 172.31.0.0/16 to host gateways where users connect to. After a user connects to this VPC/VNet via SSL VPN, they should be able to access any instances in the other VPC/VNets as long as their profile allows, without having to connect to each VPC/VNet with SSL VPN. Another requirement is split tunnel mode. That is, only traffic destined to the cloud goes through the SSL tunnel. If a user does general browsing on the Internet or watches movies, traffic should be routed via Wi-Fi > ISP > Internet so that you do not have to pay the additional cost. ### Configuration Workflow > [!NOTE] > > Hover your cursor over the fields to see their definitions. Do a software > upgrade if an upgrade alert message appears on your dashboard page. The description in the steps below provides critical fields you need to select; it may not include all fields. Make sure you have the correct VPC/VNet ID and its region for the VPC/VNet ID field and region in each step. 1. [Create a VPN Gateway](/docs/enterprise/10.1/guides/uservpn/user-vpn-gateway-guide) in VPC 172.31.0.0/16. 2. Repeat Step 1 to create more gateways with VPN enabled. Note that each gateway must have a different VPN CIDR Block and name. You may select different AZs for the Public Subnet field. 3. In CoPilot go to **Networking > Connectivity > Native Peering**. 4. Click **+ Native Peering**. * For VPC/VNet 1, select the Account Name, Region, and VPC ID of one of the gateways created. * Do the same for VPC/VNet 2. * Click **Save**. * Repeat the above steps as many times as necessary for the gateways you created. 5. Add Users and Profiles. * Go to **Cloud Fabric > UserVPN > select the Profiles tab** to create as many profiles as needed. The target field can be FQDN (DNS names or fully qualified domain name). * Go to **Cloud Fabric > UserVPN > select the Users tab** to add more users. Associate each user with a profile. Note that if no profile is associated, the user has full access to all resources. When a user is added to the database, an email with a .ovpn file will be sent to the user with detailed instructions. 6. Launch VPN connections from remote users to VPC/VNet1 (172.31.0.0/16). Once the SSL VPN connection is established, this VPN user should be able to reach all instances (in all VPC/VNets) to which they have access permission. ## Multiple VPC/VNets in Multi-Regions, Split Tunnel The network shown below has VPC/VNets in different regions. The Aviatrix Platform instance can be in the same or a different VPC/VNet. ![Multi Region VPC Reference](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/multiregionvpc-reference.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=63598235bbd5719ffbc9cce85cdc83a9) In this example, Aviatrix encrypted peering is used for connecting to remote VPCs. You can also use AWS peering/Azure peering to accomplish the task. Assume you have created four VPC/VNets. You would like to use the VPC/VNet with CIDR 172.31.0.0/16 in us-west-2 to host gateways where users connect to. After a user connects to this VPC/VNet via SSL VPN, they should be able to access any instances in the other VPC/VNets as long as their profile allows, without having to connect to each VPC/VNet with SSL VPN. Another requirement is split tunnel mode. That is, only traffic originating from the user and destined to resources in VPC/VNets is routed through the SSL VPN tunnel. The traffic to the Internet will be routed through ISP instead of SSL VPN tunnel. ### Configuration Workflow > [!NOTE] > > Hover your cursor over the fields to see their definitions. The description in > each step does not include all fields. Make sure you have the correct VPC/VNet > ID and its region for the VPC ID field and region in each step. 1. [Create a VPN Gateway](/docs/enterprise/10.1/guides/uservpn/user-vpn-gateway-guide) in VPC 172.31.0.0/16, keeping in mind the following: \* Select Split Tunnel mode. \* For the Additional CIDRs field under Split Tunnel, enter any other VPC/VNet or any network CIDRs you want to reach beyond the VPC/VNet you are connecting to. \* In the example shown, you should enter 10.10.0.0/16,10.5.0.0/16, 10.80.0.0/16. It is a good idea to do some planning to include future VPC/VNets or network address ranges. > [!NOTE] > > In a case where you never have to worry about connecting to your corporate > VPN, you may consider entering the entire private network address range in the > Additional CIDRs range field, separated by commas: > 172.16.0.0/12,10.0.0.0/8,192.168.0.0/16. Doing so means you do not have to > reconfigure the gateway if you need to add more VPC/VNets for networking with > different CIDR ranges in the future. * (Optional) If you want to use a private DNS name to access the instance, you can use the Nameservers and Search Domain fields under Split Tunnel. Enter your private DNS name and search domain. * If you use AWS Route 53 private hosted zone and records for your host names, make sure the Nameserver is the DNS server of the VPC/VNet. In this case, you should enter "172.31.0.2". * Select **ELB** in the Load Balancer field. 2. Repeat Step 1 to create more VPN gateways. You may select different AZs for the Attach to Subnet field. 3. Build encrypted routing networks to reach other VPC/VNets. 4. Launch a gateway without VPN capability in VPC/VNet 172.31.0.0/16. This is the routing gateway. Make sure NAT is NOT enabled (since step 1 has enabled NAT function for this VPC/VNet). 5. Repeat step 3 for VPC/VNet 10.10.0.0/16, 10.5.0.0/16 and 10.80.0.0/16. Enable NAT for these gateways if you want instances in these 3 VPC/VNets to be able to reach the Internet directly. 6. Connect the VPC/VNets using [Spoke-Spoke peering](https://legacy.docs.aviatrix.com/documentation/latest/network/spoke-attach-spoke.html). Note that each VPC/VNet is represented by one or more gateways. Make sure you peer between gateways that do not have VPN capability. 7. (Optional) Set up Stateful Firewall Rules at the VPC/VNet level. Select the gateway you just created to edit Security Policies to add any policies for each VPC/VNet. These steps complete the network infrastructure setup. 8. Add Users and Profiles. * Go to Cloud Fabric > UserVPN > select the Profiles tab to create as many profiles as needed. The target field can be FQDN (DNS names or fully qualified domain name). * Go to Cloud Fabric > UserVPN > select the Users tab to add as many users as needed. * Associate each user with a profile. Note that if no profile is associated, the user has full access to all resources. When a user is added to the database, an email with an .ovpn file will be sent to the user with detailed instructions. ## Multiple VPC/VNets in Multi Regions / Full Tunnel / Your Own Firewall The network shown below has VPC/VNets are in different regions. The Aviatrix Platform instance can be in the same or a different VPC/VNet. ![Full Tunnel VPC Reference](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/fulltunnelvpc-reference.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=15322a9dcbed02ab382bc30412698754) Assume the following: * Four VPC/VNets have been created. * You would like to use the VPC/VNet with CIDR 172.31.0.0/16 in us-west-2 to host gateways where users connect to. * Full tunnel mode: that is, all traffic originated from the user is routed through SSL VPN. Your organization is required to run its own firewall function for any Internet-bound traffic. * After a user connects to this VPC/VNet via SSL VPN, they should be able to access any instances in the other VPC/VNets as long as their profile allows, without having to connect to each VPC/VNet with SSL VPN. ### Configuration Workflow > [!NOTE] > > Hover your cursor over the fields to see their definitions. The description in > each step does not include all fields. Make sure you have the correct VPC/VNet > ID and its region for the VPC/VNet ID field and region in each step. 1. [Create a VPN Gateway](/docs/enterprise/10.1/guides/uservpn/user-vpn-gateway-guide) in VPC/VNet 172.31.0.0/16. \* The VPN CIDR must be a subnet that is outside your current VPC/VNet CIDR range and your laptop or device subnet range. In the example above, you may enter 192.168.2.0/24. \* Turn **Off** Split Tunnel. * Select **ELB** from the Load Balancer field. \* Enable Policy-Based Routing (PBR). \* The PBR Subnet must be a subnet that is in the same AZ as the primary subnet (Public Subnet where the gateway is launched). \* Enter the AWS subnet default gateway for PBR Default Gateway field. For example, if PBR Subnet is 172.31.48.0/20, the default Gateway field is 172.31.48.1. \* (optionally) you can enable NAT Translation Logging to log every user's activity to every server and site. This is useful for auditing and compliance. 2. Repeat Step 1 to create more gateways with VPN enabled. You may select different AZs for the Public Subnet field. 3. (Optional) If you have your own routing network to route between the VPCs and one of your own backbone routers can route traffic to your own firewall for Internet-bound traffic, you can skip this step and the next two steps (step 4 and 5). Launch a gateway (not a VPN gateway) in VPC 172.31.0.0/16. Since this is the routing gateway, make sure that NAT is not enabled for this gateway. 4. (Optional) Repeat step 3 for VPC 10.10.0.0/16, 10.5.0.0/16 and 10.80.0.0/16. Select Enable NAT if you wish the instances in these VPCs to be able to reach Internet directly. 5. (Optional) Connect the VPC/VNets using Spoke-Spoke peering. Note that each VPC/VNet is represented by one or more gateways. Make sure you peer between gateways that do not have VPN capability. The steps above complete the network infrastructure setup. 6. Add Users and Profiles. * Go to Cloud Fabric > UserVPN > select the Profiles tab to create as many profiles as needed. The target field can be FQDN (DNS names or fully qualified domain name). * Go to Cloud Fabric > UserVPN > select the Users tab to add as many users as needed. * Associate each user with a profile. If no profile is associated, the user has full access to all resources. When a user is added to the database, an email with a .ovpn file will be sent to the user with detailed instructions. * Alternatively, you can [download](/docs/enterprise/10.1/guides/uservpn/user-vpn-client-download) the .ovpn file directly. ## Use an AWS Transit Gateway to Access Multiple VPCs in One Region You can use an AWS Transit Gateway (TGW) allow remote users to connect to multiple VPCs in the same region, as shown below. ![VPN with TGW One Region](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/vpn-with-tgw-one-region.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=4291eabfaccdb2a499100a2b0c9b012e) ## UserVPN Solution for Multi Cloud With Aviatrix multicloud support, you can build a global VPN solution that spans to multi cloud. ![VPN TGW Multi Cloud](https://mintcdn.com/aviatrix-14b37c43/8iF92JcVck-Y3RaJ/images/concepts-architectures/architecture/connectivity/vpn-tgw-multi-cloud.png?fit=max\&auto=format\&n=8iF92JcVck-Y3RaJ\&q=85\&s=51d47a132b13741c5b8aee092a905074) # VRF Design Patterns Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/vrf-design-patterns Reference topologies for VRF multi-tenancy on Aviatrix fabrics: single-transit and multi-transit VRF stitching designs. Virtual Routing and Forwarding (VRF) lets you run multiple isolated routing tables on the same Aviatrix fabric, so that tenants or business units with overlapping IP address ranges can share the same Transit and Spoke Gateways without their routes or traffic mixing. ## When to Use This Use these patterns when you must onboard multiple tenants, business units, or customers with overlapping IP address space onto a shared Aviatrix fabric, and deploying a dedicated fabric per tenant is not practical at your scale. ## Architecture Diagram Every pattern in this document is fundamentally a topology decision, and each is best understood as a diagram of Transit Gateways, Spoke Gateways, VRF segments, and (where applicable) Network Domains. * **Single-Transit Multi-VRF**: a single Transit Gateway serving multiple VRF segments ![Single-Transit Multi-VRF](https://mintcdn.com/aviatrix-14b37c43/0cJ6mdcM_TcZxo37/images/concepts-architectures/architecture/connectivity/single-transit-multi-vrf.png?fit=max\&auto=format\&n=0cJ6mdcM_TcZxo37\&q=85\&s=91d1db7729a80239cf2ecb4754aa2f82) * **Multi-Transit VRF Stitching**: VRF segments stitched across multiple Transit Gateways ![Multi-Transit VRF Stitching](https://mintcdn.com/aviatrix-14b37c43/0cJ6mdcM_TcZxo37/images/concepts-architectures/architecture/connectivity/multi-transit-vrf-stitching.png?fit=max\&auto=format\&n=0cJ6mdcM_TcZxo37\&q=85\&s=f2a7275e08c3f9b4d9fac2ee6109cf80) ## Design Principles **Isolation at the routing layer.** VRF enforces tenant isolation in the routing table itself, rather than through IP address planning or NAT. This is what allows tenants with overlapping CIDRs to coexist on the same fabric without one tenant's routes leaking into another's routing table. **Overlap-tolerant by design.** These patterns assume tenants, business units, or acquired companies frequently have colliding IP address ranges that cannot be renumbered. The design goal is to accommodate that reality directly instead of requiring NAT remediation before onboarding a tenant. **Built on the existing Network Domain construct, not a second layer beside it.** VRF does not introduce a separate segmentation model that you combine with Network Domains. In CoPilot, VRF is enabled with the **Support for VRF** toggle under **Networking > Network Segmentation > Settings**, and turning it on automatically assigns each existing Network Domain its own VRF segment. Environments that already segment Spokes with Network Domains keep that design unchanged. See the Network Domain entry under Components. **Minimize fabric sprawl.** A primary motivation for these patterns is avoiding the operational cost of deploying a dedicated Transit and Spoke fabric per tenant. A single fabric serving many isolated VRF segments reduces the number of gateways, peering relationships, and control-plane objects you must manage. **Deliberate scope boundaries for GA.** Several adjacent capabilities (FireNet inter-VRF inspection and route leaking between VRFs) are explicitly deferred past 10.1. These patterns are scoped to what is supported at GA rather than describing a larger, not-yet-delivered capability set. Terraform support for VRF is available at GA; see the sibling [VRF Configuration](/docs/enterprise/10.1/guides/connectivity/routing/vrf-configuration) guide for Terraform usage. **Disabled by default.** VRF end-to-end propagation does not activate merely by upgrading to Controller 10.1. It ships off by default and must be explicitly enabled (see the sibling [VRF Configuration](/docs/enterprise/10.1/guides/connectivity/routing/vrf-configuration) guide for the enablement procedure). ## Components * **VRF (Virtual Routing and Forwarding) segment**: The isolated routing table that keeps a tenant's or business unit's routes and traffic separate from other segments on the same fabric. In CoPilot, VRF is enabled with the **Support for VRF** toggle under **Networking > Network Segmentation > Settings**. For a conceptual introduction to VRF and the multi-tenancy problem it solves, see [VRF Multi-Tenancy Overview](/docs/enterprise/10.1/concepts-architectures/components/connectivity/vrf-overview) . * **Transit Gateway**: Hosts one or more VRF-scoped routing instances and is the point where VRF segments from different Spokes (and, in the multi-transit pattern, from peer Transit Gateways) converge on the shared fabric. * **Spoke Gateway**: Attaches a tenant's VPC/VNet to a specific VRF segment on its Transit Gateway, keeping the tenant's workload traffic scoped to that segment. * **Network Domain**: The existing Aviatrix segmentation construct that groups Spoke VPC/VNets so members can communicate with each other but not with VPC/VNets outside the domain. VRF assignment to a Network Domain is automatic: the **Create Network Domain** dialog exposes only **Name**, **Associations**, and **Connect to Network Domain**, with no manual VRF ID field, and each domain receives its own VRF automatically. A Network Domain and a VRF segment are the same boundary at GA, not two independently combinable layers of segmentation. See [Transit Network Segmentation](/docs/enterprise/10.1/concepts-architectures/components/security/transit-segmentation-faq) . * **Site2Cloud (S2C)**: The overlapping-CIDR use case that motivated first-hop VRF support: multiple S2C connections in different segments can present overlapping remote CIDRs at the Transit Gateway. See [Site2Cloud Overview](/docs/enterprise/10.1/concepts-architectures/components/connectivity/s2c-overview) . ## Traffic Flows These patterns handle three distinct east-west traffic patterns: * **Intra-tenant, spoke-to-spoke.** Traffic between two Spokes that belong to the same tenant stays within that tenant's VRF segment as it crosses the Transit Gateway. This is the baseline flow for all three patterns. * **Tenant-to-shared-services.** Traffic from a tenant VRF to a shared services VPC/VNet (for example, DNS, logging, or a shared security appliance) crosses from the tenant's VRF into a shared VRF segment or Network Domain that is explicitly permitted to reach it. * **Cross-fabric, stitched.** In the Multi-Transit VRF Stitching pattern, traffic for a given tenant must cross from that tenant's VRF segment on one Transit Gateway to the matching VRF segment on a peer Transit Gateway, preserving tenant isolation across the stitched hop. The following steps illustrate the cross-fabric flow: **1. Spoke originates traffic in its tenant VRF** A Spoke Gateway in Tenant A's VRF segment on Transit Gateway 1 sends traffic destined for a resource reachable only through Transit Gateway 2 (for example, a different region or cloud). **2. Traffic crosses the stitched VRF segment** Transit Gateway 1 forwards the traffic to Transit Gateway 2 over the VRF segment stitched between the two gateways for Tenant A, keeping the traffic scoped to Tenant A's routing table on both sides of the hop. **3. Traffic reaches the destination Spoke** Transit Gateway 2 delivers the traffic to the destination Spoke in Tenant A's VRF segment, without exposing Tenant A's routes to any other tenant's VRF segment on either Transit Gateway. ## Variants and Trade-offs > [!WARNING] > > The pattern categories below are described at a conceptual level only. The > definitive list of which specific pattern combinations are supported at 10.1 > GA has not been finalized. Confirm current support status with your Aviatrix > account team or the release notes for your Controller version before > committing to a design. Every pattern in this document shares the same underlying trade-off between VRF and Network Domain alone: enabling VRF adds datapath and Controller performance overhead, because the Controller must compute and maintain a separate routing table per VRF segment, in exchange for letting tenants with overlapping IP address ranges share the same fabric. If you leave VRF disabled and rely on Network Domains alone for segmentation, you avoid that overhead, but no two tenants anywhere in the topology can use overlapping CIDRs. The trade-offs below are specific to each pattern, on top of that baseline choice. | Pattern | Description | Best suited for | Trade-off | | - | - | - | - | | **Single-Transit Multi-VRF** | A single Transit Gateway hosts multiple isolated VRF segments, each serving a different tenant's Spokes. | Environments consolidating several tenants onto one fabric within a single region or cloud. | Simplest to reason about, but all tenants share the fault domain and capacity of one Transit Gateway. | | **Multi-Transit VRF Stitching** | A tenant's VRF segment is extended across two or more Transit Gateways, so the same tenant isolation is preserved as traffic crosses fabrics (for example, across regions or clouds). | Tenants that need consistent isolation across a multi-region or multicloud footprint. | Adds routing complexity and an extra hop across Transit Gateways. See the Sizing and Scale Considerations section for scale guidance. | ## Sizing and Scale Considerations As the number of tenants grows, the primary scaling questions are how many VRF segments a single Transit Gateway can host, how many routes each VRF segment can carry, and at what point a Single-Transit Multi-VRF design should move to Multi-Transit VRF Stitching to spread tenants across more than one Transit Gateway. Treat designs in the range of roughly 10 to 20 VRF segments per Transit Gateway as representative of what this feature is being scoped and validated against. ## Security Considerations VRF provides isolation at the routing layer: it prevents one tenant's routes and traffic from being visible to or reachable by another tenant's VRF segment on the same fabric. VRF-based isolation is not equivalent to firewall inspection between tenants. It controls reachability, not payload inspection. ## Related Resources * [VRF Multi-Tenancy Overview](/docs/enterprise/10.1/concepts-architectures/components/connectivity/vrf-overview) : the conceptual introduction to VRF and the multi-tenancy problem it solves. * [VRF Configuration](/docs/enterprise/10.1/guides/connectivity/routing/vrf-configuration) : step-by-step configuration for the patterns described in this document. * [Transit Network Segmentation](/docs/enterprise/10.1/concepts-architectures/components/security/transit-segmentation-faq) : background on the Network Domain construct and how it maps to VRF segments. # FireNet Design Patterns Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/architecture/security/firenet-design-patterns FireNet supports High Performance Encryption Mode. ## Hybrid to On-prem ![500](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/architecture/security/hybrid.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=0387c744fd4f72029c17052c4901eea6) ## Hybrid with High Performance Encryption Mode FireNet supports High Performance Encryption Mode. ![500](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/architecture/security/insane.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=eadce005074917664ac2452dc94e5e1d) ## FireNet in Multi-Regions ![500](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/architecture/security/multi-regions.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=2b613f2ab704b92ce60b98a019899290) ## Dual Transit FireNet Network You can deploy two Firewall Networks, one dedicated for East-West traffic inspection and another for egress inspection. You must follow this configuration sequence: 1. Disable the Traffic Inspection of the FireNet Gateway intended for egress control. 2. Enable Egress Control for FireNet Gateway intended for egress control. 3. Build connection policies. ![500](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/architecture/security/dual-firenet.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=6f87c2903d5eb02a7a24f8c39fb6c347) ## Distributed Egress in a Multi-Region Deployment If you need to have a distributed egress for each region, make sure you filter out the default route 0.0.0.0/0 when you build the Aviatrix Transit Gateway peering, as shown in the diagram below. ![500](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/architecture/security/multi-egress.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=b9c5b30222708bb5b770bdd6ca191a70) ## Ingress Protection via Aviatrix Transit FireNet This Ingress Protection design pattern is to have the traffic forward to firewall instances directly in Aviatrix Transit FireNet VPC/VNet as shown in the diagram below. In this design pattern, each firewall instance must configure (1) SNAT on its LAN interface that connects to the Aviatrix FireNet Gateway and (2) DNAT to the IP of application server/load balancer. The drawback of this design is that the source IP address is not preserved when traffic reaches the application. ![500](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/architecture/security/ingress-transit-firenet-topology.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=d968c29a84716cbe1e3a8b96484feb0a) For an example configuration workflow, see [Ingress Protection via Aviatrix Transit FireNet with FortiGate](https://legacy.docs.aviatrix.com/documentation/latest/security/fortigate-ingress-protection-firenet.html). # Hybrid to On-prem ![](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/architecture/security/hybrid.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=0387c744fd4f72029c17052c4901eea6) # Hybrid with High Performance Encryption Mode FireNet supports High Performance Encryption Mode. ![](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/architecture/security/insane.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=eadce005074917664ac2452dc94e5e1d) # FireNet in Multi-Regions ![multi-regions](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/architecture/security/multi-regions.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=2b613f2ab704b92ce60b98a019899290) ## Dual Transit FireNet Network You can deploy two Firewall Networks, one dedicated for East-West traffic inspection and another for egress inspection. You must follow this configuration sequence: 1. Disable the Traffic Inspection of the FireNet Gateway intended for egress control. 2. Enable Egress Control for FireNet Gateway intended for egress control. 3. Build connection policies. ![Dual FireNet](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/architecture/security/dual-firenet.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=6f87c2903d5eb02a7a24f8c39fb6c347) ## Distributed Egress in a Multi-Region Deployment If you need to have a distributed egress for each region, make sure you filter out the default route 0.0.0.0/0 when you build the Aviatrix Transit Gateway peering, as shown in the diagram below. ![Multi-Egress](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/architecture/security/multi-egress.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=b9c5b30222708bb5b770bdd6ca191a70) ## Ingress Protection via Aviatrix Transit FireNet This Ingress Protection design pattern is to have the traffic forward to firewall instances directly in Aviatrix Transit FireNet VPC/VNet as shown in the diagram below. In this design pattern, each firewall instance must configure (1) SNAT on its LAN interface that connects to the Aviatrix FireNet Gateway and (2) DNAT to the IP of application server/load balancer. The drawback of this design is that the source IP address is not preserved when traffic reaches the application. For an example configuration workflow, see [Ingress Protection via Aviatrix Transit FireNet with FortiGate](https://docs.aviatrix.com/documentation/latest/security/fortigate-ingress-protection-firenet.html). ![Ingress Transit FireNet](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/architecture/security/ingress-transit-firenet-topology.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=d968c29a84716cbe1e3a8b96484feb0a) # East-West Suricata IPS and TLS MITM Inspection Architecture Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/architecture/security/ips-east-west-design-patterns A reference architecture for inspecting east-west traffic between Aviatrix spoke gateways with Suricata IPS and TLS MITM decryption, and the symmetric routing dependency this design requires. This architecture describes how Suricata-based Intrusion Prevention System (IPS) and TLS man-in-the-middle (MITM) decryption inspect east-west traffic. East-west traffic is traffic between workloads attached to different spoke gateways, as distinct from north-south traffic leaving the network. It explains why east-west inspection depends on symmetric routing between a spoke's gateway pair, and how traffic flows through the inspection point once that dependency is satisfied. > [!NOTE] > > East-west Suricata IPS and TLS MITM inspection is in > Preview in Controller 10.1. The symmetric routing this design depends on > (see > [AZ Affinity and Symmetric Routing Architecture](/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/az-affinity-symmetric-routing) > ) is generally available (GA) in Controller 10.1. Only the east-west L7 > inspection layer described on this page is Preview. ## Overview East-west traffic between spoke gateways has no NAT step, unlike egress traffic, where NAT naturally pins a flow's return path to the gateway that handled the outbound leg. Stateful inspection (Suricata IPS and TLS MITM decryption) requires both directions of a flow to reach the same gateway instance. Without a mechanism that pins forward and return traffic to a shared gateway pair, nothing guarantees that a request and its response are ever inspected together. This architecture explains that dependency and the traffic path through the inspection point once symmetric routing supplies it. Use this architecture when you are planning IPS or TLS MITM coverage for traffic between spokes (inter-VPC/VNet, not internet-bound). You need to understand what has to be true of your routing topology before that coverage is reliable. Prior to this architecture, DCF's own documented constraints stated the limitation this page addresses: rules using Layer 7 (WebGroups, and by extension IPS and TLS MITM) did not support asymmetric traffic. East-west traffic, which has no NAT step to force symmetry, was therefore limited to Layer 4 rules. See [DCF Constraints](/docs/enterprise/10.1/reference/dcf/dcf-configuring#dcf-constraints) and the Deep Packet Inspection row group (Suricata IDS (Egress only), Suricata IPS (Egress Only)) in [Distributed Cloud Firewall Supported Capabilities](/docs/enterprise/10.1/reference/dcf/dcf-capabilities) . Symmetric routing is what lifts that limitation for east-west traffic between spokes that have it enabled. The rest of this page explains how. ## Architecture Diagram ![East-west traffic flow between two spokes through Transit, where the same spoke gateway pair carries both the request and the response so Suricata IPS and TLS MITM inspection sees the complete flow.](https://mintcdn.com/aviatrix-14b37c43/Cv6uug5wRtgMFTqU/images/concepts-architectures/architecture/security/east-west-ips-symmetric-routing.png?fit=max\&auto=format\&n=Cv6uug5wRtgMFTqU\&q=85\&s=fac09531395bdbf45c09c89ab80609b8) ![East-west traffic flow without symmetric routing, where the request and the response reach different members of a spoke gateway's HA pair, so neither gateway instance sees a complete bidirectional flow to inspect.](https://mintcdn.com/aviatrix-14b37c43/Cv6uug5wRtgMFTqU/images/concepts-architectures/architecture/security/east-west-ips-no-symmetric-routing.png?fit=max\&auto=format\&n=Cv6uug5wRtgMFTqU\&q=85\&s=017295281cc094d29fe27a751b21303b) > [!NOTE] > > Contrast with north-south: an egress or ingress flow is naturally pinned to > one gateway by NAT, so the diagram above does not apply there. See > > [Intrusion Prevention System (IPS)](/docs/enterprise/10.1/guides/security/dcf/ips) > > > > for the north-south configuration pattern, which this architecture does not repeat. ## Scope and Prerequisites East-west IPS/MITM inspection has a structural prerequisite: it only functions where the routing design that provides gateway-pair symmetry is in place and applicable. * **Symmetric routing must be enabled on both spokes in the flow.** A flow between Workload A and Workload B is only reliably inspected if the spoke fronting Workload A and the spoke fronting Workload B both have symmetric routing enabled. AZ Affinity is a separate, optional setting covered by the same architecture below. It is not required for east-west inspection. See [AZ Affinity and Symmetric Routing Architecture](/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/az-affinity-symmetric-routing) . * **Symmetric routing is supported for AWS only in Controller 10.1.** The routing mechanism this architecture depends on is supported for AWS only in this release. Azure, OCI, and GCP are not currently covered. * **East-west inspection is not supported with FireNet-attached spokes.** * **DCF must already be enabled**, with an IPS Profile and, if TLS visibility is required, TLS Decryption configured, exactly as for north-south traffic. See [Intrusion Prevention System (IPS)](/docs/enterprise/10.1/guides/security/dcf/ips) . ## Design Principles * **Symmetric routing is a prerequisite, not a feature of IPS/MITM itself.** Suricata IPS and TLS MITM decryption are stateful: they track a connection across both directions to evaluate signatures and to complete the decrypt/re-encrypt cycle. East-west inspection does not add new logic to the inspection engine; it depends entirely on the surrounding routing design pinning both directions of a flow to the same gateway pair. See [AZ Affinity and Symmetric Routing Architecture](/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/az-affinity-symmetric-routing) . * **Inspection runs on the spoke gateway, not the transit.** East-west L7 inspection features apply at the spoke gateway that a workload's subnet routes through. As a result, the routing property that matters is which spoke gateway instance a flow's forward and return legs resolve to, not which transit gateway instance carries the traffic in between. IPS specifically cannot run on a transit gateway at all in Controller 10.1; it is restricted to spoke gateways by design. * **The inspection engine is direction-agnostic.** DCF, Suricata IPS, and TLS MITM decryption do not have a separate east-west code path or a separate policy model. The same IPS Profile and the same TLS Decryption/Intrusion Analysis policy fields that apply to north-south traffic apply to east-west traffic; what differs is only whether the surrounding routing guarantees the engine sees a complete flow. * **The design is cloud-provider independent, but this release has a cloud-specific dependency.** The east-west inspection architecture itself does not assume a specific cloud service provider (CSP). In Controller 10.1, however, its only routing dependency (symmetric routing) is supported for AWS only. See [Scope and Prerequisites](#scope-and-prerequisites). * **This architecture adds no new enablement surface.** It does not introduce a CoPilot workflow of its own. See [Enabling East-West Inspection](#enabling-east-west-inspection). ## Components | Component | Role in this design | | - | - | | [Distributed Cloud Firewall (DCF)](/docs/enterprise/10.1/concepts-architectures/components/dcf/dcf-overview) | Policy engine that determines which traffic (north-south or east-west) is selected for IPS and TLS MITM inspection. | | [Suricata IPS and TLS MITM decryption](/docs/enterprise/10.1/guides/security/dcf/ips) | The inspection engine itself: Suricata signature matching plus the decrypt-inspect-re-encrypt TLS Decryption mechanism. Identical capability for north-south and east-west traffic; only the routing context around it differs. | | [Spoke Gateway](/docs/enterprise/10.1/reference/ui/cloud-fabric/gateways#spoke-gateways) | The gateway instance where east-west L7 inspection is applied, and the entity that symmetric routing pins a flow's forward and return legs to. | | [ActiveMesh](/docs/enterprise/10.1/concepts-architectures/components/connectivity/activemesh-about) | The fabric that carries traffic between spoke and transit gateway instances. Without symmetric routing, ActiveMesh's next-hop selection can resolve a flow's forward and return legs to different gateway instances. | | [AZ Affinity and Symmetric Routing](/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/az-affinity-symmetric-routing) | The routing architecture that covers both settings; its Symmetric Routing component pins a spoke's forward and return traffic to a single gateway pair, which is the property east-west IPS/MITM inspection depends on. AZ Affinity, the other setting this architecture covers, is optional for east-west inspection. | ## Traffic Flows ### East-West Flow (Inspected) **1. Workload A sends a request** A workload attached to one spoke VPC/VNet sends traffic to a workload attached to a different spoke VPC/VNet. There is no NAT step on this path: the flow retains its original source and destination addresses as it enters the fabric. **2. The outbound leg reaches Spoke Gateway 1** Symmetric routing directs the outbound leg of the flow to a specific gateway instance in Workload A's spoke gateway pair (see [AZ Affinity and Symmetric Routing Architecture](/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/az-affinity-symmetric-routing) for how that pinning is established). **3. DCF policy evaluates the flow** DCF policy on Spoke Gateway 1 determines whether the flow is selected for IPS inspection, TLS MITM decryption, or both, exactly as it would for a north-south flow. See [Intrusion Prevention System (IPS)](/docs/enterprise/10.1/guides/security/dcf/ips) for the policy fields involved. **4. Traffic crosses the fabric to Spoke Gateway 2** The flow continues over ActiveMesh to the gateway pair fronting Workload B's spoke. **5. Workload B receives the request and responds** On each side, the return leg of the flow must traverse the same gateway that handled the outbound leg: Spoke Gateway 2 on Workload B's side, Spoke Gateway 1 on Workload A's side. Only then does the connection state tracked by IPS and TLS MITM decryption remain valid across both directions. ### North-South Flow (Contrast) A north-south flow (egress to the internet, or ingress from it) is naturally pinned to a single gateway by the NAT step that egress and ingress traffic already require. As a result, the same-gateway-pair dependency described above does not need a separate mechanism on that path. North-south IPS and TLS MITM configuration is already documented in [Intrusion Prevention System (IPS)](/docs/enterprise/10.1/guides/security/dcf/ips) and is out of scope for this architecture. > [!NOTE] > > A conceptually related but architecturally distinct problem (establishing > routing symmetry for stateful inspection without relying on NAT) also appears > in Aviatrix's Transparent Inspection for AWS Transit Gateway design > (bump-in-the-wire insertion into an existing TGW network). That design makes > the same general point as this architecture: symmetry for stateful L7 > inspection has to be a property the routing configuration establishes up > front, not something the inspection engine reconstructs per packet. That > source material is specific to TGW insertion. Its source-IP-preservation > mechanics and its route-table-level implementation do not apply to this > east-west overlay design, and are not repeated here. It is cited only for the > shared architectural principle. ### East-West Flow Without Symmetric Routing (Fail-Safe Behavior) If a DCF policy rule includes an IPS or TLS MITM action and targets east-west traffic, but the spoke gateway pair handling that traffic does not have symmetric routing enabled, the policy engine does not attempt partial or best-effort inspection. Instead: * The IPS or TLS MITM action on that rule is automatically converted to a Deny action for the affected east-west traffic. * A notification tagged `DCF-l7-non-symmetric-routing` is raised in CoPilot, so the change is visible rather than failing with no signal at all. No API error is returned and no step blocks rule creation in the UI. The notification is the only signal, so watch for it if you are rolling out symmetric routing and east-west L7 policy together. This fail-safe means the traffic is blocked rather than passed through uninspected, and it also means enabling symmetric routing later can change previously-Denied east-west traffic to actively inspected traffic without any change to the DCF policy itself. ## Failover Behavior Symmetric routing preserves the same-gateway-pair guarantee across a spoke gateway failover. If one instance in a spoke's gateway pair fails, its subnets are reassigned to the surviving instance, and both directions of new flows continue to resolve to that surviving gateway. What does not survive failover is inspection state: Suricata sessions and TLS MITM decryption state are tracked per gateway instance and are not replicated between members of a gateway pair. When a failover occurs: * New connections are inspected correctly on the surviving gateway. * Connections that were already established and being inspected before the failover are reset; the client and server must re-establish them. This is the same limitation any stateful inline inspection device has across a failover. It is not specific to east-west traffic. However, east-west inspection depends on symmetric routing, and therefore on gateway-pair health, in a way that north-south inspection, pinned by NAT to a single gateway, does not. ## Variants and Trade-offs | Variant | When to use it | Trade-off | | - | - | - | | **IDS (alert-only) for east-west** | You want visibility into east-west threats before committing to enforcement, or you are validating an IPS Profile's signature coverage against real traffic. | Matching traffic is logged but not dropped, so an active threat between spokes is not blocked. | | **IPS (drop) for east-west** | You have validated your IPS Profile against representative east-west traffic and are ready to enforce. | A false-positive signature match drops legitimate inter-spoke traffic; tune the Drop Level and SID exceptions before enabling broadly. | | **TLS MITM decryption enabled** | You need visibility into encrypted east-west traffic. | Adds a decrypt/re-encrypt step per flow, requires a trusted decryption CA distributed to workloads, and, per DCF's existing Ensure TLS behavior, can affect non-TLS traffic matched by the same rule. | | **TLS MITM decryption disabled, Suricata IPS only** | You do not need visibility into encrypted payloads, or you want to avoid the decrypt/re-encrypt overhead and CA-distribution requirement TLS MITM adds. | Signature matching is limited to what is visible outside the TLS payload; encrypted threats inside inspected flows are not detected. | ## Sizing and Scale Considerations * East-west inspection adds the same per-flow inspection cost (signature matching, and decrypt/re-encrypt if TLS MITM is enabled) that north-south inspection already adds on a spoke gateway. This architecture does not introduce a second inspection engine or a separate resource pool. * Symmetric routing is scoped per spoke group. Each spoke's gateway pair independently carries the inspection load for flows terminating in that spoke; there is no shared or centralized east-west inspection point across spokes. * No performance impact numbers (added latency, throughput ceiling per gateway size) are published for this design. ## Security Considerations * What this design protects: stateful Suricata IPS signature matching and TLS MITM decryption visibility for traffic between workloads in different spokes, to the same degree of coverage already available for north-south traffic, provided the symmetric-routing prerequisite above is met. * What this design does not protect: traffic between workloads attached to the *same* spoke gateway pair is not affected by the symmetric-routing dependency described here (both directions already resolve to the same gateway), but is still subject to whatever DCF policy applies to it. This architecture does not change that. On spokes that do not have symmetric routing enabled, IPS and TLS MITM actions on east-west rules are converted to Deny rather than left to run without the same-gateway guarantee. See [East-West Flow Without Symmetric Routing (Fail-Safe Behavior)](#east-west-flow-without-symmetric-routing-fail-safe-behavior). * Inspection state does not survive gateway failover. See [Failover Behavior](#failover-behavior). * Out of scope for this architecture: IPS rule management (custom Suricata rule feeds, SID exceptions) and threat intelligence feed integration are separate capabilities, already covered in [Intrusion Prevention System (IPS)](/docs/enterprise/10.1/guides/security/dcf/ips) and not re-explained here. * Decrypted-traffic handling: TLS MITM decryption relies on the same Decryption CA Certificate and trust-bundle mechanism already documented for DCF generally. See [About Distributed Cloud Firewall Settings](/docs/enterprise/10.1/guides/security/dcf/dcf-settings#decryption-ca-certificate) . This architecture does not introduce a separate certificate or trust model for east-west traffic. ## Enabling East-West Inspection There is no separate, east-west-specific enablement workflow in CoPilot. An inspected spoke gateway is an ordinary DCF-enabled gateway; what determines whether east-west inspection is effective is the routing topology, not a distinct configuration surface. In practice, enabling east-west IPS/MITM inspection means: * Turning on Symmetric Routing on the relevant spoke gateway pair on both sides of the flow you want inspected. AZ Affinity is optional for east-west inspection. See [AZ Affinity and Symmetric Routing Architecture](/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/az-affinity-symmetric-routing) for the design Symmetric Routing implements and its current scope (AWS only, not supported with FireNet, in Controller 10.1). * Configuring Distributed Cloud Firewall IPS and TLS MITM decryption policy exactly as already documented for north-south traffic in [Intrusion Prevention System (IPS)](/docs/enterprise/10.1/guides/security/dcf/ips) . No east-west-specific IPS Profile, policy field, or configuration step exists: the same profile and policy apply regardless of traffic direction. Once both are in place, east-west traffic between the affected spokes is inspected using the same DCF policy, the same IPS Profiles, and the same TLS Decryption mechanism already in effect for that policy. No additional step is required. ## Related Resources * [Intrusion Prevention System (IPS)](/docs/enterprise/10.1/guides/security/dcf/ips) : the configuration guide for the underlying IPS and TLS MITM (TLS Decryption) mechanism, including CoPilot and Terraform procedures. Applies unchanged to east-west traffic once the routing prerequisite is met. * [DCF Constraints](/docs/enterprise/10.1/reference/dcf/dcf-configuring#dcf-constraints) : states the pre-symmetric-routing limitation this architecture lifts for east-west traffic (Layer 7 rules do not support asymmetric traffic). * [Distributed Cloud Firewall Supported Capabilities](/docs/enterprise/10.1/reference/dcf/dcf-capabilities) : the release-by-release capability matrix. Suricata IDS/IPS are listed as Egress only through Controller 9.0. * [AZ Affinity and Symmetric Routing Architecture](/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/az-affinity-symmetric-routing) : the routing architecture this design depends on. * [Secure Networking and Distributed Cloud Firewall](/docs/enterprise/10.1/concepts-architectures/components/dcf/dcf-overview) : the policy engine that selects traffic for inspection. * [About ActiveMesh](/docs/enterprise/10.1/concepts-architectures/components/connectivity/activemesh-about) : the fabric whose next-hop selection symmetric routing constrains. * [About Distributed Cloud Firewall Settings](/docs/enterprise/10.1/guides/security/dcf/dcf-settings) : Decryption CA Certificate management for TLS MITM decryption. # FireNet Ingress Traffic Inspection Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/architecture/security/planning-firenet-ingress If the FireNet deployment is for both Egress and Ingress traffic, you need to SNAT on the firewall instance to its LAN or Trusted Interface IP (eth2 interface). > [!NOTE] > > If the FireNet deployment is for both Egress and Ingress traffic, you need to > SNAT on the firewall instance to its LAN or Trusted Interface IP (eth2 > interface). The rule is that for a source IP address that comes from NLB or a > vendor load balancer such as an F5 private IP address, it is translated to a > firewall interface eth2 private IP address. ## Ingress Traffic Inspection Follow the [Ingress firewall instructions](/docs/enterprise/10.1/guides/security/firenet/ingress-firewall-example) to deploy the solution for Ingress traffic inspection. ![ingress\_firewall](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/architecture/security/ingress-firewall-network-design.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=f15f6a3d4dbf9e9fca744e36e77df672) ## Ingress Directly through Firewall Another often configured Ingress Egress design pattern is to have the traffic forward to firewall instances directly as shown in the diagram below. In this design pattern, each firewall instance must configure SNAT on its LAN interface that connects to the Aviatrix FireNet gateway. ![firenet\_ingress\_egress](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/architecture/security/firenet-ingress-egress-design.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=5bb27ee5fbe2e2240f10a055f33251b1) The drawback of this design is that the source IP address is not preserved when traffic reaches the application. If you need to preserve source IP address, refer to [this recommended design for Ingress](/docs/enterprise/10.1/guides/security/firenet/ingress-firewall-example). For more information, follow the [FireNet workflow](/docs/enterprise/10.1/guides/security/firenet/transit-firenet-workflow-aws). ## Ingress Protection via Aviatrix Transit FireNet This Ingress Protection design pattern is to have the traffic forward to firewall instances directly in Aviatrix Transit FireNet VPC/VNet as shown in the diagram below. In this design pattern, each firewall instance must configure (1) SNAT on its LAN interface that connects to the Aviatrix FireNet Gateway and (2) DNAT to the IP of application server/load balancer. The drawback of this design is that the source IP address is not preserved when traffic reaches the application. ![500](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/architecture/security/ingress-transit-firenet-topology.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=d968c29a84716cbe1e3a8b96484feb0a) For an example configuration workflow, see [Ingress Protection via Aviatrix Transit FireNet with FortiGate](https://legacy.docs.aviatrix.com/documentation/latest/security/fortigate-ingress-protection-firenet.html). [Azure Ingress Firewall Setup Solution](/docs/enterprise/10.1/guides/security/firenet/azure-ingress-firewall-example) [Ingress Protection via Aviatrix Transit FireNet with Palo Alto in GCP](/docs/enterprise/10.1/guides/security/firenet/paloalto-ingress-gcp) # Auto Right-Sizing Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/architecture/sizing/auto-right-sizing-about You can use the Auto Right-Sizing feature to apply appropriate sizing to Transit and Spoke gateway instances (AWS and Azure). This can reduce cloud provider costs, improve performance and mitigate packet loss. You can use the Auto Right-Sizing feature to apply appropriate sizing to Transit and Spoke gateway instances (AWS and Azure). This can reduce cloud provider costs, improve performance and mitigate packet loss. ![Auto Right-Sizing tab showing gateway instance recommendations](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/architecture/sizing/auto-sizing-tab.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=2c5e2cdea5806a55dfaf52944b9a8449) > [!NOTE] > > Auto Right-Sizing is only supported for AWS and Azure. > > The Auto Right-Sizing feature is disabled by default. See > [Auto Right Sizing Enable](https://legacy.docs.aviatrix.com/documentation/latest/network/auto-right-sizing-enable.html?expand=true) > for instructions on enabling the feature. The Auto Right-Sizing tab shows: * Gateway instances for which auto right-sizing is recommended. This list is replaced every monitoring period. * The urgency of each recommendation. * The potential performance gain of the gateway instance once the recommendation is implemented. * The potential cost savings once the recommendation is implemented. For more information, see [Auto Right-Sizing Optimization](https://legacy.docs.aviatrix.com/documentation/latest/network/ars-optimization.html). > [!NOTE] > > The Current Size column may display gateway instance sizes that do not map to > [Aviatrix standard sizes](/docs/enterprise/10.1/reference/gateways/auto-right-sizing/auto-right-sizing-standard-sizes) > . In that case, a related Aviatrix standard size is not displayed in brackets. The gateway instance sizing recommendations are based on Aviatrix standard gateway sizes. See [Mapping Aviatrix Standard Gateway Sizes to VM Instance Sizes](/docs/enterprise/10.1/reference/gateways/auto-right-sizing/auto-right-sizing-standard-sizes) for details on which gateway instance sizes correspond to Aviatrix standard recommendations such as XSmall, Small, Large, and XLarge for Spoke and Transit gateways. > [!NOTE] > > If you configured the Gateway Scaling feature prior to CoPilot 4.15, the Policies, Recommendations, and Events tabs for that feature will show 'Legacy'. You can continue using the Gateway Scaling feature if you want to add new gateway instances when scaling up. > > If you did not use the Gateway Scaling feature prior to CoPilot 4.15, the Legacy > tabs are not displayed. > > Aviatrix recommends migrating to the Auto Right-Sizing feature for your gateway > scaling needs. > > At this time Auto Right-Sizing only suggests modifying existing gateway sizes, > not adding or removing gateway instances. > [!NOTE] > > See > [Gateway Performance and Sizing](/docs/enterprise/10.1/reference/gateways/gateway-sizing) > for information to assist you with pre-deployment planning of your gateway > sizing. ## Considerations Before Implementing Auto Right-Sizing Recommendations * Configure the monitoring frequency in [Auto Right-Sizing Settings](/docs/enterprise/10.1/reference/gateways/auto-right-sizing/auto-right-sizing-settings). * Select VPC/VNets to exclude from monitoring in [Auto Right-Sizing Settings](/docs/enterprise/10.1/reference/gateways/auto-right-sizing/auto-right-sizing-settings) before performing resize actions. * At any time, you can manually resize your gateways from Cloud Fabric > Gateways. * Do not resize gateways in the same VPC/VNet to be different sizes; periodically check if recommendations for HA gateway instances suggest vastly different sizes. If that occurs, you should check the [health of your gateway](https://legacy.docs.aviatrix.com/documentation/latest/monitoring/managed-resources-dashboard.html) and if [routing](https://legacy.docs.aviatrix.com/documentation/latest/monitoring/resource-performance.html) is working correctly. * Do not downsize any HPE gateway instances to Small. * Do not downsize any Distributed Cloud Firewall gateway instances to unsupported sizes. * Do not resize if your CSP is out of resources in the suggested region. ## Implementing Recommendations Remember that not all gateways need resizing; you can ignore or dismiss any auto right-sizing recommendations. > [!TIP] > > Aviatrix recommends performing auto right-sizing during gateway maintenance > periods. Any user can act on the provided recommendations. The network will reach 'steady state' over time, as you implement the provided recommendations. Recommendations that have been implemented are removed from the Auto Right-Sizing tab during this update, and moved to the Cloud Fabric > Scaling > Events tab. Recommendations to preserve the current gateway instance size are not included. Each gateway instance is limited to one recommendation per monitoring period. If there are no active recommendations, a message to this effect is displayed on the Auto Right-Sizing tab, along with when the next monitoring period scan will begin. ## Viewing Auto Right-Sizing Recommendation Details Clicking a gateway instance on the Auto Right-Sizing tab shows its details. ![Auto Right-Sizing recommendation details](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/architecture/sizing/auto-sizing-details.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=6751de52cee48a97d48d5ad8f6515c9f) From here you can: * [Resize](https://legacy.docs.aviatrix.com/documentation/latest/network/auto-right-sizing-resizing.html) the gateway instance (immediately, or schedule during a maintenance window). * View the remarks about the current gateway instance. These remarks provide an explanation of why resizing is recommended. * View the [network and system metrics](/docs/enterprise/10.1/reference/gateways/auto-right-sizing/ars-metrics) for the gateway instance. * View gateway instance performance. Click [View Performance](/docs/enterprise/10.1/reference/gateways/auto-right-sizing/ars-metrics) to go to the Monitor > Performance tab. * Dismiss the recommendation. * Ignore recommendations from the selected VPC/VNet. Detailed information about auto right-sizing functionality. ## Next Steps * [Configure the monitoring period and what VPC/VNets to exclude from monitoring](/docs/enterprise/10.1/reference/gateways/auto-right-sizing/auto-right-sizing-settings) * [Resize gateway instances based on provided recommendations](https://legacy.docs.aviatrix.com/documentation/latest/network/auto-right-sizing-resizing.html?expand=true) * [View the events generated by resizing actions](https://legacy.docs.aviatrix.com/documentation/latest/network/ars-events-tab.html?expand=true) ## Related Topics * [Mapping Aviatrix Standard Gateway Sizes to VM Instance Sizes](/docs/enterprise/10.1/reference/gateways/auto-right-sizing/auto-right-sizing-standard-sizes) * [Auto Right Sizing Ignoring](https://legacy.docs.aviatrix.com/documentation/latest/network/auto-right-sizing-ignoring.html?expand=true) * [Auto Right Sizing Dismissing](https://legacy.docs.aviatrix.com/documentation/latest/network/auto-right-sizing-dismissing.html?expand=true) * [Auto Right Sizing Actions](https://legacy.docs.aviatrix.com/documentation/latest/network/auto-right-sizing-actions.html?expand=true) * [Auto Right-Sizing Optimization](https://legacy.docs.aviatrix.com/documentation/latest/network/ars-optimization.html) * [Auto Right-Sizing Maintenance Window](/docs/enterprise/10.1/reference/gateways/auto-right-sizing/auto-right-sizing-maintenance) * [Ars Events Tab](https://legacy.docs.aviatrix.com/documentation/latest/network/ars-events-tab.html?expand=true) # ActiveMesh 4.0 Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/components/connectivity/activemesh-4.0 Overview of Aviatrix ActiveMesh 4.0, the latest version of the encrypted transit network architecture with enhanced performance and availability. # ActiveMesh 4.0 > [!TIP] > > ActiveMesh 4.0 is an Early Access feature. DO NOT use ActiveMesh 4.0 in production environments. Contact [Aviatrix Support](https://support.aviatrix.com) to enable and try ActiveMesh 4.0. ActiveMesh 4.0 is an iteration of ActiveMesh. The features of ActiveMesh 4.0 are as follows: * **Primary Gateway Lifecycle Control**: Provides a delete option for the primary gateway instance. * **Dynamic HPE Toggle**: Provides an HPE toggle option for a deployed gateway group to enable or disable HPE on the fly. > [!NOTE] > > Both Primary Gateway Lifecycle Control and Dynamic HPE Toggle can be used only when no S2C connections, FireNet, or AWS TGW attachments are configured on the primary gateway. ## Primary Gateway Lifecycle Control ActiveMesh 4.0 supports deletion of the primary gateway instance. After the primary gateway instance is deleted, the next gateway instance in the sequence automatically becomes the new primary gateway. To delete the primary gateway instance: 1. From the CoPilot GUI, navigate to **Cloud Fabric** > **Gateways** > **Transit Gateways** or **Spoke Gateways**. A table of deployed gateway groups appears. 2. Click the Edit icon in the row of a gateway group. 3. Click the Delete icon in the row of the primary gateway instance. ![](https://mintcdn.com/aviatrix-14b37c43/0QYX2iFaLoliwUis/images/network/primary-gw-delete.png?fit=max\&auto=format\&n=0QYX2iFaLoliwUis\&q=85\&s=090df20f922be7af80159349f8135333) > [!NOTE] > > Ensure that S2C connections, FireNet, or AWS TGW are not configured on the primary gateway instance before deleting it. 4. Click **Save**. A success message appears to confirm the deletion of the primary gateway instance. The next gateway instance in the sequence automatically becomes the new primary gateway. ## Dynamic HPE Toggle ActiveMesh 4.0 supports enabling or disabling HPE on a deployed gateway group dynamically. If a gateway group contains instances with the HPE enabled and HPE disabled, the **High Performance Encryption** status of the group appears as **Multiple Values** in the CoPilot GUI. ![](https://mintcdn.com/aviatrix-14b37c43/0QYX2iFaLoliwUis/images/network/dynamic-hpe-toggle.png?fit=max\&auto=format\&n=0QYX2iFaLoliwUis\&q=85\&s=7d08a311b227602889328f4539809ac8) # ActiveMesh Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/components/connectivity/activemesh-about ActiveMesh is an Aviatrix Encrypted Transit Network architecture where both primary gateways and backup gateways forward packets in a load balancing fashion. The architecture statistically doubles the network throughput. ActiveMesh is an Aviatrix Encrypted Transit Network architecture where both primary gateways and backup gateways forward packets in a load balancing fashion. The architecture statistically doubles the network throughput. In addition, in ActiveMesh mode, multiple remotes sites can be connected to the Aviatrix Transit gateways. > [!NOTE] > > ActiveMesh is enabled by default, and cannot be disabled. The diagram below shows an ActiveMesh deployment between Spoke and Transit where each Spoke Gateway in a VPC/VNet builds two IPsec tunnels to the primary and backup transit gateways and forwards packets to both of them inside the tunnel. The load balance mechanism leverages ECMP protocol. ![activemesh\_spoke\_transit](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/components/networking/activemesh-spoke-transit.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=2137d977ea5a4080f9f7b2ded1fbbf18) When Aviatrix Controller detects that an ActiveMesh gateway is down, Controller automatically starts it again. Once the gateway comes up, it participates in packet forwarding again. ## Key advantages of ActiveMesh The key benefits of ActiveMesh are improved network resiliency, failover convergence time and performance. ## ActiveMesh for Transit Gateway Peering ActiveMesh can be applied to connect two Transit GWs. There are 4 tunnels established between the Transit GWs, as shown in the diagram below. ![activemesh\_transit\_transit](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/components/networking/activemesh-transit-transit.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=ba3c958c67806a5a30c54e02fd2872b0) ## ActiveMesh Connection to VGW Each Transit GW connecting to the VGW in ActiveMesh mode has two VPN tunnels to the VGW. ## Link Between two ActiveMesh Gateways The link between two ActiveMesh gateways is used to forward packets when both tunnels are down for one of the ActiveMesh gateway. For example, in a spoke VPC/VNet, virtual machine (EC2/GCE) traffic is forwarded to the ActiveMesh primary gateway which then forwards traffic to the AVX Transit GW. If both tunnels between the ActiveMesh spoke gateway and the Transit GW are down, the packet is forwarded by the ActiveMesh primary gateway to the backup ActiveMesh gateway. ![activemesh\_tunnel\_failures](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/components/networking/activemesh-tunnel-failures.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=f7f7149b84acbb3d51b3e1cddb303689) ## ActiveMesh 4.0 > [!WARNING] > > ActiveMesh 4.0 is an Early Access feature. DO NOT use ActiveMesh 4.0 in > production environments. Contact [Aviatrix > Support](https://support.aviatrix.com) to enable and try ActiveMesh 4.0. ActiveMesh 4.0 is an iteration of ActiveMesh. The features of ActiveMesh 4.0 are as follows: * **Primary Gateway Lifecycle Control**: Provides a delete option for the primary gateway instance. * **Dynamic HPE Toggle**: Provides an HPE toggle option for a deployed gateway group to enable or disable HPE on the fly. > [!NOTE] > > Both Primary Gateway Lifecycle Control and Dynamic HPE Toggle can be used only > when no S2C connections, FireNet, or AWS TGW attachments are configured on the > primary gateway. ### Primary Gateway Lifecycle Control ActiveMesh 4.0 supports deletion of the primary gateway instance. After the primary gateway instance is deleted, the next gateway instance in the sequence automatically becomes the new primary gateway. To delete the primary gateway instance: 1. From the CoPilot GUI, navigate to **Cloud Fabric** > **Gateways** > **Transit Gateways** or **Spoke Gateways**. A table of deployed gateway groups appears. 2. Click the Edit icon in the row of a gateway group. 3. Click the Delete icon in the row of the primary gateway instance. > [!NOTE] > > Ensure that S2C connections, FireNet, or AWS TGW are not configured on the > primary gateway instance before deleting it. 4. Click **Save**. A success message appears to confirm the deletion of the primary gateway instance. The next gateway instance in the sequence automatically becomes the new primary gateway. ![primary gw delete](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/components/networking/primary-gw-delete.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=fc69e1b01dc70bcca912d42259645034) ### Dynamic HPE Toggle ActiveMesh 4.0 supports enabling or disabling HPE on a deployed gateway group dynamically. If a gateway group contains instances with the HPE enabled and HPE disabled, the **High Performance Encryption** status of the group appears as **Multiple Values** in the CoPilot GUI. ![dynamic hpe toggle](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/components/networking/dynamic-hpe-toggle.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=ba1387309908e77743942c93e2e12d8e) ## ActiveMesh 2.0 ActiveMesh 2.0 is a new iteration of ActiveMesh. The main advancement of ActiveMesh 2.0 is its deterministic nature of Next Hop selection. Here is how Aviatrix Transit Gateway routing engine treats the following types of routes. | Networks | Route Type | Aviatrix Transit Gateway Route Propagation | | - | - | - | | Local TGW attached VPC/VNet CIDR | tgwvpc | Local | | Aviatrix Spoke gateway associated VPC/VNet CIDR | vpc | Local | | Azure Native Spoke associated VNet CIDR | vpc | Local | | Local TGW VPN dynamically learned network CIDR | tgwedge | Advertises TGW VPN ASN and its remote peer ASN to a remote BGP peer if it is the best route. | | Local TGW DXGW learned network CIDR | tgwedge | Advertises TGW DXGW ASN and its remote peer ASN to a remote BGP peer if it is the best route. | | Remote Aviatrix Transit Gateway Peering learned routes | peer | Advertises remote Aviatrix peer's network CIDRs to a remote BGP peer if it is the best route. | | Aviatrix Transit Gateway BGP learned from on-premises | bgp | Advertises to its remote peers by Aviatrix Transit Gateway peering if it is the best route. | | Aviatrix Transit Gateway statically learned from on-premises | static | Local | | Aviatrix Transit Gateway associated VPC/VNet CIDR | linklocal | Local | | Local Firewall Egress route (0.0.0.0/0) | transit | Local | | Aviatrix Transit Gateway SNAT IP address | linklocal | Local | With this approach, there is more visibility on learned routes regarding what paths the routes are learned from. The next hop best path selection follows the priorities listed below. 1. Local 2. Shortest number of ASN list 3. For two identical length ASN routes, selects the next hop with the lowest Metric Value. 4. For two identical ASN length and Metric Value routes, if ECMP is disabled (this is the default configuration), selects the current best route. If there is no current best route, the next hop IP addresses are compared, the lower integer IP address is selected. 5. For two identical ASN length and Metric Value routes, if ECMP is enabled, traffic is distributed to both routes using ECMP. ### Migrating to ActiveMesh 2.0 There are 3 scenarios: | Deployment | Notes | ActiveMesh 2.0 Migration | | - | - | - | | Non ActiveMesh deployment | The Aviatrix Transit Gateway in the deployment has been launched before Release 5.1 (10/1/2019) | See [Migrating from Classic Aviatrix Encrypted Transit Network to Aviatrix ActiveMesh Transit Network](https://legacy.docs.aviatrix.com/documentation/latest/network/activemesh-migration.html). | | ActiveMesh 1.0 deployment | The Aviatrix Transit Gateway was launched with ActiveMesh option enabled prior to Release 6.0 | Migrate to ActiveMesh 2.0 by going to Settings > Maintenance > Migration > ActiveMesh 2.0 Migration, click Migrate. | | ActiveMesh 2.0 deployment | The Aviatrix Transit Gateway was launched with ActiveMesh enabled after Release 6.0 | ActiveMesh 2.0 is automatically enabled for brand new deployment on Controller. | ## Related Topics * [ActiveMesh Design Notes](/docs/enterprise/10.1/concepts-architectures/components/connectivity/activemesh-design-notes) # ActiveMesh Design Notes Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/components/connectivity/activemesh-design-notes ActiveMesh is the default mode when launching an Aviatrix Transit Gateway. This document describes the supported common design patterns while deploying ActiveMesh Gateways. ActiveMesh is the default mode when launching an Aviatrix Transit Gateway. This document describes the supported common design patterns while deploying ActiveMesh Gateways. ## ActiveMesh with Aviatrix Transit GW for On-premises Connection ![Activemesh AVX On-premises](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/components/networking/activemesh-avx-onprem.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=4bf74d9f242e285520c3822be7c25e3d) ### Redundant Routers On-premises If there are two on-premises routers advertising the same network CIDR and connect to Aviatrix Transit Gateway directly, Aviatrix Transit Gateway automatically enables ECMP for traffic from cloud to on-premises. If this is not desired outcome, connect on-premises to the Aviatrix Transit Gateway through a VGW or VPN Gateway. ### Multi-Sites If Aviatrix Transit Gateways connects to multi sites on-premises directly via BGP, these sites should advertise non overlapping CIDRs to the Aviatrix Transit Gateway. ### Route Propagation The local Aviatrix Transit Gateway learned routes via BGP are propagated to the peered Aviatrix Transit Gateway. The propagated information includes network CIDRS, AS\_PATH and metrics. If the local Aviatrix Transit Gateway learned duplicate network CIDRs (there are multiple paths to reach the same network CIDRs) via BGP, it uses the following rules to decide which route is propagated to the remote Aviatrix Transit Gateway. * The route with the shortest AS\_PATH length wins. * If there are identical AS\_PATH lengths, the lowest metric route wins. * If the metrics are all the same, the smallest next hop IP address wins. In other words, there will always be one route advertised to the remote Aviatrix Transit Gateway when identical network CIDRs are learned by the local Aviatrix Transit Gateway. ### Overlapping Spoke VPC/VNet CIDRs If there are overlapping Spoke VPCs CIDRs attached to the TGWs in two regions and you wish to connect them via Aviatrix Transit Gateway Peering, use [Exclude Network CIDRs](/docs/enterprise/10.1/guides/connectivity/peering/transit-gateway-peering-workflow) on both Aviatrix Transit Gateways to exclude these overlapping Spoke VPC/VNet CIDRs. ## NAT Functions SNAT function is supported on the individual connection between the Aviatrix Transit Gateway and the remote sites. Starting in Release 5.4, SNAT and DNAT functions are supported on the Spoke Gateway tunnel interface to the Aviatrix Transit Gateway. ## Egress Routes Propagation Behavior If firewalls are deployed for Internet-bound egress traffic in either FireNet or Transit FireNet deployment, the default routes are propagated to the remote peer by Transit Gateway peering. This allows firewalls to be shared across regions. If you have regional firewalls for egress traffic, make sure you apply filter to filter out the default routes. ## Configuration Notes ### One On-premises Device In this scenario, the on-premises site has one device as the diagram below. ![Activemesh One Device](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/components/networking/activemesh-one-device.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=852c525a5f1cde4ac978be919024863b) If the backup Aviatrix Transit Gateway is launched and the Transit Gateway is launched with ActiveMesh, the configuration should include the following settings: * Enable HA - Mark the checkbox to enable HA if the remote site has two external IP addresses. * Local Tunnel IP - Include two IP addresses in this field: the first one for the primary Aviatrix Transit Gateway, and the second for the backup Aviatrix Transit Gateway (only if it is launched). ### Two On-premises Devices In this scenario, the on-premises site has two devices as the diagram below. ![Activemesh Two Devices](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/components/networking/activemesh-two-devices.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=7b73243314c115600bfb869797227046) You should check HA in the configuration and configure the second pair of inside tunnel addresses, as shown below. ![Activemesh HA Config](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/components/networking/activemesh-ha-config.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=ba76dbc5ed0d78227266f832824f862c) ## ActiveMesh with AWS TGW for On-premises Connections While AWS Transit Gateway (TGW) does not propagate routes to Spoke VPCs, TGW Direct Connect via DXGW and TGW VPN have full functions of failover, multi-path and ECMP in supporting connection to on-premises sites. This includes: * TGW prefers DXGW to TGW VPN when both advertising the same network. When DXGW goes down, one of the VPN routes take over. * When there are multiple VPN routes, TGW routing policy selects the shortest AS\_PATH length. * When there are multiple VPN routes with identical AS\_PATH length, TGW VPN distributes traffic with ECMP when it is enabled. In this case, Aviatrix Controller performs the orchestration function in managing route propagation and Aviatrix Transit Gateways are used to connect two AWS TGWs. > [!NOTE] > > Implementing TGW with VPN backup design could lead to > asymmetric routing, that is, with traffic from AWS to on-premises > traversing the DX as intended while traffic from on-premises to AWS > traversing the IPsec VPN tunnel instead. > > Traffic from AWS to on-premises prefers the AWS DXGW over the VPN connection > because the TGW effectively sets a higher "local preference" (LOCAL\_PREF) on the > DXGW BGP sessions (refer to Route Evaluation Order as outlined in the AWS > Transit Gateway documentation). > > For traffic from on-premises to AWS, the DX path should be preferred because AWS > sets a Multi Exit Discriminator (MED) value of 100 on BGP sessions over VPN > links as compared to the default value of 0 over the DX path. This works well in > the case where DX and VPN are used with a Virtual Private Gateway (VGW) as the > same AS is announced over both connections but in case of the TGW, the DX path > uses a different ASN compared to the VPN path. > > The advertised ASN over VPN is the TGW AS while the ASN over DX is the ASN of > the DXGW. Note that in case of TGW, the AS path over the DXGW path only consists > of the DXGW AS instead of AS path length of two with TGW AS + DXGW AS. This is > the result of manually setting the CIDRs to be announced by the AWS DXGW towards > on-premises which effectively causes DXGW to originate the routes resulting in a > reduced path length of one over DX which is the same AS path length as over the > VPN link but different AS path. > > To ensure that the on-premises routers always consider the MED value, set the > "bgp always-compare-med" knob. This forces the router to compare the MED if > multiple routes to a destination have the same local preference and AS path > length. The deployment is shown in the diagram below. ![ActiveMesh TGW On-premises](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/components/networking/activemesh-tgw-onprem.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=47c1d2acb48a1859f442ea2f14a4c669) ### Advertising Different Routes If on-premises sites advertise non overlapping network CIDRs to TGWs, Transit Gateway peering can proceed without issues. ### Advertising Overlapping Routes If on-premises sites advertise identical network CIDRs or overlapping CIDRs to TGWs (for example, they all advertise 10.0.0.0/8 to their respective TGWs), you must enable [Excluded Network CIDRs](/docs/enterprise/10.1/guides/connectivity/peering/transit-gateway-peering-workflow) feature on both sides of the Aviatrix Transit Gateways to filter out identical or overlapping CIDRs in order to connect the two regions. > [!WARNING] > > If you use TGW DXGW/VPN for hybrid connection to on-premises sites, you cannot > use Aviatrix Transit Gateway as the backup router for connecting to the same > on-premises network. This is because TGW propagated routes do not present > themselves in the TGW route table with any BGP information and as such, it is > not possible for the Controller to coordinate the dynamic route updates > between TGW and Aviatrix Transit Gateway. ### Overlapping Spoke VPC CIDRs If there are overlapping Spoke VPCs CIDRs attached to the TGWs in two regions and you wish to connect them via Aviatrix Transit Gateway Peering, use [Exclude Network CIDRs](/docs/enterprise/10.1/guides/connectivity/peering/transit-gateway-peering-workflow) on both Aviatrix Transit Gateways to exclude these overlapping Spoke VPC CIDRs. # IPv6 Overview Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/components/connectivity/ipv6-overview IPv6 (Internet Protocol version 6) is the next-generation IP addressing protocol designed to overcome IPv4 limitations and provide enhanced scalability, security, and performance for modern networks. IPv6 (Internet Protocol version 6) is the next-generation IP addressing protocol designed to overcome IPv4 limitations and provide enhanced scalability, security, and performance for modern networks. Aviatrix introduces IPv6 support across its multicloud networking platform to meet enterprise and federal mandates for dual-stack deployments. > [!NOTE] > > **[Preview](/docs/enterprise/10.1/reference/feature-modes/index#preview-features) in Controller 9.0.** IPv6 support on the Aviatrix platform is in Preview in Controller 9.0. The controller-level **Support for IPv6** toggle in **Settings** > **Configuration** > **General** is labeled **Preview** in CoPilot. Use IPv6 in non-production environments and contact Aviatrix support before enabling it on production deployments. ## Benefits of IPv6 * **Address Exhaustion**: The limited address space of IPv4 is nearly depleted. IPv6 offers a much larger address pool. * **Compliance and Mandates**: Federal agencies and global enterprises require IPv6 adoption to meet regulatory and operational goals. * **Future-Proofing**: IPv6 ensures long-term scalability for cloud and hybrid environments. ## IPv6 Support in Controller 8.2 Controller 8.2 introduced the foundational IPv6 capabilities for AWS and Azure: * Dual-stack for AWS and Azure * FireNet with Palo Alto Networks for AWS * Network Segmentation * BGP over IPSec and BGP over LAN (AWS and self-managed networks) * Edge support for AEP and ESXi platforms ## IPv6 Support in Controller 9.0 Controller 9.0 expands the IPv6 foundation with the following additions. All Controller 9.0 IPv6 capabilities are in Preview: * GCP transit and spoke gateway IPv6 * Government Cloud (AWS GovCloud and Azure Government) * Equinix Edge IPv6 (LAN interface for Edge Spoke; WAN interface for Edge Transit) * Global IPv6 route summarization * Unique Local Address (ULA) `fd00::/8` automatic management (RFC 4193) * Bidirectional Forwarding Detection (BFD) for IPv6 BGP * BGP over LAN for Azure (parity with AWS in Controller 8.2) * BGP over GRE for IPv6 site-to-cloud connections * Fortinet and Checkpoint FireNet IPv6 support * IKEv2 enforced for IPv6 BGP over IPSec * IPv6 diagnostic tooling enhancements The following figure shows an example of IPv6-enabled nodes that communicate over dual-stack connections: ![Example: Dual-Stack Connections](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/components/networking/ipv6-phase1-overview.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=552f202657c51e41f4b27c2ce9905998) ## Key IPv6 Features in Aviatrix Platform * **Dual-Stack Support**: Enable both IPv4 and IPv6 on VPC/VNet, Transit, and Spoke Gateways on AWS, Azure, and GCP (GCP IPv6 Gateways in Controller 9.0). * **Site2Cloud Connectivity**: IPv6 support for BGP over IPsec (BGPoIPSec) and BGP over LAN (BGPoLAN) to secure S2C connections. * **Edge Enablement**: IPv6 connectivity for Aviatrix Edge gateways to support on-premises to cloud traffic. * **Global IPv6 Route Summarization**: Controller-wide setting that advertises summarized IPv6 routes instead of individual host prefixes, reducing route table size. * **Unique Local Address (ULA) Support**: Automatic management of fd00::/8 summary routes (RFC 4193) when IPv6 is enabled on a Gateway. * **FireNet Integration**: IPv6 traffic inspection for east-west and north-south flows in centralized firewall architectures. * **Terraform Automation**: Management of IPv6 configurations through Terraform. ## IPv6 Operational Considerations The following sections describe the operational considerations for IPv6 in the Aviatrix platform. ### Global Configuration Before making IPv6-related changes on the Aviatrix platform, turn On **Support for IPv6** in CoPilot. The toggle is labeled **Preview** in Controller 9.0. To turn On the **Support for IPv6** toggle: 1. In CoPilot, go to **Settings** > **Configuration** > **General**. 2. Turn On the **Support for IPv6** toggle (labeled **Preview**). 3. Click **Save**. The IPv6-related options on the CoPilot are disabled by default. ### Enable IPv6 on Gateways Before enabling IPv6 on gateways, ensure the VPC/VNet of the Gateway supports IPv6. #### Enable IPv6 on New Gateways To enable IPv6 on new gateways: 1. Turn On the **Support for IPv6** toggle in CoPilot to enable global IPv6 settings. 2. Create a new VPC/VNet that supports IPv6.\ If the VPC/VNet already supports IPv6, skip this step. 3. Create a new Gateway on the new VPC/VNet. 4. Ensure to turn On the **Support for IPv6** toggle while creating the Gateway. > [!CAUTION] > > Ensure the Controller and CoPilot VPC/VNet are IPv4-only. Refer to the following pages to enable IPv6 on new gateways: * [Global Configuration](#global-configuration) * [Create a VPC/VNet](/docs/enterprise/10.1/reference/ui/cloud-resources/cloud-networks#create-vpc-vnet) * [Create a Transit Gateway](/docs/enterprise/10.1/reference/ui/cloud-fabric/gateways#create-a-transit-gateway) * [Create a Spoke Gateway](/docs/enterprise/10.1/reference/ui/cloud-fabric/gateways#create-a-spoke-gateway) #### Enable IPv6 on Existing Gateways To enable IPv6 on existing gateways: 1. Turn On the **Support for IPv6** toggle in CoPilot to enable global IPv6 settings. 2. If the VPC/VNet of the Gateway does not support IPv6: 1. Create a new VPC/VNet that supports IPv6. 2. Create a new Gateway on the new IPv6 VPC/VNet.\ Refer to [Enable IPv6 on New Gateways](#enable-ipv6-on-new-gateways) for more details. 3. If the VPC/VNet of the Gateway supports IPv6, edit the Gateway and turn On the **Support for IPv6** toggle. Refer to the following pages to enable IPv6 on existing gateways: * [Global Configuration](#global-configuration) * [Edit a Transit Gateway](/docs/enterprise/10.1/reference/ui/cloud-fabric/gateways#edit-a-transit-gateway) * [Edit a Spoke Gateway](/docs/enterprise/10.1/reference/ui/cloud-fabric/gateways#edit-a-spoke-gateway) ### Routing, Diagnostics, and Visibility IPv6-related information is available in the route tables, diagnostic tools, Topology, and Dashboard sections. See the following pages for more details: * [Gateway Routes](/docs/enterprise/10.1/reference/ui/cloud-fabric/gateways#view-transit-gateway-routes) * [VPC/VNet Route Tables](/docs/enterprise/10.1/reference/ui/cloud-fabric/gateways#view-vpc-vnet-route-tables) * [Gaining Visibility into Your Network Topology](/docs/enterprise/10.1/guides/monitoring/topology-visibility) * [Troubleshooting - Diagnostic Results](/docs/enterprise/10.1/guides/troubleshooting/troubleshoot-diagnostic-result) ## Limitations * IPv6-only mode is not supported; dual-stack deployments are required. * Contact Aviatrix support before enabling Edge as a Transit (EaT) with IPv6. * For **BGP over IPsec** connections with IPv6 enabled, the Controller enforces **IKEv2**.\ IKEv1 is not supported for IPv6 BGPoIPSec. ## Related topics * [Enabling GCP Global VPC Routing](/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/gcp-global-vpc) * [Gateway Routes](/docs/enterprise/10.1/reference/ui/cloud-fabric/gateways#view-transit-gateway-routes) * [VPC/VNet Route Tables](/docs/enterprise/10.1/reference/ui/cloud-fabric/gateways#view-vpc-vnet-route-tables) * [Gaining Visibility into Your Network Topology](/docs/enterprise/10.1/guides/monitoring/topology-visibility) * [Troubleshooting - Diagnostic Results](/docs/enterprise/10.1/guides/troubleshooting/troubleshoot-diagnostic-result) # Site2Cloud Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/components/connectivity/s2c-overview Traditionally enterprises host their IT applications in their own data center or at a co-location. Remote sites typically connect to the data center via an Internet-based IPsec VPN tunnel or MPLS based private network. Such a hub and spoke architecture has been prevalent in the last 15 years. ## Problem: Long Latency or Unstable Internet Connectivity Traditionally enterprises host their IT applications in their own data center or at a co-location. Remote sites typically connect to the data center via an Internet-based IPsec VPN tunnel or MPLS based private network. Such a hub and spoke architecture has been prevalent in the last 15 years. A problem with this deployment architecture is long latency or unstable Internet connectivity suffered by remote sites, especially between those in different continents. This can cause application time out, resulting in lost productivity and an unhappy user experience. The solution to this pain point has been to deploy some form of WAN optimization in both the remote sites and data center to reduce application latency and reduce data bandwidth. These gears are complex, expensive and not every enterprise can afford them, and in some cases, they do not always work well. ## Solution: Bring Application to User With the many regions in the world available brought by public cloud providers, such as AWS and Azure, the application latency issue can now be solved in a brand-new way. By placing applications in a region of public cloud that your remote sites are closer to than to the data center, the long latency issue is eliminated altogether. In addition, by moving servers to the cloud, you can reduce remote sites' footprint and the amount of hardware to manage, thus reducing cost for ongoing maintenance. The comparison between the two deployment architectures is described below: ![Comparison of deployment architectures](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/components/networking/case-study.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=11264a8ad542890b9381c16957e7743e) In the diagram above, remote sites or branch offices connect to the headquarters' data center via IPsec tunnels. International sites across continents can experience hundreds or more milliseconds in latency and in some countries, connectivity to headquarters is unstable at times. The first step in deploying an application close to the user is to build a new network architecture as shown on the right side of the diagram above. A remote site now connects via an IPsec tunnel to the closest Aviatrix Gateway in a VPC or VNet in a region closest to the site. Different remote sites may connect to different Aviatrix Gateways. For example, sites in China connect to Aviatrix Gateways in the Azure China region and sites in Europe connect to an Aviatrix Gateway in a VPC in the AWS eu-west-1 region. After the new network is deployed, you can now replicate Active Directory to VPC/VNet and deploy applications such as ERP in the cloud too. The AD authentication latency and application latency can be reduced to tens of milliseconds. In addition, the remotes are simpler with fewer hardware equipment to manage. For how to set up Site2Cloud, see [Site2Cloud External Connection Workflow](/docs/enterprise/10.1/guides/connectivity/vpn/s2c-configuration-workflow) . ## Use Cases for Site2Cloud Here are common use cases: * **SaaS provider to its customer site**: If you need to move data continuously and securely from customer or partner sites to your SaaS service hosted in AWS, Azure, or Google, building an encrypted tunnel between the customer site and your SaaS service is required. * **Branch offices to cloud**: If you have branch offices that need to access applications hosted in AWS or Azure, using Site2Cloud is the most economical way to build a secure tunnel. You can use your existing Internet connection and not have to pay extra to SD-WAN vendors to go through their cloud. The Aviatrix Site2Cloud solution solves these problems: * **Overlapping IP addresses** When running a SaaS operation, the CIDR blocks at customer sites are not within your control. If a customer CIDR block overlaps with your operation VPC/VNet CIDR, you must find a way to NAT the address. The cloud provider native solution is not usable in this case. For solutions to solving overlapping IP addresses, see [Configuring Overlapping Networks with Customized SNAT and DNAT](https://legacy.docs.aviatrix.com/documentation/latest/network/s2c-overlapping-ip.html). * **Traffic Black Hole** When the tunnel on the primary gateway is down, VPC/VNet route entry still points to the primary gateway, it does not point to the backup gateway. * **AWS VPN Gateway Limitation** AWS VPN gateway supports 10 connections per VPC. For deployments with more than 10 sites, the native solution is not usable. * **Azure VPN Gateway Limitation** Azure VPN gateway supports only 1 VPN connection for IKEv1. If the office firewall device only supports IKEv1, the native solution does not meet this requirement. * **No Visibility** Cloud provider's VPN gateway is a black box, there is no visibility for troubleshooting. * **No Manual** Configuring and managing hundreds or thousands of IPsec tunnels manually, using traditional vendors such as Cisco ASA and CSR, is not possible. For configuration to external devices, see [Site2Cloud Configurations with External Devices](https://legacy.docs.aviatrix.com/documentation/latest/network/s2c-configuration-other-devices.html). * **Encryption Algorithm Mismatch** As a SaaS operator, you cannot control what VPN device a customer wishes to use. The VPN termination endpoint needs the flexibility to interoperate with customer equipment. The native solution does not have that flexibility. * **Too Slow to Onboard a Customer** VPN runs on UDP port 500/4500, and customers have to request that corporate firewall ports open. Is there a way to run the IPsec tunnel on TCP 443? * **Traffic Direction Problem** A SaaS service that requires traffic to be initiated from the cloud to the customer site cannot use AWS VPN gateway for this traffic pattern. A separate machine must constantly ping to keep the tunnel up. * **Downtime Problem** Some appliances force all IPsec tunnels to reset and go down when a new tunnel is being established, which affects business continuity and is not acceptable when the number of sites go beyond 10. * **Skill Problem** Not every organization has a team of CCIEs to handle the load. In addition, Aviatrix provides a simple point-and-click user interface for you to build and manage a large deployment. To learn how to set up Aviatrix Site2Cloud, see [Site2Cloud External Connection Workflow](/docs/enterprise/10.1/guides/connectivity/vpn/s2c-configuration-workflow) . ## Site2Cloud Frequently Asked Questions ### Does Site2Cloud support HA? You can enable high-availability when you configure a Site2Cloud connection. ### What are the encryption algorithms supported? | **Type** | **Value** | | - | - | | Phase 1 Authentication | SHA-1, SHA-512, SHA-384, SHA-256 | | Phase 1 DH Groups | 1, 2, 5, 14, 15, 16, 17, 18, 19, 20, 21 (20 & 21 IKEv2 Only) | | Phase 1 Encryption | AES-256-CBC, AES-256-GCM-64, AES-256-GCM-96, AES-256-GCM-128, AES-192-CBC, AES-128-CBC, AES-128-GCM-64, AES-128-GCM-96, AES-128-GCM-128, 3DES | | Phase 2 Authentication | HMAC-SHA-1, HMAC-SHA-512, HMAC-SHA-384, HMAC-SHA-256, NO-AUTH | | Phase 2 DH Groups | 1, 2, 5, 14, 15, 16, 17, 18, 19, 20, 21 (20 & 21 IKEv2 Only) | | Phase 2 Encryption | AES-128-CBC, AES-192-CBC, AES-256-CBC, AES-256-GCM-64, AES-256-GCM-96, AES-256-GCM-128, AES-128-GCM-64, AES-128-GCM-96, AES-128-GCM-128, 3DES, NULL-ENCR | > [!NOTE] > > Site2Cloud supports AES-256-GCM and DH21 that comply with the NIST encryption > standards. ### Is IKEv2 supported? IKEv1 and IKEv2 are supported. If you already have a Transit to External Device connection using IKEv1, you can create another one using IKEv2. For IKEv2, you need to create the first Transit to External Device connection with IKEv2 enabled. If your current Transit gateway already has a connection using IKEv1 either is created by attaching the Spoke Gateway or is built in MultiCloud Transit > Attach/Detach tab, you need to delete it first before creating the Transit to External Device connection with IKEv2. For more information, see [About External Connection Settings](/docs/enterprise/10.1/guides/connectivity/gateways/external-connection-settings) . ### How frequently are keys rotated? Re-key for IKE phase 1 is every 8 hours. Re-key for IKE phase 2 is every hour. ### How to troubleshoot Site2Cloud connection with IKEv2? Refer to [Troubleshooting IPsec VPN connection with IKEv2](/docs/enterprise/10.1/guides/connectivity/vpn/troubleshooting-ipsec-vpn-ikev2) . ### Can you configure a Site2Cloud connection using the same public IP address for the remote gateway and the remote subnet? In a Site2Cloud connection, the same IP address in the remote gateway peer and the remote subnet is supported. This is useful when configuring a Site2Cloud connection to a third-party environment where only one public IP is exposed. This feature is supported only for policy-based unmapped Site2Cloud connections in AWS and GCP for standalone gateways, not ActiveMesh gateways. # Smart Gateways Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/components/connectivity/smart-gateways-overview An overview of Smart Gateways, how it is built, how it works, its key capabilities, and when to use it. > [!WARNING] > > Smart Gateways is a **Preview** feature in Controller 10.1. ## Overview Smart Gateways moves overlay route resolution from the centralized Controller to the gateways themselves. Each gateway can respond to tunnel and peer failures locally without waiting for a Controller roundtrip, and continues to update its forwarding table even if communication with the Controller is lost. Use Smart Gateways when your fabric requires faster failure convergence or continued routing survivability when the connection to the Controller is lost. ## Design ### Core Design In the centralized model, the Controller computes all overlay routes and pushes them to every gateway. When a tunnel or gateway fails, the Controller must detect the event, recompute routes, and distribute updates. During that window, traffic can blackhole. If the Controller is unreachable, gateways are stuck with stale routes indefinitely. Smart Gateways solves this by giving each gateway its own view of peer reachability. Each gateway tracks which peers are up through a BGP Labeled Unicast (BGP-LU) underlay mesh. The Controller still defines overlay route policy, but gateways no longer depend on it for real-time failure response. ### How It Works Smart Gateways adds two layers on top of the existing Aviatrix data plane: **Underlay Mesh.** Each gateway forms BGP-LU sessions with its peers. The mesh continuously tracks which peer gateways are reachable. Sessions use a 5-second keepalive and a 15-second hold timer. **Route Resolver.** Each gateway receives overlay route mappings from the Controller and cross-references them against underlay reachability. When a peer becomes unreachable, the gateway removes affected next-hops from its forwarding table locally. The gateway also withdraws BGP advertisements for destinations that no longer have any viable overlay path (FRR withdrawal), signaling peer gateways to re-converge without waiting for the Controller. When the Route Resolver is active, it is the sole component that writes routes to the gateway forwarding table. ### Operation Model How quickly the gateway converges depends on the failure type: | Failure type | What happens | Expected timeline | | - | - | - | | **Active tunnel link down** | Once the tunnel is detected as down, the gateway fails over to the HA path automatically. No Route Resolver involvement needed. | Sub-second after detection | | **Gateway or peer failure** | The BGP-LU hold timer expires and the Route Resolver removes the unreachable next-hops from the forwarding table. | Approximately 20 seconds on large fabrics (15-second hold timer plus processing) | > [!NOTE] > > In cloud environments, there is no physical link-down event. Tunnel failure is > detected by mechanisms such as Dead Peer Detection (DPD) or BFD, which adds > convergence time before the sub-second kernel failover begins. Do not assume > zero-loss failover. When both the active and HA paths for a destination are unreachable, the Route Resolver selects an alternate path through remaining transit gateways so that traffic continues to flow during multi-failure scenarios. ## Key Features ### Gateway-Local Convergence Each gateway detects failures through the BGP-LU underlay mesh and updates its own forwarding table without waiting for the Controller. * Removes unreachable next-hops on failure detection * Withdraws BGP advertisements for unreachable destinations (FRR withdrawal), causing peer gateways to stop forwarding traffic for those CIDRs to this gateway * Restores routes automatically when peers recover ### Progressive Enablement Smart Gateways uses a three-flag hierarchy that you enable incrementally. Each flag builds on the previous one, and you can verify the fabric at each stage before proceeding. | Flag | Scope | What it enables | Depends on | | - | - | - | - | | **Smart Gateways** (Flag 1) | Controller-wide | Master switch for the entire feature. Disabling forces Flags 2 and 3 off everywhere. | None | | **Underlay Mesh Connectivity** (Flag 2) | Per gateway group | BGP-LU sessions between gateways. Gateways begin tracking peer reachability. | Flag 1 enabled | | **Route Resolver** (Flag 3) | Per gateway group | Gateway-local route resolution. The gateway takes over forwarding table updates from the Controller. | Flag 1 enabled **and** Flag 2 enabled on **all** gateway groups | The flags enforce a strict dependency chain: disabling Flag 1 forces Flags 2 and 3 off on all gateway groups. Disabling Flag 2 forces Flag 3 off for that group. Enabling Flag 3 requires Flag 2 to be enabled on all groups in the fabric, not only the target group. > [!NOTE] > > For enablement steps, see the Smart Gateways section in the CoPilot UI > reference. ### Alternate Path Selection When both the active and HA paths for a destination fail, the Route Resolver selects an alternate path through remaining transit gateways, providing continued reachability without Controller intervention. ### Network Segmentation Support When Network Segmentation is enabled, the Route Resolver maintains separate route tables per network domain for each gateway group. ### Route Type Bypass Routes learned through BGP over IPsec, BGP over GRE, BGP over LAN, and Static External connections are not subject to Route Resolver pruning. These connection types have next-hops outside the BGP-LU underlay mesh, so their routes are installed unchanged. ### Supported Gateway Types Smart Gateways in 10.1 Preview supports transit gateways, BGP-enabled spoke gateways, Edge as Spoke (EAS), and Edge as Transit (EAT). ## When to Use ### Large Multi-Region Fabrics In fabrics that span many regions, the Controller roundtrip for route recomputation after a failure can take tens of seconds. Smart Gateways reduces this to local convergence bounded by BGP-LU timers. ### Controller-Resilient Routing If the Controller becomes temporarily unreachable (for example, during a network partition), gateways with Smart Gateways enabled continue to respond to failures using their local BGP-LU state. Without Smart Gateways, gateways cannot update their routing tables until the Controller reconnects. ### Fabrics with Strict Convergence Requirements For deployments where the time between a failure and traffic recovery must be minimized, Smart Gateways provides sub-second kernel failover once a local tunnel failure is detected, and approximately 20-second convergence for peer failures detected by BGP-LU. ## Best Practices * **Enable flags incrementally.** Start with Flag 1 (controller-wide), then enable Flag 2 on one gateway group and verify BGP-LU sessions are established before rolling out to additional groups. Enable Flag 3 only after Flag 2 is active on all groups. * **Use appropriately sized instances for spoke gateways.** The Route Resolver adds memory overhead. Small-sized spoke gateways may experience memory pressure. Use medium or larger instance sizes where Smart Gateways is active. * **Verify underlay mesh health before enabling the Route Resolver.** With Flag 2 enabled, confirm that all expected BGP-LU sessions are established and that peer counts match your expected topology before enabling Flag 3. * **Plan for the emergency disable path.** Disable Flag 1 to force all flags off everywhere. To re-enable, work top-down: Flag 1 first, then Flag 2 on each group, then Flag 3. * **Monitor with CLI and API in 10.1.** The CoPilot Smart Gateway Health page requires Controller 10.2 or later. In 10.1, use CLI and API commands for diagnostics. ## Limitations * **Preview only.** GA is targeted for a future release. * **No CoPilot diagnostics page in 10.1.** The Smart Gateway Health page requires Controller 10.2 or later. * **Terraform support is partial.** The `aviatrix_config_feature` resource can toggle the fabric-wide flag (Flag 1) only. Per-group flags (Flags 2 and 3) require direct API calls. * **Pruning suspension is not observable.** During underlay convergence events (such as a BGP-LU peer restart), the Route Resolver temporarily suspends next-hop pruning as a safety measure. There is no metric or log indicating when pruning is suspended. * **Convergence is not uniformly sub-second.** Sub-second kernel failover applies only after a local tunnel failure is detected. Detection itself depends on DPD or BFD timers and adds convergence time. Peer failures detected by BGP-LU take approximately 20 seconds on large fabrics. ## Related Features | Feature | Relationship | | - | - | | Less-Disruptive Gateway Resize and Upgrade (Preview) | Make Before Break uses the drain/undrain mechanism that Smart Gateways also builds on. Both are independent Preview features in 10.1. | | [ActiveMesh](/docs/enterprise/10.1/concepts-architectures/components/connectivity/activemesh-about) | ActiveMesh provides active-active forwarding across primary and HA gateways. Smart Gateways adds gateway-local convergence on top of the ActiveMesh data plane. | # VRF Multi-Tenancy Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/components/connectivity/vrf-overview How Aviatrix VRF isolates traffic between tenants and business units with overlapping IP addresses, and propagates that segment identity end-to-end across transit peerings. VRF is an Aviatrix segmentation construct that gives each tenant, business unit, or customer its own routing context, including support for identical or overlapping IP address ranges between segments. It extends the first-hop VRF capability introduced in Controller 9.0 end-to-end across transit peerings, so a segment retains its identity from an Edge or Spoke attachment, across one or more Transit Gateways, to its destination. [Network Domains](/docs/enterprise/10.1/concepts-architectures/components/security/transit-segmentation-faq#what-is-a-network-domain-in-multicloud-transit) and [Multicloud Transit Segmentation](/docs/enterprise/10.1/concepts-architectures/components/security/transit-segmentation-faq) already isolate traffic using cloud-native VPC/VNet route tables, but both assume that a prefix advertised into the transit fabric is unique. VRF removes that assumption: use it when segments must be isolated and one or more of them use IP address ranges that overlap with another segment. ## Design ### Core Design VRF removes the implicit constraint of Network Domains and Transit Segmentation. It gives each segment its own routing context: a separate forwarding table on the gateway itself, in addition to the cloud-native VPC/VNet route table. As a result, the same prefix can exist in two segments at once, and each instance resolves independently in its own table. This is the same technical problem that first-hop VRF solves in traditional on-premises networking, now applied to Aviatrix Gateways. ### How It Works VRF is delivered in two stages that build on each other: * **First-hop VRF** (Controller 9.0) scoped this per-segment forwarding table to a single gateway hop, for Site2Cloud connections terminating on a Transit Gateway. Each domain got its own routing context at that one gateway, which was enough to resolve overlapping CIDRs arriving from different Site2Cloud connections, but the isolation did not extend past that gateway. * **End-to-end segment identity propagation across transit peerings** (Controller 10.1) extends that same per-segment forwarding table across a transit-to-transit or Edge/Spoke-to-transit peering. When traffic for a segment crosses a peering, the receiving Transit Gateway has to know which segment the packet belongs to, so it can select the matching per-segment table on its own side. The segment identity has to survive the trip across the peering, not only the first hop. Segments with the same name on both sides of a peering are treated as the same segment (name-based mapping); the peering itself is what has to be explicitly VRF-aware for this propagation to take effect. > [!NOTE] > > Enable the **Support for VRF** toggle at **Networking > Network > Segmentation > Settings** in CoPilot. Each Network Domain gets its own VRF > automatically; there is no manual VRF ID field to configure. ### Operation Model ![VRF Operation Model](https://mintcdn.com/aviatrix-14b37c43/Cv6uug5wRtgMFTqU/images/concepts-architectures/components/connectivity/vrf-operation-model-1.png?fit=max\&auto=format\&n=Cv6uug5wRtgMFTqU\&q=85\&s=849f8a89802dc9f145c0a1341717c501) 1. A packet enters a VRF-enabled gateway and is marked with its segment identity based on the connection or attachment it arrived on (for example, a specific Site2Cloud connection, BGP peer, or spoke attachment). 2. The gateway looks up the packet's destination in the forwarding table that belongs to that segment, not the gateway's shared/main table. 3. If the destination is reachable through a VRF-aware transit peering, the segment identity travels with the packet across that peering. 4. The receiving Transit Gateway reads the segment identity, matches it by name to its own local segment, and continues forwarding using that segment's own table, including out to a third-hop Spoke or Edge attachment. 5. Routes learned dynamically (for example, over BGP) are installed into, and withdrawn from, the matching per-segment table rather than the shared table, so route churn in one segment does not affect another. Cross-segment connectivity does not happen unless you explicitly allow it. A connection policy between two segments (the same policy construct used by Network Domains) permits bidirectional traffic between them; removing the policy restores isolation. ## Key Features ### Per-Segment Route Isolation Each segment gets its own forwarding table, independent of any other segment's table on the same gateway. * Identical or overlapping CIDRs in different segments do not collide, because each segment resolves destinations in its own table. * A segment's cloud-native VPC/VNet route table only ever contains that segment's own routes. It never carries another segment's overlapping CIDRs. * Dynamic routes (BGP-learned) install into and withdraw from the correct per-segment table automatically. ### Segment Identity Propagation Across Transit Peerings This release adds a new capability: a segment's isolation extends beyond the gateway it entered on. * A VRF-aware transit-to-transit or Edge/Spoke-to-transit peering preserves segment identity across the peering. * Segments are mapped between peered gateways by name: a segment named the same way on both sides of the peering is treated as one continuous segment. * Isolation holds even when the two sides of the peering are in different clouds or regions. ### Controlled Cross-Segment Connectivity Segments are isolated by default; you decide when two segments should talk to each other. * A connection policy between two segments allows bidirectional traffic between them, using the same policy model as Network Domains. * Removing a connection policy restores isolation without requiring any other configuration change. ### Compatibility with Encryption and Path Features Segment isolation is validated to keep working alongside Aviatrix's performance and resiliency features on the same peering, not only on a plain tunnel. * Segment identity propagation is validated against High Performance Encryption, which spreads a single peering's traffic across multiple encrypted subtunnels. * Segment identity propagation is validated against ActiveMesh multi-path tunnels. * IPv6 traffic is supported within a VRF-enabled segment. ## When to Use ### Consolidating Multiple Tenants Onto Shared Transit Infrastructure You operate a shared transit backbone, as a service provider or as an internal platform team serving multiple internal customers. Your tenants' networks were never coordinated with each other, so some of them use identical or overlapping private address ranges. Use VRF to isolate each tenant's routing without renumbering any of them, while still using one shared set of Transit Gateways rather than a separate Controller or Transit deployment per tenant. ### Segmenting Business Units After a Merger or Acquisition Two business units, divisions, or recently acquired subsidiaries need to be kept isolated from each other on the network, but at least one of their address ranges overlaps with the other's. Network Domains alone cannot represent that overlap in a shared transit route table; VRF can, because each business unit resolves its own destinations in its own table. ### Extending an Existing On-Premises VRF-Lite Design Into the Cloud Your on-premises network already segments traffic per business unit or customer using VRF-lite or a similar per-VRF WAN design. You want the cloud side of your Aviatrix deployment to preserve that same per-segment boundary once it reaches the cloud, rather than collapsing everything into one flat routing domain. First-hop VRF already lets a Transit Gateway preserve segment boundaries for Site2Cloud connections; this feature extends that boundary further, across transit peerings, toward the rest of your Aviatrix fabric. ## Best Practices * **Name segments consistently across every transit you peer.** Segment mapping across a VRF-aware peering is name-based. A typo or inconsistent naming convention between two transits results in two segments that fail to map to each other rather than a visible error. * **Reserve non-overlapping CIDRs for anything shared across every segment.** Shared services that every segment needs to reach are simplest to design with a non-overlapping address range, since a shared destination that itself overlaps with a segment's own range is harder to route unambiguously. * **Treat connection policies between segments as a deliberate exception, not a default.** Segments are isolated by default specifically so that overlapping or sensitive address space in one segment is never reachable from another segment without an explicit decision. * **Plan for a bounded number of segments, and confirm the actual limit for your target release before committing to a design.** Design for no more than 10 to 20 segments per Controller. This range reflects current design guidance, not a load-tested maximum for this release; confirm the published limit for your target release before finalizing a design. * **VRF is supported on AWS, Azure, and GCP.** Confirm your target segments run in one of these clouds before you design around VRF. * **Adopt first-hop VRF before relying on cross-peering propagation.** Since end-to-end propagation extends the same per-segment table that first-hop VRF establishes at the gateway, confirm your Site2Cloud or BGP-facing gateways are already using first-hop VRF correctly before layering transit peering propagation on top of it. ## Comparison with Network Domains and Transit Segmentation VRF does not replace Network Domains or Transit Segmentation. It addresses the one case they cannot: segments whose address space overlaps. | | Network Domains / Transit Segmentation | VRF | | - | - | - | | Isolation enforced at | Cloud-native VPC/VNet route tables and Aviatrix-programmed transit route tables | A separate forwarding table per segment on the gateway itself, in addition to the cloud-native route table | | Overlapping CIDRs across segments | Not supported: a shared route table cannot hold two next hops for the same prefix | Supported: each segment resolves the prefix independently, in its own table | | Cross-segment connectivity | Connection policy | Connection policy (same model) | | Scope of isolation | Within the Transit Segmentation topology (Spoke and Transit VPC/VNets) | A segment scoped at the gateway, extendable end-to-end across VRF-aware transit peerings | If none of your segments need overlapping address space, Network Domains and Transit Segmentation alone are sufficient and simpler to reason about. Reach for VRF specifically when overlapping IP addresses across segments are a requirement, not a preference. # Controller Overview Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/components/controlplane/controller-overview Overview of Aviatrix Controller capabilities, deployment model, and related product guides. Aviatrix Controller is the control plane of the Aviatrix platform. It centrally orchestrates gateway provisioning, routing policy, and cloud account management across your multicloud network. The Controller provides a full REST API and a certified Terraform provider that enable network and platform engineering teams to automate the deployment and lifecycle management of Aviatrix Transit Gateways, Spoke Gateways, and Edge Gateways. It maintains network state, drives gateway configuration, and distributes policy to the data plane across AWS, Azure, Google Cloud, and Oracle Cloud Infrastructure (OCI). All gateways provisioned by the Controller remain within your own cloud accounts and network perimeter. [Aviatrix CoPilot](/docs/enterprise/10.1/concepts-architectures/components/controlplane/copilot-overview) connects to the Controller to deliver operational visibility and day-2 management capabilities across your multicloud environment. The Controller handles gateway provisioning and policy control; [CoPilot](/docs/enterprise/10.1/concepts-architectures/components/controlplane/copilot-overview) surfaces the resulting network telemetry, topology, and health data. To use CoPilot, you must have an operational Aviatrix Controller. ## Controller User Interface Aviatrix CoPilot is the primary web-based management interface for the Aviatrix platform. Starting in Controller 10.1, the Controller also ships with a Controller UI for administrative break-glass access when CoPilot is unavailable, such as before CoPilot is deployed or during CoPilot downtime. For new deployments, use [Launch Aviatrix](https://launch.aviatrix.com) or Terraform to deploy Controller and CoPilot together rather than the Controller UI. The Controller UI supports the following administrative tasks: * Deploying a new CoPilot instance * Backing up and restoring the Controller * Opening a remote support session * Resetting your password For Controller functionality not yet available in CoPilot, click **Legacy Controller UI** to open the previous Controller UI at the `/legacy/` path. > [!NOTE] > > Aviatrix recommends performing configuration and monitoring tasks in CoPilot. > Use the legacy Controller UI only when CoPilot is unavailable or does not yet > support the task you need. ## Controller Deployment Model The Controller is deployed as a virtual appliance and is available on AWS, Azure, GCP, and OCI marketplaces. For information about base image releases that become available in the marketplaces, see the [Aviatrix Controller Image Release Notes](/docs/enterprise/10.1/reference/release-notices/release-notes/controller/controller-software-release-notes) . For AWS and Azure deployments, the Launch Aviatrix guided experience at [launch.aviatrix.com](https://launch.aviatrix.com) automates Controller and CoPilot provisioning, IAM configuration, security group setup, and initial platform integration. For Google Cloud and OCI, Terraform is the recommended deployment method. Aviatrix publishes certified modules for all supported cloud providers at the [Terraform Registry](https://registry.terraform.io/namespaces/terraform-aviatrix-modules). For information about sizing requirements, high availability, and deployment prerequisites, see [Planning Your Controller Deployment](/docs/enterprise/10.1/guides/controlplane/planning-deployment) . ## Controller Licensing and Trials A valid license (customer ID) is required to activate Aviatrix Controller. You obtain the customer ID by subscribing to the Aviatrix Enterprise Contract offer in your cloud provider marketplace. See [Aviatrix Licensing](/docs/enterprise/10.1/guides/controlplane/aviatrix-licensing) for information about the required marketplace offers and supported cloud providers. If you are interested in a trial of Aviatrix Controller, contact Aviatrix through the [online form](https://aviatrix.com/contact), or contact your Aviatrix Sales representative. ## Controller Product Guides Controller product guides are as follows: | **Controller Guide Name** | **Description** | | - | - | | [AWS Getting Started Guide](/docs/enterprise/10.1/guides/controlplane/getting-started-guide-aws) | The complete deployment walkthrough for Controller and CoPilot on **AWS** using the Launch Aviatrix guided experience. | | [Azure Getting Started Guide](/docs/enterprise/10.1/guides/controlplane/getting-started-guide-azure) | The complete deployment walkthrough for Controller and CoPilot on **Azure** using the Launch Aviatrix guided experience. | | [Planning Your Controller Deployment](/docs/enterprise/10.1/guides/controlplane/planning-deployment) | Prerequisites, sizing requirements, and deployment options for **Google Cloud** and **OCI**. | | [Controller HA on AWS](/docs/enterprise/10.1/guides/controlplane/controller-ha-aws) | Instructions for configuring Controller High Availability on AWS to protect against instance failure. | | [Aviatrix Licensing](/docs/enterprise/10.1/guides/controlplane/aviatrix-licensing) | Information about required marketplace subscriptions and the customer ID activation flow. | | [Controller Release Notes](/docs/enterprise/10.1/reference/release-notices/release-notes/controller/controller-software-release-notes) | Information about Controller software releases and Controller base image releases. | # CoPilot Features Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/components/controlplane/copilot-features This section provides an overview of Aviatrix CoPilot key features. This section provides an overview of Aviatrix CoPilot key features. ## CoPilot Feature Summary The table below highlights some of the features in CoPilot and provides links to associated documentation. | **CoPilot Key Feature** | **Description** | | - | - | | **Auto Right-Sizing** | Optimize your Aviatrix Gateways based on traffic demands, ensuring performance and reducing costs with resizing recommendations. [About Auto Right-Sizing](/docs/enterprise/10.1/concepts-architectures/architecture/sizing/auto-right-sizing-about) | | **Network Health Monitor (Dashboard)** | Real-time cloud network resource inventory and status. [Managed Resources Dashboard](/docs/enterprise/10.1/reference/ui/dashboard/index#viewing-resource-dashboard) | | **Dynamic Topology Map** | Accurate, multicloud network topology, layout control, and full search of managed resources. [Gaining Visibility into Your Network Topology](/docs/enterprise/10.1/guides/monitoring/topology-visibility) | | **FlowIQ** | Detailed application traffic flow analysis, global heat map and trends, and filter traffic by native Cloud tags. [Traffic Visibility](/docs/enterprise/10.1/reference/ui/monitor/flowiq#view-flowiq-overview) | | **Cloud Routes** | Detailed searchable routing tables across cloud providers. [Troubleshooting Routing Issues in Your Aviatrix-Managed Network](/docs/enterprise/10.1/guides/troubleshooting/diagnose-cloud-routes) | | **Notifications** | Alerts on the status of managed cloud resources across clouds with policies for triggering alerts based on telemetry data and other metrics. [Notifications (Alerts) About Network Events](/docs/enterprise/10.1/reference/ui/monitor/notifications) | | **AppIQ** | Comprehensive reports to analyze network path between any two cloud instances. [Diagnose Instance To Instance](/docs/enterprise/10.1/reference/ui/diagnostics/diagnostic-tools#run-connectivity-test) | | **Inventory Reports** | Create detailed reports showing your inventory of managed resources in a single cloud or across all clouds in your multicloud network. [Network Reports](https://legacy.docs.aviatrix.com/documentation/latest/monitoring/network-reports.html) | | **Network Behavior Analytics (Anomalies)** | Alert on anomalies detected that deviate from the custom routine traffic patterns of your own cloud workloads. [Detecting Network Anomalies using Network Behavior Analytics](/docs/enterprise/10.1/guides/security/network-anomalies) | | **Network Segmentation** | Network segmentation for inter-VPC/VNet connectivity (reachability) of your Aviatrix multicloud transit. [Network Segmentation Secured](/docs/enterprise/10.1/guides/security/network-segmentation-secured) | | **Distributed Cloud Firewall** | Embedded security to support desired application reachability. [Secure Networking and Distributed Cloud Firewall](/docs/enterprise/10.1/concepts-architectures/components/dcf/dcf-overview) | | **Groups** | Configure SmartGroups, ExternalGroups, and WebGroups to be used in Distributed Cloud Firewall rules. [About Groups](/docs/enterprise/10.1/concepts-architectures/components/dcf/hostname-smartgroups-about) | | **Egress Control** | Use [Distributed Cloud Firewall](/docs/enterprise/10.1/concepts-architectures/components/dcf/dcf-overview) with [WebGroups](/docs/enterprise/10.1/concepts-architectures/components/dcf/webgroups-about) to configure your Egress traffic. | | **Audit** | Audit history of account users that made changes to your Aviatrix managed networks. [Changes By User Audit](/docs/enterprise/10.1/reference/ui/administration/audit#view-audit-logs) | | For new features introduced in each CoPilot release, see our what's new page. | [What's New in CoPilot](/docs/enterprise/10.1/reference/release-notices/what-is-new/what-is-new-copilot) | ## Legacy Features There are some features that are no longer available in CoPilot. You can see them in the CoPilot console only if you configured them prior to Controller version 7.2.4820. These legacy features include the following: * CostIQ * ThreatIQ * Geoblocking ## Related Topics * [Aviatrix Feature Availability by Controller Version](/docs/enterprise/10.1/reference/release-notices/what-is-new/what-is-new-controller) # CoPilot Overview Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/components/controlplane/copilot-overview Overview of Aviatrix CoPilot capabilities, deployment model, and related product guides. Aviatrix CoPilot is the primary management interface for the Aviatrix platform, extending visibility into complex, multicloud networks managed by Aviatrix Controller. Aviatrix CoPilot provides an operational view of your multicloud network. CoPilot helps enterprises implement cloud networking that embeds security into their networks, with monitoring and diagnostic tools that support application availability. CoPilot offers visibility into cloud networking costs, showing how network resources are consumed by different business units or cost centers for billing purposes. Enterprise IT teams use CoPilot's dynamic topology mapping to maintain an accurate topology of their multicloud networks, and FlowIQ to analyze network traffic flows through heat maps and time series trend charts that pinpoint and troubleshoot traffic anomalies. CoPilot uses network and security services delivered by Aviatrix's multicloud network platform to provide enterprise cloud network operations teams with familiar day-2 operational capabilities, such as packet capture, trace route, and ping, built for multicloud network environments. ## CoPilot Deployment Model CoPilot is deployed as an all-in-one virtual appliance and is available on AWS, Azure, GCP, and OCI marketplaces. For information about base image releases that become available in the marketplaces, see the [Aviatrix CoPilot Image Release Notes](/docs/enterprise/10.1/reference/release-notices/release-notes/copilot-images/index) . CoPilot works in tandem with Aviatrix Controller; to use CoPilot, you must have an operational Aviatrix Controller. In the Controller, you set up user accounts and the CoPilot service account with required permissions for CoPilot usage. See [CoPilot Accounts and Users](https://legacy.docs.aviatrix.com/documentation/latest/platform-administration/accounts-and-users/index.html). Aviatrix Controller and CoPilot are not required to be collocated. It is possible to run them in separate VPC/VNets or separate cloud providers (in multicloud environments). If CoPilot is located in the same VPC/VNet as your Controller, specifying a private IP can increase operational bandwidth and potentially save on cost. To deploy Controller and CoPilot, you must subscribe to an Aviatrix offer in a cloud provider marketplace. See the discussion about planning your CoPilot deployment in [Planning Your CoPilot Deployment](/docs/enterprise/10.1/guides/controlplane/copilot-planning-deployment) for information about Aviatrix offers. For information about deploying Aviatrix CoPilot, including deployment methods and virtual machine (instance) minimum requirements, see [CoPilot Planning Deployment](/docs/enterprise/10.1/guides/controlplane/copilot-planning-deployment) . ## New and Existing Deployments For a new Aviatrix deployment, the [Launch Aviatrix](https://launch.aviatrix.com) guided experience deploys Controller and CoPilot together and directs you to the CoPilot login page when deployment completes. If you are upgrading an existing Controller that does not yet have CoPilot deployed, the upgrade process requires you to deploy CoPilot before the upgrade can proceed. If you already have CoPilot deployed, your existing CoPilot instance continues to operate and connects to the upgraded Controller. If your associated CoPilot instance does not meet the minimum supported size, Aviatrix Controller displays the following warning: > [!WARNING] > > CoPilot Instance size is smaller than the minimum supported size. Upsize your > CoPilot. Read more about CoPilot Size Support in Aviatrix Documentation. [CoPilot Requirements](/docs/enterprise/10.1/reference/controlplane/copilot-platform-requirements#instance-virtual-machine-system-requirements-for-copilot) describes the minimum supported instance size for CoPilot. ## CoPilot Licensing and Trials If you are interested in a trial of CoPilot, contact Aviatrix through the [online form](https://aviatrix.com/contact), or contact your Aviatrix Sales representative. ## CoPilot Product Guides CoPilot product guides are as follows: | **CoPilot Guide Name** | **Description** | | - | - | | [CoPilot Overview](/docs/enterprise/10.1/concepts-architectures/components/controlplane/copilot-overview) | An overview of CoPilot as it relates to other components in the Aviatrix platform and a summary of key features. | | [Getting Started](https://legacy.docs.aviatrix.com/documentation/latest/getting-started/index.html) | Information about deploying CoPilot, logging in, setting user interface preferences, and tasks that are common across pages, editors, and tables in the user interface. | | [Accounts and Users](https://legacy.docs.aviatrix.com/documentation/latest/platform-administration/accounts-and-users/index.html) | Information about user account permissions that are required for performing different actions in the user interface. | | [Building Your Network](https://legacy.docs.aviatrix.com/documentation/latest/network/index.html) | Information about deploying Aviatrix gateways and constructs onto your network environments via the CoPilot user interface. | | [Networking and Security](https://legacy.docs.aviatrix.com/documentation/latest/security/index.html) | Information about implementing Aviatrix secured networking and enhanced security services via the CoPilot user interface. | | [Monitoring and Troubleshooting](https://legacy.docs.aviatrix.com/documentation/latest/monitoring/index.html) | Information about monitoring your Aviatrix-managed cloud resources, looking out for security threats, diagnosing network problems, and monitoring the performance of Aviatrix gateways and resources. | | [Platform Administration](https://legacy.docs.aviatrix.com/documentation/latest/platform-administration/copilot/index.html) | Key topics and instructions for CoPilot administration tasks, including backing up and restoring CoPilot configuration and index data, application tuning, and lifecycle management tasks. | | [CoPilot Release Notes](/docs/enterprise/10.1/reference/release-notices/release-notes/copilot/index) | Information about CoPilot software releases and CoPilot base image releases. | For a summary of CoPilot key features, see [CoPilot Features](/docs/enterprise/10.1/concepts-architectures/components/controlplane/copilot-features) . # Distributed Cloud Firewall Overview Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/components/dcf/dcf-overview Distributed Cloud Firewall (DCF) uses micro-segmentation to provide granular network security rules for distributed applications in the Cloud. Distributed Cloud Firewall enables network policy enforcement between SmartGroups, WebGroups, and ExternalGroups you define in a single cloud or across... Distributed Cloud Firewall (DCF) uses micro-segmentation to provide granular network security rules for distributed applications in the Cloud. Distributed Cloud Firewall enables network policy enforcement between [SmartGroups, WebGroups, and ExternalGroups](/docs/enterprise/10.1/concepts-architectures/components/dcf/smartgroups-about) you define in a single cloud or across multiple clouds. ![Micro-Seg Topology](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/components/dcf/microseg-topology.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=b4025314ff3c20a27b240570dbf3a89b) Currently, DCF contains both Preview and GA features. See [Distributed Cloud Firewall Supported Capabilities by Controller Version](/docs/enterprise/10.1/reference/dcf/dcf-capabilities) for more information. > [!NOTE] > > Aviatrix strongly recommends using DCF and its integration with ExternalGroups > to monitor for threats and block certain countries. If you are an existing > user of ThreatIQ and/or Geoblocking (prior to Controller version 7.2.4820), > use these > [migration recommendations](/docs/enterprise/10.1/guides/security/threatiq-geo-dcf-migration-guidelines) > to start using DCF with ExternalGroups. You cannot use ThreatIQ and/or > Geoblocking in conjunction with DCF and ExternalGroups. ## Supported Cloud Providers and Gateways Distributed Cloud Firewall is supported for the following clouds: * AWS, AWS GovCloud, AWS China * Azure, Azure Government, Azure in China * GCP, Google for Government * OCI, Oracle Government Cloud, Oracle China The following gateway types are supported: * Spokes attached to a Transit Gateway * Spokes detached from a Transit Gateway * Public Subnet Filtering Gateways ([enable PSF Gateways with DCF here](/docs/enterprise/10.1/guides/security/dcf/dcf-settings#psf-gateways-enforce)) * External connections (Site2Cloud) ([enable External Connections with DCF here](/docs/enterprise/10.1/guides/security/dcf/dcf-settings#external-connections-enforce)): * Terminating on a Spoke Gateway * Terminating on a Transit Gateway ([L4](/docs/enterprise/10.1/reference/general/general-glossary#firewall) only) * [Edge as Spoke Gateway](/docs/enterprise/10.1/concepts-architectures/components/edge-gateways/edge-spoke-overview) ( [L4 only](/docs/enterprise/10.1/reference/general/general-glossary#firewall) ; non-CSP tag) > [!NOTE] > > DCF rules will not be applied to traffic originating or terminating on Aviatrix gateways. > > If you want to use External Connections in your DCF rules, ensure that the Spoke > or Transit gateway that the DCF rule terminates on is upgraded to 7.2.4820. DCF > rules that use External Connections will not be evaluated or enforced on 7.1 > gateways. ## Minimum Spoke Gateway Sizing * 3583MB required for L7 with IDS * 3583MB required for IDS and TLS Decryption * 8192MB (8 GB) recommended for L7 with IPS Minimum Spoke gateway sizing is provided in the following table. | Cloud | L4 only | L4 with [External Groups](/docs/enterprise/10.1/reference/general/aviatrix-glossary#external-group) | L7 with TLS Decryption only | L7 with TLS Decryption and IDS | L7 with IPS | | - | - | - | - | - | - | | AWS | t3.micro or equivalent (non-production only) | t3.medium or larger | t3.small or equivalent | t3.medium or larger | t3.large or larger | | Azure | Standard\_B1ms | Standard\_B2s | Standard\_B1ms | Standard\_B2ms | Standard\_B2ms or larger | | GCP | n1-standard-1 | n1-standard-2 | n1-standard-1 | n1-standard-2 | n1-standard-2 or larger | | OCI | VM.Standard.2.2 | VM.Standard.2.2 | VM.Standard.2.2 | VM.Standard.2.2 | VM.Standard.2.2 | > [!NOTE] > > When IPS is enabled, spoke gateways require a minimum of 8 GB memory for > production workloads. Gateways with 4 GB memory may experience connection > timeouts and increased failure rates under sustained IPS inspection. See > > [Intrusion Prevention System (IPS)](/docs/enterprise/10.1/guides/security/dcf/ips) > > > > for more information. DCF can be used to control Egress traffic by [enabling Source NAT](https://legacy.docs.aviatrix.com/documentation/latest/network/snat-dnat-settings.html?expand=true) on the gateway configuration, and creating policies with [WebGroups](/docs/enterprise/10.1/concepts-architectures/components/dcf/webgroups-about) . > [!NOTE] > > As of Controller 7.1, DCF with WebGroups is the recommended method for configuring and implementing Egress Security. > > This document describes Egress functionality available in the Aviatrix > Controller in Controller 7.1 and later. For information on configuring the > legacy Egress FQDN solution, click > > [Egress FQDN Filtering (Legacy)](/docs/enterprise/10.1/guides/security/egress/egress-legacy-overview) > . > > DCF is in Preview mode for earlier Controller versions. See > > [Aviatrix Feature Availability by Controller Version](/docs/enterprise/10.1/reference/dcf/dcf-capabilities) > for more information. ## Use Cases Use cases where you might implement DCF are: * Workload isolation: in a typical tiered application, you may want to isolate tiers that do not require access to each other. For example, in a Shopping Cart application, there could be workloads for product inventory, billing, and a Product Logging app. Since the Shopping Cart application does not need to communicate with the Product Logging app, this traffic should be blocked. * Quarantine compromised machines: You can isolate a compromised machine by placing it in its own SmartGroup and blocking communication to that SmartGroup. ## Related Topics * [DCF Rule Details](https://legacy.docs.aviatrix.com/documentation/latest/security/dcf-rule-details.html?expand=true) * [Configuring Distributed Cloud Firewall](/docs/enterprise/10.1/reference/dcf/dcf-configuring) # Distributed Cloud Firewall Rulesets Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/components/dcf/dcf-rulesets-about Your Distributed Cloud Firewall (DCF) policies are essential for managing and securing network traffic across various parts of an organization. Your Distributed Cloud Firewall (DCF) policies are essential for managing and securing network traffic across various parts of an organization. Your DCF policies consists of multiple rulesets or other policies for easier management. You can use policies to apply a consistent security posture across multiple accounts, VPCs, or environments. Each ruleset consists of a set of DCF rules. Rulesets allow for logical grouping of rules based on function, environment, geography, application, or any user-determined type. Each ruleset has a unique priority range (typically between 1000 and 8999) and is evaluated in order of priority. It is crucial for different groups within an organization to collaborate and decide which rulesets take precedence. This collaboration ensures that the most critical rules are applied first. > [!NOTE] > > You can switch between rulesets, but it is important to save changes on the > Policies tab for a specific ruleset before switching to another. A rule can be > used in more than one ruleset, providing further flexibility in managing network > security. Rules are the most granular unit in DCF. A rule defines a specific traffic control action, such as Permit or Deny, and is based on criteria such as source/destination IPs, ports, protocols, and SmartGroups. Each rule has a priority value that determines its evaluation order within a ruleset. > [!NOTE] > > DCF rulesets are supported in Controller 8.0 or later. If you started using DCF > rulesets in Controller 8.0, two predefined rulesets were created: V1 Policy List > (editable) and Post Rules Policy List (non-editable). The former contained all > existing legacy rules (created prior to Controller 8.0) and the Greenfield Rule. > The latter contained the DefaultDenyAll rule. In Controller 8.0, two predefined > rulesets will be available: V1 Policy List (editable) and Post Rules Policy List > (non-editable). The former will contain all existing legacy rules (created prior > to Controller 8.0) and the Greenfield Rule. The latter will contain the > DefaultDenyAll rule. After monitoring or protecting VPC/VNets, rules are added > to the Egress Protection Policy List ruleset. This ruleset is created > automatically when you monitor or protect VPC/VNets, and it contains the rules > that are created as part of the monitoring or protection workflows. The legacy > DefaultDenyAll rule is moved from the Post Rules Policy List (8.0) to the Legacy > Default Deny Ruleset, which can be deleted. The Greenfield Rule will no longer > be created by default and is being replaced by the zero trust [Default Action > Rule](/docs/enterprise/10.1/guides/security/dcf/dcf-enable-feature#default-action-rule). ## Security Control Templates Aviatrix provides Security Control templates built from Aviatrix-provided threat intelligence feeds. Use a template to add threat-detection rules to your environment without writing custom rules. To explore available templates, go to **Dashboard > Security Control**, then click **Explore Recommended Security Control**. Current templates include **Common Threats**. Click **Add** on a template to apply it. Each Security Control you add starts in the **Monitored** state, which logs matching traffic without blocking it. Review the matched traffic on the Dashboard, and when you are ready to block that traffic, switch the control to **Protect**. You can switch a control back to **Monitored** at any time. > [!NOTE] > > The Dashboard's traffic details depend on **Log Enrichment**, which is in > Preview and turned off by default. Turn on **Log Enrichment** on \*\*Security > > > Distributed Cloud Firewall > Settings\*\* to see enriched traffic detail for > > a Security Control's matched traffic. A Security Control is separate from the predefined rulesets described earlier on this page (**V1 Policy List**, **Post Rules Policy List**, and **Egress Protection Policy List**), and from the Premium IPS Feed, which drops traffic based on **Drop Level** severity tiers instead of **Monitored**/**Protect** states. See [Intrusion Prevention System (IPS)](/docs/enterprise/10.1/guides/security/dcf/ips) for more information. ## Related Topics * [Create a DCF Ruleset](https://legacy.docs.aviatrix.com/documentation/latest/security/dcf-ruleset-create.html?expand=true) * [Manage DCF Rulesets](https://legacy.docs.aviatrix.com/documentation/latest/security/dcf-ruleset-manage.html?expand=true) * [Create DCF Rules](https://legacy.docs.aviatrix.com/documentation/latest/security/dcf-ruleset-create.html?expand=true) * [Configuring Distributed Cloud Firewall](/docs/enterprise/10.1/reference/dcf/dcf-configuring) # Enforcing Zero Trust with Distributed Cloud Firewall and the Default Action Rule Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/components/dcf/dcf-zero-trust Aviatrix Distributed Cloud Firewall (DCF) provides a foundational framework for implementing zero trust principles across multi-cloud environments. A key component of this framework is the Default Action Rule, which establishes a baseline security posture by controlling how traffic is handled... Aviatrix Distributed Cloud Firewall (DCF) provides a foundational framework for implementing zero trust principles across multi-cloud environments. A key component of this framework is the Default Action Rule, which establishes a baseline security posture by controlling how traffic is handled in the absence of explicit rules. ## Default Action Rule Behavior > [!NOTE] > > Starting with Controller version 8.1, the Default Action Rule is automatically > created when DCF is enabled. Legacy rules such as Greenfield and > DefaultDenyAll are no longer generated; the Default Action Rule replaces their > function. When Distributed Cloud Firewall is enabled in Aviatrix CoPilot, a Default Action Rule is automatically created in the Post Rules Policy List. This global rule governs the default behavior for traffic that does not match any user-defined rules. Administrators can configure the Default Action Rule to either *Permit* or *Deny* traffic. To align with zero trust best practices, you must set this rule to *Deny*. The rule then becomes an "implicit deny" rule that remains at the bottom of the ruleset and is enforced globally. > [!TIP] > > Before setting the Default Action Rule to Deny, or deleting the legacy DefaultDenyAll Rule if it exists, you should define Permit rules for trusted SmartGroups that specify allowed traffic flows. This ensures that critical services remain accessible and avoids unintended disruptions. > > (Only applicable if you used DCF prior to Controller 8.1) Once traffic patterns > have been reviewed and appropriate rules are in place, the optional Greenfield > Rule—used to allow all traffic during initial setup—can be removed. ## SmartGroups and Policy Enforcement SmartGroups enable dynamic, tag-based policy enforcement. These groups are defined using cloud-native metadata such as instance tags or labels. Rules can refer to these dynamic cloud-based SmartGroups instead of groups of static IP addresses, allowing for scalable and context-aware access control. For example, a rule might permit HTTP and HTTPS traffic from a SmartGroup representing frontend web servers to another SmartGroup representing backend APIs. This approach supports micro-segmentation and limits lateral movement within the network—key objectives of a zero trust architecture. ## Additional Distributed Cloud Firewall Features Supporting Zero Trust Distributed Cloud Firewall includes several features that enhance zero trust enforcement: * [Geolocation Filtering](/docs/enterprise/10.1/reference/externalgroup-about#countries) : Administrators can restrict traffic based on country or region, reducing exposure to high-risk geographies. * [Threat Intelligence Feeds](/docs/enterprise/10.1/reference/externalgroup-about#threat-feeds) : Integration with threat feeds allows for automatic blocking of known malicious IP addresses and domains. * [SaaS services](/docs/enterprise/10.1/reference/externalgroup-about#saas-services) can be protected by defining policies that restrict access to specific applications or services, ensuring that only authorized users and devices can interact with certain resources. * TLS Decryption and [URL Filtering](/docs/enterprise/10.1/concepts-architectures/components/dcf/webgroups-about): These capabilities provide visibility into encrypted traffic and enable fine-grained control over web access. * [Logging and Monitoring](https://legacy.docs.aviatrix.com/documentation/latest/security/dcf-monitor.html?expand=true): Selective logging of Permit and Deny actions supports auditing and incident response without overwhelming storage resources. ## Operational Considerations The Default Action Rule cannot be deleted and serves as a persistent enforcement mechanism. It is important to regularly audit policy rules and SmartGroup definitions to ensure they reflect current organizational requirements and security posture. By combining the Default Action Rule with SmartGroups and other DCF capabilities, Aviatrix enables organizations to implement a zero trust model that is both comprehensive and adaptable to evolving cloud environments. ## Implementation Methodology A structured approach to zero trust implementation with Aviatrix includes: * Traffic Analysis: Use CoPilot's monitoring capabilities to document legitimate traffic patterns between application components. * Policy Development: Create SmartGroups to accurately represent application components, then establish explicit DCF Permit rules for required communications. * Validation Testing: Create and validate your policy with the Default Action set to Permit before changing the Default Action to Deny. This ensures that all necessary traffic is allowed before enforcing the deny-by-default posture. * Production Deployment: Transition production environments to the deny-by-default configuration while monitoring for any unexpected issues. * Continuous Refinement: Review and optimize policies regularly. ## Related Topics * [Distributed Cloud Firewall Setup and Default Action Rule](/docs/enterprise/10.1/guides/security/dcf/dcf-enable-feature) # About DNS Hostname SmartGroups Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/components/dcf/hostname-smartgroups-about DNS Hostname SmartGroups enable you to filter non-HTTP/TLS traffic by fully qualified domain name (FQDN). The gateway proactively resolves hostnames to IP addresses and enforces them as firewall rules in DCF policies. DNS Hostname SmartGroups let you define [Distributed Cloud Firewall (DCF)](/docs/enterprise/10.1/concepts-architectures/components/dcf/dcf-overview) policies using fully qualified domain names (FQDNs) instead of IP addresses. This is useful when you need to allow or deny traffic to destinations whose IP addresses change frequently — such as SaaS APIs, cloud services, or partner endpoints — but the traffic is not HTTP or TLS and therefore cannot be matched by [WebGroups](/docs/enterprise/10.1/concepts-architectures/components/dcf/webgroups-about). When you add a DNS Hostname resource to a SmartGroup, the Aviatrix gateway resolves the hostname to its current IP addresses and enforces matching traffic as /32 CIDR rules. The resolved addresses are visible in the SmartGroup detail view alongside any static CIDR entries. ![SmartGroup detail view showing both CIDR and DNS Hostname resource types](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/components/dcf/hostname-sg-detail.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=63e427c909f06879299376c1c6b1c510) > [!NOTE] > > You must enter the DNS Hostname resource type as an FQDN (for example, > `api.example.com`). Entering just a hostname without a domain is not > supported. The FQDN can only include characters that are valid in DNS > hostnames. Wildcards are not supported. ## How Hostname Resolution Works Aviatrix gateways resolve hostname SmartGroup entries **proactively** — not at the time traffic arrives. This means: 1. When a DCF policy containing a hostname SmartGroup is pushed to a gateway, the gateway immediately resolves all configured FQDNs using forward DNS (A record) lookups. 2. The resolved IP addresses are installed as /32 CIDR entries in the gateway's firewall rules. 3. The gateway periodically re-resolves each FQDN before its DNS cache entry expires, keeping the firewall rules current as IP addresses change. This proactive approach ensures that firewall rules are already in place when traffic arrives. There is no per-packet DNS lookup or inspection — the gateway simply matches traffic against the pre-resolved IP addresses. ### TTL and Refresh Behavior The gateway respects DNS TTL (Time to Live) values from the authoritative DNS server, with the following bounds: | Parameter | Value | | - | - | | Minimum TTL | 60 seconds | | Maximum TTL | 3600 seconds (1 hour) | | Refresh timing | Before the cached entry expires | If a DNS server returns a TTL below 60 seconds, the gateway caches the result for 60 seconds. If the TTL exceeds 3600 seconds, the gateway re-resolves after 1 hour regardless of the advertised TTL. > [!NOTE] > > When a hostname SmartGroup is first deployed to a gateway, there is a brief > window (typically a few seconds) while the initial DNS resolution completes. > During this window, traffic to the hostname destination may not yet match the > DCF rule. ### Resolution Limitations * **A records only** — The gateway resolves IPv4 (A) records. IPv6 (AAAA) records are not supported. * **No CNAME chasing** — If a hostname resolves to a CNAME, only the final A record addresses are used for rule enforcement. Intermediate CNAME targets are not independently tracked. * **Forward DNS only** — Reverse DNS (PTR) lookups are not performed. ## DNS Server Configuration By default, hostname SmartGroup entries are resolved using each gateway's individually configured management DNS server. You can override this with custom DNS servers that apply globally to all gateways enforcing hostname SmartGroup rules. | Setting | Description | | - | - | | **Gateway's Management DNS Server** (default) | Each gateway uses its own management DNS server, as configured in the [gateway DNS settings](/docs/enterprise/10.1/reference/gateways/transit-spoke-dns). | | **Custom DNS Servers** | Up to 2 custom DNS server IPv4 addresses. Applied globally to all gateways that enforce hostname SmartGroup rules. | To configure the DNS server for hostname resolution: 1. In CoPilot, go to **Groups > Settings**. 2. In the **DNS Server for Hostname Resolution** card, select **Custom DNS Servers**. 3. Enter up to 2 DNS server IP addresses. ![DNS Server for Hostname Resolution settings](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/components/dcf/hostname-sg-dns-settings.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=81a447898da796915f0c9b53bc9ba1c2) > [!WARNING] > > Custom DNS server settings are global — they apply to all gateways enforcing > DCF rules that contain hostname SmartGroups. You cannot configure different > DNS servers for different gateways or different SmartGroups. > [!NOTE] > > Edge gateways use the DNS server configured in their own gateway settings and do not use the custom DNS server setting configured in Groups > Settings. ## When to Use Hostname SmartGroups vs WebGroups Hostname SmartGroups and [WebGroups](/docs/enterprise/10.1/concepts-architectures/components/dcf/webgroups-about) both let you write DCF policies using domain names, but they work differently and apply to different traffic types. | | Hostname SmartGroups | WebGroups | | - | - | - | | **Traffic type** | Non-HTTP/TLS traffic (SMTP, SSH, database connections, custom protocols) | HTTP, HTTPS, and TLS traffic | | **Matching method** | DNS resolution — FQDN resolved to IP addresses, enforced as /32 CIDRs | SNI/Host header inspection — matches the domain name in the TLS handshake or HTTP request | | **Wildcard support** | No | Yes (`*.example.com`) | | **Protocol scope** | Any IP protocol | HTTP/HTTPS/TLS only | | **IPv6 support** | No | No | | **Typical use cases** | SMTP relay to a SaaS mail provider, SSH to a bastion by hostname, database connections to cloud-managed endpoints | Allow/deny web browsing to specific domains, SaaS application access, API gateway egress | > [!TIP] > > If you want to filter web (TLS, HTTP, or HTTPS) traffic, Aviatrix strongly > recommends using WebGroups instead of hostname SmartGroups. WebGroups inspect > the SNI or Host header directly, which is more reliable for web traffic than > DNS-based IP matching. ## Feature Availability | Version | Status | | - | - | | Controller 7.2 | Preview | | Controller 8.1+ | Generally Available (GA) | See [Distributed Cloud Firewall Supported Capabilities by Controller Version](/docs/enterprise/10.1/reference/dcf/dcf-capabilities) for the full feature matrix. ## Limitations * **No wildcard support** — You must enter exact FQDNs. Patterns like `*.example.com` are not supported. * **A records only** — Only IPv4 addresses are resolved. AAAA (IPv6) records are not supported. * **Global DNS configuration** — Custom DNS server settings apply to all gateways. Per-gateway or per-SmartGroup DNS configuration is not available. * **Edge gateway DNS** — Edge gateways use their own gateway DNS settings and do not use the custom DNS server configured in Groups > Settings. * **Initial resolution delay** — When a hostname SmartGroup is first deployed, there is a brief delay while DNS resolution completes. ## AWS GuardDuty Considerations If you use AWS GuardDuty in the same account as your Aviatrix gateways, GuardDuty may generate DNS-related findings (such as `Trojan:EC2/DNSDataExfiltration` or `Backdoor:EC2/DenialOfService.Dns`) for the gateway instances. This is expected behavior — the Aviatrix gateway is performing periodic DNS lookups to resolve the FQDNs configured in your hostname SmartGroups and keep firewall rules current. These DNS queries originate from the gateway's elastic network interface (ENI) and are a normal part of hostname SmartGroup enforcement. **Recommendations:** * **Suppress findings** for the Aviatrix gateway ENIs in GuardDuty to reduce noise. * **Use custom DNS servers** — If you configure custom DNS servers for hostname resolution, the gateway queries go to your specified servers instead of the VPC default resolver, which may reduce GuardDuty findings related to VPC DNS. ## Terraform Support You can manage hostname SmartGroups using the `aviatrix_smart_group` resource in the [Aviatrix Terraform Provider](https://registry.terraform.io/providers/AviatrixSystems/aviatrix/latest/docs). Use the `fqdn` key in `match_expressions` to specify DNS Hostname entries: ```hcl theme={null} resource "aviatrix_smart_group" "mail_relay" { name = "mail-relay-endpoints" selector { match_expressions { fqdn = "smtp.mailprovider.com" } match_expressions { fqdn = "smtp2.mailprovider.com" } } } ``` ## Related Topics * [Managing SmartGroups](/docs/enterprise/10.1/concepts-architectures/components/dcf/smartgroups-about) * [About WebGroups](/docs/enterprise/10.1/concepts-architectures/components/dcf/webgroups-about) * [Distributed Cloud Firewall Overview](/docs/enterprise/10.1/concepts-architectures/components/dcf/dcf-overview) * [Groups UI Reference](/docs/enterprise/10.1/reference/ui/groups) (create SmartGroups, ExternalGroups, WebGroups; Groups Settings) * [DNS Server Configuration for Aviatrix Services](/docs/enterprise/10.1/guides/platform-administration/dns-server-general) * [DNS Server Configuration for Aviatrix Gateways](/docs/enterprise/10.1/reference/gateways/transit-spoke-dns) * [DCF Supported Capabilities by Controller Version](/docs/enterprise/10.1/reference/dcf/dcf-capabilities) # Kubernetes Firewall Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/components/dcf/kubernetes-overview Aviatrix Kubernetes Firewall extends Distributed Cloud Firewall (DCF) security policies to containerized applications by including Kubernetes Pod and Service addresses in SmartGroups. > [!TIP] > > Currently, DCF contains both Preview and GA features. See > [Distributed Cloud Firewall Supported Capabilities by Controller Version](/docs/enterprise/10.1/reference/dcf/dcf-capabilities) > for more information. Aviatrix Kubernetes Firewall extends network security, cost efficiency, and deployment flexibility to containerized applications. It integrates with Kubernetes and adapts automatically to scale changes without manual intervention. Using DCF's features, users can establish security policies based on native container workload identities. By extending SmartGroup capabilities to include Kubernetes attributes such as Pod and Service addresses, DCF rules can provide security coverage across both Kubernetes and VM environments. Aviatrix optimizes IP address use within Kubernetes clusters, preventing address exhaustion and simplifying deployment processes. NAT capabilities enable connectivity for Pods, spanning both cloud and on-premises environments. > [!NOTE] > > Currently Aviatrix Kubernetes Firewall only enforces DCF rules on traffic from > the Kubernetes clusters to anywhere outside the VPC/VNet. To implement Aviatrix Kubernetes Firewall: * [Kubernetes Prerequisites and Permissions](/docs/enterprise/10.1/reference/dcf/kubernetes-prerequisites) * [Discovery of Kubernetes Resources](/docs/enterprise/10.1/guides/security/dcf/kubernetes-resource-discovery) * [Onboard Kubernetes Clusters](/docs/enterprise/10.1/guides/security/dcf/kubernetes-onboard) * [Create Kubernetes SmartGroups](/docs/enterprise/10.1/guides/security/dcf/dcf-kubernetes) * [Create Distributed Cloud Firewall Rules](/docs/enterprise/10.1/concepts-architectures/components/dcf/dcf-rulesets-about) # Security Group Orchestration for Distributed Cloud Firewall Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/components/dcf/security-group-orchestration After configuring your SmartGroups and Distributed Cloud Firewall rules, you can enable Security Group Orchestration for workload traffic that will remain in your virtual network. After configuring your SmartGroups and Distributed Cloud Firewall rules, you can enable Security Group Orchestration for workload traffic that will remain in your virtual network. > [!NOTE] > > Security Group Orchestration is available for AWS/AWS GovCloud (VMs only) and Azure. > > You can use the > > [AppIQ FlightPath troubleshooting tool](/docs/enterprise/10.1/reference/ui/diagnostics/diagnostic-tools#run-connectivity-test) > to perform connectivity tests on the AWS or Azure instances secured with > Security Group Orchestration. > [!TIP] > > **Preview in Controller 10.1.** Starting in Controller 10.1, Security Group > Orchestration also supports AWS GovCloud as a Preview feature. See > [Aviatrix Feature Modes](/docs/enterprise/10.1/reference/feature-modes/index) > for guidance on adopting Preview features. > > Security Group Orchestration for AWS GovCloud works the same as for commercial > AWS. Orchestration does not function if **Manage Enforcement on Clouds** is > turned off. This feature utilizes cloud-native security features (such as network and application security groups within Azure, and security groups within AWS) to provide security control within the virtual network. Based on the SmartGroup and Distributed Cloud Firewall rule configuration, Aviatrix will push these security group policies to provide L4 access control for the workloads in the virtual network. The examples later in this document illustrate how the Security Group Orchestration feature can be configured in your environment. The first example for each cloud describes how you can configure Layer 4 access policies between two applications in the same subnet, and the second example for each cloud describes how you can configure secured communication between applications in the same VNet but different subnets. > [!WARNING] > > Security Group Orchestration is not performed on subnets that contain Aviatrix gateways. However, if a source Group within a DCF rule contains a CIDR that is located in a subnet that contains Aviatrix gateways, the rule is executed. > > Security Group Orchestration is not currently supported for Kubernetes clusters > or shared VPCs. It is only supported for VPC/VNets, subnets, and VMs. ## System Prerequisites * Upgrade to Controller version 7.0 * [Enable the Distributed Cloud Firewall feature](/docs/enterprise/10.1/guides/security/dcf/dcf-enable-feature) ## Enabling Security Group Orchestration Before enabling Security Group Orchestration you must already have Spoke VNets deployed with Aviatrix Gateways. 1. In CoPilot, navigate to Security > Distributed Firewall > Settings. 2. In the [Security Group Orchestration panel](/docs/enterprise/10.1/guides/security/dcf/dcf-settings) , click **Manage**. The Manage Security Group Orchestration on VPC/VNets dialog displays. ![Manage Security Group Orchestration on VPC/VNets dialog](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/components/dcf/virtual-network-ui.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=270878b0f6cce52945d7b665985d6c2e) This shows all the VPC/VNets in your environment that contain an Aviatrix gateway; if they are Enabled or Disabled; and the Security Group Orchestration Status. 3. Enable one or more VPC/VNets for Security Group Orchestration. 4. Select the **I understand the network impact of the changes** checkbox. 5. Click **Save**. 6. Create your [SmartGroups](/docs/enterprise/10.1/reference/ui/groups/index#create-smartgroup) and [rules](/docs/enterprise/10.1/reference/ui/security/distributed-cloud-firewall#create-a-dcf-ruleset) . After the rules are committed, Security Group Orchestration is applied to the SmartGroups in those rules. > [!NOTE] > > When you create the rule, set the **SG Orchestration** toggle to **On**. If > this toggle is **Off**, Security Group Orchestration does not apply to that > rule. See > [Create Firewall Rule](/docs/enterprise/10.1/reference/ui/security/distributed-cloud-firewall#create-firewall-rule) > for the cases where the SG Orchestration toggle defaults to Off for new rules. * [AWS Examples](#aws-examples) * [Azure Examples](#azure-examples) Click [here](#disabling-security-group-orchestration) for steps on how to disable Security Group Orchestration. ## AWS Examples ![AWS Intra-VPC Multi-Subnet Topology](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/components/dcf/intravpc-aws-multisubnet-nosg.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=7cf45516f1179d3fd132b050fc4bbdb5) This topology diagram shows a three-tier application/architecture, consisting of a Web server, an Application, and a Database. The Web server and the Application are deployed in Subnet 1, and the Database is deployed in Subnet 2. When Security Group Orchestration is enabled in AWS: * VMs are secured immediately * The maximum allowed number of Security Groups (SGs) is created on a single NIC for the AWS account * The SGs are attached to the VMs * Your existing SGs are stored > [!WARNING] > > * Aviatrix strongly recommends that you not modify any of the SGs created by Aviatrix. > > * Any VMs or services containing a public IP address (the traffic path of which > is controlled by the NSG or Security Group) must be accounted for with an > appropriate > [Distributed Cloud Firewall rule](/docs/enterprise/10.1/reference/ui/security/distributed-cloud-firewall#create-a-dcf-ruleset) > that allows inbound access to the VM or service. ### AWS Example 1 This example will explain how to restrict communication between the Web server and the App to be only over port 80, and traffic between App to DB to be only over port 3306. This assumes you have already [enabled the Distributed Cloud Firewall feature](/docs/enterprise/10.1/guides/security/dcf/dcf-enable-feature) . ![AWS intra-VPC single-subnet topology with Web server, Application, and Database](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/components/dcf/intravpc-aws.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=faab7ecc406dfd3b6989913d0c2b9cd5) 1. Follow the steps in [Creating SmartGroups](/docs/enterprise/10.1/guides/security/dcf/dcf-create-groups) to create one SmartGroup that contains the Application, another that contains the Web server, and another that contains the Database. 2. Follow the steps in [Creating Distributed Cloud Firewall Rules](https://legacy.docs.aviatrix.com/documentation/latest/security/dcf-create-rules.html) to: * Create a rule with the Web server as the source, and the Application as the destination with an allow action on protocol TCP and port 80. * Create a rule with Application as the source and Database as the Destination with an allow action on protocol TCP and port 3306. 3. On the Policies tab, commit the policies. ![AWS Distributed Cloud Firewall policies for Web, App, and Database SmartGroups](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/components/dcf/virtual-network-policies-aws.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=57cf8fe18e9e37f97db329a4e47e67b7) > [!NOTE] > > If you already have SGs attached to the application VMs, Aviatrix stores the > SG information before pushing Aviatrix-configured policies. #### Rule Changes After the change is committed, the Aviatrix platform computes the necessary cloud native policy and pushes/programs security group configuration as shown below: * An SG is created for each of the SmartGroups (one for the Web server, one for the Application, and one for the Database). * An SG corresponding to the Web server is attached to all VMs belonging to the Web server. * An SG corresponding to the Application is attached to all VMs belonging to the App. * An SG corresponding to the Database is attached to all VMs belonging to the Database. ![AWS intra-VPC topology with Aviatrix-created security groups attached to each tier](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/components/dcf/intravpc-aws-sgadded.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=e0465d3592818dee35646b65bac92094) ### AWS Example 2 This example is similar to Example 1, except that the Database resides in a different Subnet (see the [topology diagram at the beginning of the AWS section](#aws-examples)). It will explain how to restrict communication between the Web server and the Application residing in the same subnet to be over port 80, while restricting communication between the Application and the Database residing in a different subnet (Subnet 2) to be only over port 3306. This assumes you have already [enabled the Distributed Cloud Firewall feature](/docs/enterprise/10.1/guides/security/dcf/dcf-enable-feature) . 1. Follow the steps in [Creating SmartGroups](/docs/enterprise/10.1/guides/security/dcf/dcf-create-groups) to create one SmartGroup that contains the Application, another that contains the Web server, and another that contains the Database. 2. Follow the steps in [Creating DCF Rules](https://legacy.docs.aviatrix.com/documentation/latest/security/dcf-create-rules.html) to: * Create a rule with the Web server as the source, and the Application as the destination with an allow action on protocol TCP and port 80. * Create a rule with Application as the source and Database as the Destination with an allow action on protocol TCP and port 3306. 3. On the Policies tab, commit the policies. After the commit, the same Aviatrix-created rule changes occur as described in Example 1. ![AWS intra-VPC multi-subnet topology with Database in a separate subnet](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/components/dcf/intravpc-aws-multisubnet.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=cfa36b110a5eeed37e45d7e3a4edd787) ## Azure Examples The examples in this section illustrate how the Security Group Orchestration feature can be configured in your environment. The first example describes how you can configure Layer 4 access policies between two applications in the same subnet, and the second example describes how you can configure secured communication between applications in the same VNet but different subnets. When Security Group Orchestration is enabled for Azure: * The Aviatrix platform creates a default application security group (ASG) called AVXASG-DEFAULT in the selected VNets. * The Aviatrix platform uses Azure's cloud native network security group (NSG) at the subnet level to enforce the Distributed Cloud Firewall policies for the resources in that subnet. * To enforce zero-trust architecture, a default deny rule with a priority of 4095 is programmed at the bottom of the NSG rule set to ensure that all communication is blocked until a specific rule that allows access is configured. * To ensure that this does not break traffic flow for VMs on subnets that are not part of the Distributed Cloud Firewall rule model, the Aviatrix platform assigns the default ASG to these VMs, along with adding a rule called AVX-RULE-DEFAULT in the subnet ASG that allows communication to these VMs from any source. * Original resources that are within an Aviatrix-controlled subnet are kept, but disassociated. The Aviatrix-created ASGs and NSGs are associated instead. When Security Group Orchestration is disabled, the Aviatrix-created ASGs and NSGs are disassociated, and the original resources are re-associated. > [!WARNING] > > * Aviatrix strongly recommends that you not modify any of the ASGs or NSGs created by Aviatrix. > > * Any VMs or services containing a public IP address (the traffic path of which > is controlled by the NSG or Security Group) must be accounted for with an > appropriate > [Distributed Cloud Firewall rule](/docs/enterprise/10.1/reference/ui/security/distributed-cloud-firewall#create-a-dcf-ruleset) > that allows inbound access to the VM or service. > [!NOTE] > > In Controller versions later than 7.1, ASGs and NSGs will automatically use > the tag (key-value pair) "Aviatrix-Created-Resource". Versions of Controller > 7.1 or earlier will use the tag "Aviatrix-Created-Resource Value". If you are > using an older version of Controller and you want to change this tag to > "Aviatrix-Created-Resource", you must contact [Aviatrix > Support](https://support.aviatrix.com/). > [!NOTE] > > Aviatrix-created NSGs are blocked from attaching to the following types of > Azure subnets: AzureBastionSubnet, GatewaySubnet, AzureFirewallSubnet, and > RouteServerSubnet. ### Example 1 This example will explain how to restrict communication between the Web server and the App to be only over port 80, and traffic between App to DB to be only over port 3306. This assumes you have already [enabled the Distributed Cloud Firewall feature](/docs/enterprise/10.1/guides/security/dcf/dcf-enable-feature) . ![Azure VNet topology with Web server, Application, and Database in the same subnet](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/components/dcf/virtual-network-ex2.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=4e0c50d4d814668c673e947ca06c1164) 1. Follow the steps in [Creating SmartGroups](/docs/enterprise/10.1/guides/security/dcf/dcf-create-groups) to create one SmartGroup that contains the Application, another that contains the Web server, and another that contains the Database. 2. Follow the steps in [Creating DCF Rules](https://legacy.docs.aviatrix.com/documentation/latest/security/dcf-create-rules.html) to: * Create a rule with the Web server as the source, and the Application as the destination with an allow action on protocol TCP and port 80. * Create a rule with Application as the source and Database as the Destination with an allow action on protocol TCP and port 3306. 3. On the Policies tab, commit the policies. ![Azure Distributed Cloud Firewall policies for Web, App, and Database SmartGroups](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/components/dcf/virtual-network-policies.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=16b710f961572ff1d49fd12c3f5445c1) > [!NOTE] > > If you already have ASGs attached to the application VMs, Aviatrix will store > the ASG information before pushing Aviatrix-configured policies. #### Rule Changes After the change is committed, the Aviatrix platform computes the necessary cloud native policy and pushes/programs security group configuration as shown below: * An application security group is created for each of the SmartGroups (one for the Web server, one for the Application, and one for the Database). * An ASG corresponding to the Web server is attached to all VMs belonging to the Web server. * An ASG corresponding to the Application is attached to all VMs belonging to the App. * An ASG corresponding to the Database is attached to all VMs belonging to the Database. ![Azure VNet with Aviatrix-created ASGs attached to each tier](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/components/dcf/virtual-network-asgs.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=b622fbad3b11f46dcb454e89707c30de) * The Aviatrix platform creates and pushes/programs a new NSG at the subnet level. ![Azure subnet NSG created by Aviatrix before DCF rules are added](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/components/dcf/virtual-network-rule-before.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=26fb0507c77fbcc5d36442b60791aac7) * Aviatrix adds rules to the NSG to enforce the Distributed Cloud Firewall rule configuration. These are the Aviatrix-configured rules on Aviatrix-created NSGs: * AVX-RULE-0 rule allows traffic between the Web server and App on TCP port 80. * AVX-RULE-1 rule allows traffic between the App to the Database on TCP port 3306. * AVX-RULE-AZLB-INBOUND rule allows communication between an Azure load balancer and your resources. * AVX-RULE-DEFAULT rule is configured for the VMs that are part of this subnet but not subject to any Distributed Cloud Firewall micro-segmentation rule. More details on this rule are provided in Example 2. * AVX-RULE-VNET rule denies any intra-VNet traffic. * AVX-RULE-1918 allows Security Group Orchestration connectivity. Controls will be applied through Aviatrix Security Group Orchestration before hitting this rule. ![Azure NSG populated with Aviatrix-configured AVX-RULE entries enforcing DCF policies](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/components/dcf/virtual-network-rules.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=cd498e66cfd9be8b73d0851009bddbb0) ### Example 2 This example is similar to Example 1, except that the Database resides in a different Subnet. It will explain how to restrict communication between the Web server and the Application residing in the same subnet to be over port 80, while restricting communication between the Application and the Database residing in a different subnet (Subnet 2) to be only over port 3306. This assumes you have already [enabled the Distributed Cloud Firewall feature](/docs/enterprise/10.1/guides/security/dcf/dcf-enable-feature) . 1. Follow the steps in [Creating SmartGroups](/docs/enterprise/10.1/guides/security/dcf/dcf-create-groups) to create one SmartGroup that contains the Application, another that contains the Web server, and another that contains the Database. 2. Follow the steps in [Creating Rules](https://legacy.docs.aviatrix.com/documentation/latest/security/dcf-create-rules.html) to: * Create a rule with the Web server as the source, and the Application as the destination with an allow action on protocol TCP and port 80. * Create a rule with Application as the source and Database as the Destination with an allow action on protocol TCP and port 3306. 3. On the Policies tab, commit the policies. After the commit, the same Aviatrix-created rule changes occur as described in Example 1. The additional change is that AVX-RULE-DEFAULT is created and attached to the Logging app in Subnet 2. This is to ensure that the traffic for the Logging application is unaffected. Any application that is not subject to a rule via the Aviatrix platform will use AVX-RULE-DEFAULT. > [!NOTE] > > AVX-RULE-DEFAULT is only added to the Aviatrix NSG if there are orphaned VMs > in the VNet. If there are no orphaned VMs in the VNet, AVX-RULE-DEFAULT is not > created in the Avatrix NSG. ![Azure VNet with Aviatrix ASGs across multiple subnets including Logging app in Subnet 2](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/components/dcf/virtual-network-asgs2.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=f239e34440e6d88d705bd529482a4618) ## Disabling Security Group Orchestration 1. In CoPilot, navigate to Security > Distributed Firewall > Settings. 2. In the Security Group Orchestration panel, click **Manage**. The Manage Security Group Orchestration on VPC/VNets dialog displays. 3. Disable one or more VPC/VNets for Security Group Orchestration. When Security Group Orchestration is disabled: * AWS: Aviatrix-created SGs are disassociated. * Azure: ASGs and NSGs are disassociated. * The original resources are re-associated. ## Limitations * Azure only: This feature is not supported with natively peered VNets. * Azure only: Any rules with a deny action applied to a CIDR-based SmartGroup as a destination where the CIDR does not belong to a VNet will not be enforced via Azure NSG orchestration. * AWS only: If a Distributed Cloud Firewall Allow rule with protocol set to Any and ports set to All conflicts with a higher priority Deny rule, the Allow rule is ignored. # SmartGroup Use Cases Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/components/dcf/smartgroup-use-cases The following are examples of how to use SmartGroups in Distributed Cloud Firewall (DCF) policies. The following are examples of how to use SmartGroups in Distributed Cloud Firewall (DCF) policies. ## SmartGroups Use Case 1 Alex creates three SmartGroups: * Smart Group 1 = Data\_Analytics * Smart Group 2 = Customer\_Relationships * Smart Group 3 = Public CIDR of Hosted Cloud\_Service Alex has the following business objectives: * Allow Data Analytics and Customer Relationships to communicate with the Cloud Service on port 443. * Deny all traffic between Data Analytics and Customer Relationships. To achieve Distributed Cloud Firewall (DCF) objectives, Alex does the following in **CoPilot Home > Security > Distributed Cloud Firewall**: * Builds firewalling policies that allow traffic from Smart Group 1 and 2 to communicate on port 443 with Smart Group 3. * Creates a Deny All Policy for Smart Group 1 to communicate with Smart Group 2. Note that after a SmartGroup is part of a policy, all traffic for that SmartGroup is denied unless explicitly allowed by DCF rules. ## SmartGroup Use Case 2 (External Connection) ![S2C DCF e2e](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/components/dcf/S2C-DCF-e2e.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=29a9bee932819af3553c1a40f6635568) 1. Create a SmartGroup for RS1 (Remote Site 1). 2. Create a SmartGroup for RS2 (Remote Site 2). 3. Create a DCF rule to allow traffic from RS1 to VPC2. 4. Create a DCF rule to deny traffic from RS2 to VPC2. 5. Create a DCF Egress rule to allow specific web domains from RS1 to the Internet. 6. Create a DCF rule to allow TCP/22 traffic from VPC1 to RS2. ## SmartGroup Use Case 3 (Kubernetes) Taylor manages a Kubernetes cluster with multiple node groups for different workloads. Taylor creates two SmartGroups: * Smart Group 1 = K8s-Prod-Nodes (includes all nodes labeled as "environment=production") * Smart Group 2 = K8s-Dev-Nodes (includes all nodes labeled as "environment=development") To create these SmartGroups: 1. Go to CoPilot Home > Security > SmartGroups. 2. Click **Create SmartGroup**. 3. For K8s-Prod-Nodes, select **Kubernetes Node** as the resource type and set the label filter to `environment=production`. 4. For K8s-Dev-Nodes, select **Kubernetes Node** as the resource type and set the label filter to `environment=development`. 5. Save each SmartGroup. Taylor has the following business objectives: * Allow only specific traffic (e.g., port 443) from K8s-Dev-Nodes to K8s-Prod-Nodes for application testing. * Deny all other traffic between development and production nodes. * Monitor network usage between these node groups for auditing. To achieve these objectives, Taylor creates Distributed Cloud Firewall (DCF) rules: 1. Go to CoPilot Home > Security > Distributed Cloud Firewall. 2. Create a rule to **allow** TCP traffic on port 443 from SmartGroup **K8s-Dev-Nodes** (source) to SmartGroup **K8s-Prod-Nodes** (destination). 3. Create a rule to **deny** all other traffic from **K8s-Dev-Nodes** to **K8s-Prod-Nodes**. 4. (Optional) Add a rule to **log** or **monitor** all traffic between these SmartGroups for auditing purposes. This approach ensures that only approved application testing traffic is permitted between development and production Kubernetes nodes, while all other traffic is blocked and monitored. ## Related Topics * [General Guidelines for Migrating from Legacy ThreatIQ and Geoblocking to Distributed Cloud Firewall](/docs/enterprise/10.1/guides/security/threatiq-geo-dcf-migration-guidelines) # Managing SmartGroups Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/components/dcf/smartgroups-about This section describes SmartGroups and how they can be used for implementing different Aviatrix features. This section describes SmartGroups and how they can be used for implementing different Aviatrix features. ## What is a SmartGroup? A SmartGroup is a reusable construct created in CoPilot that is a logical grouping of your resources that are managed by Aviatrix. The grouping of resources may represent various departments or business units, or other aspects of your organization based on how you group your resources. The resource(s) you include in a SmartGroup can span different subscriptions, cloud accounts, regions, and VPC/VNets within your Aviatrix multicloud network. When you create your SmartGroups, you can classify them based on the following resource types: * Virtual Machines * Subnets * VPC/VNets > [!NOTE] > > The above resource types are only supported in public AWS, Azure, and GCP. For each of the above, you can match by Cloud Tags (configured in your cloud provider) or Properties (Name, Region, Account Name). Cloud Tags identify resources you can group. This is the preferred classification method, as this automatically includes new resources created in the Cloud with the same set of tags. > [!NOTE] > > In GCP you configure 'labels' that can be selected as tags when creating your > SmartGroup. * IP addresses or CIDRs: for resources that are not tagged, you can directly specify IP addresses or CIDRs. Enter FQDNs in the DNS Hostnames resource type field. > [!NOTE] > > At this time the maximum number of CIDRs that can be enforced in a SmartGroup > is 10,000. This includes both CIDRs in CIDR groups and resolved CIDRs in > tagged groups. See > [DCF Capabilities](/docs/enterprise/10.1/reference/dcf/dcf-capabilities) > for details on ranges supported in the latest Controller release. * DNS Hostnames: Enter Fully Qualified Domain Names (FQDNs). * External connections (S2C): select the previously created external connection (ensure that the [Enforcement on External Connections option](/docs/enterprise/10.1/guides/security/dcf/dcf-settings#enforcement-on-external-connections) is enabled first). You should only select an External Connection resource type if you plan to use this SmartGroup in a DCF rule, and if Enforcement on External Connections is enabled in Security > Distributed Cloud Firewall > Settings. * Kubernetes Workloads (Namespace, Cluster, and/or Service). * Kubernetes Nodes > [!NOTE] > > Aviatrix Gateway IP addresses will not be included in any SmartGroup, even if > a SmartGroup filter matches an Aviatrix Gateway IP address. If a subnet or > VPC/VNet is added to an app domain, the Aviatrix Gateway IP addresses are > removed from the corresponding CIDRs. ## System-Defined SmartGroups For convenience, CoPilot provides two system-defined (default) SmartGroups: * Anywhere (0.0.0.0/0) - Represents all CIDR ranges or IP addresses. * Public Internet - Represents non-RFC 1918 IP ranges, or the public Internet. System-defined SmartGroups cannot be deleted. ## Viewing SmartGroup Details You can click the SmartGroup name in the list to view its Group information (VM, IP/CIDR, or External Connection), Resources, and Rule References in the right-hand pane. On the Rule References tab, clicking on a rule opens this rule on the Distributed Cloud Firewall > Policies tab. ## Features that use SmartGroups Aviatrix features that use SmartGroups include: * Aviatrix Distributed Cloud Firewall (DCF) Distributed Cloud Firewall uses micro-segmentation to provide granular network security policies for distributed applications in the Cloud. Distributed Cloud Firewall enables network policy enforcement between SmartGroups you define in a single Cloud or across multiple Clouds. You can configure policies to filter traffic between applications residing in the SmartGroups. For more information about using SmartGroups for DCF, see [Secure Networking with Distributed Cloud Firewall](/docs/enterprise/10.1/concepts-architectures/components/dcf/dcf-overview) . ## Related Topics * [SmartGroup Use Cases](/docs/enterprise/10.1/concepts-architectures/components/dcf/smartgroup-use-cases) * [Creating a SmartGroup](/docs/enterprise/10.1/reference/ui/groups/index#create-smartgroup) * [Tips for Creating SmartGroups](https://legacy.docs.aviatrix.com/documentation/latest/security/smartgroups-tips.html) * [Creating Kubernetes SmartGroups](/docs/enterprise/10.1/reference/ui/groups/index#create-smartgroup) * [Editing a SmartGroup](/docs/enterprise/10.1/reference/ui/groups/index#edit-smartgroup) * [Deleting a SmartGroup](/docs/enterprise/10.1/reference/ui/groups/index#delete-smartgroup) * [Working with SmartGroups](https://legacy.docs.aviatrix.com/documentation/latest/security/smartgroups-workflow.html) * [Cloning a SmartGroup](https://legacy.docs.aviatrix.com/documentation/latest/security/groups-cloning.html) # Managing WebGroups Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/components/dcf/webgroups-about Enable the Distributed Cloud Firewall (DCF) feature to avail the WebGroups. See Distributed Cloud Firewall Overview for more information. > [!NOTE] > > Enable the Distributed Cloud Firewall (DCF) feature to avail the WebGroups. > See > [Distributed Cloud Firewall Overview](/docs/enterprise/10.1/concepts-architectures/components/dcf/dcf-overview) > for more information. WebGroups are groupings of domains or URLs, inserted into [DCF](/docs/enterprise/10.1/concepts-architectures/components/dcf/dcf-overview) rules, that filter (and provide security to) Internet-bound traffic. WebGroups in DCF are only supported on Spoke Gateways and Public Subnet Filtering (PSF) Gateways. From this tab you can save views, filter intrusion results, and download the results in a CSV file. > [!NOTE] > > To filter HTTP or HTTPS traffic with a URL-based WebGroup, TLS Decryption must be enabled in the rule where the WebGroup is used. > > Non-TLS or non-HTTP traffic will not match the rule that uses the WebGroup and > will be evaluated against later rules. ## Considerations * A TLS packet with no SNI header will only match the All-Web default WebGroup. You can create a DCF rule that uses a URL-type WebGroup to capture the packet, but only exact match URLs are supported at this time. > [!NOTE] > > As an alternative to using a URL-based WebGroup, you can configure an L4 > rule without WebGroups to allow the traffic based on IP address, and insert > it before the first L7 rule. * For non-TLS encrypted HTTP traffic, there is no SNI header. The following values are used instead: * **Domain-based WebGroups**: the value of the HTTP Host Header. * **URL-based WebGroups**: the URL value, as it would be for HTTPS (as long as TLS Decryption is enabled). * Non-TLS, non-HTTP traffic will not match any WebGroup. * Wildcard support varies by Controller version and WebGroup type: * **Domain-based WebGroups**: Full wildcards (for example, `*.example.com`) are supported on all supported Controller versions. * **URL-based WebGroups**: Wildcard support is limited. On Controller 7.x and 8.0, the **Create Web Group** dialog rejects URL entries that contain wildcards (including the `*.example.com` example shown in the in-product help); use a domain-based WebGroup if you need wildcard matching on these versions. On Controller 8.1 and later, hostname-style wildcards (such as `*.example.com`) are accepted in URL-based WebGroups, but more complex patterns with wildcards in the URL path (such as `www.example.com/*/dashboard`) may be rejected by the Controller with an `[AVXERR-SMARTGROUP-0007]` error; use a simpler URL pattern or a domain-based WebGroup in these cases. ## WebGroup Syntax Reference When you enter domains or URLs into a WebGroup, the entries must follow the syntax described below. Malformed entries cause the Controller to reject the WebGroup. ### Allowed Characters A domain or URL entry can use only the following characters: * Alphanumeric (a-z, A-Z, and 0-9) * Dots (`.`) * Dashes (`-`) * Underscores (`_`) * Asterisks (`*`) A domain name can start with any of these characters. Any TLS protocol is supported for domain entries, along with HTTP. ### Wildcard Patterns The asterisk (`*`) is a wildcard. Where wildcards are supported (see [Considerations](#considerations) for the per-type and per-version rules), the following patterns are valid: | Pattern | Example | Matches | | - | - | - | | Leading wildcard | `*.example.com` | `app.example.com`, `test.example.com` | | Subdomain wildcard | `app*.example.com` | `app1.example.com`, `app2.example.com` | | Host suffix wildcard | `ftp-*.example.com` | `ftp-1.example.com`, `ftp-2.example.com` | | Complex multi-wildcard | `patch*-*-*.*.example.com` | `patch1-a-26.s1.example.com`, `patch2-b-27.s2.example.com` | Consecutive asterisks (for example, `**.example.com`) are not supported. If the Controller detects malformed wildcard usage, the WebGroup is not created and you must correct the entry before saving. ### Scheme Prefixes Do not include a scheme such as `http://` or `https://` in URL filter entries. The Controller rejects any entry containing `:` or `//` and returns `[AVXERR-SMARTGROUP-0007] Malformed JSON input: invalid character or format in url filter` for the offending value. Enter URL filters starting with the hostname, for example `www.example.com/path`, not `https://www.example.com/path`. ### URL Examples The following URL entries are valid for URL-based WebGroups: * `github.com/AviatrixFieldEng/` * `www.example.com/*/dashboard` * `downloads.example.com/*.zip` * `*.updates.example.com/` > [!NOTE] > > Wildcards inside the URL path (for example, `/*/dashboard`) are not supported > on every Controller version. See [Considerations](#considerations) for the > version-specific behavior and error messages. ## System-Defined WebGroup When you navigate to **Security** > **Distributed Cloud Firewall** > **WebGroups**, a system-defined WebGroup, 'All-Web', has already been created for you (if no other WebGroups exist). This predefined WebGroup cannot be deleted. ![webgroup default](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/components/dcf/webgroup-default.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=05ba949be03ff9cb3efb29b4e01ccc25) This is an "allow-all" WebGroup that you must select in a Distributed Cloud Firewall rule if you do not want to limit the Internet-bound traffic for that rule, but you still want to log the FQDNs that are being accessed. > [!NOTE] > > Prior to Release 7.1.3006, the default WebGroup was named 'Any-Web' and was > created by CoPilot. If you still have this WebGroup, you can modify it (if it > is being used by Distributed Cloud Firewall rules) or delete it (if it is not > used by any Distributed Cloud Firewall rules) so that it is not confused with > the default 'All-Web' WebGroup created by Controller. ## SNI Verification > [!NOTE] > > SNI Verification is only present if using Controller 8.0 or later. You can enable SNI verification for a DCF rule if a WebGroup is also selected in that rule. SNI verification ensures that the Common Name (CN) or Subject Alternative Name (SAN)s of the certificate sent by the origin server contain the SNI field in the request sent by the DCF rule. If this condition is not met, the traffic is dropped. You can change the enforcement level for the certificate in the Distributed Cloud Firewall settings on the [Decryption CA Certificate card](/docs/enterprise/10.1/guides/security/dcf/dcf-settings#decryption-ca-certificate) . ## Related Topics * [Creating a WebGroup](/docs/enterprise/10.1/reference/ui/groups/index#create-web-group) * [Editing a Group](/docs/enterprise/10.1/reference/ui/groups/index#edit-smartgroup) * [Deleting a Group](/docs/enterprise/10.1/reference/ui/groups/index#delete-smartgroup) # Edge Platform Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/components/edge-gateways/edge-aep Aviatrix Edge Platform is an Aviatrix turnkey solution that enables cloud orchestration of edge hardware and Aviatrix Edge Gateways for deployment in customer on-premises locations. The hardware is recommended by Aviatrix and comes pre-staged before it is shipped to a customer site. ## Overview Aviatrix Edge Platform is an Aviatrix turnkey solution that enables cloud orchestration of edge hardware and Aviatrix Edge Gateways for deployment in customer on-premises locations. The hardware is recommended by Aviatrix and comes pre-staged before it is shipped to a customer site. ![Aviatrix Edge Platform](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/components/edge-gateways/edge-aviatrix-platform.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=e297f1660ff5327d1399f0b3391ee574) Aviatrix Edge Platform connects datacenters, remote and branch sites to the cloud and offers these key benefits: * Extends the cloud operational model to the edge. * Remote orchestration of edge hardware and software with full lifecycle management. * Encrypt high-speed circuits at line-rate with Aviatrix High Performance Encryption (HPE). * Secure edge traffic with [Aviatrix Distributed Cloud Firewall (DCF)](/docs/enterprise/10.1/concepts-architectures/components/dcf/dcf-overview) and network segmentation. * High availability Edge Gateways for failover. * Designed for multicloud connectivity. * Flexible form factors to support data center high throughput needs. * Single unified control and management plane. [Edge Gateway Multi-Peer BGP over LAN Connection](/docs/enterprise/10.1/guides/connectivity/edge/edge-multipeer-bgpolan) [Installing Edge OS and Onboarding Dell Hardware](/docs/enterprise/10.1/guides/connectivity/edge/edge-device-onboarding) # Hybrid Cloud Edge Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/components/edge-gateways/edge-overview Aviatrix Hybrid Cloud Edge is an enterprise grade solution that enables customers to extend the Aviatrix Cloud Networking architecture to the edge of their networks for consistent and repeatable architecture, management, visibility, security, and control. ## Overview Aviatrix Hybrid Cloud Edge is an enterprise grade solution that enables customers to extend the Aviatrix Cloud Networking architecture to the edge of their networks for consistent and repeatable architecture, management, visibility, security, and control. This cloud-out architecture enables enterprises to leverage the Aviatrix Platform's ubiquitous support for edge connectivity. The result is a high performance encrypted, seamless connectivity to edge locations such as data centers, colocations, remote sites and provider locations such as Equinix and Megaport. ## Why is cloud capable secure edge important? During their public cloud journey, IT leaders often ask themselves, "How do we — as securely and as seamlessly as possible — bring public clouds closer to end-users and services that continue to reside on-premises?". Applications are developed at pace and scale, and the ability to connect public cloud outward to the edge of on-premises becomes reliant on infrastructure that needs to always be available, performant, scalable, secure, and cost efficient. Key challenges: * Encryption is a key challenge and setting up traditional IPSec networks is prone to errors and often don't meet performance requirements. Implementing MACSec is complex and requires specific hardware support. * Installation and setup of on-premises network edge infrastructure is often time-consuming and costly, resulting in significant delays. * Network automation (in cloud vs on-premises) is different, which increases the time to onboard. * The on-premises network architecture and cloud teams may be disconnected, creating unnecessary delays and dissatisfied end consumers. * Fragmented stacks in cloud and on-premises increase troubleshooting time and resources that need to be engaged. This fragmentation leads to network downtime and increased MTTR. * More tools to maintain for monitoring, troubleshooting, and visibility increase overhead and support costs. * Security risks due to lack of consistent security policies and procedures between cloud and on-premises can compromise the network and create an ever-expanding attack surface. * Crowded and inconsistent management consoles because of network stacks that are different across cloud and on-premises, which also increases challenges to troubleshoot problems. IT teams need trained experts in each cloud and technology concerned. IT leaders have accepted that public cloud infrastructure is a separate entity to on-premises data center infrastructure — not only physically separate, but also organizationally separate. They are linked with private circuits either directly or via a colocation, but with different governance, operational models, controls, and feature sets. If you manage to solve the challenges, one key thing remains. You will be without an edge infrastructure that looks, feels, and behaves like a VPC, VNet, or VCN. ## What are the benefits of Aviatrix Hybrid Cloud Edge solution? With clouds as a new home for business-critical applications, modern enterprises have pivoted their architectural and operational center of gravity to the cloud. Aviatrix Hybrid Cloud Edge solution leverages Aviatrix cloud native network and network security software for customer edge locations. Its cloud-out architecture enables it to extend the cloud operating model out to on-premises. That means your branch locations, collocations, and on-premises must be secure, operational and networked the same as another VPC or VNet. This allows security posture, policies, and controls to remain consistent end-to-end and gives the impression of the cloud being closer to the end users and services that reside in the data center. The solution enables enterprises to have the same governance, consistent cloud network and security controls between public cloud and on-premises, but with enterprise grade visibility and management into the edge locations. If the cloud edge is a data center or branch external to the cloud, you get a unified network stack within cloud and on-premises with complete network visibility and single console to manage and operate the network in cloud and on-premises. Aviatrix Hybrid Cloud Edge solution offers advanced analytics and reporting capabilities, with the ability to track network performance and security metrics across multiple cloud platforms making it easy to identify and address issues before they impact the performance or security of your cloud environments. ![edge benefits](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/components/edge-gateways/edge-benefits.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=cbeba889ab0baaa02af80b97b4cc4751) Aviatrix Hybrid Cloud Edge solution is a powerful software-defined cloud networking solution that offers advanced networking and embedded security features for edge locations. It offers: * **A single way of doing things**. It extends the cloud model to on-premises, allows VPCs and VNets to be extended into the data center, Equinix colocation, or remote branch locations while retaining the cloud operational model and bringing the end users and services closer to the cloud. * **Simple zero-touch provisioning**. You can deploy Aviatrix Hybrid Cloud Edge solution either by using Terraform or its easy-to-use Aviatrix CoPilot to improve time to deploy for applications and workloads without worrying about the underlying environment. * **Cloud centric management and advanced analytics capabilities**. This makes it easy to retain consistent end-to-end (cloud to edge) networking and security policies and controls. The result is an overall reduction in complexity and cost of managing multiple cloud platforms. * **Complexity extraction**. This leads to reduced skill set requirement — no multi-vendor, SDWAN, X Y Z clouds, simplicity that empowers agility. In summation: * Go-to platform for all hybrid connectivity. * Centralized control plane across multicloud networks and edge locations reducing operational complexity. * Single pane of glass for cloud visibility, monitoring, and troubleshooting. * Encrypted connectivity and routing between multicloud networks with a private path that uses standard architecture. * High Performance Encryption (HPE) support over public and private networks. * Zero-touch provisioning (ZTP) for automated Edge deployments. * Multiple form factors to support various edge requirements. ## What does Aviatrix Hybrid Cloud Edge solution comprise? The solution comprises the following components: * The **Aviatrix Edge Transit Gateway** deployed as a virtual network function in edge locations. The Edge Transit Gateway provides high performance encryption across datacenters, connectivity to partner locations via IPSec, establishes encrypted connectivity to and across provider locations such as Equinix and Megaport and to multiple clouds. It seamlessly integrates with cloud environments and provides efficient routing for resilient architectures. Edge Transit Gateway advanced features provides support for network segmentation, NAT, and visibility to efficiently operate and manage hybrid cloud environments. See [About Aviatrix Edge Transit Gateway](/docs/enterprise/10.1/concepts-architectures/components/edge-gateways/edge-transit-overview) . * **Aviatrix Edge Spoke Gateway** deployed as a virtual network function in edge locations. The Edge Spoke Gateway provides high performance encryption from on-premises to cloud and supports features such as Aviatrix Distributed Cloud Firewall (DCF) for L4 filtering. It is suitable for deployment that are primarily edge to cloud. It seamlessly integrates with cloud environments and provides efficient routing for resilient architectures. Leverage features such as network segmentation, VLAN connectivity to LAN networks and visibility across edge to cloud for robust hybrid cloud connectivity solution. See [About Aviatrix Edge Spoke Gateway](/docs/enterprise/10.1/concepts-architectures/components/edge-gateways/edge-spoke-overview) . * **Aviatrix Controller and CoPilot** deployed in cloud to provide the unified management and control plane. ## What are the Aviatrix Hybrid Cloud Edge use cases? The solution provides hybrid and multicloud connectivity over private and public networks. See: * [Edge Transit Gateway Design Patterns](/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/edge-transit-design-patterns) * [Edge Spoke Gateway Design Patterns](/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/edge-spoke-design-patterns) * [Edge Spoke Gateway Use Cases](/docs/enterprise/10.1/concepts-architectures/components/edge-gateways/edge-spoke-usecases) ## How do I deploy Aviatrix Edge Transit and Spoke Gateway? You can deploy Aviatrix Edge Transit and Spoke Gateways on these platforms: * [Aviatrix Edge Platform](/docs/enterprise/10.1/concepts-architectures/components/edge-gateways/edge-aep) (on a hardware appliance with Aviatrix managed software) * [Equinix Network Edge](/docs/enterprise/10.1/concepts-architectures/components/connectivity/edge-equinix) * [Megaport Virtual Edge](/docs/enterprise/10.1/concepts-architectures/components/connectivity/edge-megaport) * [Self Managed Platform](/docs/enterprise/10.1/guides/connectivity/edge/edge-spoke-workflow-selfmanaged) # Edge Spoke Gateway Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/components/edge-gateways/edge-spoke-overview The Aviatrix Edge Spoke Gateway provides a secure, high-performance networking solution for on-premises to cloud. It enables dynamic routing, end-to-end encryption, segmentation and supports advanced NAT for connectivity to various cloud providers and on-premises environments while providing... ## Overview The Aviatrix Edge Spoke Gateway provides a secure, high-performance networking solution for on-premises to cloud. It enables dynamic routing, end-to-end encryption, segmentation and supports advanced NAT for connectivity to various cloud providers and on-premises environments while providing centralized management and operational visibility. The Edge Spoke Gateway simplifies hybrid cloud networking by: * **Horizontal scalability**: Supports scaling out by deploying multiple gateways and peering them with Transit Gateways or Edge Transit Gateways across clouds and regions. * **Advanced NAT functions**: Provide customized SNAT and DNAT to resolve overlapping CIDR issues between cloud and on-premises networks. The Advanced NAT functions work on both HPE-enabled and HPE-disabled Spoke Gateways. * **Local Internet Breakout**: Allows direct Internet egress from the edge with SNAT-enabled NAT interfaces. * **Built-in security**: Delivers L4 distributed firewall, end-to-end encryption, and network segmentation, all managed from a unified dashboard. * **VLAN connectivity**: Enables multiple VLAN interfaces with VRRP for high availability, along with LAN segmentation using network domains and policies. * **Flexible routing**: Supports transitive routing between multiple Transit Gateways and multicloud Transit peering over the public Internet. ## Does Aviatrix Edge Spoke Gateway support scalability? The Edge Spoke Gateway can be scaled horizontally and peered to the Transit Gateway or Edge Transit Gateway across single or multiple clouds and regions. See [Edge Spoke Gateway Horizontal Scaling](/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/edge-spoke-scaling). ## Does Aviatrix Edge Spoke Gateway support NAT functions? For the use case where the Cloud Service Provider (CSP) network CIDR overlaps with the on-premises network CIDR, you can enable customized SNAT and DNAT on the Edge Spoke Gateway to resolve the overlapping CIDR issue. See [Customized SNAT and DNAT on Edge](/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/edge-spoke-snat-dnat). ## Does Aviatrix Edge Spoke Gateway support Local Internet Breakout? You can set up an Edge Spoke Gateway as the default gateway and enable SNAT to route egress connectivity directly through the NAT interface of the Internet. See [About Local Internet Breakout at Network Edge](/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/edge-spoke-local-internet-breakout). ## How does Aviatrix Edge Spoke Gateway provide security? The Aviatrix Hybrid Cloud solution is an extension of the Aviatrix cloud-native networking and security platform which has security built into the data plane. Security capabilities include: * L4 Distributed Firewall * Network Segmentation * End-to-End Encryption * Unified single dashboard for security management. ## Does Aviatrix Edge Spoke Gateway support VLAN connectivity? The Edge Spoke Gateway supports VLAN connectivity with multiple VLAN interfaces that can be enabled on Edge Gateway with VRRP support. Aviatrix Hybrid Cloud solution also enables you to segment your on-premises network traffic for your LAN network segments through network domains and connection policies. See [About Extending VLAN Segmentation to the Cloud](/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/edge-spoke-vlan-segmentation). ## How can I do transitive routing with Aviatrix Edge Spoke Gateway? You can enable transitive routing on an Edge Spoke Gateway to forward traffic between multiple Transit Gateways that are connected to it. See [Transitive Routing with Edge Spoke Gateway](/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/edge-spoke-transitive-routing). ## How can I do transit peering across multicloud with Aviatrix Edge Spoke Gateway? You can create Transit Gateway peering as a secondary path for forwarding traffic over the public network. See [Transit Peering Over Public Network with Edge Spoke Gateway](/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/edge-spoke-transit-peering). # Edge Spoke Gateway Use Cases Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/components/edge-gateways/edge-spoke-usecases This document illustrates these common use cases for deploying Aviatrix Edge Spoke Gateway. This document illustrates these common use cases for deploying Aviatrix Edge Spoke Gateway. * [Extend Aviatrix to Edge Locations](#extend-aviatrix-to-edge-locations) * [Aviatrix Edge Spoke Gateway for On-Premises - Cloud Orchestration, Simplified Onboarding](#aviatrix-edge-spoke-gateway-for-on-premises) * [Aviatrix Edge Spoke Gateway for On-Premises - Increased Performance, Reduced Operational Overhead, Bottlenecks Removed](#increased-performance-reduced-operational-overhead-and-bottlenecks-removed) * [Edge Spoke Gateways for Multiple Edge Sites](#multiple-edge-spoke-gateways-for-multiple-edge-sites) * [Multiple Edge Spoke Gateways for Single Edge Site](#multiple-edge-spoke-gateways-for-single-edge-site) * [Edge Spoke Gateway Connectivity to Different LAN Routers (Multiple Sites)](#edge-spoke-gateway-connectivity-to-different-lan-routers-multiple-sites) * [Edge Spoke Gateway Connectivity to Different LAN Routers (Single Site)](#edge-spoke-gateway-connectivity-to-different-lan-routers-single-site) * [Edge Spoke Gateway to AWS TGW Domain with FireNet Enabled](#edge-spoke-gateway-to-aws-tgw-domain-with-firenet-enabled) ## Extend Aviatrix to Edge Locations This diagram illustrates Aviatrix Edge Gateways deployed at multiple edge locations. ![Aviatrix Edge Gateways deployed at multiple edge locations](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/components/edge-gateways/edge_usecase_edge_location.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=8e9156f7b0e8fdaa04ff817e0c427fba) ## Aviatrix Edge Spoke Gateway for On-Premises ### Cloud Orchestration and Simplified Onboarding Ease of onboarding * Instantly onboard HW orchestrated via Cloud * Manage and operate the Edge HW and network software via cloud * No IT expertise required to connect and deploy the Edge HW Drive better cost & FTE efficiencies * Simplified deployment with automation * Deploy Edge network software to multiple devices at scale via Cloud * Single automation stack across cloud and on-prem * Faster deployment and onboarding of connectivity from on-prem to cloud and on-prem to multi-cloud ![Edge on-premises design for cloud orchestration and simplified onboarding](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/components/edge-gateways/edge-onpremise-design-1.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=c04cfbe5918c25bf1123bdbe6df3bcd0) ### Increased Performance, Reduced Operational Overhead, and Bottlenecks Removed * IPSec Encrypted line rate throughput from cloud to on-prem via public or private connectivity * Repeatable architecture * Same overlay from Cloud to Edge * Consistent security protocols * Config driven via cloud and consistent secure IPSec state across Edge * E2E visibility – Within Cloud and Cloud to Edge ![Edge on-premises design for increased performance](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/components/edge-gateways/edge-onpremise-design-2.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=cafb7c7dde45ca2360c402e7f658db1d) ## Multiple Edge Spoke Gateways for Multiple Edge Sites This use case illustrates Edge Gateways with BGP connection to a VIP. The Edge Gateways can be set up at multiple edge sites as shown in the diagram with Site A and Site B. ![Edge Spoke Gateways for multiple edge sites](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/components/edge-gateways/edge-no-horizontal-scale.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=44cd040e52b41491a3bad64ba2af6390) ## Multiple Edge Spoke Gateways for Single Edge Site This use case illustrates multiple Edge Gateways on the same edge site for horizontal scaling or multiple Active-Active connections from cloud to edge. ![Multiple Edge Spoke Gateways for single edge site](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/components/edge-gateways/edge-horizontal-scale.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=f12435becc2cefe551df4262ded8a699) ## Edge Spoke Gateway Connectivity to Different LAN Routers (Multiple Sites) This use case illustrates Edge Gateways on different edge sites with active and standby peering connections to different LAN routers. ![Edge Spoke Gateway connectivity to different LAN routers across multiple sites](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/components/edge-gateways/edge-no-hs-lan-connection.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=b1a0f46eeda3c1a9b71d206385a6e9d0) ## Edge Spoke Gateway Connectivity to Different LAN Routers (Single Site) This use case illustrates multiple Edge Gateways on the same edge site with peering connections to different LAN routers. > [!NOTE] > > Only Active-Active high availability mode is supported with more than two Edge > Gateways on the same site. ![Edge Spoke Gateway connectivity to different LAN routers on a single site](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/components/edge-gateways/edge-hs-lan-connection.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=3ca09de74fe32a12b3ba5f6fbb7fb7cd) ## Edge Spoke Gateway to AWS TGW Domain with FireNet Enabled When FireNet inspection is enabled for an AWS TGW domain, the Aviatrix Edge domain cannot be associated to that domain. Because of a limitation that is attributed to how AWS configures its TGW domains, the on-premises routes from the Edge domain are not installed in the Spoke TGW route table via the Aviatrix Transit Gateway. ![Edge Spoke Gateway to AWS TGW domain with FireNet enabled - limitation](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/components/edge-gateways/edge-aws-tgw-firenet-1.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=5edb854e205c260b390a2ec8628d5427) To overcome the AWS TGW domain limitation, create another pair of Edge Gateways and associate the gateways to the Transit Gateways with the associated TGWs without segmentation on Edge. ![Edge Spoke Gateway to AWS TGW domain with FireNet enabled - workaround](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/components/edge-gateways/edge-aws-tgw-firenet-2.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=a06a2402790d919c83527d1dd7cdc170) # Edge Transit Gateway Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/components/edge-gateways/edge-transit-overview The Aviatrix Edge Transit Gateway provides a secure, high-performance networking solution designed to simplify and accelerate hybrid and multicloud connectivity. ## Overview The Aviatrix Edge Transit Gateway provides a secure, high-performance networking solution designed to simplify and accelerate hybrid and multicloud connectivity. It enables seamless routing and end-to-end encryption across various cloud providers and on-premises environments while providing centralized management and operational visibility. The Edge Transit Gateway simplifies hybrid cloud networking by: * **Unified cloud-native routing**: Seamlessly connects cloud environments and on-premises datacenters using dynamic routing protocols, reducing operational overhead. * **High Performance Encryption (HPE)**: Secure end-to-end encrypted connections, ensuring data privacy and compliance. * **Centralized visibility**: Provides real-time insights and monitoring across multicloud and hybrid networks, improving operational efficiency. * **Automated deployments**: Integration with tools such as Terraform, allows automated and repeatable infrastructure provisioning, reducing the need for manual intervention. ## What makes Aviatrix Edge Transit Gateway different from other alternatives? The following differentiates the Edge Transit Gateway from other alternatives: * **Cloud-native and multicloud ready**: The Aviatrix Platform was built for the cloud, and its integration with AWS, Azure, GCP, and other cloud providers allows to provide a consistent experience across all environments. * **High Performance Encryption (HPE)**: Aviatrix's patented encryption technology ensures minimal performance overhead while maintaining secure connections. * **Automation-first approach**: With Infrastructure as Code (IaC) capabilities and Terraform integration, Aviatrix enables faster, scalable, and automated network provisioning. * **Partner integrations**: Aviatrix's partnerships with infrastructure providers such as Equinix and Megaport enable seamless hybrid cloud connectivity across distributed locations. ## Does Aviatrix Edge Transit Gateway support high availability? The Edge Transit Gateway supports active-active peering to the Aviatrix Transit Gateway in cloud and to Aviatrix Edge Spoke Gateway for high availability. ## Does Aviatrix Edge Transit Gateway support connecting to multiple clouds? The Edge Transit Gateway supports connecting to AWS, Azure, GCP and OCI cloud environments. An Edge Transit Gateway pair deployed in high availability (HA) can connect to all these cloud service providers simultaneously. ## Does Aviatrix Edge Transit Gateway support interconnectivity provider environments? The Edge Transit Gateway is integrated with Equinix Network Edge and Megaport Virtual Edge environments where the service is deployed as a VM in these environments and bootstrapped via zero-touch provisioning (ZTP). The Edge Transit Gateway can then be leveraged to build virtual connections to cloud service providers and build encrypted backbone across these provider environments. ## Does Aviatrix Edge Transit Gateway support connecting to remote datacenter or on-premises locations? The Edge Transit Gateway supports remote sites and provides end-end encryption by peering with Edge Spoke Gateways deployed at remote locations. You can deploy Edge Spoke Gateways at your datacenter and on-premises locations in an appliance form factor or as a VM and peer the Edge Spoke Gateway to the Edge Transit Gateway for high performance encrypted connectivity. ## How do I deploy Aviatrix Edge Transit Gateway? See: * [Deploy an Edge Transit Gateway on the Aviatrix Edge Platform](/docs/enterprise/10.1/guides/connectivity/edge/edge-transit-workflow-onpremise) * [Deploy an Edge Transit Gateway on Equinix Network Edge](/docs/enterprise/10.1/guides/connectivity/edge/edge-transit-workflow-equinix) # Egress Security Score Protection Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/components/egress/egress-overview Controller 8.0 and the enablement of the DCF feature is required to view the Security > Egress > Overview tab. > [!NOTE] > > Controller 8.0 and the enablement of the DCF feature is required to view the Security > Egress > Overview tab. The Security > Egress > Overview tab shows the following: * Egress Security Score: Shows the egress security score for your VPC/VNets. The higher the score, the better your VPC/VNets are protected. * How is Egress Security Score Calculated: Learn more about how your Egress Security Score is calculated, and the scores assigned to VPC/VNets with varying protection statuses. * Protected VPC/VNets: View the breakdown of Unprotected, Monitored, Partially Protected, and Protected VPC/VNets. The chart also shows the number of VPC/VNets that have No Egress or that have an Unknown state. ![Egress Overview Tab](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/components/egress/egress-overview-tab.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=e72b4fa431ee9a1858570f8e91009b42) ## Egress Security Score The Egress Security Score card shows the current calculated protection score for your protected VPC/VNets. ![Egress Security Score Card](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/components/egress/egress-security-score-card.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=9937eeb8b6420ffdd750e4c2004a3a51) From here you can click *Manage VPC/VNets* to open the Egress VPC/VNets tab and manage your VPC/VNet protection. ## How is Egress Security Score Calculated? Click *Learn More* on the How is Egress Security Score Calculated? card to open the [Egress Security Score Explained](/docs/enterprise/10.1/reference/security/egress/egress-security-score-calculated) dialog. This dialog provides insights into how your Egress Security Score is calculated, including the status of your VPC/VNets and the scores assigned to them based on their protection status. ## Protected VPC/VNets ![Protected VPC/VNets Card](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/components/egress/protected-vpc-vnets-card.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=94bc5b3f236dd6e3ad573f6e7569664a) The Protected VPC/VNets card indicates the protection status of the VPC/VNets in your onboarded cloud accounts. ## Related Topics * [About Egress Security Score](/docs/enterprise/10.1/reference/egress-filtering/egress-security-score) * [How is Egress Security Score Calculated](/docs/enterprise/10.1/reference/security/egress/egress-security-score-calculated) * [Egress Traffic Overview](/docs/enterprise/10.1/reference/ui/security/egress#view-egress-overview) * [Analysis of Egress Traffic](/docs/enterprise/10.1/reference/ui/security/egress#view-egress-overview) * [Monitoring Egress Traffic](/docs/enterprise/10.1/guides/security/egress/egress-monitor-vpc-vnets) * [Protecting Egress Traffic with the Egress Protection Workflow](/docs/enterprise/10.1/guides/security/egress/egress-protect-vpc-vnets) # Native Peering Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/components/native-peering-about You can use Aviatrix CoPilot to create native peering connections between cloud VPCs or VNets. There are two types of native peering: VPC/VNet to VPC/VNet, and Transit VNet to Spoke VNet (previously referred to as Azure ARM Spoke through Native Peering). You can use Aviatrix CoPilot to create native peering connections between cloud VPCs or VNets. There are two types of native peering: VPC/VNet to VPC/VNet, and Transit VNet to Spoke VNet (previously referred to as Azure ARM Spoke through Native Peering). The first type of native peering (AWS VPC peering or Azure VNet peering) allows you to connect two VPCs or VNets within the same cloud account or across different cloud accounts, enabling private traffic flow between them. This type of native peering is ideal for east-west traffic flow, where you want to connect VPCs or VNets in the same region or across regions without routing traffic through a Transit Gateway. The second type of native peering (Transit VNet to Spoke VNet) allows you to connect a Transit VNet to a Spoke VNet in Azure, enabling private traffic flow between them. This type of native peering is useful for scenarios where you want to connect a central hub (Transit VNet) to multiple spoke networks (Spoke VNets) for centralized management and routing. Native peering is supported for AWS and Azure. ## AWS VPC Peering AWS VPC peering allows you to connect VPCs within the same AWS account or across different AWS accounts. This connection enables private traffic flow between the VPCs, allowing resources in one VPC to communicate with resources in another VPC without going through the public internet. * You can peer two VPCs together. ![AWS VPC Peering - Two VPCs](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/components/native-peering1.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=f140112eb6b59f119977fdba892eb6af) * You can peer three VPCs together, where each VPC will have access to the resources from the other two VPCs because of its peering connections. ![AWS VPC Peering - Three VPCs Fully Connected](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/components/native-peering2.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=b00047bf2d88276c9797c5aedc8ce777) * You can peer three VPCs together where one is in the middle, meaning that the middle VPC will have access to the resources from the other two VPCs, but the other two VPCs will not have access to each other. ![AWS VPC Peering - Three VPCs Hub and Spoke](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/components/native-peering3.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=fee121e033359c0e9e89d3b26eeef36e) ### Use Cases for AWS VPC Peering * **Inter-Departmental Connectivity**: For instance, two different departments within an organization needing full access to each other's resources. * **Customer Segregation**: When serving multiple customers, VPC Peering ensures that each customer's resources remain isolated while still allowing access to shared resources. ### Example Scenario * **Inter-Departmental Connectivity**: The Finance and HR departments each have their own VPCs in different AWS accounts for security and compliance reasons. By creating a VPC peering connection between these VPCs, both departments can securely share payroll and benefits data, while keeping their core applications isolated from each other and the public internet. Steps to create this VPC peering connection: 1. In CoPilot, navigate to **Connectivity** > **Native Peering** tab, and click **+ Native Peering**. 2. For **Cloud** type, select **AWS** (choose **Standard**, **GovCloud**, or **China** as appropriate). 3. From the **VPC/VNet 1** dropdown menu, select the Finance department VPC. 4. From the **VPC/VNet 2** dropdown menu, select the HR department VPC (in the other AWS account). 5. (Optional) To select custom route tables, turn **Select Route Tables** switch **On** and choose the route tables to peer for each VPC. Otherwise, leave it **Off** to peer all VPC CIDRs. 6. Click **Save** to create the peering connection. The new VPC peering connection will appear in the table, and resources in both VPCs can now communicate privately for secure data sharing. ## Azure VNet Peering Azure VNet peering allows you to connect two VNets within the same Azure subscription or across different Azure subscriptions. This connection enables private traffic flow between the VNets, allowing resources in one VNet to communicate with resources in another VNet without going through the public internet. ### Use Cases for Azure VNet Peering * **Business Unit Segregation**: When serving multiple business units or subsidiaries, VNet peering ensures that each unit's resources remain isolated in their own VNets, while still allowing access to shared services (such as a central authentication or logging service) hosted in another VNet. * **Cross-Environment Integration**: For securely connecting environments like staging and production for testing, migration, or blue/green deployments. ### Example Scenario * **Business Unit Segregation**: You have a production VNet and a development VNet in different Azure subscriptions. By creating a VNet peering connection between them, you enable secure, private communication for application testing and data sharing, without exposing resources to the public internet. Steps to create this VNet peering connection: 1. In CoPilot, navigate to **Connectivity** > **Native Peering** tab, and click **+ Native Peering**. 2. For **Cloud** type, select **Azure** (choose **Global**, **GovCloud**, or **China** as appropriate). 3. From the **VPC/VNet 1** dropdown menu, select the production VNet. 4. From the **VPC/VNet 2** dropdown menu, select the development VNet (in the other Azure subscription). 5. Click **Save** to create the peering connection. The new VNet peering connection will appear in the table, and resources in both VNets can now communicate privately. ## Transit VNet to Spoke VNet Peering > [!NOTE] > > Previously this was referred to as Azure ARM Spoke through Native Peering. Transit VNet to Spoke VNet peering enables private connectivity between a central Transit VNet and one or more Spoke VNets in Azure. This approach is ideal for centralized management, routing, and secure communication between hub and spoke networks. ### Use Cases for Transit VNet to Spoke VNet Peering * **Centralized Network Management**: When you want to manage multiple Spoke VNets from a central hub (Transit VNet) for consistent routing, security policies, and monitoring. * **Scalable Network Architecture**: For scenarios where you need to connect multiple Spoke VNets to a single Transit VNet, allowing for efficient routing and management of traffic between them. ### Example Scenario * **Centralized Security and Routing**: An enterprise has a Transit VNet acting as a central hub for security inspection, routing, and monitoring. Multiple Spoke VNets are deployed for different business units or applications. By peering each Spoke VNet with the Transit VNet, all outbound and inter-spoke traffic is routed through the Transit VNet, enabling centralized policy enforcement and simplified management. Steps to create this Transit VNet to Spoke VNet peering connection: 1. In Aviatrix CoPilot, go to **Cloud Fabric** > **Gateways > Transit Gateways**. 2. In the table, locate the Azure-based Transit Gateway that will act as the central hub for security and routing and click the **Manage Gateway Attachments** icon on the right side of its row. 3. In the Manage Gateway Attachments dialog, click the Azure VNet tab. 4. From the Azure VNets dropdown, select one or more Spoke VNets for specific business units or applications. 5. Click **Save** to create the attachments (peering connections). All Spoke VNets will now have private connectivity to the Transit VNet, enabling centralized security inspection, routing, and monitoring for the enterprise. ## Related Topics * [Creating AWS VPC Peering Connection](/docs/enterprise/10.1/guides/connectivity/peering/aws-vpc-peering) * [Creating Azure VNet Peering Connection](/docs/enterprise/10.1/guides/connectivity/peering/azure-vnet-peering) * [Creating an Azure-Based Transit Gateway to Azure VNet Attachment](https://legacy.docs.aviatrix.com/documentation/latest/network/transit-attach-vnet.html) * [View an External Connection Details](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-details.html) # About High Performance Encryption Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/components/security/hpe-about Aviatrix High Performance Encryption tunneling techniques establish multiple tunnels between the two virtual routers, thus allowing all CPU cores to be used for performance scaling with the CPU resources, as shown below. Aviatrix High Performance Encryption tunneling techniques establish multiple tunnels between the two virtual routers, thus allowing all CPU cores to be used for performance scaling with the CPU resources, as shown below. ![insane\_tunnel\_diagram](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/components/security/insane-tunnel-diagram.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=f4b6eefa3f6903883fb154847f6461fa) With Aviatrix High Performance Encryption Mode tunneling, IPsec encryption can achieve 10Gbps, 25Gbps and beyond, using the multiple CPU cores in a single instance, VM or host. > [!NOTE] > > NAT on Transit Gateways is not supported when High Performance Encryption (HPE) is enabled. > > For Spoke Gateways, Single IP SNAT, customized SNAT, and DNAT are supported on > HPE-enabled gateways in AWS, Azure, and Edge. For Azure Spoke Gateways with > Subnet Groups enabled (Controller 10.1 and later), Single IP SNAT, customized > SNAT, and DNAT are supported. ## Why is Transit VPC/VNet performance capped at 1.25Gbps? In the current Transit VPC/VNet solution, the throughput is capped at 1.25Gbps regardless of whether you have a 10Gbps connection between an on-premises network and the cloud (Direct Connect (DX)/ExpressRoute/FastConnect/InterConnect) link. This is because the Transit VPC/VNet deployment relies on an IPsec session between the Virtual Private Gateway (VGW)/VPN Gateway and Transit Gateway, and the VGW/VPN Gateway itself has a performance limitation. [AWS VGW](https://aws.amazon.com/vpc/faqs/) and other Cloud Service Providers' IPsec VPN solutions have a published performance cap of 1.25Gbps. Most virtual routers or software-based routers are built with general purpose CPUs. Despite the vast CPU technology advancement, why does IPsec performance not scale further? It turns out the problem lies in the nature of tunneling, a common technique in networking to connect two endpoints. When two general purpose server or virtual machine-based routes are connected by an IPsec tunnel, there is one UDP or ESP session going between the two machines, as shown below. ![tunnel\_diagram](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/components/security/tunnel-diagram.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=f65522aa53527ded97a8fee00e918e1d) In the above diagram, the virtual router has multiple CPU cores, but since there is only one tunnel established, the Ethernet Interface can only direct incoming packets to a single core, thus the performance is limited to one CPU core, regardless how many CPU cores and memory you provide. This applies to all tunneling protocols, including IPsec, GRE, and IPIP. ## How does Aviatrix High Performance Encryption work? When a gateway is launched with [High Performance Encryption](https://legacy.docs.aviatrix.com/documentation/latest/network/transit-gateway-settings.html?expand=true#hpe) enabled, a new /26 public subnet is created where the High Performance Encryption Mode gateway is launched on. Aviatrix High Performance Encryption builds high performance encryption tunnel over private network links. The private network links are Direct Connect (DX)/AWS Peering (PCX), Azure ExpressRoute, GCP FastConnect, and OCI InterConnect. For High Performance Encryption between two gateways, between a Transit Gateway and a Spoke Gateway, or between two Transit Gateways (Transit Peering), the Aviatrix Controller automatically creates the underlying peering connection and builds the tunnels over it. Since High Performance Encryption tunnels are over private network links, the VPC/VNet route architecture is described as below, where virtual machine (EC2/GCE/OC) instances associated route entry to the remote site point to Aviatrix Gateway, and the Aviatrix Gateway instance associated route entry to remote site points to PCX or VGW. ![insane\_routing](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/components/security/insane-routing.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=22ff09a6a1a36ef9752ee77124b3d104) ## What are the use cases for High Performance Encryption? * High performance [Encrypted Transit](/docs/enterprise/10.1/guides/connectivity/transit/transitvpc-workflow) * High performance encryption over Direct Connect/ExpressRoute/FastConnect/InterConnect * Overcome VGW performance limit and 100 route limits ## How can you deploy Aviatrix High Performance Encryption? Aviatrix High Performance Encryption Mode is integrated into the Transit Network solution to provide 10Gbps performance between on-premises networks and Transit VPC/VNet with encryption. For VPC/VNet to VPC/VNet, High Performance Encryption Mode can achieve 25 - 30Gbps. High Performance Encryption Mode can also be deployed in a flat (as opposed to Transit VPC/VNet) architecture for 10Gbps encryption. The diagram below illustrates the high performance encryption between Transit VPC/VNet and on-premises networks, between Transit VPC/VNet and Spoke VPC/VNet. ![insane\_transit](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/components/security/insane-transit.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=c69b158a1934f7f89473c2a6bc9f8d3a) # High Performance Encryption for GCP Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/components/security/hpe-for-gcp Because GCP network infrastructure is different from AWS and Azure, Aviatrix High Performance Encryption mode for GCP differs from AWS and Azure in the following ways: Because GCP network infrastructure is different from AWS and Azure, Aviatrix High Performance Encryption mode for GCP differs from AWS and Azure in the following ways: * Only Spoke and Transit Gateway types are supported. * Only Multicloud Transit functionality is supported with High Performance Encryption Mode gateways * The Advertise Transit VPC Network CIDR(s) feature is not supported with a High Performance Encryption Mode Gateway. * GCP High Performance Encryption Mode supports only ActiveMesh 2.0. For GCP High Performance Encryption, MTU is 1460 and MSS is 1330 bytes. ## Subnet Prefix Length for GCP High Performance Encryption Mode For GCP High Performance Encryption, the gateway subnet prefix length cannot be greater than /24. > [!NOTE] > > Aviatrix highly recommends that customers utilize a subnet exclusively for > deploying High Performance Encryption enabled gateway without any other > instances in the subnet. * [High Performance Encryption Performance Benchmarks](/docs/enterprise/10.1/reference/gateways/hpe-perf-benchmark) * [About Transit Network Segmentation](/docs/enterprise/10.1/concepts-architectures/components/security/transit-segmentation-faq) # Transit Network Segmentation Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/components/security/transit-segmentation-faq Aviatrix Multicloud Transit Segmentation provides network isolation through network domains and connection policies to Aviatrix Transit network where both Spoke and Transit networks deploy Aviatrix Gateways across multi-region and multicloud. ## What is Multicloud Transit Segmentation? Aviatrix Multicloud Transit Segmentation provides network isolation through network domains and connection policies to Aviatrix Transit network where both Spoke and Transit networks deploy Aviatrix Gateways across multi-region and multicloud. The concept is described in the below diagram. ![Transit Segmentation](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/components/security/transit-segmentation.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=dc9e2bb6ae6ef1f49aabceec4a7d4417) Where Spokes associated with the blue domain can communicate with each other while Spokes associated with the green domain can communicate with each other. But there is no cross communication between blue domain and green domain unless there is connection policy. The concept is the same as [Network Domains](/docs/enterprise/10.1/concepts-architectures/components/tgw/tgw-orchestrator-faq#network-domain-connection-policy) and [Connection Policies](/docs/enterprise/10.1/concepts-architectures/components/tgw/tgw-orchestrator-faq#network-domain-connection-policy) defined in TGW Orchestrator, except this is implemented with Aviatrix Transit where both Spokes and Transit VPC/VNet deploy Aviatrix gateways. (Note the segmentation works with Azure native Spoke VNets.) ## What is a Network Domain in Multicloud Transit? A Network Domain is an Aviatrix enforced network of VPC/VNet members, where VPC/VNets in the Network Domain can communicate with each other, and VPC/VNets not in the network domain cannot communicate with VPC/VNets in the Network Domain. One or more Spoke VPC/VNets are members in a network domain. Spokes in a network domain can communicate with each other via an Aviatrix Transit Gateway. The Aviatrix Controller dynamically programs and updates both VPC/VNet route tables so that instances in different Spoke VPC/VNets in the same domain can communicate with each other. Two network domains are not connected, i.e., a Spoke in one domain has no connectivity to another Spoke in a different domain. Connection policy must be specified to connect the two domains so that Spokes in each domain can communicate with each other. The Network Domain also applies to the hybrid connection from Aviatrix Transit Gateway to on-prem or remote sites. Each BGP peer or connection can be associated with one Network Domain. ## What is a Connection Policy? A connection policy is a rule enforced by Aviatrix for cross Network Domain connectivity. ## What are the benefits of using Network Domains and Connection Policies? The key use case for building Network Domains is to segment traffic for enhanced security posture. Using Network Domains and Connection Policies allow you to identify groups of Spokes and Edges with the same requirements from a networking point of view and then apply connection policies at the group level. This avoids having to individually specify connections at the Spoke level. The Aviatrix Controller takes care of route programming of all route tables. ## Can an Aviatrix Transit Network Domain work with TGW Orchestrator Network Domain? They do not work together at this time, however we have plan to integrate them in the future. ## How do I set up Multicloud Transit Segmentation? Follow the [Transit Segmentation Workflow](/docs/enterprise/10.1/guides/security/network-segmentation-secured) . ## How many Network Domains are supported in Multicloud Transit Segmentation? The configuration of Network Domain is managed as a global setting. Each Controller can support a maximum of 200 network domains. The number of network domains is independent of the number of gateways. ## What is the difference in implementation of Segmentation between Release 6.1 and Release 6.0? In Release 6.1 and later, each Network Domain is implemented as an individual route table on the Aviatrix Transit Gateway. This allows better handling for the default route (0.0.0.0/0) traffic if different domains require different egress next hop. In addition, duplicate Spoke CIDRs attached to different Aviatrix Transit Gateways can co-exist if they belong to different domains. ## What is the limitation of Segmentation? * Duplicated CIDRs that cross domains or cross transits may not work all the time. Aviatrix does not support duplicated CIDRs that cross domains or cross transits. * Overlapping CIDRs advertised from on-prem to different Spoke network domains connected to one Aviatrix Transit Gateway is not supported. # AWS Transit Gateway Orchestrator Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/components/tgw/tgw-landing-page Landing page for AWS Transit Gateway (TGW) Orchestrator resources, including overview, FAQ, design patterns, and attachment workflows. ## Overview * [Overview of AWS Transit Gateway Orchestrator Features](https://legacy.docs.aviatrix.com/documentation/latest/network/tgw-create.html/tgw/tgw-overview) * [Aviatrix AWS Transit Gateway Orchestrator FAQ](/docs/enterprise/10.1/concepts-architectures/components/tgw/tgw-orchestrator-faq) * [TGW Design Patterns](/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/tgw-design-patterns) * [AWS Transit Gateway Route Limit Test Validation](https://legacy.docs.aviatrix.com/documentation/latest/network/tgw-attach-transit-gw-to-tgw.html/tgw/tgw-route-limit) ## AWS TGW Attachments * [Creating an AWS TGW](https://legacy.docs.aviatrix.com/documentation/latest/network/tgw-vpc-attachment-create.html) * [Attaching a Transit Gateway to an AWS TGW](https://legacy.docs.aviatrix.com/documentation/latest/network/tgw-vpn-connection-create.html) * [VPC Attachments for AWS TGW](https://legacy.docs.aviatrix.com/documentation/latest/network/tgw-direct-connect.html) * [Creating an AWS Transit Gateway VPN Connection](https://legacy.docs.aviatrix.com/documentation/latest/network/tgw-peering-attachment-create.html) * [Setting up AWS Transit Gateway Direct Connect](https://legacy.docs.aviatrix.com/documentation/latest/network/tgw-connect.html) * [Creating a TGW Peering Attachment](https://legacy.docs.aviatrix.com/documentation/latest/network/tgw-edge-segmentation.html) * [Creating AWS TGW Network Domains](/docs/enterprise/10.1/guides/connectivity/tgw/tgw-overview#tgw-network-domain) * [Creating TGW Connection Attachments](/docs/enterprise/10.1/guides/connectivity) * [Configuring AWS TGW Edge Segmentation](/docs/enterprise/10.1/guides/connectivity) ## AWS TGW Migration * [Migrating a CSR Transit to AWS Transit Gateway (TGW)](/docs/enterprise/10.1/guides/connectivity/tgw/tgw-csr-migrate) * [Migrating a DIY TGW to Aviatrix Managed TGW Deployment](/docs/enterprise/10.1/guides/connectivity/tgw/tgw-migrate-aviatrix-tgw) # AWS Transit Gateway Orchestrator FAQ Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/components/tgw/tgw-orchestrator-faq Aviatrix AWS TGW (Transit Gateway) Network Orchestration creates a single hub for transit networks across multiple AWS regions. The AWS TGW feature in Aviatrix CoPilot connects your Aviatrix platform to this AWS feature and enables you to attach Spoke VPCs to your transit network. Aviatrix AWS TGW (Transit Gateway) Network Orchestration creates a single hub for transit networks across multiple AWS regions. The AWS TGW feature in Aviatrix CoPilot connects your Aviatrix platform to this AWS feature and enables you to attach Spoke VPCs to your transit network. ## AWS TGW Functions AWS TGW simplifies, abstracts, and extends the latest AWS Transit Gateway service. The Aviatrix platform makes Transit Gateway-based Transit architecture deployable by overcoming [Transit Gateway limitations](https://legacy.docs.aviatrix.com/documentation/latest/network/tgw-limits.html). * **Functional Completeness** Aviatrix makes AWS Transit Gateway functionally deployable. This feature programs and updates both VPC route tables and TGW route tables so the routes are dynamically propagated to the Spoke VPCs. Read [Why should I use Aviatrix AWS TGW to build a transit network architecture?](#why-should-i-use-aviatrix-aws-tgw-to-build-a-transit-network-architecture) for more details. * **Segmentation** The AWS TGW abstracts the route domain and route propagation concepts in Transit Gateway that allows you to create network segmentation by policy and intent. * **Multi-Account** AWS TGW automates the AWS Resource Access Manager (RAM) to allow you to manage multi account VPC attachments to TGW. * **Scaling** Aviatrix solution overcomes Transit Gateway route limits to scale the hybrid deployment to hundreds/thousands of VPCs. * **Hybrid** AWS TGW extends the Transit Gateway capability to include Direct Connect support for connecting to on-prem data center. 1. Orchestrates VPC to VPC and on-prem to VPC connectivity via AWS Transit Gateway. 2. Automates AWS Resource Access Manager (RAM) for multi-account support. 3. Creates security boundaries between groups of VPCs to achieve network segmentation. 4. Out-of-the-box integration of AWS Transit Gateway and Direct Connect and Internet to re-use what has been built. 5. Provides [High Performance Encryption Mode high performance](https://legacy.docs.aviatrix.com/documentation/latest/network/gateway-hpe-enable.html) and features rich hybrid network for connecting to on-prem. 6. Supports Bring Your Own Firewall to TGW deployment for inline traffic inspection ([Firewall Network](/docs/enterprise/10.1/guides/security/firenet/firewall-overview)). 7. Orchestrate AWS TGW Inter Region Peering and expand the Security Domains to be global. 8. Advanced mode for end-to-end encryption where Aviatrix Gateways are deployed in the AWS Spoke VPCs and Azure Spoke VNets. The AWS TGW is illustrated in the diagram below. ![tgw\_overview](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/components/tgw/tgw-overview.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=38c151f8b95cdd7ee846f93d1ab29bd3) In the above diagram, AWS VPCs are grouped into four domains: Dev domain, Prod domain, Shared Service domain and Aviatrix Edge domain. Each VPC in the same domain can communicate with each other via AWS Transit Gateway. VPCs in Prod domain cannot communicate with VPCs in Dev Domain, while all VPCs in Dev domain and Prod domain can communicate with Shared service domain and Aviatrix Edge domain. Through the Aviatrix Transit GW in the Aviatrix Edge domain, Spoke VPCs can communicate with on-prem over Direct Connect. In the deployment, the VPC in the Aviatrix Edge domain is a Spoke VPC from the Transit Gateway point of view. However, it serves as Transit VPC from the Aviatrix Transit Network point of view. No Aviatrix gateways are deployed in Spoke VPCs except in the Transit VPC. Aviatrix Transit GW serves as hub connecting to Azure and GCP networks. ## Transit Gateway (TGW) complements the AWS Transit Gateway service * **Dynamic Route Propagation** Using Aviatrix TGW is the only guaranteed way to ensure your on-prem routes are properly propagated to Spoke VPCs. AWS Transit Gateway propagates VPC CIDR and IPSec VPN routes to the Transit Gateway route table. But the routes are not propagated to the VPC route table. It is the account owner's responsibility to program VPC route tables. The Aviatrix AWS TGW feature dynamically updates route entries in the VPC route tables. * **Policy Abstraction** An AWS Transit Gateway provides the capability to allow two Transit Gateway route tables to propagate routes to each other, but the actual route entry programming is left to the owner. AWS TGW builds on that and allows customers to define policies that form a security boundary. * **Multi-Account Support** Automate the RAM resource sharing process to seamlessly manage multi-account VPC attachment. * **Troubleshooting** FlightPath allows a single pane of glass for troubleshooting connectivity with expert diagnostics capabilities. * **Hybrid and Multicloud Support** Native AWS Transit Gateway VPN and Direct Connect support. Furthermore, Aviatrix allows customers to bridge multiple Transit Gateways together in different and public clouds. * **Traffic Visibility** Netflow log support for traffic between on-prem and all VPCs. * **Stateful Firewall** Enforce security policy for all traffic between on-prem and all VPCs. * **10Gbps Transit** Support 10Gbps Transit network throughput. ## How does Transit Gateway work with Transit VPC? The AWS TGW leverages the Aviatrix Transit Network workflow for the hybrid connectivity function to an on-prem data center and branches. It enables the AWS TGW to connect with on-prem over Direct Connect or Internet. The Aviatrix AWS TGW feature can also be used as a stand-alone function for orchestrating VPC to VPC connections. When using the AWS TGW feature for hybrid connectivity, no gateways are deployed in the Spoke VPCs for hybrid function. ## How does the TGW Transit compare with Aviatrix VPC? Transit VPC refers to the transit deployment model where an Aviatrix Gateway is deployed in a Spoke VPC. This is now called "advanced mode" in the AVX Transit. The AWS TGW can be deployed with some Spoke VPCs run Aviatrix gateways. When is the right use case to run Aviatrix Spoke gateway? 1. If you need a packet in flight to be encrypted, launch an Aviatrix gateway in the Spoke VPC. 2. If you need various NAT functions between Spoke and Transit VPC, use an Aviatrix gateway in the Spoke VPC. 3. If you need to obtain Netflow and log information from the Spoke and Transit, use Aviatrix gateway. 4. If you want to build a fully isolated Transit network where there is no inter-VPC connectivity by default. ## There is AWS CloudFormation and Terraform support for Transit Gateway. Why should I use Aviatrix AWS TGW? AWS CloudFormation for [Transit Gateway](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ec2-transitgateway.html) is a resource construct for Transit Gateway. So is the [Terraform example](https://www.terraform.io/docs/providers/aws/r/ec2_transit_gateway_route_table.html). They are all useful solutions, but these constructs may not be sufficient to run your network. For example, a Transit Gateway does not propagate routes from on-prem to the VPC route table, meaning there is no guarantee that your VPC instances can reach a specific on-prem server or host. Even if you hard coded the list of CIDRs to shuffle them down to Transit Gateway, what happens when a new VLAN or Subnet is stood up on-prem. Who is going to notify you? A modern distributed network either requires BGP to dynamically propagate the routes or a CoPilot that dynamically updates the routes. No matter what approach you use, it is the only way to guarantee the network actually functions. At Aviatrix, we choose a software defined approach with our platform. Unless you plan to develop a platform like ours, you should consider using our product. Learn more about [Transit Gateway limitations](https://legacy.docs.aviatrix.com/documentation/latest/network/tgw-limits.html). ## TGW/Transit Segmentation Interoperability The TGW/Transit Segmentation Interoperability feature enables you to extend your AWS TGW network domains into other CSPs. When you are setting up your multicloud transit structure, you can attach an AWS TGW domain to an Aviatrix Transit network domain. Attachments in the AWS TGW network domains can then communicate with Aviatrix Spoke gateways and Site2Cloud connections in the same and in connected Transit network domains. ![tgw-interop](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/components/tgw/tgw-interop.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=0f41c043ac8054a246b9283ff6a4ec51) ### Prerequisites Ensure that the following conditions are met before connecting an AWS TGW domain to a Transit network: * The AWS TGW domains you want to attach to a Transit Gateway must already be connected to the [Edge Domain](#what-is-the-aviatrix-edge-domain). * The network domains must already be created in the Aviatrix CoPilot under **Networking > Network Segmentation > Network Domains**. > [!WARNING] > > Errors will occur if the prerequisites are not met. Go to [Associating a Spoke/Edge Gateway or a TGW Domain](/docs/enterprise/10.1/guides/security/network-segmentation-secured) for attachment instructions. ### Limitations * Not supported with TGW FireNet. * Segmentation is not supported for peered AWS TGWs. * If duplicate CIDRs exist when associating an AWS TGW domain with a Transit domain (for example, a TGW domain CIDR is the same as one in a Transit domain, because now the routes in the main route table are duplicated) an error is raised. ## What are Network Domains and Connection Policies? See [Implementing Network Segmentation in an Aviatrix-Managed Network](/docs/enterprise/10.1/guides/security/network-segmentation-secured) for information about Network Domains and Connection Policies. ## What is the Default\_Domain? When an Aviatrix Transit Gateway is created, the Default\_Domain is created and a route table corresponding to the Default\_Domain is created on the Transit Gateway. If you do not plan on building any network segmentation, you can use Default\_Domain for inter Spoke VPC and hybrid communications. ## What is the Shared\_Service\_Domain? When an Aviatrix Transit Gateway is created, the Shared\_Service\_Domain is created and a route table corresponding to the Shared\_Service\_Domain is created on Transit Gateway. You can attach a Spoke VPC to this domain and host your shared service instances such as your DevOps tools. Shared\_Service\_Domain is always connected to Default\_Domain and Aviatrix\_Edge\_Domain. ## What is the Aviatrix Edge Domain? When an Aviatrix Transit Gateway is created, the Aviatrix\_Edge\_Domain is created and a route table corresponding to the Aviatrix\_Edge\_Domain is created on the Transit Gateway. Aviatrix\_Edge\_Domain is designated for connecting VPCs managed by the AWS TGW to on-prem network. There must be one VPC attached to this domain. In the VPC, an Aviatrix Transit GW is deployed and used for data traffic forwarding between Spoke VPCs and on-prem network. Aviatrix\_Edge\_Domain is always connected to the Shared\_Service Domain and the Default\_Domain. ## Deploy the AWS TGW For information about how to deploy the AWS TGW in Aviatrix, see [Overview of AWS Transit Gateway Orchestrator Features](/docs/enterprise/10.1/guides/connectivity/tgw/tgw-overview) . ## AWS TGW deployment scenarios Here are some [design patterns](/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/tgw-design-patterns) that address your requirements. ## Can I change my plan or VPC attachment on AWS TGW? Yes, you can change your design any time. ## Migration Unlike a VPC, where once you have created it and launched instances in the VPC you cannot delete the VPC or move the instances easily, a Transit Gateway and its attachments can all be changed without making changes to the instances and VPC CIDRs. Simply detach the VPCs from the current Transit Gateway, launch a new Transit Gateway, and build it out again. The Aviatrix AWS TGW manages the entire life cycle of the network, including Network Domains, all Transit Gateway and attachments should be created and managed by AWS TGW. ## How does the CSR based Transit VPC solution compare with the Transit Gateway? Transit Gateway significantly simplifies building VPC connections. But the Transit Gateway itself is functionally incomplete for hybrid connection. For example, the Transit Gateway does not propagate routes to Spoke VPCs, which means using a Transit Gateway alone does not offer a functional hybrid solution. The example below illustrates how CSR based Transit VPC provides an end-to-end solution while a Transit Gateway alone leaves Spoke VPC route table all empty. ![tgw\_transit\_vpc\_compare](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/components/tgw/tgw-transit-vpc-compare.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=1a8c463649a465c0ef5d3aa3a8374e09) The missing function of Transit Gateway is listed as below: * Not able to propagate routes from on-prem to the Spoke VPCs. * Not able to connect with Direct Connect. * The Transit Gateway VPN has 100 route limits. * The Transit Gateway route table cannot summarize routes to advertise to Transit Gateway VPN. While you may think you can gather the on-prem routes and program the Spoke VPC tables, it is in reality not so simple. The on-prem routes change from time to time as new networks are added or removed, which means you need a reliable way to monitor the route changes, handle exceptions and deal with errors and duplicate routes — essentially a function carried by BGP or AWS TGW. ## Why should I use Aviatrix AWS TGW to build a transit network architecture? Aviatrix AWS TGW fulfills the need to propagate on-prem routes to the Spoke VPCs. This function is either carried by BGP or is software defined. In the Aviatrix case, it is software defined and performed by the Aviatrix Controller. The diagram below shows how the CSR Transit VPC, the Transit Gateway and the Aviatrix AWS TGW compare for route propagation function. As can be seen, in the CSR Transit VPC case, CSR propagates on-prem routes to Spoke VPC via BGP to VGW; the Transit Gateway has no route propagation to Spoke VPC. The Aviatrix Controller propagates routes to Spoke VPC through a software-defined mechanism. ![tgw\_transit\_orchestrator\_compare](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/components/tgw/tgw-transit-orchestrator-compare.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=cba1c6a0a7c534022243bebd9e109d4b) ## Solutions that an Aviatrix Gateway provide in the AWS TGW An Aviatrix Gateway deployed at the edge/transit VPC provides the following values: * Ensure the correctness of connectivity by monitoring and dynamically programming on-prem network address ranges to Spoke VPCs' route tables. * Avoid network outages by detecting and alerting overlapping and conflicting network address ranges between on-prem and all VPCs. * Avoids AWS VGW or Transit Gateway VPN 100 route limits by summarizing Spoke VPC CIDRs advertisements to on-prem network. * Provides traffic visibility by supporting Netflow logs between on-prem network and all VPCs. * Provides stateful firewall to enforce policy between on-prem network and all VPCs. * Out-of-the-box integration to support Direct Connect. * Connects multi-region Transit Gateway deployment. * Supports Transit DMZ architecture by inserting third party firewalls at the edge/transit VPC. * Supports 10Gbps Transit network throughput. ## When a VPC is attached to a TGW, why can't I simply program the default route in the VPC route table to point to the TGW? In some cases, you absolutely can. For example, if you have a group of VPCs that need to be connected to each other, you can attach each VPC to the same TGW route table with propagation enabled. Then program each VPC route table with the default route (0.0.0.0/0) to point to TGW. But in other cases you may not. Using the above example, if there is public subnet in a Spoke VPC, then you cannot simply program each route table with the default route pointing to TGW, as a public subnet already must have its default route pointing to the IGW. Even a Spoke VPC route table for private subnet may already have the default route point to an AWS NAT gateway. This is quite a common situation and as it happens, you cannot program the default route to the TGW. However, in the example scenarios above, you maybe able to program RFC 1918 (10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16) routes of the Spoke VPCs to point to TGW. This is a viable solution you can use to address the issues mentioned above and one that works in a lot of situations. ## Can the Aviatrix platform orchestrate VPN attachment to AWS Transit Gateway? Yes. The Aviatrix platform allows you [set up a VPN attachment](/docs/enterprise/10.1/reference/ui/networking/connectivity#edit-aws-tgw) from CoPilot directly. ## Can the Aviatrix platform orchestrate Direct Connect Gateway to AWS Transit Gateway? Yes. If you would like to connect your Direct Connect directly into Transit Gateway, the Aviatrix platform allows you to [configure](/docs/enterprise/10.1/reference/ui/networking/connectivity#edit-aws-tgw) an association between the Direct Connect Gateway and AWS Transit Gateway in Aviatrix CoPilot. ## Migrate from Aviatrix Transit Gateway to on-prem to TGW + DXGW 1. Create a DirectConnect Gateway on the AWS Console and figure out the cloud VPCs summary prefixes. 2. Disconnect the Aviatrix Transit Gateway from the VGW. 3. Connect the Aviatrix Transit Gateway to the DirectConnect Gateway. ## How does Aviatrix TGW feature compare with AWS Serverless TGW Orchestrator? AWS Serverless TGW Orchestrator is a solution published by AWS. It orchestrates VPC attachment to a TGW by programming both the TGW route table and VPC route table. The deployment is a Cloudformation Template that contains many AWS services such as Amazon DynamoDB, Amazon EventBridge, Amazon Simple Notification, AWS Lambda function. | Feature | Aviatrix TGW Orchestrator | Serverless TGW Orchestrator | | - | - | - | | Single pane of glass for orchestration | Yes | No. Orchestration is done by VPC tag | | Single pane of glass for visualization | Yes (View, List) | No. Each region must have its own deployment | | Inter region peering | Yes | No | | Orchestration consistency checking | Yes (Audit) | No | | Configuration for TGW DXGW | Yes | No | | Configuration for TGW VPN | Yes | No | | Troubleshooting connectivity | Yes (Test) | No | | Onboard secondary account | Automated | Manual | | Connection Policies between Domains | Flexible Connection Policies | 4 Policies defined (Flat, Isolated, Infrastructure & On-premises) | | Integrate Firewall deployment | Yes | No | ## Edge Segmentation Edge Segmentation allows you to further specify on each edge connection which domain it can communicate with. When you create a Transit Gateway, you can go to **CloudFabric > Gateways > select the Transit Gateways tab > select the Transit Gateway > select the Settings tab > open the CSP Related Settings section > find AWS TGW Edge Segmentation** and click on the Connection dropdown menu to select a connection. When this option is selected, you can then use [Build Your Domain Connection Policies](/docs/enterprise/10.1/guides/security/network-segmentation-secured) to specify which Network Domain this edge connection can communicate with, as shown in the diagram below. ![edge\_segmentation](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/components/tgw/edge-segmentation.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=4e61a7be5dc47d951498a19d66bbcd74) In the above diagram, Site 1 can communicate with Prod domain but not with Dev domain and Shared Service domain. Site 2 can communicate with Dev domain but not with Prod domain and Shared Service domain. Site 3 can communicate with Shared Service domain but not with Dev domain and Prod domain. Edge Segmentation works across Connection Policies for [AWS TGW Peered](/docs/enterprise/10.1/reference/ui/networking/connectivity#edit-aws-tgw) Network Domains. > [!NOTE] > > The Edge Segmentation is only applicable to the Aviatrix AWS TGW deployed > Spoke VPCs. It does not apply to Aviatrix Encrypted Transit. It also does not > apply to Aviatrix Transit Gateway peering. ## Enable multicast capability function on TGW Multicast capability function is able to be turned on when users launch AWS TGW. This is API support only. # UserVPN Overview Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/components/uservpn/user-vpn-feature-description The Aviatrix UserVPN feature provides a cloud-native and feature-rich client VPN solution, leveraging OpenVPN® compatibility to ensure seamless integration with all OpenVPN® clients. The Aviatrix UserVPN feature provides a cloud-native and feature-rich client VPN solution, leveraging OpenVPN® compatibility to ensure seamless integration with all OpenVPN® clients. Aviatrix also offers its own [client that supports SAML authentication](/docs/enterprise/10.1/guides/uservpn/user-vpn-saml-authentication) directly from the client, enhancing security and user authentication capabilities. ![Aviatrix Cloud VPN](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/components/uservpn/aviatrix-cloud-vpn.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=7621db5a179f8f31d90c264349557186) > [!NOTE] > > Only AWS is drawn in the diagram, but this feature applies equally to Azure and Google Cloud. ## VPN Management * **Centrally Managed** - A single pane of glass allows you to manage all VPN users, VPN certificates, and VPN user visibility. * **Split Tunnel** - Supports split tunnel mode where only specified CIDRs ranges go through the VPN tunnel. * **Full Tunnel** - Supports full tunnel mode where all user IP sessions including Internet browsing go through the VPN tunnel. * **PKI Management** - Supports Bring Your Own (BYO) PKI management system. * **Force Disconnect** - Any admin can force disconnect a VPN user from the Controller. * **Dashboard** - View all active VPN users and their connection history from the Controller dashboard. * **API** - Support API for all management activities. ## Authentication Options There are multiple authentication options to choose from when you create your VPN gateway: * **LDAP/AD Integration**: Authenticates VPN user from Aviatrix Gateways in addition to VPN certificate authentication. * **Duo Integration**: Authenticates VPN user from Aviatrix Gateways in addition to VPN cert authentication. * **Okta Integration**: Authenticates VPN user from Aviatrix Gateways in addition to VPN cert authentication. * **MFA Integration**: Combines LDAP and Duo for multi-factor authentication. * **Shared Certificate** (Client Certificate Sharing): Supports a shared certificate arrangement among VPN users. (When this option is enabled, you should select an additional authentication option to ensure secure access.) * **Client SAML Integration** - Authenticates a VPN user directly from the Aviatrix VPN client to any IDP via SAML protocol. ## Authorization * **Profile-Based Access Control** - Each VPN user can be assigned to a profile that is defined by access privileges to network, host, protocol and ports. The access control is dynamically enforced when a VPN user connects to the public cloud via an Aviatrix VPN Gateway. For more description, refer to [Aviatrix UserVPN FAQ](/docs/enterprise/10.1/concepts-architectures/components/uservpn/user-vpn-faq). For how to assign VPN users to profiles, see [Assigning a User Profile](/docs/enterprise/10.1/guides/uservpn/user-vpn-client-user-guide). ## Scale Out Performance * **TCP-based VPN** - For a universal/no firewall/no fuss UserVPN solution, use an Aviatrix-integrated Network Load Balancer to load balance multiple Aviatrix VPN gateways. When NLB is used, OpenVPN® client software runs on TCP port 443. TCP-based VPN requires no special corporate firewall rules when VPN client is on-prem. * **UDP-based VPN** - For a high performance UserVPN solution, use Aviatrix-integrated AWS Route53 round-robin routing to load balance multiple Aviatrix VPN gateways. When Route53 round-robin routing is used, OpenVPN® client software runs on UDP port 1193. UDP-based VPN has improved file transfer performance. * **Geo VPN** - For TCP-based VPN, you can use Aviatrix-integrated AWS Route53 latency-based routing to load balance clients residing in different geographic locations. ## Logging Integration * **VPN User** - VPN user connection history and bandwidth usage can be logged to Splunk, SumoLogic, ELK, Remote Syslog and DataDog. * **User Activity** - Each VPN user TCP/UDP session can be logged to Splunk, SumoLogic, ELK, Remote Syslog and DataDog. ## Client Software * **UserVPN Client Software** - The supported clients are macOS, Windows, iOS, Android, Chromebook, Linux and BSD. * **Aviatrix VPN Client** - Aviatrix VPN Client supports macOS, Windows and Linux Debian distribution and BSD distribution. Choose Aviatrix VPN Client if you require SAML authentication directly from VPN client software. To download and install Aviatrix VPN Client, see [Download VPN Client](/docs/enterprise/10.1/guides/uservpn/user-vpn-client-download). # Aviatrix Platform Overview Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/platform-overview Aviatrix® cloud networking software allows you to easily build your cloud network backbone, connect gateways in different regions, cloud service providers (CSPs), or data centers, and extend your applications to the edge of your network. Aviatrix® cloud networking software allows you to easily build your cloud network backbone, connect gateways in different regions, cloud service providers (CSPs), or data centers, and extend your applications to the edge of your network. Aviatrix allows you to uniformly create and manage CSP native objects and Aviatrix solutions across your secured multi-cloud network. A suite of embedded Aviatrix security solutions protects your network, and highly customizable monitoring tools provide you with mile-high and granular views of activity anywhere in your network at any point in time. ![Product Overview](https://mintcdn.com/aviatrix-14b37c43/vKpoXYmi49a-T1vW/images/concepts-architectures/product-overview.png?fit=max\&auto=format\&n=vKpoXYmi49a-T1vW\&q=85\&s=ec1984d7f2cf00c45fae57f9063805b7) ## Aviatrix Cloud Networking Platform The Aviatrix Cloud Networking Platform provides safe and reliable connections between users, devices, cloud native services and applications. You can easily add Aviatrix spoke gateways to your cloud network backbone creating secure ingress and egress points for your network. Aviatrix supports standard application layer protocols used to manage traffic and access to your critical applications. The same high-availability, performance, security, and visibility options available on your Aviatrix cloud network backbone extend to your spoke gateways. For more information, see [Building Your Network](/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/multi-cloud-design-patterns). ## Cloud Network Backbone Aviatrix brings all your enterprise network resources together to create a resilient and secure cloud network backbone using a hub-and-spoke topology. After deploying an Aviatrix software instance in AWS, Azure, or GCP you can begin to create Aviatrix transit gateways in one or more CSPs to form the core of your cloud network. AWS and GCP virtual private clouds (VPCs), Azure virtual networks (VNets), and OCI virtual cloud networks (VCNs) can then seamlessly integrate into your Aviatrix secure cloud backbone. Aviatrix software gives you advanced cloud network capabilities and features not offered by individual CSPs. Features such as BGP path prepending, alternative path routing, full-mesh ECMP high-availability, and auto-scaling allow you to design a resilient cloud network infrastructure that dynamically prioritizes application performance and ensures connectivity when primary routes are not available. For more information, see [Security](/docs/enterprise/10.1/concepts-architectures/components/dcf/overview). ## Edge Networking Aviatrix edge solutions extend your network and cloud applications to data centers, branch locations, and IoT nodes. The enterprise level performance, security, and visibility capabilities required for your Aviatrix cloud network backbone are also available on the edge of your network. Deploy Aviatrix gateways on edge devices anywhere you need processing power with core network reliability and security. You can create edge gateways for high-speed cloud connections, including AWS Direct Connect, Azure ExpressRoute, GCP Cloud Interconnect, and OCI FastConnect from locations around the world. For more information, see [Aviatrix Edge Gateways](/docs/enterprise/10.1/concepts-architectures/components/edge-gateways/edge-overview). ## Embedded Security Security is built into every level of your Aviatrix network. With Aviatrix high-performance encryption (HPE) you are not bound by standard IPSec throughput limitations, so you do not need to sacrifice performance for security. Build encrypted transit connections between CSPs and CSP regions to avoid the complexity and high latencies of direct connections. Aviatrix Distributed Cloud Firewall protects your network while reducing cost and complexity. You can also integrate firewalls from partners like Check Point, F5, Fortinet and Palo Alto Networks as part of your security implementation. Network micro-segmentation and role-based user polices allow you to control access to network resources and applications. With egress FQDN filtering you are protected from public internet attacks on your VMs and applications. In addition to the security enforcement capabilities you implement, Distributed Cloud Firewall protects your network from the latest security threats. The Aviatrix Threat database is constantly updated with information about active threats around the world. Allow Aviatrix to automatically block newly detected threats and regions with malicious activity or create custom threat and geoblocking policies at the IP address or geographic region level. For more information, see [Securing your Network](/docs/enterprise/10.1/concepts-architectures/components/dcf/overview). ## Secure Cloud Network Operations The Aviatrix data plane provides the CoPilot™ management UI with complete visibility into your secure cloud network. With Aviatrix CoPilot you configure all your network connections, policies, and monitor all gateways and traffic on your network. Customizable monitoring tools give you views of network resource usage, performance, security threats, and financial data. Start with the CoPilot dashboard overviews and use filters to drill down on any of your Aviatrix network resources to view the history or real-time data. You can use CoPilot to design and monitor your network and the Aviatrix Terraform provider to replicate your designs and automate deployment. For more information, see [CoPilot Features](/docs/enterprise/10.1/concepts-architectures/components/controlplane/copilot-features) and [Terraform](https://registry.terraform.io/providers/AviatrixSystems/aviatrix/latest/docs). Follow these links to learn about the latest features and updates available in the Aviatrix Platform offerings: * [What's New in Controller?](/docs/enterprise/10.1/reference/release-notices/what-is-new/what-is-new-controller) * [What's New in CoPilot?](/docs/enterprise/10.1/reference/release-notices/what-is-new/what-is-new-copilot) * [Deleting a CoPilot instance](/docs/enterprise/10.1/guides/controlplane/delete-copilot) * [Aviatrix Licensing](/docs/enterprise/10.1/guides/controlplane/aviatrix-licensing) # Validated Designs ✅ Source: https://docs.aviatrix.com/docs/enterprise/10.1/concepts-architectures/validated-designs/index Welcome to the Validated Designs section! Here you'll find proven, battle-tested architectural patterns and design solutions that have been validated in production environments. Welcome to the **Validated Designs** section! Here you'll find proven, battle-tested architectural patterns and design solutions that have been validated in production environments. ## What You'll Find Here > **Proven Patterns** > > Discover architectural patterns that have been tested and validated in > real-world scenarios > **Reference Implementations** > > Explore complete implementation examples with detailed explanations > **Performance Benchmarks** > > Review performance metrics and scalability data from production deployments > **Best Practice Guidelines** > > Learn the recommended approaches based on validated results and lessons > learned ## Why Validated Designs Matter These designs have been: * **Tested in Production** - Real-world validation with actual user traffic * **Performance Optimized** - Benchmarked for scalability and efficiency * **Security Audited** - Reviewed for potential vulnerabilities and compliance * **Community Approved** - Peer-reviewed by experienced architects and developers > [!WARNING] > > **Important**: While these designs are validated, always consider your > specific requirements, constraints, and context when implementing them. ## Getting Started > Browse through our validated designs to find patterns that match your use > case. Each design includes implementation details, performance > characteristics, and deployment considerations. > [!TIP] > > **Pro tip**: Start with the reference implementations to understand the > complete picture before adapting the patterns to your specific needs! # Installing Edge OS and Onboarding Dell Hardware Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/edge/edge-device-onboarding To deploy an Aviatrix Edge Transit or Spoke Gateway, you must first procure and onboard your edge device onto the Aviatrix Edge Platform. To deploy an Aviatrix Edge Transit or Spoke Gateway, you must first procure and onboard your edge device onto the Aviatrix Edge Platform. This following topics provide the prerequisite and deployment steps to install Aviatrix Edge OS and onboard Dell hardware onto the Aviatrix Edge Platform. **Onboarding Dell Hardware** This topic provides the instructions to install Edge OS and onboard edge device. ## Prerequisites The prerequisites to install Edge OS and onboard edge device. ### 1. Procure the Edge Device If you do not already have your edge device, you need to evaluate the hardware that you require for your edge computing and procure the hardware. Contact your Aviatrix account manager for the hardware you may require. For a list of Aviatrix supported hardware, see [Supported Edge Hardware for the Aviatrix Edge Platform](/docs/enterprise/10.1/reference/gateways/edge/edge-hardware-specs). ### 2. Obtain the Device Serial Number Obtain the Serial Number of your edge device. To onboard and register the edge device with the Aviatrix Edge platform, you will be required to provide the device Serial Number. The device Serial Number can be found from the iDRAC. Login to the iDRAC. On the Dashboard, look for *System Information/Service Tag*: ![edge-idrac-serial-number](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/edge/edge-idrac-serial-number.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=c89927685357ccb114c127805fa3a010) ### 3. You must have Aviatrix Controller and CoPilot deployed. ### 4. Set Up the Aviatrix Edge Platform Accout To onboard and register the edge device onto the platform, you will need to create an Aviatrix Edge Platform Account. To set up the account, see [Setting up Accounts for Edge Platforms](/docs/enterprise/10.1/guides/connectivity/edge/edge-platform-account). ### 5. Set Up Proxy Profile If you are using a Proxy setup, you can configure a proxy profile and associate the profile with the edge device. To create a proxy profile, see [Using a Proxy Profile on the Edge Device](/docs/enterprise/10.1/guides/connectivity/edge/edge-proxy-profile). ### 6. Obtain iDRAC, MGMT, WAN, LAN IP Address/Mask, and ASN and LAN Router Details 1. To Configure iDRAC, iDRAC port requires IPv4 address, its subnet and gateway IP information. Refer to [Integrated Dell Remote Access Controller 9 User's Guide](https://www.dell.com/support/manuals/en-us/idrac9-lifecycle-controller-v4.x-series/idrac9_4.00.00.00_ug_new/setting-up-idrac-ip-address?guid=guid-23b48a9f-7afe-42e0-8398-ad3bedb82226\&lang=en-us). 2. MGMT interface is used to establish connectivity to AEP backend. MGMT interface can either get DHCP IP address from customer network or can be statically configured. MGMT requires IPv4 network, mask and gateway. > [!NOTE] > > When using static IP on MGMT port, an image file is requried to be downloaded > from CoPilot and loaded onto the device. Steps are documented in the [Using > iDRAC Remote Console and ISO](#using-idrac-remote-console-and-iso) and > [USB](#usb) tabs. 3. LAN port is used by Edge gateway for connectivity to datacenter on-premises devices. The Edge gateway LAN interface requires IPv4 address, subnet length and gateway IP (which is the downstream datacenter on-premises router). 4. WAN port is used by Edge gateway to connect to upstream network over which connectivity to Transit in cloud is established. WAN interface on Edge gateway requires IPv4 address, subnet length and gateway IP (which is the downstream datacenter on-premises router). ### 7. Provide Network Access The Management Port needs internet access to communicate with the Aviatrix Platform. Ensure your firewalls allow access to the IP addresses in the *Aviatrix Edge Platform access (Edge Gateway deployed on hardware form factors which are Aviatrix managed)* table at [Aviatrix Products: Required Access for External Sites](https://aviatrix.zendesk.com/hc/en-us/articles/4417312119437-Aviatrix-Products-Required-Access-for-External-Sites) on the Aviatrix Customer Support website. ## Edge OS Installation and Device Onboarding The Aviatrix Edge OS can be installed on the Dell hardware using Dell Remote Access Controller (iDRAC) and ISO, or using USB. Before you install the Edge OS, you need to enable the required BIOS settings on the Dell hardware. See [Dell PowerEdge BIOS Configuration](/docs/enterprise/10.1/reference/gateways/edge/edge-device-bios-config). If you prefer to install Edge OS using iDRAC and ISO, follow the instruction in [Using iDRAC Remote Console and ISO](#using-idrac-remote-console-and-iso). If you prefer to use a USB installer to install Aviatrix Edge OS, follow the instructions in [USB](#usb). **Using iDRAC Remote Console and ISO** This topic provides instructions for installing Aviatrix Edge OS on Dell hardware by using iDRAC remote console and ISO. > [!TIP] > > These instructions require that a Dell iDRAC license is applied on the Dell > server, which would allow the operator to access the server's remote virtual > console and load virtual media. The license is typically applied at factory by > Dell, please choose enterprise iDRAC as described in Dell tech specs for > Aviatrix Edge. > [!NOTE] > > This document does not cover user management for Dell iDRAC. You are > responsible to configure user access on the Dell server. You will need an > admin user or one with equivalent access to install Aviatrix Edge OS on the > server. ## 1. Prerequisites Before you start to onboard and install the Edge OS on the edge device, make sure these [prerequisite steps](#edge-device-prerequisites) (in the **Onboarding Dell Hardware** tab) are completed. ## 2. Onboard Edge Device onto Aviatrix Edge Platform 1. In Aviatrix CoPilot, go to **Cloud Fabric** > **Hybrid Cloud** > **Devices** tab. 2. Click **Onboard Device**, then provide the following information. | Field | Description | | - | - | | Name | A name to identify the edge device. | | Platform | The Aviatrix Edge Platform Account where to onboard the device. | | Device Configuration | Dell HW with single ethernet controller for Edge Spoke Gateway, select **DELL Poweredge 1x E810**. Dell HW with two ethernet controllers for Edge Spoke or Transit Gateway, select **DELL Poweredge 2x E810**. | | Serial Number | The serial number of the edge device. | 3. Expand **Device Interface Configuration** and configure the device's **Function MGMT** (Management) interface. 1. If you plan to use DHCP for the device Management interface, turn **DHCP** toggle to **On**. 2. If you plan to configure a Static IP address for the device Management interface, leave **DHCP** toggle to **Off**. 3. If **DHCP** is **Off**, enter the Static IP address for the Management interface. | Field | Description | | - | - | | Interface CIDR | The Static IP address of the Management interface. | | Gateway IP | The Management Default Gateway IP address. | | Primary DNS | The primary DNS server IP address. | | Secondary DNS | (Optional) The secondary DNS server IP address. | 4. If you are using a proxy server to route internet traffic, from the **Proxy Profile** dropdown menu, select the proxy profile to upload to the edge device (see [Using a Proxy Profile on the Edge Device](/docs/enterprise/10.1/guides/connectivity/edge/edge-proxy-profile)). 4. Under **Advanced**, specify the Edge Management CIDR that will be used on the Edge gateway interface. If a CIDR is not specified, the system auto assigns 10.100.101.0/24 CIDR range and assigns 10.100.101.2 IP address on the interface. > [!TIP] > > If the auto assigned CIDR is used, ensure that this range does not overlap > with any other network. ![edge-onboard-mgmt-static-ip.png](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/edge/edge-onboard-mgmt-static-ip.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=b85eb6621f239c60be5cfca359539acd) 5. Save the configuration. * When DHCP is selected for MGMT interface, click **Save**. * When Static IP is selected for MGMT interface, from the **Save and Download Config** drop-down menu, click **Save**. This will download a **bootstrap-config.img** file (for example, 3QDDWX3-bootstrap-config.img). Your device will show up as **Provisioned** in Copilot. ![edge-onboard-device-provisioned.png](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/edge/edge-onboard-device-provisioned.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=72ca87d0d33af4fad40cb9d964df68f8) 6. If you need to change the management interface's configurations (from DHCP to Static or vice versa), you can click Edit icon and update the configuration. Save the changes you made to the device. ## 3. Install Aviatrix Edge OS on the Edge Device Install Aviatrix Edge OS on the edge device. 1. Obtain the latest **iso** file published by Aviatrix from the [Aviatrix Support site](https://support.aviatrix.com/downloads). 2. Download the latest, **production** Aviatrix Edge OS **.iso** file to a jumphost which is in the same network as the Dell server. The jumphost enables a fast upload of the ISO file to the server as compared to getting it over the internet. If the server has fast internet access, you can mount the ISO file from your laptop. > [!TIP] > > If the ISO file is fetched over the internet and if the download speed is > poor, it may increase the overall installation time. It is recommended that > the file be downloaded to a laptop/jumphost in the same network as the server. 3. From the jumphost, login to the Dell server's iDRAC UI. ![edge-idrac-ui.png](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/edge/edge-idrac-ui.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=07971f7ef74c2e4171cea7af867d6590) 4. Open the Virtual Console to the server. On the Virtual Console window, click **Virtual Media** > **Connect Virtual Media** > **Connect Virtual Media**. Virtual Media should show up as connected. ![edge-idrac-virtual-console1.png](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/edge/edge-idrac-virtual-console1.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=051525faac53f76c0cc28539b79aee7e) 5. Under **Map CD/DVD**, choose the Edge OS ISO file and click **Map Device**. Under **Map Removable Disk**, select the **bootstrap-config.img** file and again click **Map Device**. The removable disk is a virtual USB. > [!NOTE] > > If you selected DHCP for the device's management interface, you do not need to > map a file to the removable disk. You only need the Edge OS ISO file. Your virtual media dialog box should look like this: ![edge-idrac-virtual-console2.png](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/edge/edge-idrac-virtual-console2.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=2d56ed18e0b7cf2005dc190eb87fe6d9) Close this dialog box. 6. From the Virtual Console, click **Boot** and under **Boot Controls**, and select **Virtual CD/DVD/ISO**. A dialog box will open, asking you to confirm the boot action. Click **Yes**. ![edge-idrac-virtual-console3.png](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/edge/edge-idrac-virtual-console3.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=e13ac040221a08a065d20bcb00064fc2) 7. From the Virtual Console, click **Power** and select the **Reset System (warm boot)** option. This will start the installation of Aviatrix Edge OS (AEP EvE) on the server. > [!WARNING] > > Do not close the Virtual console window. ![edge-idrac-virtual-console4.png](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/edge/edge-idrac-virtual-console4.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=7a90aec3e7e4333f783d60eb434bb886) ![edge-idrac-virtual-console5.png](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/edge/edge-idrac-virtual-console5.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=cf15ade9463606e314a98575f3eca39d) 8. If you have SSH access to the iDRAC, you can SSH to it and type **console com2**. This will connect you to the serial console of the server and you will see the Aviatrix Edge OS (AEP EvE) installation logs. ![edge-idrac-install-log1.png](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/edge/edge-idrac-install-log1.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=bfcd9cb67483ff025acbbed0db62c8dc) 9. The installation typically takes 10 to 15 minutes. It may vary depending on where the ISO file is being uploaded to the server via the Virtual Console. Once the installation is complete, the server will automatically power down. This can also be confirmed from the EvE installation logs in the serial console. ![edge-idrac-install-log2.png](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/edge/edge-idrac-install-log2.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=856d37db74c8e1d860e4943e624e860a) ![edge-idrac-power-off.png](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/edge/edge-idrac-power-off.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=1017f8d9da9b6991548b3c22f9cdf305) 10. From the Virtual Console, go to **Virtual Media** and under **Map CD/DVD**, click **Un-Map Device**. Confirm the un-map action by clicking **Yes**. This will remove the ISO file from the Virtual Console. If you had mapped a **bootstrap-config.img** file to the removable disk/virtual USB, keep it as is. We will configure the server's management interface with the static IP information in this virtual USB. ![edge-idrac-virtual-console6.png](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/edge/edge-idrac-virtual-console6.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=346a88e761dab87d808e776d3870fea5) Click **Close** to save the changes. 11. From the Virtual Console, go to **Power**. You will see only one option this time, **Power On System**. Confirm the power action by clicking on **Yes**. ![edge-idrac-power-on1.png](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/edge/edge-idrac-power-on1.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=e3da90cb647e645a94fe3ebb479beedd) 12. Allow the server to power on. It will boot AEP EvE. ![edge-idrac-power-on2.png](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/edge/edge-idrac-power-on2.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=c410aca24c91107be864b0c57cc4024d) 13. If you have access to the server's serial console, you will see the boot logs. Aviatrix Edge OS will read the static IP configuration from the virtual USB and configure it on the server. The server will dial out to AEP (controller.edge.aviatrix.com) and connect to it. Look for logs (the actual log text may vary) with these entries: * **PASS: port eth0 fully connected to EV controller controller.edge.aviatrix.com** * **1 out of 4 ports specified to have EV controller connectivity passed test** ![edge-idrac-boot-log1.png](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/edge/edge-idrac-boot-log1.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=e8571f5526f57c9439678a0cf4eac479) ![edge-idrac-boot-log2.png](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/edge/edge-idrac-boot-log2.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=d402c6f4c8ef080041bec201680c1106) 14. The Dell device will show up as **Connected** in **Copilot (go to Cloud Fabric** > **Hybrid Cloud** > **Devices)** and can now be used to launch an Edge gateway. For details on how to launch an Edge Transit or Edge Spoke Gateway, see [Hybrid Cloud Connectivity](https://legacy.docs.aviatrix.com/documentation/latest/network/edge-connectivity.html). ![edge-device-connected.png](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/edge/edge-device-connected.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=954ef3364f1e3cfd85ff9b0717389666) 15. Return to the Virtual Console, go to **Virtual Media** > **Disconnect Virtual Media**. Click **Yes**. This will unmap and disconnect all virtual media files. Close the virtual media box. You can now also close the Virtual Console. AEP EvE installation is complete and it will persist across server reboots. **USB** This topic provides instructions for installing the Aviatrix Edge OS on Dell hardware using USB. If you use iDRAC with virtual media to install Edge OS, we recommend using the remote console to configure a static IP on the device. See the [Using iDRAC Remote Console and ISO](#using-idrac-remote-console-and-iso) tab. ## 1. Prerequisites * Before you start to onboard and install the Edge OS on the edge device, make sure these [prerequisite steps](#edge-device-prerequisites) (in the **Onboarding Dell Hardware** tab) are completed. * Download and install a software on your laptop which you can use to Flash an OS to a USB drive or SD card, like [balenaEtcher - Flash OS images to SD cards & USB drives](https://etcher.balena.io/). ## 2. Prepare a USB Installer with Aviatrix Edge OS Prepare a USB installer with Aviatrix Edge OS. 1. Download the latest Aviatrix Edge OS from [Aviatrix Support](https://support.aviatrix.com/downloads). 2. Connect a USB drive to a laptop. 3. Open Balena Etcher 1. Select the Aviatrix Edge OS image 2. Select the USB drive 3. Flash The USB installer is ready to use. ## 3. Onboard Edge Device onto Aviatrix Edge Platform Onboard the edge device onto the Aviatrix Edge Platform. 1. In Aviatrix CoPilot, go to **Cloud Fabric** > **Hybrid Cloud** > **Devices** tab. 2. Click **Onboard Device**, then provide the following information. | Field | Description | | - | - | | Name | A name to identify the edge device. | | Platform | The Aviatrix Edge Platform Account where to onboard the device. | | Device Configuration | Dell HW with single ethernet controller for Edge Spoke Gateway, select **DELL Poweredge 1x E810**. Dell HW with two ethernet controllers for Edge Spoke or Transit Gateway, select **DELL Poweredge 2x E810**. | | Serial Number | The serial number of the edge device. | 3. Expand **Device Interface Configuration** and configure the device's **Function MGMT** (Management) interface. 1. If you plan to use DHCP for the device Management interface, turn **DHCP** toggle to **On**. 2. If you plan to configure a Static IP address for the device Management interface, leave **DHCP** toggle to **Off**. 3. If **DHCP** is **Off**, enter the Static IP address for the Management interface. | Field | Description | | - | - | | Interface CIDR | The Static IP address of the Management interface. | | Gateway IP | The Management Default Gateway IP address. | | Primary DNS | The primary DNS server IP address. | | Secondary DNS | (Optional) The secondary DNS server IP address. | 4. If you are using a proxy server to route internet traffic, from **Proxy Profile** dropdown menu, select the profile to upload to the edge device. 4. Under **Advanced**, specify the Edge Management CIDR that will be used on the Edge gateway interface. If a CIDR is not specified, the system auto assigns 10.100.101.0/24 CIDR range and assigns 10.100.101.2 IP address on the interface. > [!TIP] > > If the auto assigned CIDR is used, ensure that this range does not overlap > with any other network. ![edge-onboard-mgmt-static-ip.png](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/edge/edge-onboard-mgmt-static-ip.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=b85eb6621f239c60be5cfca359539acd) 5. Save the configuration. * When DHCP is selected for MGMT interface, click **Save**. * When Static IP is selected for MGMT interface, from the **Save and Download Config** drop-down menu, click **Save**. This will download a **bootstrap-config.img** file (for example, 3QDDWX3-bootstrap-config.img). Your device will show up as **Provisioned** in Copilot. ![edge-onboard-device-provisioned.png](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/edge/edge-onboard-device-provisioned.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=72ca87d0d33af4fad40cb9d964df68f8) 6. Connect a USB drive to your laptop. > [!TIP] > > This USB drive should be different from the one which was Flashed with the > Aviatrix Edge OS image. 7. Open Balena Etcher. 1. Select the device's **bootstrap-config.img** image file. 2. Select the USB drive. 3. Flash ## 4. Installing Aviatrix Edge OS with a USB Installer Install Aviatrix Edge OS on the edge Device using the USB installer. > [!NOTE] > > This procedure is based on Aviatrix OS installer on a USB drive. When we have > iDRAC license to enable virtual media functions, we can use an ISO installer > file over a network (see the **Using iDRAC Remote Console and ISO** tab). 1. Connect VGA/serial and keyboard to Dell server, and power it on. 2. From the Dell server's VGA/serial console, from **iDRAC Settings** > **Network** to setup iDRAC with Static IP address or desired IP address settings, apply the new settings and then reboot the system. ![edge-usb-idrac-settings.png](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/edge/edge-usb-idrac-settings.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=5347e8f4a48d97a90133fe52a7c04e7e) 3. Identify the Dell server's face panel with dual USB ports and plug in Aviatrix OS installer USB disk to the USB port labeled with **SS**; then browse to iDRAC's IP address to login to the iDRAC UI with default credentials, which can be found from the Dell service tag label. > [!TIP] > > Do not reset the root password when prompted to do so. ![edge-usb-idrac-login.png](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/edge/edge-usb-idrac-login.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=e21055d3fceec02b57976b706db720a6) 4. After you login to iDRAC UI, go to **IDRAC Settings** > **Users** add a default login user account. ![edge-usb-idrac-user-settings.png](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/edge/edge-usb-idrac-user-settings.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=c08ec1e7fd9aec3303f680ac724f3afc) 5. From iDRAC9 UI **Configuration** > **BIOS Settings**, follow Aviatrix's recommended BIOS configuration in [Dell PowerEdge BIOS Configuration](/docs/enterprise/10.1/reference/gateways/edge/edge-device-bios-config). Apply BIOS setting changes, go to the bottom of the page and click **Apply AND Reboot** and **OK**. ![edge-usb-bios-config.png](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/edge/edge-usb-bios-config.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=6c2435a71875cb6ce59a35e1abb5cc4f) 6. Go to **Maintenance** > **Job Queue**, you can see current Job queue and task status. ![edge-usb-idrac-maintenance.png](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/edge/edge-usb-idrac-maintenance.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=b9406ce02c7dc5345ea2b15055d1be2c) 7. Login to iDRAC9 console using SSH. ```bash theme={null} ssh admin@idrac9_mgmt_ip ``` and issue the following command to enter Eve device console: ```bash theme={null} console com2 ``` ![edge-usb-idrac-ssh-console.png](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/edge/edge-usb-idrac-ssh-console.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=cd5b581bbd701731f7e3b17b86413c0c) Wait 5 to 10 minutes until the installation is done and you should see the log below log from iDRAC9 console. ![edge-usb-idrac-install-log.png](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/edge/edge-usb-idrac-install-log.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=4feb4e2cc95cf41b9f539576b93d4c76) 8. Detach the USB installer from the Dell R450 and power it on from **Configuration** > **Power Management**, from **Power Control** dropdown, choose **Power On System** and click **Apply**. ![edge-usb-idrac-power-on.png](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/edge/edge-usb-idrac-power-on.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=8cdf9048b1b3ae746a12c7552d800fe4) Wait 5 to 10 minutes while Aviatrix OS boots up. 9. The device will power off once the installation is done. At this stage, connect the USB drive which has the static IP information to the device and power on the device. Aviatrix Edge OS will read the static IP information from the USB and configure the device. To view the configuration logs, you can SSH to the device and connect to the serial console. > [!NOTE] > > If you power on the device without inserting the USB drive which has the > static IP information, you can do it at a later time. Connect the USB to the > device and reset the system (warm boot). Aviatrix Edge OS will read the > information from the USB and configure the device. 10. After the static IP is configured, the device should be able to dial out to AEP (controller.edge.aviatrix.com) and register with the service. The device should show up as **Connected** in CoPilot and can now be used to launch an Edge gateway. For details on how to launch an Edge Transit or Edge Spoke Gateway, see [Hybrid Cloud Connectivity](https://legacy.docs.aviatrix.com/documentation/latest/network/edge-connectivity.html). ![edge-device-connected.png](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/edge/edge-device-connected.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=954ef3364f1e3cfd85ff9b0717389666) # Edge Gateway Image Upgrade and Post-Upgrade Tasks Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/edge/edge-gateway-upgrade This document provides information about Aviatrix Edge Gateway image upgrade and post-upgrade tasks. For complete instructions on upgrading Aviatrix Controller and gateways, see Upgrade Aviatrix Controller and Gateways. This document provides information about Aviatrix Edge Gateway image upgrade and post-upgrade tasks. For complete instructions on upgrading Aviatrix Controller and gateways, see [Upgrade Aviatrix Controller and Gateways](/docs/enterprise/10.1/guides/maintenance/controller-gateway-upgrade-latest-image). * [Edge Gateway Image Upgrade](#edge-gateway-image-upgrade) * [Edge Gateway Post-Upgrade Tasks](#edge-gateway-post-upgrade-tasks) ## Edge Gateway Image Upgrade ### Edge Gateway VM Image Upgrade (Aviatrix Edge Platform) To upgrade your Edge Gateway VM image, see [Upgrade your Controller and Gateways to the Latest Aviatrix Supported Images (AWS Only)](/docs/enterprise/10.1/guides/maintenance/controller-gateway-upgrade-latest-image). > [!NOTE] > > Edge Gateway VM image upgrade is required for Controller release version > 7.1.3958 or later. ### Edge Gateway VM Image Upgrade (Equinix and Self Managed Platforms) The Edge Gateway VM image upgrade is not supported for Equinix and Self Managed platforms. If you have Edge Gateways deployed on VMware ESXi or KVM self-managed hypervisor or Equinix Network Edge, you will need to delete and re-deploy your Edge gateways after you upgrade the Avaitrix Controller to the latest supported images. For instructions on Edge Gateway deployment, see: * [Deploy an Edge Spoke Gateway on Equinix Network Edge](/docs/enterprise/10.1/guides/connectivity/edge/edge-spoke-workflow-equinix) * [Edge Spoke Gateway Deployment Workflow on Self-Managed VMware ESXi and KVM](/docs/enterprise/10.1/guides/connectivity/edge/edge-spoke-workflow-selfmanaged) ### Edge Gateway Selective Upgrade (Equinix and Self Managed Platforms) Aviatrix Hybrid Cloud Edge does not support selective gateway image update and software rollback for Equinix and Self Managed platforms. ## Edge Gateway Post-Upgrade Tasks After you upgrade to the lastest Aviatrix Controller version, perform the following post-upgrade tasks for Edge gateways. * [Upgrading the Edge Gateway's Platform Type](#upgrading-the-edge-gateways-platform-type) * [Upgrading Active-Standby Edge Gateways](#upgrading-active-standby-edge-gateways) ### Upgrading the Edge Gateway's Platform Type > [!NOTE] > > You only need to perform this post-upgrade task if you are upgrading from > Controller release versions prior to 7.1.3958 release. In Aviatrix Controller releases prior to the latest version of 7.1, the Platform Type for an Edge Spoke Gateway on the Equinix Network Edge Platform was *Self-Managed*. In the latest Controller release, the Platform Type for an Edge Spoke Gateway on the Equinix Network Edge Platform is *Equinix*. There are additional features, such as underlay BGP, supported for Edge Spoke Gateways with Platform Type as *Equinix*. To be able to use the new features, after you upgrade your Edge Spoke Gateway, perform the following post-upgrade task to update the Edge Spoke Gateway's Platform Type from *Self-Managed* to *Equinix*. In Aviatrix CoPilot: 1. Go to **Cloud Fabric > Hybrid Cloud > Edge Gateways** tab. 2. Click **Spoke Gateways**. 3. In the table, locate the Edge Spoke Gateway that was migrated and verify the gateway's **Platform Type** is **Self-Managed**. ![Edge Equinix Gateway Migration Step 1](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/edge/edge-equinix-gateway-migration1.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=629dc3d9b01d3bd39f16dc192576b7ac) 4. Click the gateway's Edit icon on the right side of the row. 5. In Edit Spoke Gateway, verify **Platform** is **edge\_admin**. ![Edge Equinix Gateway Migration Step 2](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/edge/edge-equinix-gateway-migration2.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=43f8c17d59325f9b60c4d49da48e814e) 6. Go to **Cloud Fabric > Hybrid Cloud > Platforms** tab. 7. Click **+Platform**. 8. In **Name**, enter a name to identify the Equinix platform account. 9. In **Type**, select **Network Service Provider (NSP)**. 10. From the **Existing Self Managed Equinix Gateways** dropdown menu, select the Edge Spoke Gateway that was migrated and click **Save**. This adds a new Equinix platform account to the existing Edge platform accounts and updates the platform account for the Edge Spoke Gateway that was migrated to the new Equinix platform account. ### Upgrading Active-Standby Edge Gateways There are changes with the auto-upgrade of Aviatrix Edge Spoke Gateways from Aviatrix Controller 7.0 and 7.1 release to the latest version of Controller 7.1. Due to an architecture change, if you have two Edge Spoke Gateways that are deployed in *active-standby* mode in the same edge site and perform an upgrade, the Edge Spoke Gateways are upgraded to *active-active* mode. If the active-standby gateways were originally in non-preemptive mode, then after the upgrade they will become preemptive. The primary Edge Spoke Gateway becomes the active gateway, and to avoid traffic disruption, the Controller prepends the AS-PATH of the second Edge Spoke Gateway, which causes the gateways to function in preemptive failover mode. > [!NOTE] > > This behavior is different from an active-standby Edge deployment. You can > continue to run the Edge Spoke Gateways in active-active mode, however, you > may not be able to scale the gateways. For an active-standby Edge deployment, after you have upgraded the Edge Spoke Gateways, Aviatrix highly recommends that you enable active-standby high availability mode for the primary Edge Spoke Gateway to create the new highly available (HA) gateway, then delete the second Edge Spoke Gateway that was created during the upgrade. # Edge Gateway Multi-Peer BGP over LAN Connection Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/edge/edge-multipeer-bgpolan This document provides information about multi-peer BGP over LAN connections from an Aviatrix Edge gateway to on-premises LAN BGP routers. This document provides information about multi-peer BGP over LAN connections from an Aviatrix Edge gateway to on-premises LAN BGP routers. ## Overview The Aviatrix Platform supports multi-peer BGP over LAN connections on Aviatrix Edge Transit and Edge Spoke Gateways designed to provide resilient and scalable architecture. Multi-Peer BGP over LAN peering allows Edge gateways to establish full mesh BGP sessions with two highly available (HA) BGP neighbors and provide failover redundancy. For instance, if an Edge gateway loses connection with one LAN router, a full mesh peering allows the Edge gateway to route network traffic to the other LAN router. Without the full mesh capability, traffic would need to be routed through multiple hops (to another Edge gateway and LAN router) causing delay in latency. To create full mesh BGP peering to HA LAN routers, you must enable BGP ActiveMesh for the Edge gateway connection to the LAN routers. You can set up the BGP connections from the Edge gateway's LAN interface. The Edge gateway's LAN interface and the LAN routers IP addresses must be in the same subnet. ## Deployment Models You can set up multi-peer BGP over LAN connections from the Edge gateway LAN interfaces towards on-premises router. ### Primary Edge Gateway to HA Remote Devices ![Edge BGP over LAN Peering](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/edge/edge-bgpolan-peering4.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=a530ca6ddcdc90fd138ebeddf5bea36a) When only the primary Edge gateway is deployed with HA LAN routers: * With ActiveMesh, two BGP connections are created (Edge gateway to Primary and Secondary routers). * Without ActiveMesh, only one BGP connection is created (Edge gateway to Primary router). ### HA Edge Gateways to HA Remote Devices Edge gateways can also be deployed in high availability (HA) Active-Active or Active-Standby mode. ![HA Edge Gateways to HA Remote Devices](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/edge/edge-bgpolan-peering1.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=62945dbccfda1572d0544fcd8b05f572) In this architecture: * With ActiveMesh, four BGP connections are created (Edge 1 to router 1 and 2, Edge 2 to router 1 and 2). * Without ActiveMesh, only two BGP connections are created (Edge 1 to router 1, Edge 2 to router 2). The Edge gateways must have the same BGP ASN and the LAN BGP Routers must have the same BGP ASN. If the LAN routers advertise the same routes, then the Edge gateways consider them to be ECMP. When routes are advertised by only one LAN router, then the secondary HA Edge gateway functions as a backup route. ## Create Multi-Peer BGP over LAN Connections for Edge Gateway To create BGP over LAN connections from an Edge gateway to multiple LAN BGP routers: 1. In Aviatrix CoPilot, go to **Networking > Connectivity > External Connections (S2C)**. 2. From the **+ External Connection** dropdown menu, select **External Device**. 3. In Create External Connection to External Device, provide the following information. | **Field** | **Description** | | - | - | | Name | A name for the connection. | | Type | Select **BGP over LAN**. | | Local Gateway | Select the Edge gateway that you want to connect to the LAN-side remote device. | 4. In the **BGP Configuration** section, provide the following information. | **Field** | **Description** | | - | - | | Local ASN | The BGP AS Number of the Edge gateway. | | ActiveMesh | Set ActiveMesh toggle to **On**. ActiveMesh enables full mesh BGP connections to the remote devices. | 5. In the **LAN Configuration** section, provide the following information. | **Field** | **Description** | | - | - | | Remote Device 1 IP | The remote device interface IP address. | | Remote ASN | The BGP AS Number of the remote device. The Remote ASN should be the same for the Primary and HA Edge gateways. | | Local LAN IP | The Edge gateway interface IP address. This is auto populated. | | Remote LAN IP | The remote device interface IP address. | 6. To connect the Edge gateway to another remote device, click **+ Remote Device**, then provide the IP address and ASN of the remote device. 7. Click **Save**. ## Related Links * [Edge Transit Gateway External Connection Workflow](/docs/enterprise/10.1/guides/connectivity/edge/edge-s2c-workflow) * [Aviatrix Edge Platform](/docs/enterprise/10.1/concepts-architectures/components/edge-gateways/edge-overview) # Setting up Accounts for Edge Platforms Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/edge/edge-platform-account Edge platform accounts determine the features and functions that are available for creating and deploying a gateway on an edge platform. For instance, setting up a platform account for the Aviatrix Edge Platform allows the Aviatrix Controller to register with the Aviatrix Edge infrastructure... Edge platform accounts determine the features and functions that are available for creating and deploying a gateway on an [edge platform](/docs/enterprise/10.1/concepts-architectures/components/edge-gateways/edge-overview) . For instance, setting up a platform account for the Aviatrix Edge Platform allows the Aviatrix Controller to register with the Aviatrix Edge infrastructure for managing and onboarding edge devices and Edge gateways. Before you can deploy an Aviatrix Edge Gateway, you must set up an account for the edge platform where you want to deploy the gateway. > [!NOTE] > > For Self Managed platform, the default platform account, *edge\_admin*, is > automatically created for deploying Aviatrix Edge Gateways in customer managed > ESXi and KVM environments. ## Creating the Edge Platform Account 1. In CoPilot, go to **Cloud Fabric > Hybrid Cloud > Platforms** tab. 2. Click **+ Platform**. 3. Enter a name for the account. For example, *OnPremises*, *Equinix* or *Megaport*. 4. To set up an account for the Aviatrix Edge Platform, select **Aviatrix Edge** type. 5. To set up an account for a network service provider, select **Network Service Provider (NSP)** type. 1. For Equinix Network Edge platform, from the dropdown menu select **Equinix**. 2. For Megaport Virtual Edge platform, from the dropdown menu select **Megaport**. 6. Click **Connect**. # Using a Proxy Profile on the Edge Device Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/edge/edge-proxy-profile The edge devices supported on the Aviatrix Edge Platform requires egress over the public internet to communicate with the Aviatrix Platform. If internet traffic is routed through a proxy server, you can configure a proxy profile and associate the profile with the edge device. The edge devices supported on the Aviatrix Edge Platform requires egress over the public internet to communicate with the Aviatrix Platform. If internet traffic is routed through a proxy server, you can configure a proxy profile and associate the profile with the edge device. ## Creating a Proxy Profile To configure a proxy profile, follow these steps. 1. In Aviatrix CoPilot, go to **Cloud Fabric > Hybrid Cloud > Devices** tab. 2. Click **Proxy Profiles**. 3. In Proxy Profiles, click **+ Proxy Profile**, then provide the following information. | **Field** | **Description** | | - | - | | Name | A name for the proxy profile. | | Transparent Mode | If you use a Transparent proxy server, set Transparent Mode toggle to **On**. A Proxy CA Certificate is required when Transparent Mode is On. | | Proxy Server IP | The IP address of the Proxy server. | | Port | The port number used for the Proxy server. | | Proxy CA Certificate | The certificate authority (CA) certificate to use on the edge device. The Proxy CA certificate must be in Privacy Enhanced Mail (PEM) file format. | 4. Click **Save**. You can now use the proxy profile to configure the Management interface of the edge device when you onboard the device. ## Modifying or Deleting a Proxy Profile > [!NOTE] > > If a proxy profile is associated with an edge device, modifying the proxy file > may result in communication disruptions with the Aviatrix platform. To modify a proxy profile: 1. In Aviatrix CoPilot, go to **Cloud Fabric > Hybrid Cloud > Devices** tab. 2. Click **Proxy Profiles**. 3. In the table, locate the proxy profile you want to edit and click the **Edit** icon on the right. 4. Make any changes needed. 5. Click **Save**. To delete a proxy profile: 1. In Aviatrix CoPilot, go to **Cloud Fabric > Hybrid Cloud > Devices** tab. 2. Click **Proxy Profiles**. 3. In the table, locate the proxy profile you want to delete and click the **Delete** icon on the right. > [!NOTE] > > The Delete icon is disabled if a proxy profile is associated with any edge > device. You must first remove the proxy profile association for the edge > devices. 4. To confirm the deletion, click **Delete**. ## Removing a Proxy Profile Association for an Edge Device To remove a proxy profile association for an edge device: 1. In Aviatrix CoPilot, go to **Cloud Fabric > Hybrid Cloud > Devices** tab. 2. In the table, locate the edge device and click the **Edit** icon on the right. 3. Delete the proxy profile associated with the MGMT interface. 4. Click **Save**. # Creating the Default RBAC Access Account for Edge Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/edge/edge-rbac-account You have the option to create a default RBAC group and assign users to this group with permissions to create, delete, and manage Aviatrix Edge Gateways. You have the option to create a default RBAC group and assign users to this group with permissions to create, delete, and manage Aviatrix Edge Gateways. ## Creating the Default RBAC Access Account for Edge To create an RBAC group with permissions to create, delete, and manage Edge gateways, follow these steps. 1. In Aviatrix CoPilot, go to **Administration** > **User Access** > **Permission Group**. 2. Click **+ Permission Group**. 3. In the **Name** field, enter a name for this Permission group. 4. From the **Users** dropdown list, select the users to add to this Permission group. 5. From the **Access Accounts** dropdown list, select the Edge Platform account this group should be able to access. 6. Click **CoPilot Visibility** tab. 7. Under **Cloud Fabric > Hybrid Cloud**, select the permissions for the users in this Permission group. 8. Click **Save**. ![Edge RBAC Configuration](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/edge/edge-rbac.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=1c1f3281b1e3323f1ea1cd1c0f7665d6) You can now create or assign a user account with the newly created RBAC group. 9. Go to **Administration** > **User Access** > **Users** tab. 10. Click **+ User**. 11. Enter the user credentials, then from the **Permissions Groups** dropdown list, select the RBAC Permission group. # Edge Transit Gateway External Connection Workflow Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/edge/edge-s2c-workflow This document describes how to connect an Edge gateway in hybrid or multicloud environment to an on-premises remote device, such as a remote router or firewall that terminate VPN connection. This document describes how to connect an Edge gateway in hybrid or multicloud environment to an on-premises remote device, such as a remote router or firewall that terminate VPN connection. ## Configuration Workflow To build an external connection (S2C) between an Edge gateway and an external remote device, you need to configure one side of the tunnel connection on the Edge gateway and the other side of the tunnel connection on the remote device. To create an external connection: 1. [Configure the tunnel on the Edge gateway](#configure-the-tunnel-on-the-edge-gateway). 2. [Download the Edge gateway tunnel configuration file](#download-an-external-connection-configuration). 3. [Configure the tunnel on the remote device](#configure-the-tunnel-on-the-remote-device). 4. [Verify the connection is up](#verify-the-connection-status). ### Configure the Tunnel on the Edge Gateway For BGP connection to the remote device, configure the tunnel using one of the options below: * [BGP over IPsec](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-create-bgp-over-ipsec.html) * [BGP over GRE](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-create-bgp-over-gre.html) * [BGP over LAN](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-create-bgp-over-lan.html) ### Download an External Connection Configuration If you are connecting an Aviatrix gateway and an on-premises router or firewall, Aviatrix can generate a configuration file that you can apply to your remote router or firewall. The configuration file contains the Aviatrix gateway tunnel details, such as the Public IPv4 and IPv6 addresses, VPC/VNet CIDR, pre-shared key, and encryption algorithm. You can download the configuration file and then import the details to your remote router or firewall to configure the other end of the VPN tunnel. After creating an external connection, to download an external connection configuration: 1. In Aviatrix CoPilot, go to **Networking > Connectivity > External Connections (S2C)** tab. 2. On the External Connections (S2C) tab, locate the connection you created and click the vertical ellipsis ![vertical ellipsis icon](/docs/enterprise/10.1/guides/connectivity/vpn/images/vertical-ellipsis-icon.png) icon on the right side of the row. 3. Select the following values: * **Vendor**: Select your remote site device. * Select **Generic** for anything that is not an Aviatrix gateway. * Select **Aviatrix**, if you are connecting two Aviatrix gateways. * **Platform and Software**: * If you selected a Generic vendor, the Platform field is populated as Generic, and the Software field is populated with Vendor Independent. * If you selected the Aviatrix vendor, the Platform field is populated with UCC, and the Software version is 1.0. * If you selected a specific hardware vendor (such as Cisco), select from the available platforms belonging to that vendor are displayed in the Platform field (ISR, ASR, and CSR are for Cisco routers), and the Software field is populated with the related software version. 4. Click **Download**. ### Configure the Tunnel on the Remote Device On the external remote device, configure the other end of the tunnel by using the Edge gateway's connection details provided in the configuration file which you downloaded. If you need to download the configuration file, refer to [Download the External Connection Configuration](#download-an-external-connection-configuration). ### Verify the Connection Status To verify the connection status: 1. In CoPilot, go to **Diagnostics > Cloud Routes > External Connections** tab. 2. In the table, locate the connection that you created. The **Status** should be **Established**. If the connection is not established, you can run [diagnostics](/docs/enterprise/10.1/reference/ui/diagnostics/diagnostic-tools#run-connectivity-analysis) to perform an analysis or review logs. To troubleshoot issues with BGP connections, you can go to **Diagnostics > Diagnostics Tools > BGP Diagnostics** tab and run BGP commands to retrieve BGP information from the Edge Transit Gateway. ## Disconnect the External Connection To disconnect an external connection: 1. In Aviatrix CoPilot, go to **Networking > Connectivity > External Connections (S2C)** tab. 2. In the table, locate the external connection and click the Delete icon ![delete icon](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/edge/delete-icon.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=53573a210df48afba2fc09ced51e8fac) on the right side of the row. 3. To confirm that you want to delete the external connection, click **Delete**. The external connection is deleted. # Create Edge Spoke Gateway Attachments Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/edge/edge-spoke-attachments An Edge Spoke Gateway can be attached to multiple gateways for hybrid, multicloud, and LAN connectivity. An Edge Spoke Gateway can be attached to multiple gateways for hybrid, multicloud, and LAN connectivity. * For encrypted connectivity from remote site to edge location, you can attach an Edge Spoke Gateway deployed at the remote site to an Edge Transit Gateway deployed at the edge location. See [Attach an Edge Spoke Gateway to an Edge Transit Gateway](#attach-an-edge-spoke-gateway-to-an-edge-transit-gateway). * For encrypted connectivity from edge location to cloud, you can attach an Edge Spoke Gateway deployed at the edge location to a Transit Gateway in cloud. See [Attach an Edge Spoke Gateway to a Transit Gateway](#attach-an-edge-spoke-gateway-to-a-transit-gateway). ## Attach an Edge Spoke Gateway to an Edge Transit Gateway If you want to use Jumbo Frames for the attachment peering connection between the Edge gateways, make sure to enable Jumbo Frames on the Edge gateways before you create the attachment. To attach an Edge Spoke Gateway to an Edge Transit Gateway: 1. In Aviatrix CoPilot, go to **Cloud Fabric > Hybrid Cloud > Edge Gateways > Spoke Gateways** page. 2. In the table, locate the Edge Spoke Gateway to which you want to attach the Edge Transit Gateway, then click the **Manage Gateway Attachments** icon ![manage attachment link icon](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/edge/manage-attachment-link-icon.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=15215abb903bdeccf81c995db67abdce) on the right side of the row. 3. In Manage Gateway Attachments > Transit Gateway, click **+ Attachment** and provide the following information. | Field | Description | | - | - | | Transit Gateway | The remote Edge Transit Gateway to attach. | | Local Edge Gateway Interface | The WAN interface of the local Edge Spoke Gateway to use for the attachment. | | Remote Edge Gateway Interface | The WAN interface of the remote Edge Transit Gateway to use for the attachment. | 4. In the **Advanced** section, enter or select the following: | Field | Description | | - | - | | Attach Over | The connection between the Edge gateways. It can be over a **Private Network** or the **Public Network**. On the Aviatrix Edge Platform (AEP), Edge Transit Gateway to Edge Spoke Gateway peering for high performance encryption is supported over private and public networks. On the Equinix and Megaport platforms, Edge Transit Gateway to Edge Spoke Gateway peering for high performance encryption is supported over private networks only. Regular encryption is supported over both private and public networks for all edge environments. | | Jumbo Frame | Jumbo Frame improves performance for the connection between the Edge gateways. Jumbo Frame is applicable when the attachment is over a Private network. To use Jumbo Frames for the connection , set **Jumbo Frame** toggle to **On**. | | ActiveMesh | ActiveMesh enables full mesh peering between the local and remote Edge gateways. For full mesh peering, set **ActiveMesh** toggle **On**. | | Number of HPE Tunnels | The number of High Performance Encryption (HPE) tunnels to create for the attachment peering. **Single** creates a single tunnel. **Maximum** creates the maximum tunnels based on the gateway sizes and the number of interface IPs on the peering gateway. This option is available only for connection over a Private network. **Custom** allows you to specify the number of tunnels to create. | 5. To attach the Edge Spoke Gateway to another Edge Transit Gateway, click **+ Attachment** again and provide the required information. You can attach an Edge Spoke Gateway to multiple Edge Transit Gateways. Each attachment can be configured with different parameters, such as connecting WAN interfaces, connection over private or public networks, and enabling high-performance encryption. 6. Click **Save**. ## Attach an Edge Spoke Gateway to a Transit Gateway To attach an Edge Spoke Gateway to a Transit Gateway, perform the prerequisites then create the attachment. ### Prerequisites Before you create the attachment: * Ensure **[Local ASN Number](https://legacy.docs.aviatrix.com/documentation/latest/network/bgp-connection-settings.html#local-as-number)** is configured on Edge and Transit Gateway. * If the Edge to Transit Gateway attachment is over public network, you need to update the WAN Public IP on the Edge Gateway. 1. Go to **Cloud Fabric** > **Hybrid Cloud** > **Edge Gateways** tab. 2. Click **Spoke Gateways**. 3. Locate the Edge Gateway, and click its Edit icon on the right. 4. In **Edit Edge Gateway**, go to **Interface Configuration** and click **WAN**. 5. In **Public IP**, click **Discover**. ![edge discover public ip](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/edge/edge-discover-public-ip.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=83302044cfae414a8d1fc3ae96526b0b) 6. Verify the WAN Public IP and click **Save**. ### Attach Edge Spoke Gateway to Transit Gateway To create a High Performance Encryption attachment peering, make sure the Transit Gateway is created with High Performance Encryption enabled. If you want Jumbo Frame enabled for the attachment peering, make sure to enable Jumbo Frame on the Edge Gateway before you attach it to the Transit Gateway. To create the attachment: 1. In Aviatrix CoPilot, go to **Cloud Fabric** > **Hybrid Cloud** > **Edge Gateways** tab. 2. Click **Spoke Gateways**. 3. Locate the Edge Spoke Gateway, and click **Manage Gateway Attachments** icon ![manage attachment link icon](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/edge/manage-attachment-link-icon.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=15215abb903bdeccf81c995db67abdce) on the right side of the row. 4. In **Manage Gateway Attachments > Transit Gateway** tab, click **+Attachment** and provide the following information. | Field | Description | | - | - | | Transit Gateway | The Transit Gateway in cloud to attach. | | Local Edge Gateway Interface | The WAN interface of the local Edge Spoke Gateway to use for the attachment. | | Attach Over | The connection between the Edge gateways. It can be over a **Private Network** or the **Public Network**. | | ActiveMesh | ActiveMesh enables full mesh peering between the local and remote Edge gateways. For full mesh peering, set **ActiveMesh** toggle **On**. | | Jumbo Frame | Jumbo Frame improves performance for the connection between the Edge gateways. Jumbo Frame is applicable when the attachment is over a Private network. To use Jumbo Frames for the connection , set **Jumbo Frame** toggle to **On**. | | Number of HPE Tunnels | The number of High Performance Encryption (HPE) tunnels to create for the attachment peering. **Single** creates a single tunnel. **Maximum** creates the maximum tunnels based on the gateway sizes and the number of interface IPs on the peering gateway. This option is available only for connection over a Private network. **Custom** allows you to specify the number of tunnels to create. | 5. To attach the Edge Spoke Gateway to another Transit Gateway, click **+ Attachment** again and provide the required information. You can attach an Edge Spoke Gateway to multiple Transit Gateways. Each attachment can be configured with different parameters, such as connecting interfaces, connection over private or public network, high-performance encryption, and Jumbo Frame. 6. Click **Save**. ## Manage Gateway Attachments You can attach an Edge Spoke Gateway to multiple Transit Gateways. Each attachment can be configured with different parameters, such as connecting interfaces, connection over private or public network, high-performance encryption, and Jumbo Frame. Click **+ Attachment** and provide the required information. | Field | Description | | - | - | | Transit Gateway | The Transit Gateway in cloud to attach. | | Local Edge Gateway Interface | The WAN interface of the local Edge Spoke Gateway to use for the attachment. | **Advanced** | Field | Description | | - | - | | Attach Over | The connection between the Edge gateways. It can be over a **Private Network** or the **Public Network**. | | ActiveMesh | ActiveMesh enables full mesh peering between the local and remote Edge gateways. For full mesh peering, set **ActiveMesh** toggle **On**. | | Jumbo Frame | Jumbo Frame improves performance for the connection between the Edge gateways. Jumbo Frame is applicable when the attachment is over a Private network. To use Jumbo Frames for the connection , set **Jumbo Frame** toggle to **On**. | | Number of HPE Tunnels | The number of High Performance Encryption (HPE) tunnels to create for the attachment peering. **Single** creates a single tunnel. **Maximum** creates the maximum tunnels based on the gateway sizes and the number of interface IPs on the peering gateway. This option is available only for connection over a Private network. **Custom** allows you to specify the number of tunnels to create. | # Deploy an Edge Spoke Gateway on Equinix Network Edge Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/edge/edge-spoke-workflow-equinix This document provides instructions to deploy a primary Edge Spoke Gateway on Equinix Network Edge. This document provides instructions to deploy a primary Edge Spoke Gateway on [Equinix Network Edge](/docs/enterprise/10.1/concepts-architectures/components/connectivity/edge-equinix#edge-equinix-platform) . For an overview of Aviatrix Edge, see [About Aviatrix Hybrid Cloud Edge](/docs/enterprise/10.1/concepts-architectures/components/edge-gateways/edge-overview) . ## Topology The following diagram shows an example of network connectivity for Aviatrix Edge Spoke Gateway to Transit Gateway in AWS. This topology shows Aviatrix Edge Spoke Gateway connection to an upstream WAN router which is used to terminate CSP underlay private connections. ![Edge Equinix Topology 1](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/edge/edge-equinix-topology-1.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=e78d4f24dda4b01c8ca364c1891bff35) The topology below shows Aviatrix Edge Spoke Gateway used to terminate CSP underlay private connections (which does not require upstream WAN router). ![Edge Equinix Topology 2](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/edge/edge-equinix-topology-2.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=babc22f9b8b589b12bb4dd83ab8dce09) > [!NOTE] > > Aviatrix Edge Spoke Gateway requires the latest versions of Aviatrix > Controller 7.1 and Aviatrix Edge Image 7.1 or higher to support BGP underlay > connectivity to CSP. ## Prerequisites Before you deploy an Aviatrix Edge Spoke Gateway on the Equinix platform, ensure the [prerequisite steps](/docs/enterprise/10.1/reference/gateways/edge/edge-spoke-transit-planning#equinix) are complete. ## Deploy the Primary Edge Spoke Gateway (Equinix) To deploy an Edge Spoke Gateway, first you need to generate the cloud-init file in CoPilot, then deploy the Edge gateway in Equinix using the cloud-init file: 1. [Create the Primary Edge Spoke Gateway cloud-init ZTP file](https://legacy.docs.aviatrix.com/documentation/latest/network/edge-spoke-create-equinix.html). 2. [Deploy the Edge Spoke Gateway in Equinix using the cloud-init ZTP file](https://legacy.docs.aviatrix.com/documentation/latest/network/edge-vm-create-equinix.html). 3. [Verify the Edge Spoke Gateway creation](https://legacy.docs.aviatrix.com/documentation/latest/network/edge-create-verify.html). ## Next Steps * For high availability (HA) and redundancy, you can create and deploy HA Edge Spoke Gateway. See [Deploy a HA Edge Spoke Gateway (Equinix)](https://legacy.docs.aviatrix.com/documentation/latest/network/edge-spoke-ha-create-equinix.html). * For hybrid, multicloud, or LAN-side connectivity, you can attach the Edge gateway to multiple Aviatrix gateways. See [Edge Gateway Attachments](/docs/enterprise/10.1/guides/connectivity/edge/edge-spoke-attachments) . * For remote site connectivity, you can connect the Edge gateway to remote devices. See [Edge Gateway External Connection](https://legacy.docs.aviatrix.com/documentation/latest/network/edge-connect-external-device.html). # Deploy an Edge Spoke Gateway on Megaport Virtual Edge Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/edge/edge-spoke-workflow-megaport This document provides instructions to deploy a primary Edge Spoke Gateway on Megaport Virtual Edge (MVE). This document provides instructions to deploy a primary Edge Spoke Gateway on [Megaport Virtual Edge (MVE)](/docs/enterprise/10.1/concepts-architectures/components/connectivity/edge-megaport#edge-megaport-platform). For an overview of Aviatrix Edge, see [About Aviatrix Hybrid Cloud Edge](/docs/enterprise/10.1/concepts-architectures/components/edge-gateways/edge-overview). ## Topology The design below shows a high-level hybrid cloud solution with Aviatrix Edge leveraging Megaport fabric. ![Edge Megaport Topology](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/edge/edge-megaport-topology.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=5a47fef508538626fecca2a97908bb0b) ## Prerequisites Before you deploy an Edge Spoke Gateway on the Megaport platform, ensure the [prerequisite steps](/docs/enterprise/10.1/reference/gateways/edge/edge-spoke-transit-planning#megaport) are complete. ## Deploy the Primary Edge Spoke Gateway (Megaport) To create and deploy the primary Edge Spoke Gateway in Megaport Virtual Edge (MVE), perform these steps. 1. [Create the Primary Edge Spoke Gateway ZTP Cloud-Init File (Megaport)](https://legacy.docs.aviatrix.com/documentation/latest/network/edge-spoke-create-megaport.html). 2. [Deploy the Edge Spoke Gateway in MVE using the cloud-init ZTP file](https://legacy.docs.aviatrix.com/documentation/latest/network/edge-vm-create-megaport.html). 3. [Verify the Edge Spoke Gateway creation](https://legacy.docs.aviatrix.com/documentation/latest/network/edge-create-verify.html). ## Next Steps * For high availability (HA) and redundancy, you can create and deploy HA Edge Spoke Gateway. See [Deploy a HA Edge Spoke Gateway (Megaport)](/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/edge-spoke-ha). * For hybrid, multicloud, or LAN-side connectivity, you can attach the Edge Spoke Gateway to multiple Aviatrix gateways. See [Edge Gateway Attachments](/docs/enterprise/10.1/guides/connectivity/edge/edge-spoke-attachments). * For remote site connectivity, you can connect the Edge Spoke Gateway to remote devices. See [Edge Gateway External Connection](/docs/enterprise/10.1/guides/connectivity/edge/edge-s2c-workflow). # Deploy an Edge Spoke Gateway on Aviatrix Edge Platform Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/edge/edge-spoke-workflow-onpremise This document provides instructions for deploying a primary and secondary highly available (HA) Edge Spoke Gateways on the Aviatrix Edge Platform. This document provides instructions for deploying a primary and secondary highly available (HA) Edge Spoke Gateways on the Aviatrix Edge Platform. For an overview of Aviatrix Edge, see [About Aviatrix Hybrid Cloud Edge](/docs/enterprise/10.1/concepts-architectures/components/edge-gateways/edge-overview). The following deployment scenarios are supported: * Single VLAN connected to the Edge Gateway via a single vNIC. * Multiple VLANs connected to the Edge Gateway via a single vNIC (Trunk Port) and sub-interfaces for each VLAN. * VRRP on Edge Gateway. * LAN-side BGP. * Connectivity to single or multiple Transit Gateways from Edge Gateway. ## Topology ![Edge Network Connectivity](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/edge/edge-network-connectivity.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=bf70ac62bbd07f9e2af66fcb0fe7072b) ## Prerequisites Before you deploy an Edge Spoke Gateway on the Aviatrix Edge Platform, ensure the [prerequisite steps](/docs/enterprise/10.1/reference/gateways/edge/edge-spoke-planning-aep) are complete. ## Create the Primary and Secondary Edge Spoke Gateway (Aviatrix Edge Platform) Follow the steps below to create and deploy a primary and secondary highly available (HA) Edge Transit Gateways on the Aviatrix Edge platform. This diagram provides a high-level view of the process for deploying Edge Spoke Gateway using Aviatrix CoPilot. ### Step 1: Gateway Configuration 1. In Aviatrix CoPilot, go to **Cloud Fabric > Hybrid Cloud > Edge Gateways > Spoke Gateways** tab. 2. Click **+ Spoke Gateway**, then provide the following information. | Field | Description | | - | - | | **Name** | A name for the Edge Gateway. The name must begin with a letter, may include only letters, numbers, and dashes (excluding special characters or spaces), and must not exceed 50 characters in length. | | **Platform** | The platform account where you want to deploy the Edge gateway. You can create and edit platform accounts in CoPilot by going to **Cloud Fabric** > **Hybrid Cloud** > **Platforms** tab. | | **Site** | Identifies the edge location. You can select an existing name or enter a new name for the edge location. | | **High Availability** | The high availability mode. **Off** creates only the primary Edge Gateway with one active peering. **On (Active Standby Mode)** enables Edge gateway connection with one active peering and one standby peering. Only the active peering forwards network traffic. The network switches to the standby peering when the primary peering goes down. **On (Active Active Mode)** enables Edge Gateway connections with all active peerings to perform load sharing and forward network traffic. | | **Preemptive** | Determines the network's behavior when the primary gateway goes down. Preemptive is applicable only when High Availability is set to **On** with **Active Standby Mode**. The Preemptive is set on the primary gateway. **On** enables the network to automatically switch back to the primary gateway when the primary gateway connection is back up. **Off** enables the network to continue to use the standby gateway even after the primary gateway is back up, until you initiate a manual switchover. | | **Primary Device** | The edge device where you want to deploy the primary Edge Gateway. | | **Secondary Device** | The edge device where you want to deploy the secondary (HA) Edge Gateway. The primary and secondary devices must have the same hardware configuration. | | **Gateway Resource Size** | The gateway size. **Small** - 2 vCPU - 4GB. **Medium** - 4 vCPU - 8GB. **Large** - 8 vCPU - 16GB. **X-Large** - 16 vCPU - 32GB. | 3. Click **Next** to [configure the gateway interfaces](#step-2-interface-configuration). ### Step 2: Interface Configuration In the Interface Configuration section, configure the WAN, LAN, and Management interfaces. By default, an Edge Spoke Gateway has three interfaces: one WAN interface on eth0, one LAN interface on eth1, and one Management interface on eth2. You will need [these IP addresses](/docs/enterprise/10.1/reference/gateways/edge/edge-spoke-planning-aep) to configure the interfaces. If High Availability mode is selected, then configure both the primary and secondary Edge gateway interfaces. #### Configure the WAN Interface To configure the WAN interface: In **Interface Configuration**, click **WAN**, then provide the following information. | Field | Description | | - | - | | **Interface** | This is set to the Edge Gateway's logical interface. | | **IP Assignment** | The default is **Static** for static IP assignment. **DHCP** for dynamic IP address assignment is not supported. | | **Interface Labels** | A name to identify the WAN interface. | | **Interface CIDR** | The CIDR for the WAN interface. Interface CIDR must be in the format interface\_ip/netmask (for example, 192.18.20.1/24). | | **Default Gateway IP** | The Default Gateway IP address for the WAN interface. For CSP underlay, this is the remote side IP address of the BGP session on CSP VNG or VGW. If **Link-Local Underlay CIDR** is configured, the Default Gateway IP should be in the same subnet as the Link-Local Underlay CIDR, otherwise, it should be in the same subnet as the WAN Interface CIDR. | | **Public IP** | The public IP for the WAN interface. The public IP of the WAN interface is used for peering connections over the public network. | #### Configure the LAN Interface To configure the LAN interface: Click **LAN**, then provide the following information. | Field | Description | | - | - | | **Interface** | The Edge Gateway's logical interface name. | | **IP Assignment** | The default is **Static** for static IP assignment. **DHCP** for dynamic IP address assignment is not supported. | | **VRRP** | To enable Virtual Router Redundancy Protocol (VRRP) on the Edge Gateway, set this switch to **On**. VRRP can be configured only if HA Edge Gateway is configured. | | **Support for IPv6** | To enable IPv6 on the Edge Gateway, set this switch to **On**. Enable **Support for IPv6** on the Aviatrix Controller (**Settings** > **Configuration** > **General** in CoPilot) to see this option. For more information, see the [IPv6 overview](/docs/enterprise/10.1/concepts-architectures/components/connectivity/ipv6-overview). | | **Interface CIDR** | The native VLAN interface IPv4 address. This interface is where untagged packets are sent. | | **IPv6 CIDR** | The native VLAN interface IPv6 address. This interface is where untagged packets are sent. | | **VRRP Gateway IP** | The Virtual IP for the VRRP Gateway, when VRRP is enabled. | | **Default Gateway IP** | The Default Gateway IPv4 address for the native VLAN interface. | | **Default Gateway IPv6** | The Default Gateway IPv6 address for the native VLAN interface. | | **Interface Labels** | Name to identify the native VLAN interface. | If your LAN is segmented into virtual LANs (VLANs), click **+ VLAN Interface** to add one or more VLAN sub-interfaces, then provide the following information for each VLAN sub-interface. > [!WARNING] > > You cannot edit the VLAN ID after the Edge Gateway is created. To edit the > VLAN sub-interface attributes, it is highly recommended to delete and recreate > the VLAN sub-interface configuration. | Field | Description | | - | - | | **VLAN ID** | The VLAN ID of the VLAN segment. VLAN ID must be a number between 2 and 4092. | | **VLAN Interface CIDR** | The VLAN IP address of the VLAN segment. | | **Default Gateway IP** | The Default Gateway IP address of the VLAN segment. | | **Sub-Interface Tag** | Name to identify the VLAN segment. | When a secondary HA Edge Gateway is configured, the VLAN configurations that are shared between the primary and secondary gateway and are non-editable on the secondary gateway. #### Configure the MGMT Interface To configure the Management interface, click **MGMT**, then provide the following information. | Field | Description | | - | - | | **Interface** | The Edge Gateway's logical interface name. | | **IP Assignment** | The MGMT interface defaults to **DHCP**. This setting cannot be changed. The Edge Gateway will automatically NAT out of the physical MGMT interface of the edge node when using the Aviatrix Edge platform. | | **Private Network** | Leave this setting to **Off**. The Edge Gateway on the edge hardware requires public Internet reachability to connect to the Aviatrix Controller and Aviatrix Edge infrastructure in the cloud. | | **Egress CIDR (Primary)** | The Egress CIDR is the public IP address which the Management interface uses. If the Public IP is used from Edge Gateway Management interface to establish connectivity to Aviatrix Controller, then configure the Public IP as the CIDR. The CIDR is then added to the Controller security group to allow incoming traffic from the Edge Gateway. | | **Egress CIDR (Secondary)** | The Egress Public IP for the secondary Edge Gateway's Management interface when High Availability is configured. | If a required field is missing, the interface tab is highlighted to indicate there is an error. ### Step 3: Interface Mapping Interface Mapping is provided for reference purposes (view only). The table shows a mapping of the Edge Gateway's logical interface names to the Edge Gateway's Linux interface names. These interface names are used in the Aviatrix log files and error messages. When performing diagnostic or troubleshooting on the Aviatrix Platform, you can refer to this mapping for the interface name. | Field | Description | | - | - | | **Interface** | The Edge Gateway's logical interface name. | | **Primary Gateway Ethernet Interface** | The primary Edge Gateway's Linux interface name. | | **Secondary Gateway Ethernet Interface** | The secondary (HA) Edge Gateway's Linux interface name. | The secondary gateway interface mappings are shown only when high availability is configured for the primary Edge Gateway. ### Step 4: Verify the Edge Gateway Creation See, [Verify the Edge Gateway Creation](https://legacy.docs.aviatrix.com/documentation/latest/network/edge-create-verify.html). ## Next Steps * For hybrid, multicloud, or LAN-side connectivity, you can attach the Edge gateway to multiple Aviatrix gateways. See [Edge Gateway Attachments](/docs/enterprise/10.1/guides/connectivity/edge/edge-spoke-attachments). * For remote site connectivity, you can connect the Edge gateway to remote devices. See [Edge Gateway External Connection](https://legacy.docs.aviatrix.com/documentation/latest/network/edge-connect-external-device.html). # Edge Spoke Gateway Deployment Workflow on Self-Managed VMware ESXi and KVM Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/edge/edge-spoke-workflow-selfmanaged This document provides instructions for deploying a primary and secondary highly available (HA) Edge Spoke Gateways on self-managed VMware ESXi or an open-source Kernal-based Virtual Machine (KVM). This document provides instructions for deploying a primary and secondary highly available (HA) Edge Spoke Gateways on self-managed VMware ESXi or an open-source Kernel-based Virtual Machine (KVM). For an overview of Aviatrix Edge, see [About Aviatrix Hybrid Cloud Edge](/docs/enterprise/10.1/concepts-architectures/components/edge-gateways/edge-overview) . > [!NOTE] > > The deployment workflow on this page applies to all currently supported > Aviatrix Edge VM image generations, including **g3**, **g4**, and later. > Specific image generations may have different VM sizing, disk, and host > requirements; always confirm the requirements for the image generation you are > deploying before you provision the VM. See > [About Aviatrix Edge Gateway Requirements](/docs/enterprise/10.1/reference/gateways/edge/edge-vm-requirements) > for the current per-image-generation specifications. ## Topology The following diagram shows an example of network connectivity for Aviatrix Edge Gateway to Transit Gateway in AWS. ![Edge Network Connectivity](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/edge/edge-network-connectivity.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=bf70ac62bbd07f9e2af66fcb0fe7072b) ## Prerequisites Before you deploy an Aviatrix Edge Gateway on VMware ESXi or KVM, ensure the prerequisite requirements are complete, see [Prerequisites for Edge Spoke Deployment on VMware ESXi and KVM](/docs/enterprise/10.1/reference/gateways/edge/edge-spoke-planning-selfmanaged) . ## Aviatrix Edge Spoke Gateway Deployment Workflow To deploy Aviatrix Edge Spoke Gateway, first you need to procure and onboard your edge device on the platform of your choice (see [Prerequisites for Edge Spoke Deployment on VMware ESXi and KVM](/docs/enterprise/10.1/reference/gateways/edge/edge-spoke-planning-selfmanaged) ). Next, you deploy the Aviatrix Edge Gateway on the edge device and attach the Edge Gateway to the Aviatrix Transit Gateway for cloud connectivity. Then, configure the Edge Gateway for LAN-side connectivity. The diagram below provides a high-level view of the four-step process for deploying Aviatrix Edge Spoke Gateway in Aviatrix CoPilot. You have the option to use either VMware ESXi or an open-source Kernel-based Virtual Machine (KVM) to deploy the Edge Spoke Gateway VM and attach the ISO file. The ISO file is the equivalent of the Zero-Touch Provisioning (ZTP) token. ZTP allows to remotely deploy and provision network devices at remote locations. ![Edge Deployment Workflow](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/edge/edge-deploy-workflow.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=8e168b3f40f824d109ec949dcb0479a4) This workflow provides the steps to create a primary and secondary (HA) Edge Gateway in VMware ESXi and KVM. It also provides the steps to attach the Edge Gateways to a Transit Gateway and connect the Edge Gateways to an external device, such as a LAN BGP router. 1. [Create the ZTP ISO for the primary Edge Gateway](#creating-the-ztp-iso-for-the-edge-gateway-self-managed-platform). 2. [Deploy the primary Edge Gateway Virtual Machine and Attach ZTP ISO](#deploying-the-edge-gateway-virtual-machine-instance-and-attaching-the-ztp-iso). 3. [Create the ZTP ISO for the secondary Edge Gateway](#creating-a-highly-available-edge-gateway-self-managed-platform). 4. [Deploy the secondary Edge Gateway Virtual Machine and Attach ZTP ISO](#deploying-the-edge-gateway-virtual-machine-instance-and-attaching-the-ztp-iso). 5. [Attach the primary Edge Gateway to a Transit Gateway](#attach-an-edge-spoke-gateway-to-a-transit-gateway). 6. [Connect the Edge Gateway to an external device](#connecting-edge-spoke-gateway-to-an-external-device-bgp-over-lan). ## Creating the ZTP ISO for the Edge Gateway (Self-Managed Platform) You must have port 443 open to the IP address of the Aviatrix Controller. For the required port access for Edge Gateway deployment, refer to [Aviatrix Edge Gateway Ports and Protocols](/docs/enterprise/10.1/reference/gateways/edge/edge-spoke-planning-selfmanaged) . In Aviatrix CoPilot: 1. Go to **Cloud Fabric > Hybrid Cloud > Edge Gateways** tab. 2. Click **+ Spoke Gateways**, then provide the following information. Provide the following information. | Parameter | Description | | - | - | | Name | Name for the Edge Gateway. | | Platform | Select **edge\_admin**. | | Site | Select an existing name or enter a new name to identify the edge location. Site names cannot contain spaces. | | ZTP File Type | Select the ZTP file type. For VMware ESXi, select **iso**. For KVM, select **iso** or **cloud-init**. | | High Availability | High Availability is set to **Off** for the primary Edge Gateway. For the secondary (HA) Edge Gateways, select **Active-Active** or **Active-Standby** mode. | > [!NOTE] > > Deploying multiple Edge Gateways for the same site is supported. A maximum of > 8 Edge Gateways are supported. 3. [Configure the WAN, LAN, and Management interfaces](#configuring-the-edge-gateway-interfaces). ### Configuring the Edge Gateway Interfaces By default, an Aviatrix Edge Gateway has three interfaces: one WAN interface on eth0, one LAN interface on eth1, and one Management interface on eth2. You will need these configuration information to configure the interfaces. In the **Interface Configuration** section, configure the **WAN**, **LAN**, and **Management** interfaces for the Edge Gateway. #### Configuring the WAN Interface Click **WAN**, then provide the following information. For IP and DNS settings, enter using the applicable format. For example, if the Edge Gateway's WAN IP is 10.1.1.151, enter 10.1.1.151/24 or what your netmask is. | Parameter | Description | | - | - | | IP Assignment | The default is **Static** for static IP assignment. **DHCP** for dynamic IP address assignment is not supported. | | Interface Labels | (optional) A name to identify the WAN interface. | | Interface CIDR | The CIDR for the WAN interface. | | Default Gateway IP | The Default Gateway IP address for the WAN interface. | | Public IP | (optional) The Public IP address of the WAN interface. | > [!NOTE] > > To change or update the Edge Gateway WAN connectivity to Transit Gateway, you > will need to first detach the Edge-to-Transit gateway attachment, if there is > an attachment. #### Configuring the LAN Interface Click **LAN**, then provide the following information. | Parameter | Description | | - | - | | IP Assignment | The default is **Static** for static IP assignment. **DHCP** for dynamic IP address assignment is not supported. | | VRRP | If you have Virtual Router Redundancy Protocol (VRRP) configured for the LAN router redundancy, set this switch to **On**. | | Interface CIDR | The CIDR for the LAN interface. | | VRRP Gateway IP | The Virtual IP (VIP) address, when VRRP is enabled. | | Default Gateway IP | (optional) The Default Gateway IP for the LAN interface. | | Interface Labels | (optional) A name to identify the LAN interface. | **VLAN Interface** If your LAN is segmented into virtual LANs (VLANs), click **+ VLAN Interface** to add one or more VLAN sub-interfaces, then provide the following information for each VLAN sub-interface. > [!NOTE] > > You cannot edit the VLAN ID after the Edge Gateway is created. To edit the > VLAN sub-interface attributes, it is highly recommended to delete and recreate > the VLAN sub-interface configuration. | Parameter | Description | | - | - | | Interface CIDR | The native VLAN interface IP address. This is the interface where the untagged packets are sent. | | VRRP Gateway IP | The Virtual IP for the VRRP Gateway, when VRRP is enabled. | | Default Gateway IP | The Default Gateway IP address for the native VLAN interface. | | Interface Labels | (optional) A name to identify this native VLAN interface. | | **VLAN Sub-Interfaces** | | | VLAN ID | The VLAN ID. VLAN ID must be a number between 2 and 4092. | | VLAN Interface CIDR | The VLAN sub-interface IP address. | | VRRP Gateway IP | The Virtual IP for the VRRP Gateway, when VRRP is enabled. | | Default Gateway IP | The Default Gateway IP address for this VLAN sub-interface. | | Sub-Interface Tag | (optional) A name to identify this VLAN sub-interface. | #### Configuring the MGMT Interface Click **MGMT**, then provide the following information. | Parameter | Description | | - | - | | IP Assignment | Select **DHCP** or **Static**, depending on your environment. This setting cannot be changed after the gateway is created. | | Private Network | If the Management interface connection to the Aviatrix Controller is over a private network, set this switch to **On**. Leave the setting to **Off**, if the connection is over the public internet. | | Parameter | Description | | - | - | | Egress CIDR (Optional) | The CIDR range for the egress flow for the Management interface. | CoPilot creates the ISO file and downloads the file to your downloads folder. Next, log in to your VMware ESXi or KVM host and upload the ISO or cloud-init file to a datastore or storage device. Then, deploy the Edge Gateway VM instance and attach the ISO or cloud-init image file to complete the Edge Gateway creation and authentication with the Aviatrix Controller. > [!NOTE] > > The ISO file expires after 24 hours. You cannot download it again and will > have to repeat the above steps. You must mount the ISO file to an Edge VM to > complete the Edge Gateway registration within 24 hours. See [Deploying the > Edge Gateway Virtual Machine > Instance](#deploying-the-edge-gateway-virtual-machine-instance-and-attaching-the-ztp-iso). ## Creating a Highly Available Edge Gateway (Self-Managed Platform) Before you can create the highly available Edge Gateway, the primary Edge Gateway must be deployed and its status must be **Up**. You must have port 443 open to the IP address of the Aviatrix Controller. For the required port access for Edge Gateway deployment, refer to [Aviatrix Edge Gateway Ports and Protocols](/docs/enterprise/10.1/reference/gateways/edge/edge-spoke-planning-selfmanaged) . To create a secondary (HA) Edge Gateway, follow these steps. 1. In Aviatrix CoPilot, go to **Cloud Fabric** > **Edge** > **Gateways** tab. 2. In the table, locate the primary Edge Gateway for which you want to create the HA gateway and click its Edit icon. 3. In the **Edit Edge Gateway** dialog box, from the **High Availability** dropdown menu, select **Active-Active** or **Active-Standby** mode. 4. In the Interfaces section, configure the WAN, LAN, and Management interfaces for the secondary (HA) Edge Gateway. **WAN Interface** Click **WAN**, then provide the following information. For IP and DNS settings, enter using the applicable format. For example, if the Edge Gateway's WAN IP is 10.1.1.151, enter 10.1.1.151/24 or what your netmask is. | Parameter | Description | | - | - | | IP Assignment | The default is **Static** for static IP assignment. **DHCP** for dynamic IP address assignment is not supported. | | Interface Labels | (optional) A name to identify the WAN interface. | | Interface CIDR | The CIDR for the WAN interface. | | Default Gateway IP | The Default Gateway IP address for the WAN interface. | | Public IP | (optional) The Public IP address of the WAN interface. | **LAN Interface** Click **LAN**, then provide the following information. | Parameter | Description | | - | - | | IP Assignment | The default is **Static** for static IP assignment. **DHCP** for dynamic IP address assignment is not supported. | | VRRP | If you have Virtual Router Redundancy Protocol (VRRP) configured for the LAN router redundancy, set this switch to **On**. | | Interface CIDR | The CIDR for the LAN interface. | | VRRP Gateway IP | The Virtual IP (VIP) address, when VRRP is enabled. | | Default Gateway IP | (optional) The Default Gateway IP for the LAN interface. | | Interface Labels | (optional) A name to identify the LAN interface. | **VLAN Interface** Provide the following information for each VLAN sub-interface. | Parameter | Description | | - | - | | Interface CIDR | The native VLAN interface IP address. This is the interface where the untagged packets are sent. | | Default Gateway IP | The Default Gateway IP address for the native VLAN interface. | | Interface Labels | (optional) A name to identify this native VLAN interface. | | **VLAN Sub-Interfaces** | | | VLAN Interface CIDR | The VLAN sub-interface IP address. | | Default Gateway IP | The Default Gateway IP address for this VLAN sub-interface. | > [!NOTE] > > VLAN configurations are added to the primary Edge Gateway. On the secondary > Edge Gateway, some fields are disabled and non-editable, the field value > appears when it is selected. CoPilot creates the ISO file and downloads the file to your downloads folder. Next, log in to your VMware ESXi or KVM host and upload the ISO or cloud-init file to a datastore or storage device. Then, deploy the Edge Gateway VM instance and attach the ISO or cloud-init image file to complete the Edge Gateway creation and authentication with the Aviatrix Controller. > [!NOTE] > > The ISO file expires after 24 hours. You cannot download it again and will > have to repeat the above steps. You must mount the ISO file to an Edge VM to > complete the Edge Gateway registration within the 24-hour timeframe. See [Deploying the Edge Gateway Virtual Machine Instance](#deploying-the-edge-gateway-virtual-machine-instance-and-attaching-the-ztp-iso). ## Deploying the Edge Gateway Virtual Machine Instance and Attaching the ZTP ISO See: * [Deploying the Edge Gateway Virtual Machine in VMware ESXi](#deploying-the-edge-gateway-virtual-machine-in-vmware-esxi) * [Deploying the Edge Gateway Virtual Machine in KVM](#deploying-the-edge-gateway-virtual-machine-in-kvm) ### Deploying the Edge Gateway Virtual Machine in VMware ESXi To deploy the Edge Gateway virtual machine, follow these steps. 1. If you have not downloaded the ESXi OVA file, download the file by using the link provided to you by Aviatrix Support. See [Download the Aviatrix Secure Edge Image File](/docs/enterprise/10.1/reference/gateways/edge/edge-vm-requirements) . 2. Log in to VMware vSphere Web client to access the ESXi host. You can use vSphere Web client to manage ESXi host, launch a VM, mount ISO files, and start and stop the Aviatrix Edge Gateway. 3. Load the OVA file into the ESXi using vSphere, go to **ESXi** > **Virtual Machines** > **Create/Register VM**. 4. Select **Deploy a virtual machine from an OVF or OVA file** and click **Next**. 5. Enter a name for the Aviatrix Secure Edge VM and drag the OVA file into the blue pane, then click **Next**. 6. In the Select storage page, select the storage device where to create the VM instance (the OVA is installed in this instance) and click **Next**. 7. In the Deployment options window, enter the **Network mappings** for WAN, LAN, and MGMT network interfaces and select the **Deployment type**. (Refer to the pull-down menu or see [Virtual Machine CPU and Memory Configurations](/docs/enterprise/10.1/reference/gateways/edge/edge-vm-requirements) .) If necessary, you can change the network interface mappings after deployment. 8. Click **Next**. 9. In the Ready to complete page, click **Finish**. Next, [attach the ISO file](#attaching-the-iso-file-to-the-edge-gateway-virtual-machine-in-vmware-esxi) to Edge Gateway VM, which will auto-mount the media with the configuration file to be provision the Edge Gateway. #### Attaching the ISO File to the Edge Gateway Virtual Machine in VMware ESXi > [!NOTE] > > The ZTP ISO file can only be used for a single Aviatrix Secure Edge VM > instance, and only one time for that instance. > [!NOTE] > > The ZTP token expires after 24 hours. If you wait too long to boot up the VM > with the attached ISO image, it will not work. In that case, delete the Edge > Gateway in the Aviatrix CoPilot and create a new Edge Gateway to receive a new > ISO file. 1. Upload the ISO file downloaded from Aviatrix CoPilot to your VMware datastore. 2. In vSphere, select the Aviatrix Secure Edge VM you created and click **Edit settings**. 3. Select the **Virtual Hardware** tab. 4. Next to CD/DVD Drive 1, click the dropdown menu and select **Datastore ISO file**. 5. Next to CD/DVD Drive 1, ensure the **Connect** box is checked and click **Save**. **Connect at power on** is required when you attach the ISO image to the VM for the first time. If the VM is powered on at the time you attach the ISO image, select the ISO file and save the configuration to make the ISO available to ZTP. 6. Next to the CD/DVD Media, click **Browse**, locate the datastore and select the ISO file you uploaded. 7. Click **Save**. ZTP auto-mounts the ISO file and deploys the Edge Gateway on the VM. Verify the Edge Gateway is Up (see [Verifying the Edge Gateway Creation](https://legacy.docs.aviatrix.com/documentation/latest/network/edge-create-verify.html)). Next, [attach the Edge Gateway to the Transit Gateway](#attach-an-edge-spoke-gateway-to-a-transit-gateway). ### Deploying the Edge Gateway Virtual Machine in KVM Aviatrix Edge Gateway can be deployed on KVM on Linux. There are numerous user-space front-ends to KVM. The Edge Gateway VM does not depend on any specific user-space tools, but you must understand how to configure your choice of KVM tools. Refer to your management tool's documentation for details. If these requirements are unclear, consider adopting [Aviatrix Edge Platform](/docs/enterprise/10.1/concepts-architectures/components/edge-gateways/edge-aep) instead, which manages this for you. **Requirements for Edge Gateway on KVM:** * The network must use `virtio` drivers. Emulated physical drivers do not provide adequate performance. * Multi-queue networking should be enabled, with the number of queues equal to the number of CPUs. * Storage must use `virtio` or `nvme`. Emulated physical drivers do not provide adequate performance. * At least 64 GB of disk space is required for production use cases. The images support smaller deployments, but this is only appropriate for lab scenarios. * The LAN, WAN, and MGMT network bridges must be associated with the physical Ethernet interfaces on the KVM host. Refer to your KVM product documentation. > [!NOTE] > > Even with `virtio` networking, self-managed Edge Gateway deployments on KVM > may run into CPU saturation under heavy load. For higher performance, consider > [Aviatrix Edge Platform](/docs/enterprise/10.1/concepts-architectures/components/edge-gateways/edge-aep) > , which uses interface passthrough to deliver bare-metal network speeds. After you have configured your KVM environment to meet the requirements above, deploy the Edge Gateway VM using your chosen KVM management tool: 1. Download the KVM QCOW2 image file using the link provided by Aviatrix Support. See [Download the Aviatrix Secure Edge Image File](/docs/enterprise/10.1/reference/gateways/edge/edge-vm-requirements) for the current per-image-generation disk, vCPU, and memory specifications (these vary across image generations such as g3 and g4). 2. Create the Edge Gateway VM from the QCOW2 image, attaching the WAN, LAN, and MGMT virtual bridge interfaces using the `virtio` device model. 3. Attach the ZTP ISO file you downloaded from Aviatrix CoPilot to the VM. 4. Start the VM. ZTP auto-mounts the ISO file and provisions the Edge Gateway. Verify the Edge Gateway is Up (see [Verifying the Edge Gateway Creation](https://legacy.docs.aviatrix.com/documentation/latest/network/edge-create-verify.html)). Next, [attach the Edge Gateway to the Transit Gateway](#attach-an-edge-spoke-gateway-to-a-transit-gateway). ## Attach an Edge Spoke Gateway to a Transit Gateway To attach an Edge Spoke Gateway to a Transit Gateway, perform the prerequisites then create the attachment. ### Prerequisites Before you create the attachment: * Ensure **Local ASN Number** is configured on Edge and Transit Gateway. * If the Edge to Transit Gateway attachment is over public network, you need to update the WAN Public IP on the Edge Gateway. 1. Go to **Cloud Fabric** > **Hybrid Cloud** > **Edge Gateways** tab. 2. Click **Spoke Gateways**. 3. Locate the Edge Gateway, and click its Edit icon on the right. 4. In **Edit Edge Gateway**, go to **Interface Configuration** and click **WAN**. 5. In **Public IP**, click **Discover**. 6. Verify the WAN Public IP and click **Save**. ### Attach Edge Spoke Gateway to Transit Gateway > [!NOTE] > > To create a High Performance Encryption attachment peering, make sure the > Transit Gateway is created with High Performance Encryption enabled. > [!NOTE] > > If you want Jumbo Frame enabled for the attachment peering, make sure to > enable Jumbo Frame on the Edge Gateway before you attach it to the Transit > Gateway. To create the attachment: 1. In Aviatrix CoPilot, go to **Cloud Fabric** > **Hybrid Cloud** > **Edge Gateways** tab. 2. Click **Spoke Gateways**. 3. Locate the Edge Spoke Gateway, and click **Manage Gateway Attachments** icon on the right side of the row. 4. In **Manage Gateway Attachments > Transit Gateway** tab, click **+Attachment** and provide the following information. | Field | Description | | - | - | | Transit Gateway | The Transit Gateway in cloud to attach. | | Local Edge Gateway Interface | The WAN interface of the local Edge Spoke Gateway to use for the attachment. | | Attach Over | The connection between the Edge gateways. It can be over a **Private Network** or the **Public Network**. | | ActiveMesh | ActiveMesh enables full mesh peering between the local and remote Edge gateways. For full mesh peering, set **ActiveMesh** toggle **On**. | | Jumbo Frame | Jumbo Frame improves performance for the connection between the Edge gateways. Jumbo Frame is applicable when the attachment is over a Private network. To use Jumbo Frames for the connection , set **Jumbo Frame** toggle to **On**. | | Number of HPE Tunnels | The number of High Performance Encryption (HPE) tunnels to create for the attachment peering. **Single** creates a single tunnel. **Maximum** creates the maximum tunnels based on the gateway sizes and the number of interface IPs on the peering gateway. This option is available only for connection over a Private network. **Custom** allows you to specify the number of tunnels to create. | To attach the Edge Spoke Gateway to another Transit Gateway, click **+ Attachment** again and provide the required information. > [!NOTE] > > You can attach an Edge Spoke Gateway to multiple Transit Gateways. Each > attachment can be configured with different parameters, such as connecting > interfaces, connection over private or public network, high-performance > encryption, and Jumbo Frame. 5. Click **Save**. ### Manage Gateway Attachments You can attach an Edge Spoke Gateway to multiple Transit Gateways. Each attachment can be configured with different parameters, such as connecting interfaces, connection over private or public network, high-performance encryption, and Jumbo Frame. Click **+ Attachment** and provide the required information. | Field | Description | | - | - | | Transit Gateway | The Transit Gateway in cloud to attach. | | Local Edge Gateway Interface | The WAN interface of the local Edge Spoke Gateway to use for the attachment. | **Advanced** | Field | Description | | - | - | | Attach Over | The connection between the Edge gateways. It can be over a **Private Network** or the **Public Network**. | | ActiveMesh | ActiveMesh enables full mesh peering between the local and remote Edge gateways. For full mesh peering, set **ActiveMesh** toggle **On**. | | Jumbo Frame | Jumbo Frame improves performance for the connection between the Edge gateways. Jumbo Frame is applicable when the attachment is over a Private network. To use Jumbo Frames for the connection , set **Jumbo Frame** toggle to **On**. | | Number of HPE Tunnels | The number of High Performance Encryption (HPE) tunnels to create for the attachment peering. **Single** creates a single tunnel. **Maximum** creates the maximum tunnels based on the gateway sizes and the number of interface IPs on the peering gateway. This option is available only for connection over a Private network. **Custom** allows you to specify the number of tunnels to create. | Next, [connect the Edge Gateway to the external device](#connecting-edge-spoke-gateway-to-an-external-device-bgp-over-lan). ## Connecting Edge Spoke Gateway to an External Device (BGP over LAN) For LAN-side connectivity, you can connect the Edge Spoke Gateway to an external device, such as a LAN BGP router. To connect the Edge Gateway to the LAN BGP router, follow these steps. 1. In CoPilot, navigate to **Networking > Connectivity** > **External Connections (S2C)** tab. 2. From **+ External Connection To** dropdown menu, select **External Device**, then provide the following information. | Field | Description | | - | - | | Name | Name to identify the connection to the LAN router. | | Connect Using | Select **BGP**. | | Type | Select **LAN**. | | Local Gateway | The Edge Gateway that you want to connect to the LAN router. | | Local ASN | The Local AS number that the Edge Gateway will use to exchange routes with the LAN router. This is automatically populated if the Edge Gateway is assigned an ASN already. | 3. In **LAN Configuration**, provide the following information. | Field | Description | | - | - | | Remote ASN | The BGP AS number that is configured on the LAN router. | | Remote LAN IP | The IP address of the LAN router. | | Local LAN IP | This is automatically populated with the Edge Gateway's LAN interface IP address. | 4. Click **Save**. # Create Edge Transit Gateway Attachments Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/edge/edge-transit-attachments An Edge Transit Gateway can be attached to multiple Aviatrix gateways for hybrid, multicloud, and LAN connectivity. An Edge Transit Gateway can be attached to multiple Aviatrix gateways for hybrid, multicloud, and LAN connectivity. * For encrypted connectivity across network service providers (NSPs), such as Equinix and Megaport, and Datacenter fabric, [create an Edge Transit Gateway to Edge Transit Gateway attachment](#attach-an-edge-transit-gateway-to-edge-transit-gateway). * For encrypted connectivity to cloud, [create an Edge Transit Gateway to a Transit Gateway attachment](#attach-an-edge-transit-gateway-to-transit-gateway). * For encrypted connectivity to remote site or on-premises, [create an Edge Transit Gateway to an Edge Spoke Gateway attachment](#attach-an-edge-transit-gateway-to-edge-spoke-gateway). ## Attach an Edge Transit Gateway to Edge Transit Gateway > [!NOTE] > > If you want to use Jumbo Frames for the attachment peering connection between > the Edge Gateways, make sure to enable Jumbo Frames on the Edge Gateways > before you create the attachment. To attach an Edge Transit Gateway to an Edge Transit Gateway: 1. In Aviatrix CoPilot, go to **Cloud Fabric > Hybrid Cloud > Edge Gateways > Transit Gateways** tab. 2. Locate the Edge Transit Gateway to which you want to attach an Edge Transit Gateway, then click the **Manage Gateway Attachments** icon ![manage attachment link icon](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/edge/manage-attachment-link-icon.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=15215abb903bdeccf81c995db67abdce) on the right side of the row. 3. In **Manage Gateway Attachments > Transit Gateway** tab, click **+ Attachment** and provide the following information. | Field | Description | | - | - | | Transit Gateway | The remote Edge Transit Gateway to attach. | | Local Edge Gateway Interface | The WAN interface of the local Edge Transit Gateway to use for the attachment. | | Remote Edge Gateway Interface | The WAN interface of the remote Edge Transit Gateway to use for the attachment. | In the **Advanced** section, enter or select the following: | Field | Description | | - | - | | Attach Over | The connection between the Edge gateways. It can be over a **Private Network** or the **Public Network**. On the Aviatrix Edge Platform (AEP), Edge Transit Gateway to Edge Transit Gateway peering for high performance encryption is supported over private and public networks. On the Equinix and Megaport platforms, Edge Transit to Edge Transit peering for high performance encryption is supported over private network only. Regular encryption is supported over both private and public networks for all environments. | | Jumbo Frame | Jumbo Frame improves performance for the connection between the Edge gateways. Jumbo Frame is applicable when the attachment is over a Private network. To use Jumbo Frames for the connection, set **Jumbo Frame** toggle to **On**. | | ActiveMesh | ActiveMesh enables full mesh peering between the local and remote Edge gateways. For full mesh peering, set **ActiveMesh** toggle **On**. | | Number of HPE Tunnels | The number of High Performance Encryption (HPE) tunnels to create for the attachment peering. **Single** creates a single tunnel. **Maximum** creates the maximum tunnels based on the gateway sizes and the number of interface IPs on the peering gateway. This option is available only for connection over a Private network. **Custom** allows you to specify the number of tunnels to create. | | Excluded Network CIDRs | If you want to exclude CIDRs from the local Edge Gateway from being propagated to the remote Edge Gateway, set Excluded Network CIDRs toggle to **On**. In **Excluded Network CIDRs** field, enter the CIDRs to be excluded. | 4. To attach the Edge Transit Gateway to another Edge Transit Gateway, click **+ Attachment** again and provide the required information. You can attach an Edge Transit Gateway to multiple Edge Transit Gateways. Each attachment can be configured with different parameters, such as connecting WAN interfaces, connection over private or public networks, and enabling high-performance encryption. 5. Click **Save**. ## Attach an Edge Transit Gateway to Transit Gateway > [!NOTE] > > To create a High Performance Encryption (HPE) attachment, make sure the > Transit Gateway in the cloud is created with HPE enabled. > [!NOTE] > > If you want to use Jumbo Frames for the attachment, make sure to enable Jumbo > Frames on the Edge Transit Gateway and the Transit Gateway in the cloud before > you create the attachment. To attach an Edge Transit Gateway to a Transit Gateway in the cloud: 1. In Aviatrix CoPilot, go to **Cloud Fabric > Hybrid Cloud > Edge Gateways > Transit Gateways** tab. 2. Locate the Edge Transit Gateway to which you want to attach the Transit Gateway, then click the **Manage Gateway Attachments** icon ![manage attachment link icon](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/edge/manage-attachment-link-icon.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=15215abb903bdeccf81c995db67abdce) on the right side of the row. 3. In **Manage Gateway Attachments > Transit Gateway** tab, click **+ Attachment** and provide the following information. | Field | Description | | - | - | | Transit Gateway | The Transit Gateway in cloud to attach. | | Local Edge Gateway Interface | The WAN interface of the local Edge Transit Gateway to use for the attachment. | | Attach Over | The connection between the Edge gateways. It can be over a **Private Network** or the **Public Network**. | | Jumbo Frame | Jumbo Frame improves performance for the connection between the Edge gateways. Jumbo Frame is applicable when the attachment is over a Private network. To use Jumbo Frames for the connection, set **Jumbo Frame** toggle to **On**. | | Number of HPE Tunnels | The number of High Performance Encryption (HPE) tunnels to create for the attachment peering. **Single** creates a single tunnel. **Maximum** creates the maximum tunnels based on the gateway sizes and the number of interface IPs on the peering gateway. This option is available only for connection over a Private network. **Custom** allows you to specify the number of tunnels to create. | | Excluded Network CIDRs | If you want to exclude CIDRs from the local Edge Gateway from being propagated to the remote Edge Gateway, set Excluded Network CIDRs toggle to **On**. In **Excluded Network CIDRs** field, enter the CIDRs to be excluded. | 4. To attach the Edge Transit Gateway to another Transit Gateway, click **+ Attachment** again and provide the required information. You can attach an Edge Transit Gateway to multiple Transit Gateways. Each attachment can be configured with different parameters, such as connecting WAN interfaces, connection over private or public networks, and enabling high-performance encryption. 5. Click **Save**. ## Attach an Edge Transit Gateway to Edge Spoke Gateway > [!NOTE] > > If you want to use Jumbo Frames for the attachment peering connection between > the Edge Gateways, make sure to enable Jumbo Frames on the Edge Gateways > before you create the attachment. To attach an Edge Transit Gateway to an Edge Spoke Gateway: 1. In Aviatrix CoPilot, go to **Cloud Fabric > Hybrid Cloud > Edge Gateways > Transit Gateways** tab. 2. Locate the Edge Transit Gateway to which you want to attach the Edge Spoke Gateway, then click the **Manage Gateway Attachments** icon ![manage attachment link icon](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/edge/manage-attachment-link-icon.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=15215abb903bdeccf81c995db67abdce) on the right side of the row. 3. In **Manage Gateway Attachments > Spoke Gateway** tab, click **+ Attachment** and provide the following information. | Field | Description | | - | - | | Spoke Gateway | The remote Edge Spoke Gateway to attach. | | Local Edge Gateway Interface | The WAN interface of the local Edge Transit Gateway to use for the attachment. | | Remote Edge Gateway Interface | The WAN interface of the remote Edge Spoke Gateway to use for the attachment. | | Attach Over | The connection between the Edge gateways. It can be over a **Private Network** or the **Public Network**. On the Aviatrix Edge Platform (AEP), Edge Transit Gateway to Edge Spoke Gateway peering for high performance encryption over private and public networks is supported. On the Equinix and Megaport platforms, Edge Transit Gateway to Edge Spoke Gateway peering for high performance encryption is supported over private networks only. Regular encryption is supported over both private and public networks for all environments. | | Jumbo Frame | Jumbo Frame improves performance for the connection between the Edge gateways. Jumbo Frame is applicable when the attachment is over a Private network. To use Jumbo Frames for the connection, set **Jumbo Frame** toggle to **On**. | | ActiveMesh | ActiveMesh enables full mesh peering between the local and remote Edge gateways. For full mesh peering, set **ActiveMesh** toggle **On**. | | Number of HPE Tunnels | The number of High Performance Encryption (HPE) tunnels to create for the attachment peering. **Single** creates a single tunnel. **Maximum** creates the maximum tunnels based on the gateway sizes and the number of interface IPs on the peering gateway. This option is available only for connection over a Private network. **Custom** allows you to specify the number of tunnels to create. | 4. To attach the Edge Transit Gateway to another Edge Spoke Gateway, click **+ Attachment** again and provide the required information. You can attach an Edge Transit Gateway to multiple Spoke Gateways. Each attachment can be configured with different parameters, such as connecting WAN interfaces, connection over private or public networks, and enabling high-performance encryption. 5. Click **Save**. # Deploy an Edge Transit Gateway on Equinix Network Edge Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/edge/edge-transit-workflow-equinix This document provides instructions to deploy a primary Edge Transit Gateways on Equinix Network Edge. This document provides instructions to deploy a primary Edge Transit Gateways on Equinix Network Edge. For an overview of Aviatrix Edge, see [About Aviatrix Hybrid Cloud Edge](/docs/enterprise/10.1/concepts-architectures/components/edge-gateways/edge-overview). ## Topology The following diagram shows an example of network connectivity for Edge Transit Gateway to Transit Gateways in the cloud and Edge Spoke Gateway at remote site. ![Network connectivity topology for Edge Transit Gateway to Transit Gateways in the cloud and Edge Spoke Gateway at remote site](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/edge/eat-equinix-megaport-topology.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=3f95f255590679c73d4d52de762cfbe7) ## Prerequisites Before you deploy an Edge Transit Gateway on the Equinix platform, ensure the [prerequisite steps](/docs/enterprise/10.1/reference/gateways/edge/edge-spoke-transit-planning#equinix) are complete. ## Deploy the Primary Edge Transit Gateway (Equinix) To deploy an Edge Transit Gateway, first you need to generate the cloud-init file in CoPilot, then deploy the Edge gateway in Equinix using the cloud-init file: 1. [Create the Primary Edge Transit Gateway cloud-init ZTP file](https://legacy.docs.aviatrix.com/documentation/latest/network/edge-transit-create-equinix.html). 2. [Deploy the Edge Transit Gateway in Equinix using the cloud-init ZTP file](https://legacy.docs.aviatrix.com/documentation/latest/network/edge-vm-create-equinix.html). 3. [Verify the Edge Transit Gateway creation](https://legacy.docs.aviatrix.com/documentation/latest/network/edge-gateway-verify.html). ## Next Steps * For high availability (HA) and redundancy, you can create and deploy HA Edge Transit Gateway. See [Deploy a HA Edge Transit Gateway (Equinix)](https://legacy.docs.aviatrix.com/documentation/latest/network/edge-transit-ha-create-equinix.html). * For hybrid, multicloud, or LAN-side connectivity, you can attach the Edge Transit Gateway to multiple Aviatrix gateways. See [Edge Gateway Attachments](/docs/enterprise/10.1/guides/connectivity/gateways/edge-gateway-attachments). * For remote site connectivity, you can connect the Edge Transit Gateway to remote devices. See [Edge Gateway External Connection](/docs/enterprise/10.1/guides/connectivity/gateways/edge-s2c). # Deploy an Edge Transit Gateway on Megaport Virtual Edge Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/edge/edge-transit-workflow-megaport This document provides instructions to deploy a primary Edge Transit Gateway on Megaport Virtual Edge (MVE). This document provides instructions to deploy a primary Edge Transit Gateway on Megaport Virtual Edge (MVE). For an overview of Aviatrix Edge, see [About Aviatrix Hybrid Cloud Edge](/docs/enterprise/10.1/concepts-architectures/components/edge-gateways/edge-overview). ## Topology ![eat-equinix-megaport-topology.png](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/edge/eat-equinix-megaport-topology.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=3f95f255590679c73d4d52de762cfbe7) ## Prerequisites Before you deploy an Edge Transit Gateway on the Megaport platform, ensure the [prerequisite steps](/docs/enterprise/10.1/reference/gateways/edge/edge-spoke-transit-planning#megaport) are complete. ## Deploy the Primary Edge Transit Gateway (Megaport) To create and deploy the primary Edge Transit Gateway in Megaport Virtual Edge (MVE), first you need to generate the cloud-init file in CoPilot, then deploy the Edge gateway in MVE using the cloud-init file: 1. [Create the Primary Edge Transit Gateway cloud-init ZTP file](https://legacy.docs.aviatrix.com/documentation/latest/network/edge-transit-create-megaport.html). 2. [Deploy the Edge Transit Gateway in MVE using the cloud-init ZTP file](https://legacy.docs.aviatrix.com/documentation/latest/network/edge-vm-create-megaport.html). 3. [Verify the Edge Transit Gateway creation](https://legacy.docs.aviatrix.com/documentation/latest/network/edge-gateway-verify.html). ## Next Steps * For high availability (HA) and redundancy, you can create and deploy HA Edge Transit Gateway. See [Deploy a HA Edge Transit Gateway (Megaport)](https://legacy.docs.aviatrix.com/documentation/latest/network/edge-transit-ha-create-megaport.html). * For hybrid, multicloud, or LAN-side connectivity, you can attach the Edge Transit Gateway to multiple Aviatrix gateways. See [Edge Gateway Attachments](/docs/enterprise/10.1/guides/connectivity/gateways/edge-gateway-attachments). * For remote site connectivity, you can connect the Edge Transit Gateway to remote devices. See [Edge Gateway External Connection](/docs/enterprise/10.1/guides/connectivity/gateways/edge-s2c). # Deploy an Edge Transit Gateway on the Aviatrix Edge Platform Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/edge/edge-transit-workflow-onpremise This document provides instructions to deploy a primary and secondary highly available (HA) Edge Transit Gateways on the Aviatrix Edge platform. This document provides instructions to deploy a primary and secondary highly available (HA) Edge Transit Gateways on the Aviatrix Edge platform. For an overview of Aviatrix Edge, see [About Aviatrix Hybrid Cloud Edge](/docs/enterprise/10.1/concepts-architectures/components/edge-gateways/edge-overview) . ## Topology The following diagram shows an example of network connectivity for Edge Transit Gateway to Transit Gateways in the cloud and Edge Spoke Gateways at remote sites. ![Network connectivity topology for Edge Transit Gateway](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/edge/eat-onpremise-topology.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=553e76ce1ec2b44d03385795be6cc99a) ## Prerequisites Before you deploy an Edge Transit Gateway on the Aviatrix Edge Platform, ensure the [prerequisite steps](/docs/enterprise/10.1/reference/gateways/edge/edge-spoke-transit-planning#aep) are complete. ## Create the Primary and Secondary Edge Transit Gateway (Aviatrix Edge Platform) Follow the steps below to create and deploy a primary and secondary highly available (HA) Edge Transit Gateways on the Aviatrix Edge platform. ### Step 1: Gateway Configuration In Aviatrix CoPilot, go to **Cloud Fabric > Hybrid Cloud > Edge Gateways > Transit Gateways** tab. Click **+ Transit Gateway**, then provide the following information. | Field | Description | | - | - | | **Name** | A name for the Edge Gateway. The name must begin with a letter, may include only letters, numbers, and dashes (excluding special characters or spaces), and must not exceed 50 characters in length. | | **Platform** | The platform account where you want to deploy the Edge gateway. You can create and edit platform accounts in CoPilot by going to **Cloud Fabric** > **Hybrid Cloud** > **Platforms** tab. | | **Site** | Identifies the edge location. You can select an existing name or enter a new name for the edge location. | | **High Availability** | The high availability mode. **Off** creates only the primary Edge gateway with one active peering. **On (Active Active Mode)** enables Edge gateway connections with all active peerings to perform load sharing and forward network traffic. | | **Primary Device** | The edge device where you want to deploy the primary Edge Gateway. | | **Secondary Device** | The edge device where you want to deploy the secondary (HA) Edge Gateway. The primary and secondary devices must have the same hardware configuration. | | **Gateway Resource Size** | The gateway size. **Small** - 2 vCPU - 4GB. **Medium** - 4 vCPU - 8GB. **Large** - 8 vCPU - 16GB. **X-Large** - 16 vCPU - 32GB. | Click **Next** to [configure the gateway interfaces](#step-2-interface-configuration). ### Step 2: Interface Configuration In the Interface Configuration section, configure the WAN and Management interfaces. By default, an Edge Transit Gateway has two interfaces: one WAN interface on eth0 and one Management interface on eth2. You will need [these configuration information](/docs/enterprise/10.1/reference/gateways/edge/edge-spoke-transit-planning#aep) to configure the interfaces. If High Availability mode is selected, then configure both the primary and secondary Edge gateway interfaces. #### Configure the WAN Interface Edge Transit Gateway on the Aviatrix Edge Platform supports up to 4 WAN interfaces. In **Interface Configuration**, click **WAN**, then provide the following information. | Field | Description | | - | - | | **Interface** | This is set to the Edge Gateway's logical interface. Adding multiple WAN interfaces is applicable when the Edge Gateway is set up for BGP underlay to cloud service provider (CSP) or other Edge Transit Gateways. A maximum of four WAN interfaces per Edge Gateway is supported. Add an interface per CSP underlay (such as Direct Connect or Express Route). When Edge Gateway is not terminating CSP underlay, use one interface per Edge Gateway to connect to upstream router. | | **IP Assignment** | The default is **Static** for static IP assignment. **DHCP** for dynamic IP address assignment is not supported. | | **Interface Labels** | A name to identify the WAN interface. | | **BGP** | Enables BGP underlay connection to cloud service provider (CSP) or other Edge Transit Gateways on the WAN interface. Set BGP toggle **On** to set up BGP connection to cloud routers such as VGW, VNG, and Google cloud router. Edge Gateway WAN support for BGP underlay to CSP is supported for AWS, Azure, and GCP. | | **Interface Primary CIDR** | The CIDR for the WAN interface. **DHCP** for dynamic IP address assignment is not supported. Interface CIDR must be in the format interface\_ip/netmask (for example, 192.18.20.1/24). Interface CIDR cannot be link-local CIDR. If you need to create a BGP underlay connection to cloud service provider (CSP) with a link-local IP address, you must enter the link-local IP address in the **Link-local Underlay CIDR** setting of the WAN interface. | | **Interface Secondary CIDRs** | The secondary CIDRs for the WAN interface. Interface CIDR must be in the format interface\_ip/netmask (for example, 192.18.20.1/24). The secondary CIDRs are used for High Performance Encryption (HPE) attachment peering connections over a private network between the Edge Transit Gateway to another Edge Transit Gateway or Edge Spoke Gateway. The secondary IP addresses (based on the secondary CIDRs) are automatically assigned to create the peering connections. You can define /32 CIDR for specific secondary IP address. Secondary CIDR cannot be link-local CIDR. | | **Default Gateway IP** | The Default Gateway IP address for the WAN interface. For CSP underlay, this is the remote side IP address of the BGP session on CSP VNG or VGW. If **Link-Local Underlay CIDR** is configured, the Default Gateway IP should be in the same subnet as the Link-Local Underlay CIDR, otherwise, it should be in the same subnet as the WAN Interface CIDR. | | **Public IP** | The public IP for the WAN interface. The public IP of the WAN interface is used for peering connections over the public network. | If BGP is turned On, provide the following information. | Field | Description | | - | - | | **Link-Local Underlay CIDR (GCP only)** | The Link-Local Underlay CIDR is used for BGP underlay connections to cloud service provider (CSP). If you need to create a BGP underlay connection to CSP with a link-local IP address, you must provide the Link-Local Underlay CIDR for the WAN interface in the format of link\_local\_underlay\_ip/netmask (for example 169.254.100.3/24). This is required for GCP. If terminating GCP Interconnect and using BGP underlay on Edge, provide the WAN Default Gateway of the peer IP address. If Link-Local Underlay CIDR is configured, the **Default Gateway IP** should be in the same subnet as the Link-Local Underlay CIDR, otherwise, it should be in the same subnet as the WAN Interface CIDR. | | **Local ASN** | The **[Local ASN Number](https://legacy.docs.aviatrix.com/documentation/latest/network/bgp-connection-settings.html#local-as-number)** of the Edge Gateway. | | **Remote ASN** | The AS Number of the CSP side peering connection such as private VIF on VGW (AWS) and VNG ASN (Azure). | | **Local LAN IP** | The IP address of the Edge Gateway. This is the local peering PTP IP for BGP. | | **Remote LAN IP** | The IP address of the CSP VNG or VGW peering PTP IP. (GCP is not supported). | | **Password (optional)** | The MD5 authentication key. | If BGP is turned On, in **Gateway Configuration**, enter the **Local AS Number** of the local Edge Gateway. To add another WAN interface, click **+ WAN Interface** again and provide the required information. If a required field is missing, the interface tab is highlighted to indicate there is an error. #### Configure the Management Interface In **Interface Configuration**, click **MGMT**, then provide the following information. | Field | Description | | - | - | | **Interface** | The Edge Gateway's logical interface name. | | **IP Assignment** | The MGMT interface defaults to **DHCP**. This setting cannot be changed. The Edge Gateway will automatically NAT out of the physical MGMT interface of the edge node when using the Aviatrix Edge platform. | | **Private Network** | Leave this setting to **Off**. The Edge Gateway on the edge hardware requires public Internet reachability to connect to the Aviatrix Controller and Aviatrix Edge infrastructure in the cloud. | | **Egress CIDR (Primary)** | The Egress CIDR is the public IP address which the Management interface uses. If the Public IP is used from Edge Gateway Management interface to establish connectivity to Aviatrix Controller, then configure the Public IP as the CIDR. The CIDR is then added to the Controller security group to allow incoming traffic from the Edge Gateway. | | **Egress CIDR (Secondary)** | The Egress Public IP for the secondary Edge Gateway's Management interface when High Availability is configured. | If a required field is missing, the interface tab is highlighted to indicate there is an error. Click **Next** to [view the interface mapping](#step-3-interface-mapping). ### Step 3: Interface Mapping Interface Mapping is provided for reference purposes (view only). The table shows a mapping of the Edge Gateway's logical interface names to the Edge Gateway's Linux interface names. These interface names are used in the Aviatrx log files and error messages. When performing diagnostic or troubleshooting on the Aviatrix Platform, you can refer to this mapping for the interface name. | Field | Description | | - | - | | **Interface** | The Edge Gateway's logical interface name. | | **Primary Gateway Ethernet Interface** | The primary Edge Gateway's Linux interface name. | | **Secondary Gateway Ethernet Interface** | The secondary (HA) Edge Gateway's Linux interface name. The secondary gateway interface mappings are shown only when high availability is configured for the primary Edge Gateway. | #### Backup Peering If the Edge Gateway is deployed in Active-Active high availability mode: In **Backup Peering**, select a **WAN interface** to use to establish the attachment peering between the primary and HA Edge Gateway. Select whether the attachment is over a **Private Network** or **Public Network**. Click **Done** to create the primary and HA gateways. ### Step 4: Verify the Edge Gateway Creation See, [Verify the Edge Gateway Creation](https://legacy.docs.aviatrix.com/documentation/latest/network/edge-create-verify.html). ## Next Steps * For hybrid, multicloud, or LAN-side connectivity, you can attach the Edge gateway to multiple Aviatrix gateways. See [Edge Gateway Attachments](/docs/enterprise/10.1/guides/connectivity/gateways/edge-gateway-attachments) . * For remote site connectivity, you can connect the Edge gateway to remote devices. See [Edge Gateway External Connection](https://legacy.docs.aviatrix.com/documentation/latest/network/edge-connect-external-device-eat.html). # Troubleshooting Edge Gateway Connectivity Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/edge/edge-troubleshooting This document provides information about using the Aviatrix Edge Gateway's command-line interface to troubleshoot common connectivity issues. This document provides information about using the Aviatrix Edge Gateway's command-line interface to troubleshoot common connectivity issues. ## Using the Edge Gateway's Command Line Interface You can connect to the Aviatrix Edge Gateway's command-line interface (CLI) or console and use the Clish commands below to change the password, show gateway diagnostics, and troubleshoot Edge Gateway connectivity. > [!NOTE] > > Once the Edge Gateway registers with the Aviatrix Controller, the password for the CLI login changes to the Management IP address of the Edge Gateway. | **Command** | **Description** | | - | - | | change\_console\_password | Changes the password for the CLI login. | | check\_conduit | Check conduit state. | | check\_network \[dns]\[reachability] | Troubleshoot network connectivity. | | diagnostics | Show gateway diagnostics. | | logout | Log out of the console. | | ping \[-c count] \[dest] | Ping destination, optional parameter ping packet count. The default is 5. | | reboot | Reboot the system. | | set\_controller\_ip \[controller\_ip] | Set the Controller IP address, usually performed after Controller migration when the Controller IP address is changed. | | show\_interfaces | Show output from the command "ifconfig -a more". | | show\_routes | Show output from the command "ip route show table all". | ## Diagnosing and Resolving Edge Gateway Connectivity Issues After you deploy an Aviatrix Edge Gateway, the Edge Gateway registers with the Aviatrix Controller. The registration process takes about five minutes. If the registration is successful, the Edge Gateway status changes from *Waiting* to *Up*. If the registration fails, it is because the Edge Gateway is unable to communicate with the Aviatrix Controller. You could use the **diagnostics** Clish command to diagnose the network connectivity. ## Updating the Controller IP for Edge Gateways when Controller IP Changes If the Aviatrix Controller IP has changed, you must manually update the Controller IP configuration on the Aviatrix Edge Gateways that are connected to the Controller. Otherwise, the Edge Gateways may not be able to communicate with the Controller. To update the Controller IP configuration on the Edge Gateway: 1. Connect to the Edge Gateway's command-line interface (CLI). 2. Run the following Clish command: **set\_controller\_ip** ``, where `new_controller_ip_address` is the IP address of the new Controller. ## Verifying the Management CIDR Configuration One common reason the Aviatrix Edge Gateway is unable to communicate with the Aviatrix Controller is the MGMT CIDR is incorrect. You could use the Edge Gateway's command-line interface and run the **check\_network** Clish command to view if the Edge Gateway can reach the Controller. Ensure that the correct MGMT CIDR is configured for the gateway. 1. In Aviatrix CoPilot, go to **Cloud Fabric > Hybrid Cloud > Edge Gateways** tab. 2. Click **Spoke Gateways** or **Transit Gateways**. 3. In the table, locate the Edge Gateway and click its Edit icon on the right side of the row. 4. In Edit Edge Gateway, click **Next** to go to **Interface Configuration**. 5. Click **MGMT**. 6. Update the **Egress CIDR** with the correct public IP. 7. Click **Save**. ## Verifying Edge Gateway Access in the Controller Security Group In the Aviatrix Controller's Security Group, ensure the security group rule allows access to the Controller's HTTPS port from the Management Egress CIDR of the Edge Gateway. # Edge Gateway Attachments Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/gateways/edge-gateway-attachments The Aviatrix Edge gateways provide a secure high-performance networking solution designed to simplify and accelerate hybrid and multicloud connectivity. The Aviatrix Edge gateways provide a secure high-performance networking solution designed to simplify and accelerate hybrid and multicloud connectivity. Aviatrix Edge Transit Gateway enables you to create Edge Transit to Edge Transit Gateway peering for connectivity across network service providers, such as Equinix and Megaport, Edge Transit to Transit Gateway peering for connectivity across cloud service providers, and Edge Transit to Spoke Gateway peering for connectivity to on-premises environments. The Aviatrix Edge Spoke Gateway enables you to create Edge Spoke to Edge Transit peering for hybrid connectivity and Edge Spoke to Transit Gateway peering for cloud connectivity. ## Related Topics * [Create Edge Transit Gateway Attachments](/docs/enterprise/10.1/guides/connectivity/edge/edge-transit-attachments) * [Create Edge Spoke Gateway Attachments](/docs/enterprise/10.1/guides/connectivity/edge/edge-spoke-attachments) * [Setting up Accounts for Edge Platforms](/docs/enterprise/10.1/guides/connectivity/edge/edge-platform-account) # Edge Gateway External Connection Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/gateways/edge-s2c Edge Gateway External Connection (S2C) enables you to build secure connectivity between Edge gateways in hybrid and multicloud environments to remote sites such as on-premises, branch offices, and partner locations, by establishing connections to remote routers and firewalls that terminate VPN... Edge Gateway External Connection (S2C) enables you to build secure connectivity between Edge gateways in hybrid and multicloud environments to remote sites such as on-premises, branch offices, and partner locations, by establishing connections to remote routers and firewalls that terminate VPN connection. ## Related Topics * [About Edge Transit Gateway External Connection](/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/edge-transit-s2c-overview) * [Edge Transit Gateway External Connection Workflow](/docs/enterprise/10.1/guides/connectivity/edge/edge-s2c-workflow) * [Edge Gateway Multi-Peer BGP over LAN Connection](/docs/enterprise/10.1/guides/connectivity/edge/edge-multipeer-bgpolan) # External Connection Settings Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/gateways/external-connection-settings This document describes the options that you can configure for an Aviatrix gateway Site2Cloud (S2C) external connection to remote devices, such as on-premises router or firewall. This document describes the options that you can configure for an Aviatrix gateway Site2Cloud (S2C) external connection to remote devices, such as on-premises router or firewall. You can create external connections in Aviatrix CoPilot by going to **CoPilot** > **Networking** > **Connectivity** > **External Connections (S2C)** tab, then from the **+ External Connection** dropdown menu, select **External Device**. > [!NOTE] > > Starting with Controller version 9.0, route-based IPSec Site-to-Cloud > connections can negotiate IKE phase 2 Proxy-ID traffic selectors when Proxy-ID > is enabled on the connection. See > [Proxy-ID for Routed IPSec Connections](/docs/enterprise/10.1/guides/connectivity/vpn/proxy-id-routed-ipsec-connections) > . In this document, *Local Gateway* refers to the Aviatrix gateway that you want to connect to a remote device. ## General Configuration ### Type Type refers to the VPN tunneling and routing protocol. You can select from: * **BGP over IPsec**: Build a VPN tunnel using the IPsec tunneling protcol and Border Gateway Protocol (BGP) routing. * **BGP over GRE**: Build a VPN tunnel using the GRE tunneling protocol and BGP routing. * **BGP over LAN**: Build a VPN tunnel over the LAN interface without using a tunneling protocol (such as IPsec or GRE) and BGP routing. * **Static Routing over IPsec**: Build a VPN tunnel using the IPsec tunneling protcol and Static routing. ### Static Routing Type * **Unmapped NAT**: Connect to a VPN supported device from a non-BGP enabled Spoke Gateway or Specialty Gateway. * **Mapped NAT**: Connect overlapping networks between the cloud and on-premises from a non-BGP enabled Spoke Gateway or Specialty Gateway. * **ActiveMesh**: Connect a VPN supported device from a Transit Gateway or BGP-enabled Spoke Gateway in AWS or Azure. ### Local Gateway The Local Gateway that you want to connect to a remote device. ### Local Subnet CIDR(s) (Static Route connections only) The Local Gateway CIDR(s) to route traffic to the cloud destination. ### Real Local Subnet CIDR(s) A list of the source network CIDRs, separated by comma, that will be encrypted (for example, 172.16.1.0/24, 172.16.2.0/24). These Local Subnets are advertised to Remote Subnets that the Site2Cloud external connection can reach. ### Virtual Local Subnet CIDR(s) A list of virtual local network CIDRs that are mapped to the real local subnet. For example, for the real local subnets 172.16.1.0/24 and 172.16.2.0/24, the virtual local subnets could be 192.168.7.0/24 and 192.168.8.0/24. ### Remote Device Type The remote device type. * **Generic** device type type applies to third-party routers and firewalls. * **Aviatrix** device type applies when terminating on an Aviatrix gateway. Any other remote devices listed are only valid with Aviatrix Controller version 6.7 or lower. If using a higher Controller version, only select **Generic** or **Aviatrix**. ### Remote Subnet CIDR(s) (Static Route connections only) The remote network CIDR(s) to route traffic to the remote network destination. ### Real Remote Subnet CIDR(s) A list of the destination network CIDRs, separated by comma, that will be encrypted (for example, 10.10.1.0/24, 10.10.2.0/24). ### Virtual Remote Subnet CIDR(s) A list of virtual remote network CIDRs that are mapped to the real remote subnet. For example, for the real remote subnets 10.10.1.0/24 and 10.10.2.0/24 , the virtual remote subnets could be 192.168.1.0/24 and 192.168.2.0/24. ## IPsec Configuration Aviatrix IPsec tunnels comply NIST standards, and support AES-256-GCM and Perfect Forward Secrecy (PFS) using DH21. The supported configurations are as follows: * Spoke–Transit * Transit–Transit * Transit FQDN gateway * Transit Edge-as-Spoke Both Greenfield (new deployments) and Brownfield (existing deployments) environments are supported. ### Attach Over The underlying infrastructure of your network. * **Private Network**: Your underlying infrastructure is a private network, such as AWS Direct Connect and Azure ExpressRoute. When this option is selected, BGP over IPsec runs over private IP addresses. * **Public Network**: Your underlying infrastructure is a public network or the internet. When this option is selected, BGP over IPsec runs over public IP addresses. ### Jumbo Frame Jumbo Frame improves the performance throughput between the Local Gateway and the remote device. You must first enable Jumbo Frame on the Local Gateway before creating an external connection. Enabling Jumbo Frame on the Local Gateway enables it on the WAN and LAN interfaces of the gateway. > [!NOTE] > > * Jumbo Frame is only supported on private connections that support Jumbo > Frame. \* For Transit and Spoke Gateways: \* Jumbo Frame is supported for AWS > and OCI only; it is not supported for Azure and GCP. \* Jumbo Frame is > supported with High Performance Encryption and BGP over IPsec and BGP over LAN > connections only. High Performance Encryption must be enabled on all gateways > in the end-to-end path of the traffic flow. \* For Edge Transit and Spoke > Gateways: \* (Edge Transit Gateway) Jumbo Frames can be enabled on the Edge > Transit Gateway when creating BGP over IPsec or BGP over GRE external > connection to remote devices. \* (Edge Spoke Gateway) Jumbo Frame is enabled > (by default) for BGP over LAN connections, when the Edge Spoke Gateway is > created. ### Algorithms Aviatrix supports the following default encryption algorithms to authenticate the communication between the Aviatrix gateway and the remote device. This feature is not applicable for BGP over GRE, BGP over LAN connections. * **Phase 1 Authentication**: Default is SHA-256 * **Phase 1 DH Groups**: Default is 14 * **Phase 1 Encryption**: Default is AES-256-CBC * **Phase 2 Authentication**: Default is HMAC-SHA-256 * **Phase 2 DH Groups**: Default is 14 * **Phase 2 Encryption**: Default is AES-256-CBC You can also choose to customize or modify the default values. ### Internet Key Exchange The Internet Key Exchange (IKE) protocol is used for authentication and encryption of packets between the Aviatrix gateway and the remote device. Aviatrix gateways support IKE version 1(IKEv1) and IKE version 2(IKEv2) to establish secure connections to remote sites. > [!WARNING] > > For **BGP over IPsec** connections where **Support for IPv6** is enabled in > the BGP configuration, set Internet Key Exchange to **IKEv2**. IKEv1 is not > supported for IPv6 BGP over IPsec. ## Authentication Configuration ### Authentication Method (Static Routing over IPsec connections only) You can authenticate the connection using [Pre-Shared Key (PSK)](#pre-shared-key) or [certificate-based](#certificate-based) authentication. ### Pre-Shared Key (Optional) Pre-Shared Key(PSK) is a string of random characters that is used to authenticate the BGP tunnel connection between the Aviatrix gateway and the remote device. You must ensure that this key is the PSK that is configured for your remote device. If you do not specify one, it is auto-generated. You can view the auto-generated PSK value when you download the external connection's configuration (see [Download an External Connection Configuration](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-download-configuration.html)). For [ActiveMesh connections](#activemesh), if you want the tunnels that are created to have different PSK values, you will need to enter the PSKs separated by a comma. > [!NOTE] > > * A comma denotes two separate PSK values. Do not include a comma in a single > PSK string. \* The double quote ASCII character (") is not allowed. A tunnel > connection cannot be established if the PSK string contains the double quote > character. ### Certificate-Based You can use certificate-based authentication only if a [remote CA certificate has been uploaded or imported](/docs/enterprise/10.1/guides/connectivity/vpn/s2c-cacert). When creating your external connection, you enter the Subject Alternative Name(SAN) of the certificate for Remote Identifier SAN. If the remote endpoint is highly available(HA) devices, add a second connection row and enter the SAN of the remote HA gateway in the Remote Identifier SAN field. The SAN (Subject Alternative Name) is a way to indicate that multiple domains can be secured by the same certificate. ### Remote Identifier SAN (Static Routing over IPsec only) Aviatrix only supports IP\_ADDRESS and KEY\_ID as the IKE identity type for the remote identifier in the pre-shared key authentication. The IP\_ADDRESS must be a valid IPv4 IP address. The KEY\_ID is a remote device ID during the key authentication. By default, Aviatrix configures the public IPv4 address of the peer device as the Remote Identifier for pre-shared key authentication. You can adjust this setting to the private IPv4 address of the peer device. A Remote Identifier is used to identify the remote end of the tunnel and establish the IPsec connection. You can enter a DNS, IP address, or email address. ## BGP Configuration ### Local ASN The BGP AS number the Local Gateway will use to exchange routes with the remote device. ### ActiveMesh ActiveMesh enables full mesh peering for failover redundancy. When ActiveMesh is disabled, point-to-point tunnels are created instead of full mesh. When you have a pair of primary and highly available (HA) Local Gateways and HA remote devices: * With ActiveMesh: four tunnels are created (primary and HA Local Gateways to the primary and HA remote devices). * Without ActiveMesh: two tunnels are created (primary local to primary remote, HA local to HA remote). When you have one Local Gateway with HA remote devices or HA Local Gateways with one remote device, * With ActiveMesh: two tunnels are created (primary to primary, primary to secondary) * Without ActiveMesh: one tunnel is created (primary to primary) By default, ActiveMesh is On for Transit and Spoke Gateway BGP over IPsec and BGP over GRE connections. You can turn On ActiveMesh for these connections: * Transit and Spoke Gateway BGP over LAN connections in Azure and GCP * Edge Transit and Spoke Gateway BGP over LAN connections * Edge Transit Gateway BGP over IPsec and BGP over GRE connections over private network. > [!NOTE] > > For Edge Transit Gateways in Megaport, ActiveMesh is disabled for external > connections over private networks. Megaport only supports point-to-point > connections. ### BFD [Bidirectional Forwarding Detection (BFD)](/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/bgp-bfd) is a network protocol that enables detection of a link or node failure between the Local Gateway and its BGP peer. ### Manual CIDR Approval An approval process for gateway-learned CIDRs. This approval process improves security for your network. When Manual CIDR Approval is On, if an unapproved CIDR address attempts to access the connection, an email notification is sent to the administrator to approve or block access before the CIDRs are propagated to the Spoke VPC/VNet route tables. Manual CIDR Approval is Off and disabled by default unless the Local Gateway you select has [Learned CIDR Approval](https://legacy.docs.aviatrix.com/documentation/latest/network/bgp-connection-settings.html?expand=true#gateway-learned-cidr-approval) turned On; the Connection option selected, and the BGP connection selected. Then it is On by default (not editable). ### Advertise BGP Communities The [BGP communities](/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/bgp-communities) to advertise to the BGP peer. > [!NOTE] > > Before you can configure additional BGP communities at the connection-level, > the gateway advertise to BGP communities to peers setting must be enabled > either from the global BGP communities setting or at the gateway-level. * **Same as Gateway**: This is the default setting for the connection. The gateway advertises the BGP communities that were received or tagged on the route. * **Block Advertisement**: The gateway will not to advertise BGP Communities received from any BGP peer or attachment. * **Add Additional Communities**: The gateway will advertise the additional BGP communities specified to the external BGP peer in addition to the BGP communities that were received or tagged on the route. You can choose to enter a numeric BGP community tag or select from the dropdown menu. Aviatrix supports: * **Numeric** You can specify numeric BGP community tags specified as two 16-bit numbers (0-65535):(0-65535). * **No-Advertise** When a No-Advertise BGP community is attached to a route from an external BGP peer, the Aviatrix gateway will not advertise the route to any gateway attachments or external BGP peers. > [!NOTE] > > Aviatrix gateways only support eBGP to external BGP peers. * **No-Export** When a No-Export BGP community is attached to a route from an external BGP peer, the Aviatrix gateway will not advertise the route to any external BGP peers and advertise only to gateway attachments. * **Specify Communities**: The gateway will advertise to the external BGP peer only these BGP communities specified and ignore the BGP communities that are already tagged on the route. ### BGP Multihop External BGP connectivity requires that the remote peer is directly connected (single-hop) to the Local Gateway. When the remote peer is not directly connected (separated by multiple hops), you must enable BGP Multihop for the Local Gateway to establish a BGP session with a remote peer. ### Support for IPv6 on BGP Connections Select an IPv6-enabled Local Gateway to see this option. For more information, refer to the [IPv6 overview](/docs/enterprise/10.1/concepts-architectures/components/connectivity/ipv6-overview). Enable the **Support for IPv6** option to see IPv6-related options in the Tunnel Configuration section when creating **BGP over IPsec**, **BGP over GRE**, or **BGP over LAN** connections. ## Tunnel Configuration ### BGP over IPsec and BGP over GRE Connections #### Remote Device Tunnel Destination IP The remote device interface IP address. #### Remote ASN The BGP AS Number that the remote device will use to exchange routes with the Local Gateway. The Remote ASN should be the same for the primary and HA Local Gateways. #### Advertised IPv6 via IPv4 Peer To advertise both IPv4 and IPv6 routes to the remote device over the IPv4 BGP session, enable this option. To advertise only IPv4 routes to the remote device over the IPv4 BGP session, disable this option. For more information, refer to [Support for IPv6 on BGP Connections](#support-for-ipv6-on-bgp-connections). #### BGP Local IP (Optional) The local tunnel inner CIDR range allowed to communicate over the VPN tunnel. #### BGP Neighbor IP The remote tunnel inner IPv6 CIDR range allowed to communicate over the VPN tunnel. #### BGP Neighbor IPv6 The remote tunnel inner CIDR range allowed to communicate over the VPN tunnel. Disable the **Advertised IPv6 via IPv4 Peer** field to see the **BGP Neighbor IPv6** field. #### Tunnel Source IP (Edge Transit Gateway only). The Tunnel Source IP is the Edge Transit Gateway WAN interface IP address to use for the connection to the remote device. For Transit Gateway, this setting defaults to the eth0 IP address. #### Pre-Shared Key See [Pre-Shared Key (PSK)](#pre-shared-key). ### BGP over LAN #### Remote Device IP The remote device interface IP address. #### Remote ASN The BGP AS Number of the remote device that will be used to exchange routes with the Local Gateway. The Remote ASN should be the same for the primary and HA Local Gateways. #### Local LAN IP The Local Gateway interface IP address. #### Remote LAN IP The remote device interface IP address. #### Local LAN IPv6 (Optional) The Local Gateway interface IPv6 address. #### Remote LAN IPv6 The remote device interface IPv6 address. ### Static Routing over IPsec #### Remote Device Tunnel Destination IP (Static Routing ActiveMesh only) The remote device's interface IP address. #### Remote Device IP The remote device's interface IP address. #### Local Gateway Instance The Local Gateway's IP address. #### Local Tunnel IP (Static Routing ActiveMesh only) The local tunnel inner CIDR range allowed to communicate over the tunnel. #### Remote Tunnel IP (Static Routing ActiveMesh only) The remote tunnel inner CIDR range allowed to communicate over the tunnel. #### Pre-Shared Key See [Pre-Shared Key (PSK)](#pre-shared-key). ## Related Topics * [Transit Gateway to External Devices Workflow](/docs/enterprise/10.1/guides/connectivity/vpn/transit-externaldevice-workflow) * [Site2Cloud External Connection Workflow](/docs/enterprise/10.1/guides/connectivity/vpn/s2c-configuration-workflow) * [Overview of Transit Gateway to External Devices](/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/transit-externaldevice-usecase) * [About External BGP Connection Notification Settings](/docs/enterprise/10.1/reference/connectivity/external-connections-bgp-settings) # Transit Gateway CSP Related Settings Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/gateways/transit-gateway-csp-settings Configuration settings for Transit Gateway CSP-related features in the Aviatrix platform. This section describes the cloud service provider related settings for a Transit Gateway. ## Summarize CIDR(s) to AWS TGW Enable this setting to limit routes propagated to TGW to only three RFC 1918 CIDRs and specific non-RFC 1918 CIDRs. Limiting routes saves route propagation time. Leave this setting disabled (the default setting) to maintain better segmentation behavior without improving performance. ## AWS TGW Edge Segmentation AWS TGW Edge Segmentation allows you to further specify on each AWS TGW Aviatrix \_Edge\_Domain connection which domain the Transit Gateway can communicate with. > [!NOTE] > > This feature is not available for Transit Gateways that have FireNet added, or have Network Segmentation enabled. After you turn On this option you can build domain connection policies to specify which Network Domain this AWS TGW Aviatrix\_Edge\_Domain connection can communicate with. For more information, see [TGW Edge Segmentation](https://docs.aviatrix.com/documentation/latest/network/tgw-edge-segmentation.html). ## Configuring CSP Settings ## Configuring Transit Gateway CSP Related Settings 1. In Aviatrix CoPilot, go to *Cloud Fabric* > *Gateways* > *Transit Gateways* tab. 2. In the table, locate and select the Transit Gateway you want to edit. 3. Go to the Transit Gateway's *Settings* tab. 4. Expand the *CSP Settings* section. 5. (optional) Toggle *Summarize CIDR(s) to AWS TGW* to *On*. 6. Follow the instructions to [set up AWS TGW Edge Segmentation](https://docs.aviatrix.com/documentation/latest/network/tgw-edge-segmentation.html). # Securing and Scaling Kubernetes - AWS EKS Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/kubernetes/secure-eks-design Architect identity-based Kubernetes firewalling on Amazon EKS using Aviatrix spoke gateways, non-routable pod CIDRs, and DCF SmartGroups — enabling namespace-as-a-service and cluster-as-a-service platform models without DaemonSets, sidecars, or CNI replacement. ## Why Secure Kubernetes at the Network Fabric Traditional firewalls match on source and destination IP addresses. Kubernetes breaks this model. Pods get ephemeral IPs that change on every restart, scale event, or rolling deployment — making IP-based rules meaningless within minutes. Securing Kubernetes traffic requires a shift from address-based policy to identity-based policy. ### The Problem with IP-Based Security in Kubernetes A pod running at `100.64.3.47` right now may be gone in 30 seconds. A completely different workload may inherit that same IP moments later. Firewall rules like "allow `100.64.3.47` to `10.0.1.5` on port 443" are stale before the next deployment. * **Ephemeral IPs** — Pods receive random IPs from a CIDR pool. Every restart, scale-up, or rolling update shuffles the assignments. Static rules cannot keep pace. * **Overlapping CIDRs at scale** — When multiple clusters use the same secondary CIDR (e.g., `100.64.0.0/16`), the same IP address maps to entirely different workloads in different clusters. CIDR-based rules are ambiguous. * **IPs do not represent identity** — Kubernetes workloads are defined by namespace, labels, and service accounts. These are the primitives that express intent — not IP addresses. > [!NOTE] > > IP overlap is a real operational problem, but it is a symptom. The deeper > issue is that IP addresses do not represent workload identity in Kubernetes. > Solving overlap without solving identity leaves security gaps. ### External Firewalling: Don't Touch the Cluster Aviatrix enforces Kubernetes security at the network fabric — outside the cluster — using the spoke gateway as the policy enforcement point. This keeps the cluster clean: * **Use the cloud-native CNI** — The VPC CNI is battle-tested, AWS-supported, and deeply integrated with EC2 networking. Aviatrix works with it, not against it. No CNI replacement required. * **No DaemonSets** — In-cluster network policy engines (Calico, Cilium) require DaemonSets on every node — another component to patch, monitor, and troubleshoot inside the blast radius of the cluster itself. * **No sidecars** — Service mesh approaches (Istio, Linkerd) inject proxies into every pod, adding latency, memory overhead, and operational surface. For network-level segmentation, this is unnecessary. * **Separation of concerns** — The networking team manages firewall policy in the Aviatrix fabric. Application teams manage workloads in the cluster. Neither modifies the other's domain. A platform engineer can enforce "namespace X can only talk to namespace Y" without ever running `kubectl`. * **Consistent across clusters and clouds** — The same DCF policies work whether the cluster is EKS, AKS, or GKE. Policy follows the architecture, not the Kubernetes distribution. > [!TIP] > > For a full overview of DCF capabilities for Kubernetes — including SmartGroup > types and supported distributions — see > [Distributed Cloud Firewall for Kubernetes](/docs/enterprise/10.1/guides/security/dcf/dcf-kubernetes) > . ### Platform Engineering Models This architecture enables two common platform delivery patterns: * **Namespace as a service** — Multiple teams share a single EKS cluster, isolated by namespace. DCF uses Kubernetes-type SmartGroups to enforce per-namespace egress policy at the spoke gateway. Teams get self-service namespaces; the platform team controls what leaves the cluster. * **Cluster as a service** — Each tenant gets a dedicated EKS cluster in its own VPC with its own (potentially overlapping) pod CIDR. DCF uses VPC-type SmartGroups for cluster-level segmentation. Tenants get full cluster isolation; the platform team manages inter-cluster and egress policy centrally. * **Multi-cluster, multi-cloud** — Both models compose. An organization can run namespace-as-a-service in EKS and cluster-as-a-service in AKS, governed by a single set of DCF policies. ## Overview One of the practical challenges with EKS at scale is IP exhaustion: the VPC CNI plugin assigns one VPC IP address per pod, rapidly consuming routable RFC 1918 address space. A cluster running 500 pods consumes 500 IPs from your enterprise IP plan — and that is a single cluster. AWS documents this problem and a native solution in [Addressing IPv4 address exhaustion in Amazon EKS clusters using private NAT gateways](https://aws.amazon.com/blogs/containers/addressing-ipv4-address-exhaustion-in-amazon-eks-clusters-using-private-nat-gateways/). The AWS approach uses a private NAT gateway as the SNAT translation point between non-routable pod subnets and routable VPC subnets. When Aviatrix is the network fabric, the Aviatrix spoke gateway replaces the private NAT gateway — it becomes the SNAT translation point. This is simpler (no additional NAT gateway infrastructure), integrates with DCF for policy enforcement, and works consistently across multi-cloud environments. This architecture supports both the namespace-as-a-service and cluster-as-a-service [platform engineering models](#platform-engineering-models) described above. > [!NOTE] > > SmartGroups are enforced on cluster egress traffic. Kubernetes network > policies should be used outside of Aviatrix to restrict traffic between > namespaces within the cluster. This guide walks through end-to-end configuration: AWS VPC setup, EKS VPC CNI custom networking, Aviatrix custom SNAT policies, transit route exclusion, and DCF SmartGroup considerations. > [!TIP] > > A complete Terraform reference implementation is available in the [AWS EKS > Multi-Cluster > Blueprint](https://github.com/AviatrixSystems/aviatrix-blueprints/tree/main/blueprints/aws-eks-multicluster). ## Routable vs. Non-Routable Address Space The EKS VPC uses two CIDR blocks with distinct roles: | Layer | CIDR Source | Routable? | Purpose | | - | - | - | - | | Primary CIDR | Enterprise IP plan (e.g., `10.10.0.0/23`) | Yes — unique, advertised across the fabric | Cluster ingress, EKS control plane ENIs, node management, Aviatrix gateway | | Secondary CIDR | RFC 6598 space (e.g., `100.64.0.0/16`) | No — never advertised, can overlap across VPCs | Pod networking via VPC CNI custom networking | The primary CIDR is intentionally small — it only needs to cover infrastructure, not pods. A `/23` (512 IPs) is often sufficient. The heavy lifting moves to the secondary CIDR, which provides tens of thousands of pod IPs without consuming enterprise address space. ### What Lives on the Routable (Primary) CIDR * **Cluster ingress** — Load balancer ENIs (ALB, NLB, or any ingress controller) sit in routable subnets so they are reachable from other VPCs, on-prem, or the internet. * **EKS control plane ENIs** — The managed cross-account ENIs that EKS places in your VPC for API server connectivity. These must be in routable subnets so nodes and pods can reach the Kubernetes API. * **EKS nodes** — Node EC2 instances launch in routable private subnets. Their primary ENI gets a routable IP; secondary ENIs (for pods) are placed in non-routable subnets via ENIConfig. * **Aviatrix spoke gateway** — The gateway ENIs need routable IPs since they are the SNAT translation point. ### What Lives on the Non-Routable (Secondary) CIDR * **Pod ENIs** — Every pod gets an IP from the `100.64.0.0/16` secondary CIDR via VPC CNI custom networking. These IPs are never exposed directly outside the VPC — all egress is SNATed by the Aviatrix spoke gateway. ### Traffic Flow ![EKS pod traffic flow through Aviatrix spoke gateway with SNAT](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/kubernetes/eks-pod-snat-flow.svg?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=b1380a99e4ce9917bad2447f34cd465a) ## Step 1: AWS VPC Configuration ### 1.1 Add a Secondary CIDR Block Attach a secondary CIDR from RFC 6598 space (`100.64.0.0/10`) to the EKS VPC. A `/16` provides 65,536 pod IPs — sufficient for most clusters. ```hcl theme={null} resource "aws_vpc_ipv4_cidr_block_association" "secondary" { vpc_id = aws_vpc.this.id cidr_block = "100.64.0.0/16" } ``` This CIDR is non-routable by design. It can safely overlap with the same secondary CIDR in other VPCs because it is never advertised across the Aviatrix fabric. ### 1.2 Create Pod Subnets from the Secondary CIDR Create one pod subnet per Availability Zone from the secondary CIDR: * **AZ-a:** `100.64.0.0/17` (32,768 IPs) * **AZ-b:** `100.64.128.0/17` (32,768 IPs) These subnets must depend on the `aws_vpc_ipv4_cidr_block_association` resource. ### 1.3 Subnet Layout | Subnet | CIDR Source | Routable? | Purpose | | - | - | - | - | | Aviatrix gateway (x2 AZs) | Primary | Yes | Spoke gateway ENIs | | Load balancer / ingress (x2 AZs) | Primary | Yes | ALB, NLB, or any ingress controller ENIs | | Infrastructure / nodes (x2 AZs) | Primary | Yes | EKS nodes, control plane ENIs | | Pod networking (x2 AZs) | Secondary | No | Pod ENIs via VPC CNI custom networking | The load balancer subnets should be tagged appropriately for your ingress controller. For the AWS Load Balancer Controller: * Public-facing: `kubernetes.io/role/elb = 1` * Internal-facing: `kubernetes.io/role/internal-elb = 1` ## Step 2: EKS VPC CNI Configuration ### 2.1 Enable Custom Networking on the vpc-cni Addon Three environment variables control the behavior: | Variable | Value | Purpose | | - | - | - | | `AWS_VPC_K8S_CNI_CUSTOM_NETWORK_CFG` | `"true"` | Pods use ENIConfig-specified subnets instead of the node's subnet | | `ENI_CONFIG_LABEL_DEF` | `"topology.kubernetes.io/zone"` | CNI selects the ENIConfig matching the node's AZ label | | `AWS_VPC_K8S_CNI_EXTERNALSNAT` | `"true"` | Disables CNI-level SNAT — delegates NAT to Aviatrix | ```hcl theme={null} addons = { vpc-cni = { most_recent = true configuration_values = jsonencode({ env = { AWS_VPC_K8S_CNI_CUSTOM_NETWORK_CFG = "true" ENI_CONFIG_LABEL_DEF = "topology.kubernetes.io/zone" AWS_VPC_K8S_CNI_EXTERNALSNAT = "true" } }) } } ``` > [!WARNING] > > `AWS_VPC_K8S_CNI_EXTERNALSNAT` is critical. By default, the VPC CNI performs > SNAT on pod egress, translating the pod IP to the node's primary IP. This must > be disabled so packets leave the node with their `100.64.x.x` source intact. > The Aviatrix spoke gateway then performs SNAT at the network edge — this is > what replaces the AWS private NAT gateway. ### 2.2 Create ENIConfig Custom Resources One ENIConfig per AZ, created before node groups launch: ```yaml theme={null} apiVersion: crd.k8s.amazonaws.com/v1alpha1 kind: ENIConfig metadata: name: us-east-2a # Must match the AZ name spec: subnet: subnet-0abc123... # Pod subnet ID (100.64.x.x) in this AZ securityGroups: - sg-0def456... # Pod security group ``` > [!WARNING] > > If nodes launch before ENIConfigs exist, pods will incorrectly receive IPs > from the node's primary (routable) subnet. ### 2.3 Pod Security Group Create a dedicated security group for pod ENIs with the following rules: **Ingress:** * All traffic from node security group (node-to-pod communication) * All traffic from self (pod-to-pod within cluster) * TCP 9443 from EKS cluster primary security group (control plane webhook callbacks) **Egress:** * All traffic to `0.0.0.0/0` ### 2.4 Cluster Management Access from Pods Pods on the non-routable secondary CIDR (`100.64.x.x`) need to reach the EKS API server for normal Kubernetes operations — service account token refresh, watch streams, operator controllers, etc. This works because the control plane ENIs and pod ENIs are in the same VPC, even though they are on different CIDRs. The key security group rule is: ```hcl theme={null} # Allow pods (on 100.64.x.x) to reach the EKS control plane (on 10.x.x.x) resource "aws_vpc_security_group_ingress_rule" "cluster_from_pods" { security_group_id = module.eks.cluster_primary_security_group_id referenced_security_group_id = aws_security_group.pod.id ip_protocol = "-1" description = "Allow pods to reach EKS control plane" } ``` This rule opens the cluster's primary security group (attached to control plane ENIs) to all traffic from the pod security group (attached to pod ENIs via ENIConfig). Because security group rules are evaluated at the ENI level regardless of subnet CIDR, the cross-CIDR reference works — pods on `100.64.x.x` can reach control plane ENIs on `10.x.x.x` within the VPC. The reciprocal rule (in section 2.3 above) allows the control plane to call back to pod webhooks on port 9443. This is required for admission controllers like the AWS Load Balancer Controller or cert-manager. > [!NOTE] > > No SNAT is involved in this path — traffic stays entirely within the VPC and > does not traverse the Aviatrix gateway. ### 2.5 Cluster Ingress on the Routable CIDR Ingress load balancers must be placed in routable subnets (from the primary CIDR) so they are reachable from outside the VPC. The pods behind them are on the non-routable secondary CIDR — the load balancer bridges the gap. Target type must be `ip` regardless of which ingress controller you use. With CNI-level SNAT disabled, the `instance` target type (which routes through kube-proxy on the node) does not work correctly. The `ip` target type registers pod IPs (`100.64.x.x`) directly with the load balancer, and the load balancer reaches them within the VPC. **ALB (AWS Load Balancer Controller)** ```yaml theme={null} apiVersion: networking.k8s.io/v1 kind: Ingress metadata: annotations: alb.ingress.kubernetes.io/scheme: internet-facing alb.ingress.kubernetes.io/target-type: ip ``` **NLB Service** ```yaml theme={null} apiVersion: v1 kind: Service metadata: annotations: service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: ip spec: type: LoadBalancer ``` > [!NOTE] > > **AWS Load Balancer Controller caveat:** When pods run on the secondary CIDR, they cannot reach EC2 instance metadata (IMDS) to auto-detect VPC ID and region. The controller must be configured with explicit values: > > ``` > --set vpcId= > --set region= > ``` > > This caveat is specific to the AWS Load Balancer Controller. Other ingress > controllers may not have this dependency. ## Step 3: Aviatrix Spoke Gateway — Custom SNAT This is where Aviatrix replaces the AWS private NAT gateway. Instead of deploying a NAT gateway in the VPC and configuring route tables to point non-routable subnets at it, the Aviatrix spoke gateway performs the same SNAT function — and integrates with DCF for policy enforcement. ### 3.1 Do NOT Enable single\_ip\_snat The spoke gateway must not use `single_ip_snat = true`. That mode applies blanket NAT and cannot be combined with policy-based rules. Leave it at the default (`false`). ### 3.2 Configure Custom SNAT Policies Create an `aviatrix_gateway_snat` resource with three policies: **Policy 1 — Pod traffic toward the transit** (east-west to other VPCs, on-prem, etc.) ```hcl theme={null} snat_policy { src_cidr = "100.64.0.0/16" dst_cidr = "0.0.0.0/0" protocol = "all" connection = "" snat_ips = "" } ``` **Policy 2 — Pod traffic toward the internet** (north-south) ```hcl theme={null} snat_policy { src_cidr = "100.64.0.0/16" dst_cidr = "0.0.0.0/0" protocol = "all" interface = "eth0" snat_ips = "" } ``` **Policy 3 — Node traffic toward the internet** Since `single_ip_snat` is disabled, nodes also lose default internet NAT. Add a policy per infrastructure subnet: ```hcl theme={null} snat_policy { src_cidr = "10.10.0.160/26" # Node subnet dst_cidr = "0.0.0.0/0" protocol = "all" interface = "eth0" snat_ips = "" } ``` ### 3.3 Field Notes * `connection` and `interface` are mutually exclusive — when specifying `connection`, do not set `interface`, and vice versa. * `snat_ips` is the spoke gateway's primary private IP — the routable VPC address. * Custom SNAT policies must be configured on each gateway individually. If the spoke has an HA gateway, create a separate `aviatrix_gateway_snat` resource targeting the HA gateway with its own private IP as the `snat_ips` value. ## Step 4: Aviatrix Transit Gateway — Route Exclusion The transit must not advertise the non-routable pod CIDR to other spokes or on-prem: ```hcl theme={null} excluded_advertised_spoke_routes = "100.64.0.0/16" ``` Without this, the spoke advertises `100.64.0.0/16` into the transit routing domain. If multiple clusters use the same secondary CIDR, this creates a routing conflict. Even with a single cluster, advertising a non-routable CIDR is unnecessary — all pod traffic is SNATed to a routable address before entering the fabric. > [!TIP] > > Multiple CIDRs can be comma-separated: `"100.64.0.0/16,100.65.0.0/16"`. ## Step 5: Aviatrix DCF and SmartGroup Considerations ### How DCF Handles SNAT DCF evaluates traffic at the spoke gateway — the entry point into the Aviatrix fabric. Transit gateways are forwarders and do not perform policy evaluation. The DCF engine is NAT-aware and automatically accounts for SNAT translations when programming policies. You do not need to manually adjust rules for pre- vs. post-SNAT addresses. ### Use Dynamic-Type SmartGroups Because the pod CIDR is non-routable and potentially overlapping across VPCs, CIDR-based SmartGroups matching `100.64.0.0/16` are ambiguous — DCF cannot distinguish which cluster a `100.64.x.x` address belongs to. Use dynamic-type SmartGroups that resolve to identity: | Dynamic Type | Use Case | | - | - | | VPC | Cluster-level / VPC-level segmentation | | Kubernetes | Workload-level segmentation by namespace, label, or service account | **VPC-type SmartGroup** ```hcl theme={null} resource "aviatrix_smart_group" "eks_cluster" { name = "sg-eks-cluster" selector { match_expressions { type = "vpc" name = "eks-production" } } } ``` **Kubernetes-type SmartGroup** ```hcl theme={null} resource "aviatrix_smart_group" "api_pods" { name = "sg-api-pods" selector { match_expressions { type = "k8s" namespace = "production" label_key = "app" label_value = "api" } } } ``` ### Prerequisites for Kubernetes-Type SmartGroups EKS clusters must be [onboarded to the Aviatrix Controller](/docs/enterprise/10.1/guides/security/dcf/kubernetes-onboard) via `aviatrix_kubernetes_cluster`. The Controller needs read access to the cluster (EKS access entries with view permissions) to discover pod, namespace, and label metadata. ## Related Topics * [Using Customized SNAT and DNAT on Spoke Gateway](/docs/enterprise/10.1/guides/connectivity/vpn/spoke-gateway-snat-dnat) * [Onboarding Kubernetes Clusters](/docs/enterprise/10.1/guides/security/dcf/kubernetes-onboard) * [Distributed Cloud Firewall for Kubernetes](/docs/enterprise/10.1/guides/security/dcf/dcf-kubernetes) * [Kubernetes Prerequisites and Permissions](/docs/enterprise/10.1/reference/dcf/kubernetes-prerequisites) * [Aviatrix Kubernetes Firewall](/docs/enterprise/10.1/concepts-architectures/components/dcf/kubernetes-overview) # Tuning for Sub-10 Seconds Failover Time in Overlapping Networks Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/nat/s2c-overlapping-cidrs-with-fast-convergence The purpose of this document is to provide the instructions for tuning network configurations for sub-10 seconds failover time when network address ranges on-prem and cloud are overlapping. ## Introduction The purpose of this document is to provide the instructions for tuning network configurations for sub-10 seconds failover time when network address ranges on-prem and cloud are overlapping. The scenario is described in the following diagram. ![](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/nat/s2c-overlapping-cidr-topology.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=77b8da1194c371f0edb5f71899603bd5) In the above diagram, Client-1 and Client-2 need to communicate with on-prem network. However, both Client-1 and Client-2 network address ranges overlap with each other, and worse yet, they both overlap with on-prem network address range (10.0.0.0/16). Such scenarios happen when Client-1, Client-2 and the on-prem networks belong to three different organizations. The traditional solution is to build IPsec tunnel between the two networks and use SNAT/DNAT rules to translate each addresses, as demonstrated in this [example](/docs/enterprise/10.1/guides/connectivity/vpn/site-to-site-vpn). Such solution requires a potentially large number of SNAT/DNAT rules which is difficult to configure and maintain. With the introduction of [Mapped Site2Cloud for address overlapping networks](/docs/enterprise/10.1/guides/connectivity/vpn/overlapping-network-connectivity-solutions), you no longer need to wrestle with the individual SNAT/DNAT rules. ## Configuration Steps > [!NOTE] > > This example uses Aviatrix Gateway on client site to simulate fast convergence > environment ### Step 1: Follow the Multicloud Transit workflow to launch gateways Log in to the Controller console, go to Multi-CLOUD TRANSIT. Follow step 1, step 4 and step 6 respectively to launch transit and spoke gateways, and attach spoke gateways to transit. Create VPN tunnel between Transit Gateway and On-prem. ### Step 2: Create a Site2Cloud tunnel between Spoke Gateway and Client-1 #### 2.1 Configure S2C from Spoke Gateway to Client-1 Go to Controller Console → Site2Cloud → Setup. Click "+Add New". Fill the form and click OK. Select "Mapped" for the Connection Type field. | **Field** | **Value** | | - | - | | VPC ID/VNet Name | Choose VPC ID (Select Spoke Gateway VPC) | | Connection Type | Mapped | | Connection Name | Arbitrary (e.g. S2C-SPK-to-Client1) | | Remote Gateway Type | Aviatrix | | Tunnel Type | Route-based | | Algorithms | Uncheck this box | | IKEv2 | Uncheck this box | | Over Private Network | Uncheck this box | | Enable HA | Check this box | | Primary Cloud Gateway | Select the Aviatrix Gateway created above | | Backup Gateway | Select the Aviatrix Gateway HA | | Remote Gateway IP Address | Public IP of Client-1 Primary Gateway | | Remote Gateway IP Address (Backup) | Public IP of Client-1 Backup Gateway | | Pre-shared Key | Optional (auto-generated if not entered) | | Same Pre-shared Key as Primary | Check this box | | Custom Mapped | Uncheck this box | | Remote Subnet (Real) | 10.10.0.0/16 (Client-1 Real CIDR) | | Remote Subnet (Virtual) | 100.64.0.0/16 (Client-1 Virtual CIDR) | | Local Subnet (Real) | 10.10.0.0/16 (On-Prem Network CIDR) | | Local Subnet (Virtual) | 192.168.0.0/16 (On-Prem Virtual CIDR) | #### 2.2 Configure S2C from Client Side Go to Controller Console → Site2Cloud → Setup. Click "+Add New". Fill the form and click OK. Select "unmapped" for the Connection Type field. | **Field** | **Value** | | - | - | | VPC ID/VNet Name | Choose VPC ID (Select Client-1 VPC) | | Connection Type | Unmapped | | Connection Name | Arbitrary (e.g. S2C-Client1-to-SPK-GW) | | Remote Gateway Type | Aviatrix | | Tunnel Type | Route-based | | Algorithms | Uncheck this box | | IKEv2 | Uncheck this box | | Over Private Network | Uncheck this box | | Enable HA | Check this box | | Primary Cloud Gateway | Select the Aviatrix Gateway created above | | Backup Gateway | Select the Aviatrix Gateway HA | | Remote Gateway IP Address | Public IP of Spoke Primary Gateway | | Remote Gateway IP Address (Backup) | Public IP of Spoke Backup Gateway | | Pre-shared Key | Optional (auto-generated if not entered) | | Same Pre-shared Key as Primary | Check this box | | Remote Subnet | 192.168.0.0/16 (On-Prem Virtual CIDR) | | Local Subnet | 10.10.0.0/16 (Client-1 Local Network CIDR) | ### Step 3: Configure global parameters Go to Controller Console → Settings → Advanced 1. Click on "Tunnel" tab and change "Status Change Detection Time" and save settings. | **Field** | **Value** | | - | - | | Aviatrix Entity | Choose Controller | | Detection time (secs) | 20 | 2. Click on "Keepalive" tab and modify Keepalive Template Configuration | **Field** | **Value** | | - | - | | Keep Alive Speed | fast | ### Step 4: Configure site2cloud parameters Go to Aviatrix Controller's Console → Site2Cloud → Setup. #### 4.1 Spoke Gateway Side Select Spoke Gateway VPC, spoke gateway to client site2cloud connection and click "Edit" 1. Make sure only one tunnel is UP and HA status Active-Standby 2. DPD Timer is enabled, configure DPD timers as shown below and click "Save and Apply". | **Field** | **Value** | | - | - | | Initial Delay | 1 | | Retry | 1 | | Maxfail | 1 | 3. Forward Traffic to Transit Gateway is enabled 4. Event Triggered HA is enabled #### 4.2 Client Side Select Client VPC, client to spoke site2cloud connection and click "Edit" 1. Make sure only one tunnel is UP and HA status Active-Standby 2. DPD Timer is enabled, configure DPD timers as shown below and click "Save and Apply". | **Field** | **Value** | | - | - | | Initial Delay | 1 | | Retry | 1 | | Maxfail | 1 | 3. Active Active HA is disabled 4. Event Triggered HA is enabled ## Test site2cloud fast convergence Bring down IPSec primary tunnel and measure convergence. # Creating AWS VPC Peering Connection Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/peering/aws-vpc-peering The Aviatrix Controller integrates native AWS VPC peering to create a peering connection between two VPCs to enable traffic flow between the VPCs privately. The Aviatrix Controller integrates native AWS [VPC peering](https://docs.aws.amazon.com/vpc/latest/peering/what-is-vpc-peering.html) to create a peering connection between two VPCs to enable traffic flow between the VPCs privately. * A VPC peering connection can be created between VPCs within the same AWS region (intra-region) or across AWS regions (inter-region), where it is available. * A VPC peering connection can be created between VPCs in the same AWS account or in different AWS accounts. Aviatrix simplifies how to create VPC peering by integrating requester and acceptor and route table programming into one step. You can also decide which route table participates in the AWS VPC Peering. > [!NOTE] > > The following limitations apply: > > * You cannot create a peering connection between VPCs that have matching or > overlapping CIDRs. > * Transitive peering is not possible with VPC peering. ## Creating an AWS VPC Peering Connection **CoPilot UI** To create an AWS VPC Peering connection: 1. In CoPilot, navigate to **Networking** > **Connectivity** > **Native Peering** tab, and click **+ Native Peering**. 2. For **Cloud** type, select **AWS**, you can use the dropdown menu to select **Standard**, **GovCloud**, or **China**. 3. From the **VPC/VNet 1** dropdown menu, select the VPC you want to peer. The VPCs are listed in the format *VPC\_Name*(*VPC\_ID*) and for each VPC, the *cloud account name* and *cloud region* are also shown. > [!TIP] > > To locate or display VPCs in the list by a specific cloud account name or > cloud region, type the cloud account name or cloud region in the VPC/VNet list > field. 4. You can choose to build the peering for all VPC CIDRs or select custom route tables. * To build the peering for all VPC CIDRs, leave **Select Route Tables** switch **Off** next to the selected VPC. * To select custom route tables, turn **Select Route Tables** switch **On**, and from **Select Route Tables** dropdown menu, select the route tables(s) to peer. 5. Do the same for the other VPC with which to create the peering. 1. From the **VPC/VNet 2** dropdown menu, select another VPC to peer with. * To build the peering for all VPC CIDRs, leave **Select Route Tables** switch **Off** next to the selected VPC. * To select custom route tables, turn **Select Route Tables** switch **On**, and from **Select Route Tables** dropdown menu, select the route tables(s) to peer. 6. Click **Save**. The new AWS VPC Peering connection appears in the table. The name for the peering connection is auto-generated by CoPilot. You can use the filter to display additional connection settings. You can click the peering connection name to display VPC details. The details for VPC 1 and VPC 2 are shown in their respective columns. **Terraform** Create the peering connection with the [aviatrix\_aws\_peer](https://registry.terraform.io/providers/AviatrixSystems/aviatrix/latest/docs/resources/aviatrix_aws_peer) resource. **1. Create the AWS VPC peering connection** Add the following `aviatrix_aws_peer` resource block to your Terraform configuration file: ```hcl theme={null} resource "aviatrix_aws_peer" "aws_vpc_peer" { account_name1 = "aws_account_1" account_name2 = "aws_account_2" vpc_id1 = "vpc-abcdef" vpc_id2 = "vpc-123456" vpc_reg1 = "us-east-1" vpc_reg2 = "us-west-1" } ``` > [!NOTE] > > Both VPCs must have non-overlapping CIDRs. By default, this peers all > route tables in both VPCs — set `rtb_list1`/`rtb_list2` to peer only > specific route tables. **2. Verify and Apply the Configuration** Run `terraform plan`, review the planned changes, and run `terraform apply`. ## Related Resources * [aviatrix\_aws\_peer Resource](https://registry.terraform.io/providers/AviatrixSystems/aviatrix/latest/docs/resources/aviatrix_aws_peer) ## Deleting an AWS VPC Peering Connection To delete an AWS VPC peering connection: 1. In CoPilot, navigate to **Networking** > **Connectivity** > **Native Peering** tab. 2. In the table, locate the connection you want to delete and click the Delete icon on the right side of its row. 3. To delete the peering connection, click **Delete** to delete the selected peering connection. The table shows the updated list of active VPC peering connections. > [!NOTE] > > To delete native peering of an HPE connection, delete the gateway attachment. > > Refer to the following to delete a gateway attachment: > > * [Deleting a Transit Gateway Attachment](/docs/enterprise/10.1/reference/ui/cloud-fabric/gateways#view-transit-gateways) > * [Deleting a Spoke-to-Transit Attachment](https://legacy.docs.aviatrix.com/documentation/latest/network/spoke-gateway-attachments.html?expand=true) > * [Deleting a Spoke-to-Spoke Attachment](https://legacy.docs.aviatrix.com/documentation/latest/network/spoke-gateway-attachments.html?expand=true) # Creating Azure VNet Peering Connection Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/peering/azure-vnet-peering The Aviatrix Controller integrates native Azure Virtual network peering to create a peering connection between two VNets to enable traffic flow between the VNets privately. The Aviatrix Controller integrates native Azure [Virtual network peering](https://learn.microsoft.com/en-us/azure/virtual-network/virtual-network-peering-overview) to create a peering connection between two VNets to enable traffic flow between the VNets privately. * A peering connection can be created between VNets within the same Azure region (virtual network peering) or across Azure regions (global virtual network peering), where it is available. * A peering connection can be created between VNets in the same Azure subscription or in different Azure subscriptions, as long as both subscriptions are onboarded in Aviatrix Controller. ## Creating an Azure VNet Peering Connection **CoPilot UI** To create an Azure VNet Peering connection: 1. In CoPilot, navigate to **Networking** > **Connectivity** > **Native Peering** tab, and click **+ Native Peering**. 2. For **Cloud** type, select **Azure**, you can use the dropdown menu to select **Global**, **GovCloud**, or **China**. 3. From the **VPC/VNet 1** dropdown menu, select the VNet to peer. The VNets are listed in the format *VNet\_Name*(*Resource\_ID*) and for each VNet, the *cloud account name* and *cloud region* are also shown. > [!TIP] > > To locate or display VPCs in the list by a specific cloud account name or > cloud region, type the cloud account name or cloud region in the VPC/VNet > list field. 4. From the **VPC/VNet 2** dropdown menu, select another VNet to peer with. 5. Click **Save**. The new Azure VNet Peering connection appears in the table. The name for the peering connection is auto-generated by CoPilot. You can use the filter to display additional connection settings. You can click the peering connection name to display VNet details. The details for VNet 1 and VNet 2 are shown in their respective columns. **Terraform** Create the peering connection with the [aviatrix\_azure\_peer](https://registry.terraform.io/providers/AviatrixSystems/aviatrix/latest/docs/resources/aviatrix_azure_peer) resource. **1. Create the Azure VNet peering connection** Add the following `aviatrix_azure_peer` resource block to your Terraform configuration file: ```hcl theme={null} resource "aviatrix_azure_peer" "azure_vnet_peer" { account_name1 = "azure_account_1" account_name2 = "azure_account_2" vnet_name_resource_group1 = "VNet1:ResourceGroup1:11111111-1111-1111-1111-111111111111" vnet_name_resource_group2 = "VNet2:ResourceGroup2:22222222-2222-2222-2222-222222222222" vnet_reg1 = "West US" vnet_reg2 = "East US" } ``` > [!NOTE] > > `vnet_name_resource_group1`/`2` use the format > `"VNet_Name:Resource_Group_Name:GUID"`, where `GUID` is the Azure > subscription ID that owns the VNet. **2. Verify and Apply the Configuration** Run `terraform plan`, review the planned changes, and run `terraform apply`. ## Related Resources * [aviatrix\_azure\_peer Resource](https://registry.terraform.io/providers/AviatrixSystems/aviatrix/latest/docs/resources/aviatrix_azure_peer) ## Deleting an Azure VNet Peering Connection To delete an Azure VNet peering connection: 1. In CoPilot, navigate to **Networking** > **Connectivity** > **Native Peering** tab. 2. In the table, locate the connection you want to delete and click the Delete icon on the right side of its row. 3. To delete the peering connection, click **Delete** to delete the selected peering connection. The table shows the updated list of active VPC peering connections. > [!NOTE] > > To delete native peering of an HPE connection, delete the gateway attachment. > > Refer to the following to delete a gateway attachment: > > * [Deleting a Transit Gateway Attachment](/docs/enterprise/10.1/reference/ui/cloud-fabric/gateways#view-transit-gateways) > * [Deleting a Spoke-to-Transit Attachment](https://legacy.docs.aviatrix.com/documentation/latest/network/spoke-gateway-attachments.html?expand=true) > * [Deleting a Spoke-to-Spoke Attachment](https://legacy.docs.aviatrix.com/documentation/latest/network/spoke-gateway-attachments.html?expand=true) # Multi-Region Multicloud Transit Gateway Peering Workflow Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/peering/transit-gateway-peering-workflow Aviatrix Transit Gateway Peering enables connectivity across two or more Aviatrix Transit Gateways for communication between groups of Spoke VPCs or VNets across single or multiple clouds and regions. Aviatrix Transit Gateway Peering enables connectivity across two or more Aviatrix Transit Gateways for communication between groups of Spoke VPCs or VNets across single or multiple clouds and regions. In the diagram below, the Aviatrix Transit Gateways are deployed in AWS and Azure, where each Transit Gateway connects a group of Spoke VPCs/VNets. As a result of Transit Gateway Peering, the Spoke VPC/VNet CIDRs and on-premises routes are dynamically propagated throughout the network, which allows the groups of Spoke VPCs/VNets to communicate with each other via the Transit Gateways. ![Multi-Region Multicloud Transit Gateway Peering diagram](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/transit/multi-region.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=e4ada9877f8355c8c7ea98196902350a) [Aviatrix High Performance Encryption (HPE)](/docs/enterprise/10.1/concepts-architectures/components/security/hpe-about) is supported for Transit Gateway Peering. To create Transit Gateway Peering with HPE, you must first create the Transit Gateways with HPE mode enabled, then create the Transit Gateway peering connection. ## Creating a Transit Gateway Peering Connection To create Transit Gateway peering, do the following: 1. [**Create the Transit Gateways**](https://legacy.docs.aviatrix.com/documentation/latest/network/transit-gateway-about.html?expand=true) The Transit Gateways can be in a single or multiple clouds and regions. > [!NOTE] > > Aviatrix High Performance Encryption (HPE) mode is supported on Transit > Gateway Peering. To enable Transit Gateway Peering HPE, you must create the > Transit Gateways with HPE mode enabled. 2. [**Attach the Transit Gateways**](https://legacy.docs.aviatrix.com/documentation/latest/network/transit-gateway-attachments.html#transit-to-transit-attachment). ## Excluded Network CIDRs Excluded Network CIDRs enables you to filter CIDRs to exclude from being propagated to the peering Transit Gateway. One use case is when there are conflicting or identical CIDRs on both sides of the Transit Gateways, the peering action will be rejected. Using the filter option prevents the overlapped CIDRs from being propagated to the other Transit Gateway. By default, a Transit Gateway propagates all learned routes from Spoke VPCs and VNets and on-premises. The diagram below illustrates how Excluded Network CIDRs can be used in a two region deployment. In this case, 10.20.0.0/16 appears on both sides as VPC/VNet CIDR. To allow Transit Peering to proceed, configure on both Transit Gateways Excluded Network CIDRs with 10.20.0.0/16. ![Excluded Network CIDRs diagram showing two region deployment with overlapping CIDRs](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/transit/excluded-network-cidrs.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=e6e1c47924c9b6548cd03cd6482bcece) To configure excluded network CIDRs, see [Creating a Transit Gateway to Transit Gateway Attachment](https://legacy.docs.aviatrix.com/documentation/latest/network/transit-gateway-attachments.html#transit-to-transit-attachment). ## Excluded TGW Connections Excluded TGW Connections applies to TGW hybrid connections. This is because AWS TGW does not preserve BGP information in its learned routes. When you use AWS Transit Gateway (TGW) to connect to on-premises via DXGW or VPN, there are situations where on-premises advertise the same network CIDRs to TGW in two different regions. When you connect the two regions by the Aviatrix Transit Gateway peering, overlapping network CIDRs will occur. Excluded TGW Connections enables you to not advertise certain TGW hybrid attachment (DXGW and VPN) to the remote Aviatrix Transit Gateway and therefore the remote TGW. The diagram below illustrates one use case. In the diagram, both on-premises connects to TGW and advertise 10.0.0.0/8. Transit Gateway Peering will fail because of the conflicting routes. To solve the problem, configure on both Transit Gateways to exclude 10.0.0.0/8. With this configuration, Site-1 still accesses Prod-1/Prod-2 and Dev-1/Dev-2 via the local regional TGW and Site-2 accesses Prod-3/Prod-4 and Dev-3/Dev-4 via its local regional TGW. ![Excluded TGW Connections diagram showing two regions with conflicting on-premises routes](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/transit/excluded-tgw-connections.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=7b9de270478d4bf0199726bdfa8e0751) To configure excluded TGW connections, see [Creating a Transit Gateway to Transit Gateway Attachment](https://legacy.docs.aviatrix.com/documentation/latest/network/transit-gateway-attachments.html#transit-to-transit-attachment). ## Peering over Private Network Peering over Private Network feature enables you to build Aviatrix Transit Gateway peering connections with High Performance Encryption (HPE) over multicloud where there is a private network connectivity. For example, one Transit Gateway is in AWS and the other in Azure. One use case is two Aviatrix Transit Gateways are deployed in different public clouds where each has its private connectivity, such as AWS Direct Connect and Azure ExpressRoute, connecting to on-premises or a co-location. By building a Transit Gateway private peering, Aviatrix Transit Gateway forwards traffic over the private links to the other Aviatrix Transit Gateway and beyond. For an example configuration workflow, see [Transit Gateway Peering over Private Network Workflow](/docs/enterprise/10.1/guides/connectivity/peering/transit-peering-over-private-network-hpe). ### Single-Tunnel mode Single-Tunnel mode applies to Transit Gateway peering over private network when two Transit Gateways are launched in different clouds with High Performance Encryption Mode. For example, one Transit Gateway is in AWS and the other in Azure. When Single-Tunnel mode is selected, only a single tunnel connection is established between the two multicloud Transit Gateways (instead of building up to 50 IPSec tunnels as in Aviatrix High Performance Encryption Mode). One use case is where the underlying private network is a low speed (up to 4Gbps) link across the two cloud types. By using the Single-Tunnel mode, you do not pay the High Performance Encryption Mode license charges. When the multicloud Transit Gateways enable HA on both cloud types, the aggregate throughput via Single-Tunnel mode can reach 4Gbps. ## Peering over Public Network Aviatrix Transit Gateway Peering over the public network expands Transit Gateway peering connections across cloud service providers over the internet by using Aviatrix High Performance Encryption (HPE) tunneling. Aviatrix HPE enables high throughput performance and high performance encrypted peered connections between the intercloud Transit Gateways. To create Transit Gateway peering over the internet, see [Transit Gateway Peering over Public Network Workflow](/docs/enterprise/10.1/guides/connectivity/peering/transit-peering-over-public-network-hpe). For more information about Aviatrix HPE tunneling, see [About High-Performance Encryption](/docs/enterprise/10.1/concepts-architectures/components/security/hpe-about). ## Default Route Propagation Behavior If centralized egress is enabled by local TGW FireNet or Transit FireNet, the default route 0.0.0.0/0 is **not** propagated to the remote Aviatrix Transit Gateway via Transit Peering. On the other hand, if on-premises advertises the default route to the Aviatrix Transit Gateway, this default route is propagated to the remote Aviatrix Transit Gateway via Transit Peering. # Transit Gateway Peering Over Private Network Workflow Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/peering/transit-peering-over-private-network-hpe The Aviatrix Transit Gateway Peering over Private Network feature expands Transit Gateway peering to across multiclouds where there is a private network connectivity between the cloud providers via on-prem or a co-location. The Aviatrix Transit Gateway Peering over Private Network feature expands Transit Gateway peering to across multiclouds where there is a private network connectivity between the cloud providers via on-prem or a co-location. This enables customers to build high performance data networks while ensuring data privacy by encrypting data in motion. The solution applies to AWS Direct Connect, Azure ExpressRoute, and Google Cloud Interconnect. This document provides step-by-step instructions on how to build Aviatrix Transit Gateway Peering with Private Network over AWS Direct Connect and Azure ExpressRoute for R6.2 and later releases. In this document, you learn the following: 1. Workflow for building underlay connectivity for private network with AWS Direct Connect 2. Workflow for building underlay connectivity for private network with Azure ExpressRoute 3. Workflow for Aviatrix Transit Gateway Peering with private network > [!NOTE] > > * The Aviatrix Transit Gateway Peering over Private Network solution supports > only High Performance Encryption (HPE) mode where Aviatrix Transit Gateways > have High Performance Mode Encryption option enabled at gateway launch. \* > ActiveMesh 2.0 is required. To migrate to ActiveMesh 2.0, see [Migrating from > Classic Aviatrix Encrypted Transit Network to Aviatrix ActiveMesh Transit > Network](https://legacy.docs.aviatrix.com/documentation/latest/network/activemesh-migration.html). > * Private subnet reachability between two Transit CIDRs is your > responsibility, which is typically done via co-located providers. \* The > workflow shown here for building underlay connectivity for a private network > with AWS Direct Connect/Azure ExpressRoute is just an example. Please adjust > the topology depending on your requirements. ## Topology ![transit-gateway-peering-with-private-network-diagram](https://mintcdn.com/aviatrix-14b37c43/xzfnEI4Ko3Ysw6na/images/guides/connectivity/peering/transit-gateway-peering-network-diagram.png?fit=max\&auto=format\&n=xzfnEI4Ko3Ysw6na\&q=85\&s=a41ee3e4fdafb15f96e7d1a37c7b96c0) ### The key ideas for this solution are: * The edge (WAN) router runs a BGP session to AWS VGW via AWS Direct Connect where the edge router advertises the Azure Transit VNET CIDR and the AWS VGW advertises the AWS Transit VPC CIDR. * The edge (WAN) router runs a BGP session to Azure VNG via Azure ExpressRoute where the edge router advertises the AWS Transit VPC CIDR and the Azure VNG advertises the AZURE Transit VNET CIDR. * The edge (WAN) router redistributes AWS Transit VPC CIDR and AZURE Transit VNET CIDR. * Once the reachability between two cloud transits over private network is present, you are able to deploy Aviatrix Multi Cloud Global Transit Gateway Encrypted Peering over a Private Network. > [!NOTE] > > Reachability between two transit networks' private CIDRs is not the > responsibility of Aviatrix. ## Prerequisites Upgrade Aviatrix Controller to the latest version. In this example, we are going to deploy the below VPCs in AWS and Azure: * AWS Aviatrix Transit VPC (i.e. 10.1.0.0/16) * AWS Aviatrix Spoke VPC (i.e. 192.168.1.0/24) * Azure Aviatrix Transit VNET (i.e. 10.0.0.0/16) * Azure Aviatrix Spoke VNET (i.e. 192.168.0.0/24) ## Workflow on building underlay connectivity for private network with AWS Direct Connect Building AWS Direct Connect is your responsibility. For more information about AWS Direct Connect, see [Connect Your Data Center to AWS](https://aws.amazon.com/getting-started/projects/connect-data-center-to-aws/). Please adjust the topology depending on your requirements. ### Build AWS Direct Connect Refer to [Equinix ECX Fabric AWS Direct Connect](https://docs.equinix.com/en-us/Content/Interconnection/Fabric/connections/Fabric-aws-direct-connect.htm) if you select an Equinix solution. The below is just an example. ### Associate AWS VGW to AWS Transit VPC 1. Log into the AWS VPC Portal. 2. On the VPC dashboard, click **Virtual private gateways**. 3. Select the Virtual Private Gateway that has the private virtual interface to AWS Direct Connect. 4. From the Actions menu, select **Attach to VPC**. 5. Select the AWS Transit VPC and click **Attach to VPC**. ## Workflow on building underlay connectivity for private network with Azure ExpressRoute Building Azure ExpressRoute is your responsibility. For more information about Azure ExpressRoute, see the below documents: * [Azure ExpressRoute](https://azure.microsoft.com/en-us/services/expressroute/) * [ExpressRoute documentation](https://docs.microsoft.com/en-us/azure/expressroute/) * [Equinix ECX Fabric Microsoft Azure ExpressRoute](https://docs.equinix.com/en-us/Content/Interconnection/Fabric/connections/Fabric-ms-azure.htm) if you select the Equinix solution. The below is just an example. Please adjust the topology depending on your requirements. ### Create an ExpressRoute circuit Refer to [Tutorial: Create and modify an ExpressRoute circuit](https://docs.microsoft.com/en-us/azure/expressroute/expressroute-howto-circuit-portal-resource-manager). ### Create Azure private peering for an ExpressRoute circuit Refer to [private peering section in Create and modify peering for an ExpressRoute circuit](https://docs.microsoft.com/en-us/azure/expressroute/expressroute-howto-routing-portal-resource-manager). ### Create a virtual network gateway for an ExpressRoute circuit [Configure a virtual network gateway for ExpressRoute using the Azure portal](https://docs.microsoft.com/en-us/azure/expressroute/expressroute-howto-add-gateway-portal-resource-manager). ### Connect a virtual network to an ExpressRoute circuit [Connect a virtual network to an ExpressRoute circuit using the portal](https://docs.microsoft.com/en-us/azure/expressroute/expressroute-howto-linkvnet-portal-resource-manager). ### Check Express Route Circuits - List Routes Table on Azure portal 1. Log in to the Azure Portal. 2. Search for "ExpressRoute circuits" in the search bar. 3. Select the "ExpressRoute circuits" that you created. 4. Select the Azure private peering row. 5. Click the hyperlink "Get route table". 6. Check whether the AWS Transit VPC's CIDR appears in the table with the ASN Path of edge router and AWS VGW. ![express\_route\_circuits\_list\_routes](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/transit/express-route-circuits-list-routes.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=d09afcf58a50705d67859494af8044c4) ## Workflow on Aviatrix Transit Gateway Peering with Private Network Refer to [Global Transit Network Workflow Instructions](/docs/enterprise/10.1/guides/connectivity/transit/transitvpc-workflow) and [Aviatrix Transit Gateway Peering](/docs/enterprise/10.1/guides/connectivity/peering/transit-gateway-peering-workflow) for the below steps. Please adjust the topology depending on your requirements. ### Deploy VPCs for Transit FireNet * Create an AWS Transit VPC and Azure Transit VNET with **Transit + FireNet** selected. * Create an AWS Spoke VPC and Azure Spoke VNET as per the previous step or by manually deploying in each cloud portal. You can use your existing cloud network. ### Deploy Aviatrix Multicloud Transit Gateway and HA in AWS Create an Aviatrix Transit Gateway with High Availability and High Performance Encryption (HPE) mode. > [!NOTE] > > An instance size of at least c5.xlarge is required for High Performance Mode > Encryption for higher throughput. The recommended minimum size for Transit in > AWS is c5n.4xlarge. Please refer to > [High Performance Encryption Performance Benchmarks](/docs/enterprise/10.1/reference/gateways/hpe-perf-benchmark) > for performance details. ### Enable Route Propagation on the subnet route table for the Aviatrix Transit Gateway on AWS portal 1. Log into the AWS VPC portal. 2. Locate the subnet route table for the Aviatrix Transit Gateway. 3. Select the Route Propagation tab. 4. Click **Edit route propagation**. ![aws route propagation edit](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/transit/aws-route-propagation-edit.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=5e4ae4649cd7b359b6057fd23a8aaebd) 5. On the Edit route propagation tab, locate the AWS VGW that is associated with this Transit VPC and select the **Enable** checkbox. 6. Click **Save**. 7. On the Route propagation tab, check if the Propagation status is Yes. ![aws\_route\_propagation\_status\_yes](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/transit/aws-route-propagation-status-yes.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=4bc8323c843776fc6446bc4192810130) ### Check route propagation on AWS portal 1. In the AWS VPC portal, locate the subnet route table for the Aviatrix Transit Gateway. 2. On the Routes tab, check if there is a route entry pointing to the virtual private gateway (this is the Azure Transit VNet CIDR pointing to the virtual gateway). ![aws\_route\_propagation\_routing\_entry](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/transit/aws-route-propagation-routing-entry.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=f5947071f07829afeac0ad3adb113ed7) ### Deploy Aviatrix Multicloud Transit Gateway and HA in Azure * Deploy the Transit Aviatrix Gateway and enable HA with HPE enabled in the Azure Transit VNET. > [!NOTE] > > An instance size of at least Standard\_D5\_v2 will be required for High > Performance Encryption Mode Encryption for higher throughput. Please refer to > this > [High Performance Encryption Performance Benchmarks](/docs/enterprise/10.1/reference/gateways/hpe-perf-benchmark) > for performance details. > [!NOTE] > > An instance size of at least Standard\_D5\_v2 will be required for High > Performance Encryption Mode Encryption for higher throughput. Please refer to > this > [High Performance Encryption Performance Benchmarks](/docs/enterprise/10.1/reference/gateways/hpe-perf-benchmark) > for performance details. * Enable Transit FireNet Function (optional) ### Check Effective Routes on Azure Portal 1. Log into the Azure Portal. 2. Search for "Network interfaces" in the search bar. 3. Select the Aviatrix Transit Gateway's interface. 4. Click **Effective routes**. 5. Check if there is a route entry for the AWS Transit VPC's CIDR pointing to the Next Hop Type Virtual network gateway. ![azure\_effective\_routes\_routing\_entry](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/transit/azure-effective-routes-routing-entry.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=4f995b671e5893e8d5af5b59b6846184) ### Establish Transit Gateway Peering over Private Network 1. In Aviatrix CoPilot, go to Cloud Fabric > Gateways > Transit Gateways and click the edit ![edit icon](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/transit/edit-icon.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=d5d775ce21a056389f6c95244aff0ded) icon next to the AWS Transit Gateway. 2. Ensure that the Azure Transit Gateway is shown in the Peer to Transit Gateways field (if not, click the dropdown in that field to add it). 3. Click **Save**. ### Deploy Spoke Gateway and HA * [Deploy a Spoke Gateway](https://legacy.docs.aviatrix.com/documentation/latest/network/gateway-spoke-create.html) in the AWS Spoke VPC with HA and High Performance Encryption Mode enabled. An instance size of at least c5.xlarge will be required for [High Performance Encryption Mode Encryption](/docs/enterprise/10.1/concepts-architectures/components/security/hpe-about) for higher throughput. * [Deploy a Spoke Gateway](https://legacy.docs.aviatrix.com/documentation/latest/network/gateway-spoke-create.html) in the Azure Spoke VNet with HA and HPE enabled. An instance size of at least Standard\_D5\_v2 will be required for [High Performance Encryption Mode Encryption](/docs/enterprise/10.1/concepts-architectures/components/security/hpe-about) for higher throughput. Please refer to this [High Performance Encryption Performance Benchmarks](/docs/enterprise/10.1/reference/gateways/hpe-perf-benchmark) for performance detail. ### Attach Spoke Gateways to Transit Network See [Attach Spoke Gateways to Transit Network](https://legacy.docs.aviatrix.com/documentation/latest/network/spoke-attach-transit.html) to attach Aviatrix Spoke Gateways in AWS or Azure. # Transit Gateway Peering over Public Network Workflow Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/peering/transit-peering-over-public-network-hpe Aviatrix Transit Gateway peering over public network expands Aviatrix Transit Gateway peering across multicloud where the connection between cloud service providers is over the internet. Aviatrix Transit Gateway peering over public network expands Aviatrix Transit Gateway peering across multicloud where the connection between cloud service providers is over the internet. The Aviatrix Controller builds multiple tunnels between the peered transit gateways using High Performance Encryption (HPE) Mode, enabling high performance data throughput and data security. For more information about Multicloud Transit Gateway encrypted peering, see the following documents: * [Aviatrix Transit Gateway Peering](/docs/enterprise/10.1/guides/connectivity/peering/transit-gateway-peering-workflow) * [Transit Network Design Patterns](/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/multi-cloud-design-patterns) * [Multicloud Transit Network Workflow Instructions (AWS/Azure/GCP/OCI)](/docs/enterprise/10.1/guides/connectivity/transit/transitvpc-workflow) ## Topology ![transit\_gateway\_peering\_over\_internet\_topology](https://mintcdn.com/aviatrix-14b37c43/xzfnEI4Ko3Ysw6na/images/guides/connectivity/peering/transit-gateway-peering-over-internet.png?fit=max\&auto=format\&n=xzfnEI4Ko3Ysw6na\&q=85\&s=9a51bfe8030c7688ca2a92c924cc2091) ## Prerequisites 1. Upgrade Aviatrix Controller to the latest version. Refer to [Upgrading the Aviatrix Cloud Network Platform](/docs/enterprise/10.1/guides/maintenance/controller-upgrade-workflow) . 2. Create and launch the Aviatrix Transit Gateways with HA and High Performance Encryption Mode enabled in the clouds where you want to establish peered transit connection and attach the Spoke Gateways to the Transit Gateways. Refer to [Multicloud Transit Network Workflow Instructions (AWS/Azure/GCP/OCI)](/docs/enterprise/10.1/guides/connectivity/transit/transitvpc-workflow) . > [!NOTE] > > * The Aviatrix Transit Gateway peering over public network solution requires HPE > (High-Performance Encryption). Aviatrix Transit Gateways must have HPE enabled > when the Transit Gateway is created for peered connection over the internet. > * This solution requires ActiveMesh 2.0. To migrate to ActiveMesh 2.0, refer to > [Migrating to ActiveMesh 2.0](/docs/enterprise/10.1/concepts-architectures/components/connectivity/activemesh-about#migrating-to-activemesh-2-0) > . ## Establishing Transit Gateway Peering over Public Internet Before you begin, review [Prerequisites](#prerequisites) for transit gateway peering over public network. The steps to establish transit gateway peering across cloud service providers over the public network are as follows: **CoPilot UI** To establish transit gateway peering across cloud service providers over the public network using the CoPilot UI, follow these steps: **1. Navigate to the Transit Gateways** In Aviatrix CoPilot, go to **Cloud Fabric** > **Gateways** > **Transit Gateways**. **2. Edit the Transit Gateway** Click the edit ![edit icon](https://mintcdn.com/aviatrix-14b37c43/VY6W0iTAMUQkrobS/images/security/edit-icon.png?fit=max\&auto=format\&n=VY6W0iTAMUQkrobS\&q=85\&s=d37797a7c48eefc52ed6e66ccf9915bf) icon next to one of the Transit Gateways created in this workflow. **3. Select the Transit Gateways to Peer** In the **Attach To Transit Gateways** field, select the Transit Gateways to peer with your selected Gateway. **4. Save the Configuration** Click **Save**. The Transit Gateways are now peered together. **Terraform** To establish transit gateway peering across cloud service providers over the public network using the [Aviatrix Terraform provider](https://registry.terraform.io/providers/AviatrixSystems/aviatrix/latest), follow these steps: **1. Get the Transit Gateway Names** Use the `aviatrix_transit_gateway` data source to retrieve the names of the existing transit gateways. The data source block checks if the transit gateway exists before creating the peering between them and helps to avoid errors. ```hcl theme={null} data "aviatrix_transit_gateway" "csp1_transit_gateway" { # Replace csp1_transit_gateway_name with the name of the transit gateway gw_name = "csp1_transit_gateway_name" } data "aviatrix_transit_gateway" "csp2_transit_gateway" { # Replace csp2_transit_gateway_name with the name of the transit gateway gw_name = "csp2_transit_gateway_name" } ``` **2. Create the Transit Gateway Peering** Use the `aviatrix_transit_gateway_peering` resource block to create the peering. Reference the retrieved transit gateway names in the data source block to create dependency between the two data source blocks and the resource block. The following code snippet creates the peering between the two Transit Gateways over public network. ```hcl theme={null} resource "aviatrix_transit_gateway_peering" "transit_gateway_peering" { transit_gateway_name1 = data.aviatrix_transit_gateway.csp1_transit_gateway.gw_name transit_gateway_name2 = data.aviatrix_transit_gateway.csp2_transit_gateway.gw_name } ``` **3. Verify and Apply the Configuration** Run `terraform plan`, review the planned changes, and then run `terraform apply` to make the changes. ### Examples The following examples establish Transit Gateway peering over the public network using the Aviatrix Terraform provider. > **AWS-to-Azure Peering for a European Multicloud Hub** > > A European business runs its line-of-business app on Azure in `West Europe` and a customer-data service on AWS in `eu-west-1`. There is no ExpressRoute or Direct Connect between the two clouds, so the two Transit Gateways are peered over the public network using HPE for encrypted line-rate east-west traffic. > > ```hcl theme={null} > data "aviatrix_transit_gateway" "transit_aws" { > gw_name = "transit-aws-euw1" > } > > data "aviatrix_transit_gateway" "transit_azure" { > gw_name = "transit-azure-weu" > } > > resource "aviatrix_transit_gateway_peering" "aws_to_azure" { > transit_gateway_name1 = data.aviatrix_transit_gateway.transit_aws.gw_name > transit_gateway_name2 = data.aviatrix_transit_gateway.transit_azure.gw_name > } > ``` > **AWS Multi-Region DR Peering with Excluded CIDRs and TGW Connections** > > An organization runs an active-active deployment across AWS `us-east-1` and `us-west-2`, and uses public-network peering between the two Aviatrix Transit Gateways for cross-region failover (Disaster Recovery). Each region also has its own AWS Transit Gateway attached to on-prem over Direct Connect, and two things are excluded from the Aviatrix peering: > > * A shared-services CIDR (`10.0.0.0/24`) that both regions already learn from on-prem. Excluding it prevents the peer from re-advertising a route that should only enter the fabric from on-prem. > * The region-local AWS TGW connection toward on-prem (`onprem-dx-use1` / `onprem-dx-usw2`). Excluding it stops on-prem traffic from looping through the cross-region peer. > > ```hcl theme={null} > data "aviatrix_transit_gateway" "transit_aws_use1" { > gw_name = "transit-aws-use1" > } > > data "aviatrix_transit_gateway" "transit_aws_usw2" { > gw_name = "transit-aws-usw2" > } > > resource "aviatrix_transit_gateway_peering" "aws_use1_to_usw2" { > transit_gateway_name1 = data.aviatrix_transit_gateway.transit_aws_use1.gw_name > transit_gateway_name2 = data.aviatrix_transit_gateway.transit_aws_usw2.gw_name > > gateway1_excluded_network_cidrs = ["10.0.0.0/24"] > gateway2_excluded_network_cidrs = ["10.0.0.0/24"] > > gateway1_excluded_tgw_connections = ["onprem-dx-use1"] > gateway2_excluded_tgw_connections = ["onprem-dx-usw2"] > } > ``` > > > [!NOTE] > > > > `gateway1_excluded_tgw_connections` and `gateway2_excluded_tgw_connections` apply only to AWS Transit Gateway attachments. If one side of the peering is not AWS, set the argument only on the AWS side. > **Three-Cloud Mesh: AWS, Azure, and GCP** > > A SaaS provider runs front-end services on AWS, identity and licensing on Azure, and analytics on GCP. The three Aviatrix Transit Gateways are peered in a full mesh over the public internet so every Spoke can reach every other Spoke without hairpinning through a single hub. > > ```hcl theme={null} > data "aviatrix_transit_gateway" "transit_aws" { > gw_name = "transit-aws-euw1" > } > > data "aviatrix_transit_gateway" "transit_azure" { > gw_name = "transit-azure-weu" > } > > data "aviatrix_transit_gateway" "transit_gcp" { > gw_name = "transit-gcp-euw1" > } > > resource "aviatrix_transit_gateway_peering" "aws_azure" { > transit_gateway_name1 = data.aviatrix_transit_gateway.transit_aws.gw_name > transit_gateway_name2 = data.aviatrix_transit_gateway.transit_azure.gw_name > > } > > resource "aviatrix_transit_gateway_peering" "aws_gcp" { > transit_gateway_name1 = data.aviatrix_transit_gateway.transit_aws.gw_name > transit_gateway_name2 = data.aviatrix_transit_gateway.transit_gcp.gw_name > > } > > resource "aviatrix_transit_gateway_peering" "azure_gcp" { > transit_gateway_name1 = data.aviatrix_transit_gateway.transit_azure.gw_name > transit_gateway_name2 = data.aviatrix_transit_gateway.transit_gcp.gw_name > > > } > ``` *** ## Viewing the Peering Connections You can view the peering connections by clicking the name of the Transit Gateway, clicking the **Attachments** tab, and then clicking **Transit-Transit Peering**. This tab also displays the Connection Status for the peered Gateways. The Status may take a few minutes to update. ![Transit gateway peering connection status in CoPilot](https://mintcdn.com/aviatrix-14b37c43/xzfnEI4Ko3Ysw6na/images/guides/connectivity/peering/transit-gateway-peering-status.png?fit=max\&auto=format\&n=xzfnEI4Ko3Ysw6na\&q=85\&s=b949f7daf1b3b1f3784a4f51a8c5c567) ## Related Topics * [Aviatrix Terraform provider documentation](https://registry.terraform.io/providers/AviatrixSystems/aviatrix/latest/docs) * [Excluded Network CIDRs](/docs/enterprise/10.1/guides/connectivity/peering/transit-gateway-peering-workflow#excluded-network-cidrs) * [Excluded TGW Connections](/docs/enterprise/10.1/guides/connectivity/peering/transit-gateway-peering-workflow#excluded-tgw-connections) # BGP Route Approval Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/routing/bgp-route-approval Aviatrix Transit Gateways, BGP-enabled Spoke Gateways, and Edge Gateways dynamically learn BGP routes from remote peers. These learned routes are reported to the Aviatrix Controller, which then propagates and programs the route entries into the Spoke VPC or VNet route tables. Aviatrix Transit Gateways, BGP-enabled Spoke Gateways, and Edge Gateways dynamically learn BGP routes from remote peers. These learned routes are reported to the Aviatrix Controller, which then propagates and programs the route entries into the Spoke VPC or VNet route tables. For scenarios such as a VPN connecting to a partner network, you may require an approval process before dynamically learned CIDRs propagate to the Spoke VPC or VNet. This prevents undesirable routes, such as the default route (0.0.0.0/0), from entering your network and causing outages. The Learned CIDR Approval feature enforces this process. When enabled, dynamically learned routes from all remote peers trigger an email notification to the Controller administrator. To propagate the learned routes to the Spoke VPC or VNet route table, the Controller administrator should log in to the CoPilot and approve the learned routes. ![Transit Approval](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/routing/transit-approval.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=61a937405e5311c967d504eacf3be293) **Gateway Mode** Gateway mode is the default approval mode. In this mode, learned CIDR approval applies to all BGP connections configured on the gateway. **Connection Mode** Connection mode enables you to select a specific BGP connection for approval. > [!NOTE] > > Connection Mode is only available for Transit Gateways. BGP Spoke Gateways > only support Gateway Mode. To propagate all the dynamically learned routes to the Spoke VPC/VNet route table, set the **Manual Approval** toggle to **Off**. See [Enabling Gateway Learned CIDR Approval](#enabling-gateway-learned-cidr-approval) for more information. ## Enabling Gateway Learned CIDR Approval To set an approval process for the dynamically learned CIDRs on the Transit or Spoke Gateway, in Aviatrix CoPilot: 1. Go to **Cloud Fabric** > **Gateways** > **Transit Gateways** or **Spoke Gateways** tab, a table of gateways and their details appears. 2. In the table, click the gateway name to enable learned CIDR approval. 3. Go to the gateway's **Settings** tab and expand **Border Gateway Protocol (BGP)** section. > [!TIP] > > The gateway's **Settings** tab is in the same row as the **Details**, > **Instances**, and **Attachments** tabs. 4. Locate the **Manual Learned CIDR Approval** card and set **Manual Approval** toggle to **On**. > [!NOTE] > > If the **Manual Approval** toggle is set to **Off**, all learned routes on > the gateway from its remote peer are approved. * To approve learned CIDRs for all BGP connections on the Transit Gateway or Spoke Gateway, select **Gateway Level**. \* To approve learned CIDRs for a specific BGP connection on the Transit Gateway, select **Connection Level**. An **On External Connections** dropdown menu appears. > [!NOTE] > > Connection Level is only available for Transit Gateways. BGP Spoke Gateways > only support Gateway Level. * From the **On External Connections** dropdown menu, select one or more BGP connections to enable learned CIDR approval. \* Click **Save**. An approval process is set for the dynamically learned CIDRs on the Transit or Spoke Gateway. ## Approving Learned CIDRs in CoPilot When Gateway Learned CIDR Approval is enabled, an email notification is sent to the Aviatrix Controller administrator to approve the learned CIDRs. On approval the learned CIDRs are propagated to the Spoke VPC or VNet route table. ### Approving Learned CIDRs Enabled for a Gateway The following are the types of approval for a gateway: * **Manual Approval**: All learned CIDRs from all BGP connections on the gateway require approval. * **Pre-Approval**: You can add one, multiple, or a range of CIDRs to a pre-approved CIDRs rule. Any learned CIDR that matches a pre-approved CIDR in the rule is automatically approved and propagated to the Spoke VPC or VNet route table without requiring manual approval. #### Manual Approval For **Manual Approval** of the learned CIDR, in Aviatrix CoPilot: 1. Go to **Cloud Fabric** > **Gateways** > **Transit Gateways** or **Spoke Gateways** tab, a table of gateways and their details appears. 2. In the table, click the gateway name to approve CIDRs. 3. Click the **Route Approval** tab. A table of learned CIDRs appears. > [!NOTE] > > The **Route Approval** tab only appears for a gateway if the **Manual > Approval** toggle is set to **On** for the gateway. 4. In the table, select the CIDRs and click **+ Approval Rules**. 5. To use the Free Range Routing (FRR) syntax, set the **FRR Syntax** toggle to **On**. See [FRR Syntax](#frr-syntax) for more information. 6. To add CIDRs using a relationship operator between a Base CIDR and Prefix Length instead of using FRR syntax, set the **FRR Syntax** toggle to **Off**. 7. Click **+Rule**. A new row appears in the **Create Approval Rules** table. 8. In the new row, enter the Base CIDR, select the relationship operator, and enter the Prefix Length. 9. Click **Approve**. Examples: | Base CIDR | Relationship | Prefix Length | Description | Examples of Approved CIDRs | | - | - | - | - | - | | 10.1.0.0/16 | Exact Match | 16 | Approves only the Base CIDR. | 10.1.0.0/16 | | 10.1.0.0/16 | Greater Than or Equal To | 16 | Approves the Base CIDR and all longer CIDRs greater than the Prefix Length. For the given example, /16 and /32 are included. | 10.1.0.0/16, 10.1.1.0/24, 10.1.2.0/24, 10.1.1.0/28, continues through /32 | | 10.1.0.0/16 | Less Than or Equal To | 24 | Approves the Base CIDR and any CIDRs until the Base CIDR. For the given example, /16 and /24 are included. | 10.1.0.0/16, 10.1.1.0/17, 10.1.2.0/22, continues through /24 | | 10.0.0.0/8 | Prefix Length Range | 16-24 | Approves the CIDRs starting from 10.0.0.0/8 only if the prefix length is between /16 and /24. For the given example, /16 and /24 are included. | 10.1.0.0/16, 10.1.1.0/24, 10.1.2.0/24, continues through /24 | The CIDRs learned from the remote peer a approved. #### Pre-Approval For **Pre-Approval** of the learned CIDRs: 1. Go to **Cloud Fabric** > **Gateways** > **Transit Gateways** or **Spoke Gateways** tab, a table of gateways and their details appears. 2. In the table, click the gateway name to approve CIDRs to propagate. 3. Click the **Route Approval** tab. A table of learned CIDRs appears. 4. Click **Approval Rules**. 5. Click **+ Approval Rule**. The **Create Approval Rules** dialog appears. 6. Set the **FRR Syntax** toggle to **On** to use the Free Range Routing (FRR) syntax. See [FRR Syntax](#frr-syntax) for more information. 7. To add CIDRs using a relationship operator between a Base CIDR and Prefix Length instead of using FRR syntax, set the **FRR Syntax** toggle to **Off**. 8. Click **+Rule**. A new row appears in the **Create Approval Rules** table. 9. In the new row, enter the Base CIDR, select the relationship operator, and enter the Prefix Length. 10. Click **Save Draft**. 11. Click **Commit**. Examples: | Base CIDR | Relationship | Prefix Length | Description | Examples of Approved CIDRs | | - | - | - | - | - | | 10.1.0.0/16 | Exact Match | 16 | Approves only the Base CIDR. | 10.1.0.0/16 | | 10.1.0.0/16 | Greater Than or Equal To | 16 | Approves the Base CIDR and all longer CIDRs greater than the Prefix Length. For the given example, /16 and /32 are included. | 10.1.0.0/16, 10.1.1.0/24, 10.1.2.0/24, 10.1.1.0/28, continues through /32 | | 10.1.0.0/16 | Less Than or Equal To | 24 | Approves the Base CIDR and any CIDRs until the Base CIDR. For the given example, /16 and /24 are included. | 10.1.0.0/16, 10.1.1.0/17, 10.1.2.0/22, continues through /24 | | 10.0.0.0/8 | Prefix Length Range | 16-24 | Approves the CIDRs starting from 10.0.0.0/8 only if the prefix length is between /16 and /24. For the given example, /16 and /24 are included. | 10.1.0.0/16, 10.1.1.0/24, 10.1.2.0/24, continues through /24 | The added CIDRs are pre-approved when learned from the remote peer. ### Approving Learned CIDRs Enabled for a BGP Connection The following are the types of approval for an external connection: * **Manual Approval**: All learned CIDRs from all BGP connections on the external connection require approval. * **Pre-Approval**: You can add one, multiple, or a range of CIDRs to a pre-approved CIDRs rule. Any learned CIDR that matches a pre-approved CIDR in the rule is automatically approved and propagated to the Spoke VPC or VNet route table without requiring manual approval. If you select an external connection for [Learned CIDR Approval](https://legacy.docs.aviatrix.com/documentation/latest/network/bgp-connection-settings.html?expand=true#gateway-learned-cidr-approval) (either from the gateway or from the external connection's settings tab), CoPilot displays a **Route Approval** tab for that connection. #### Manual Approval For **Manual Approval** of the learned CIDR, in Aviatrix CoPilot: 1. Go to **Networking** > **Connectivity** > **External Connections (S2C)** tab. A table of external connections and their details appears. 2. Click the connection name to approve CIDRs. 3. Click the **Route Approval** tab, a table of learned CIDRs appears. > [!NOTE] > > The **Route Approval** tab only appears for the connection if the **Manual > Learned CIDR Approval** toggle is set to **On** for the connection. 4. In the table, select the CIDRs and click **+ Approval Rules**. 5. Set the **FRR Syntax** toggle to **On** to use the Free Range Routing (FRR) syntax. See [FRR Syntax](#frr-syntax) for more information. 6. To add CIDRs using a relationship operator between a Base CIDR and Prefix Length instead of using FRR syntax, set the **FRR Syntax** toggle to **Off**. 7. Click **+Rule**. A new row appears in the **Create Approval Rules** table. 8. In the new row, enter the Base CIDR, select the relationship operator, and enter the Prefix Length. 9. Click **Save Draft**. 10. Click **Commit**. Examples: | Base CIDR | Relationship | Prefix Length | Description | Examples of Approved CIDRs | | - | - | - | - | - | | 10.1.0.0/16 | Exact Match | 16 | Approves only the Base CIDR. | 10.1.0.0/16 | | 10.1.0.0/16 | Greater Than or Equal To | 16 | Approves the Base CIDR and all longer CIDRs greater than the Prefix Length. For the given example, /16 and /32 are included. | 10.1.0.0/16, 10.1.1.0/24, 10.1.2.0/24, 10.1.1.0/28, continues through /32 | | 10.1.0.0/16 | Less Than or Equal To | 24 | Approves the Base CIDR and any CIDRs until the Base CIDR. For the given example, /16 and /24 are included. | 10.1.0.0/16, 10.1.1.0/17, 10.1.2.0/22, continues through /24 | | 10.0.0.0/8 | Prefix Length Range | 16-24 | Approves the CIDRs starting from 10.0.0.0/8 only if the prefix length is between /16 and /24. For the given example, /16 and /24 are included. | 10.1.0.0/16, 10.1.1.0/24, 10.1.2.0/24, continues through /24 | The CIDR learned from the remote peer is approved. #### Pre-Approval For **Pre-Approval** of the learned CIDRs: 1. Go to **Networking** > **Connectivity** > **External Connections (S2C)** tab. A table of external connections and their details appears. 2. Click the connection name to approve CIDRs. 3. Click the **Route Approval** tab. 4. Click **Approval Rules**. 5. Click **+ Approval Rule**. The **Create Approval Rules** dialog appears. 6. Set the **FRR Syntax** toggle to **On** to use the Free Range Routing (FRR) syntax. See [FRR Syntax](#frr-syntax) for more information. 7. To add CIDRs using a relationship operator between a Base CIDR and Prefix Length instead of using FRR syntax, set the **FRR Syntax** toggle to **Off**. 8. Click **+Rule**. A new row appears in the **Create Approval Rules** table. 9. In the new row, enter the Base CIDR, select the relationship operator, and enter the Prefix Length. 10. Click **Save Draft**. 11. Click **Commit**. Examples: | Base CIDR | Relationship | Prefix Length | Description | Examples of Approved CIDRs | | - | - | - | - | - | | 10.1.0.0/16 | Exact Match | 16 | Approves only the Base CIDR. | 10.1.0.0/16 | | 10.1.0.0/16 | Greater Than or Equal To | 16 | Approves the Base CIDR and all longer CIDRs greater than the Prefix Length. For the given example, /16 and /32 are included. | 10.1.0.0/16, 10.1.1.0/24, 10.1.2.0/24, 10.1.1.0/28, continues through /32 | | 10.1.0.0/16 | Less Than or Equal To | 24 | Approves the Base CIDR and any CIDRs until the Base CIDR. For the given example, /16 and /24 are included. | 10.1.0.0/16, 10.1.1.0/17, 10.1.2.0/22, continues through /24 | | 10.0.0.0/8 | Prefix Length Range | 16-24 | Approves the CIDRs starting from 10.0.0.0/8 only if the prefix length is between /16 and /24. For the given example, /16 and /24 are included. | 10.1.0.0/16, 10.1.1.0/24, 10.1.2.0/24, continues through /24 | The added CIDRs are pre-approved when learned from the remote peer. ### FRR Syntax Use Free Range Routing (FRR) syntax with "ge" (greater than or equal to) and "le" (less than or equal to) to match a range of prefixes. This approves multiple routes without needing to specify each exact match. For example, `10.1.0.0/16 ge 16 le 24` matches any prefix within the 10.1.0.0/16 block that has a subnet mask between /16 and /24, where /16 and /24 are inclusive. **Special Cases** * `0.0.0.0/0 le 32` matches all CIDRs. * `0.0.0.0/0 ge 0` only matches the exact default route. Examples: | FRR syntax | Relationship | Description | Examples of approved CIDRs | | - | - | - | - | | 10.1.0.0/16 ge 16 | Greater than or equal to | Approves the base CIDR and any more specific CIDRs inside the same block. | 10.1.0.0/16, 10.1.1.0/24, 10.1.2.0/24, 10.1.1.0/28 | | 10.1.0.0/16 le 24 | Less than or equal to | Approves the base CIDR and any CIDRs in this block up to prefix length /24. | 10.1.0.0/16, 10.1.1.0/24, 10.1.2.0/24 | | 10.0.0.0/8 ge 16 le 24 | Prefix length range | Approves CIDRs only if they match the base CIDR and have prefix lengths between /16 and /24. | 10.1.0.0/16, 10.1.1.0/24, 10.1.2.0/24. The 10.0.0.0/8 CIDR is not approved | | 10.1.0.0/16 ge 24 le 24 | Exact prefix length match | Approves CIDRs only if they match the base CIDR and have prefix length /24. | 10.1.1.0/24, 10.2.1.0/24. The 10.1.0.0/16 CIDR is not approved | | 10.1.0.0/16 | Exact match | Approves only the base CIDR. | 10.1.0.0/16 | # GCP Spoke Single IP SNAT and Tagless Default Routes (Regional VPC) Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/routing/gcp-spoke-single-ip-snat-regional-vpc Starting with Controller 9.0, Single IP SNAT on a GCP Regional VPC spoke or standalone gateway supports tagless default routes so serverless workloads and untagged VMs can egress through the Aviatrix gateway without the avx-snat-noip network tag. This document describes how that behavior differs from releases before Controller version 9.0, what the Controller programs in GCP, and how to enable Single IP SNAT, verify VPC routes, and disable the feature. Starting with Controller version 9.0, enabling **Single IP SNAT** on an Aviatrix spoke or standalone gateway in a **GCP Regional VPC** programs an additional tagless default route (`0.0.0.0/0` with priority 1000) toward the gateway. That route applies to all workloads in the VPC that do not match a higher-priority tagged route, including Cloud Run, Cloud Functions, App Engine, and untagged VMs, without requiring the `avx-snat-noip` network tag on every instance. > [!NOTE] > > * **Regional VPC only:** Tagless default routing applies when the spoke uses a **Regional VPC** (Global VPC disabled). Global VPC continues to use regional tag-based routes; see [Enabling GCP Global VPC Routing](/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/gcp-global-vpc). > * **Controller 9.0 or later** is required for tagless default route programming with Single IP SNAT on Regional VPC spokes. > * In **Global VPC** mode, the SNAT tag format is `avx-snat-noip-{region}` (for example, `avx-snat-noip-us-west1`). Tagless default routing is **not** supported in Global VPC mode. For related GCP routing and VPC topics, see: * [Enabling GCP Global VPC Routing](/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/gcp-global-vpc) * [Transit Gateway Multi-Peer BGP Over LAN in GCP Workflow](/docs/enterprise/10.1/guides/connectivity/routing/transit-externaldevice-gcp-multipeer-bgpolan) ## Prerequisites * Needs Aviatrix Controller **9.0** or later. * Needs an Aviatrix Spoke or standalone Gateway in a **GCP Regional VPC** (not Global VPC mode). * The GCP account [onboarded](/docs/enterprise/10.1/guides/platform-administration/gcp-account-onboarding) in the Controller should have IAM permissions to create and delete VPC routes. ## Background and Route Behavior Before Controller version 9.0, Single IP SNAT on a GCP spoke relied on instances using the `avx-snat-noip` GCP network tag so the tagged default route applied. Network tags are not supported on serverless compute (Cloud Run, Cloud Functions, and App Engine), so those workloads could not be steered through the gateway the same way as tagged VMs. Starting with Controller version 9.0 on a **Regional VPC**, the Controller installs both the existing tagged SNAT route and a **tagless** default route. The tagged route remains for explicitly tagged VMs; the tagless route covers the rest of the VPC. ### Route priorities when Single IP SNAT is enabled When Single IP SNAT is enabled on a Regional VPC spoke gateway, Aviatrix manages these VPC routes: | GCP route priority | Network tag | Next hop | Traffic affected | | - | - | - | - | | 991 | `avx-snat-noip` | Aviatrix spoke gateway | VM instances explicitly tagged for SNAT (Regional VPC) | | 1000 | *(none — tagless)* | Aviatrix spoke gateway | All remaining instances, including serverless services and untagged VMs | ## Enable Single IP SNAT on a GCP Spoke Gateway To enable Single IP SNAT and activate tagless default route behavior: 1. In CoPilot, go to **Cloud Fabric** > **Gateways** > **Spoke Gateways**. 2. Click the GCP Spoke or standalone Gateway name to open the Gateway Details page. 3. Click the **Settings** tab. 4. Expand **Network Address Translation (NAT)** and turn **Source NAT** to **On**. 5. Click **Single IP**. 6. Click **Save**. When Single IP SNAT is enabled, the Controller automatically: * Programs a SNAT route (priority 991, tagged `avx-snat-noip`) in the VPC pointing to the Spoke Gateway. * Programs a tagless default route (priority 1000, no network tags) in the VPC pointing to the Spoke Gateway. * Adjusts tagging on the existing GCP internet gateway route so the routing hierarchy stays consistent. You do not need to assign GCP network tags on VM instances for default routing behavior through the tagless Route. ## Verify Routes in Google Cloud Console After you enable Single IP SNAT, confirm routes in Google Cloud Console: 1. In the **Google Cloud Console**, go to **VPC network** > **Routes**. 2. Filter routes by the VPC network attached to the Spoke Gateway. 3. Confirm a route to `0.0.0.0/0` with priority 991 and source tag `avx-snat-noip`, with next hop pointing to the Aviatrix Gateway instance. 4. Confirm a second route to `0.0.0.0/0` with priority 1000, **no** network tags, with next hop pointing to the same Aviatrix Gateway instance. ### Verifying Serverless Egress (Optional) From a serverless workload in the same VPC (for example, a Cloud Run service), initiate an outbound connection to an external endpoint and confirm that traffic reaches the expected destination and Source NAT behavior matches your policy. ## Disable Single IP SNAT When you disable Single IP SNAT on the gateway, the Controller removes both the tagged SNAT route and the tagless default route from the VPC. Normal GCP routing resumes through the default internet gateway. 1. Go to **Cloud Fabric** > **Gateways** > **Spoke Gateways**. 2. Click the gateway name to open the gateway details page. 3. Click the **Settings** tab. 4. Under **Network Address Translation (NAT)**, turn **Source NAT** to **Off**. 5. Click **Save**. ## Limitations * **GCP Regional VPC only.** Tagless default route programming applies only when the spoke uses a Regional VPC. Global VPC continues to use regional tag-based routes (`avx-snat-noip-{region}`); serverless workloads in Global VPC are not covered. * **Legacy VPC required.** Single IP SNAT applies only to GCP VPCs configured as legacy VPCs. * **One Single IP SNAT gateway per VPC.** Only one gateway in a VPC can have Single IP SNAT enabled. You cannot enable Single IP SNAT on a different gateway in the same VPC if FQDN egress control is already enabled on another gateway in that VPC, and vice versa. ## Related topics * [Enabling GCP Global VPC Routing](/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/gcp-global-vpc) * [Transit Gateway Multi-Peer BGP Over LAN in GCP Workflow](/docs/enterprise/10.1/guides/connectivity/routing/transit-externaldevice-gcp-multipeer-bgpolan) * [GCP Account Onboarding](/docs/enterprise/10.1/guides/platform-administration/gcp-account-onboarding) # AWS Getting Started Guide Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/controlplane/getting-started-guide-aws Aviatrix provides a product launch experience with automation and a simple UI guided flow to make it easy for you to begin using the product. Aviatrix provides a product launch experience with automation and a simple UI guided flow to make it easy for you to begin using the product. ## Prerequisites Ensure you have enough VPCs, gateways, and elastic IPs available for deployment in your AWS region. See [Preparing for Your AWS Deployment](/docs/enterprise/10.1/guides/controlplane/planning-aws-deployment) for pre-deployment considerations. ## AWS Simple Product Launch Overview The launch experience is currently available for AWS at [launch.aviatrix.com](https://launch.aviatrix.com). The launch experience automates the following: * Creation of a VPC for Controller and CoPilot (or you can select an existing VPC) * Deployment and configuration of IAM * Deployment of Controller and CoPilot into the newly created VPC * Creation of security groups securing platform access to only your IP * Initialization of Controller and CoPilot to current version * Configuration and integration between Controller and CoPilot > [!NOTE] > > The entire process completes in about 15 minutes when orchestrated with > CloudFormation. The guided flow involves four simple steps. The details for each step are outlined in [Launch Step Details](#launch-step-details). **Step 1: Aviatrix License** ![Step 1](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/controlplane/simple-launch-step-1.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=53e365ce51fd10e17e67e09803fbe998) **Step 2: Platform Subscription** ![Step 2](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/controlplane/simple-launch-step-2.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=533aa9107364629427f228382ec13a8e) **Step 3: Configuration** ![Step 3](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/controlplane/simple-launch-step-3.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=305e86b771b1f0b543edbd6245832984) **Step 4: Deployment** ![Step 4](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/controlplane/simple-launch-step-4.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=6cfefd912416f5db466dc461bebc0e2f) **Log in to CoPilot** ![CoPilot login screen](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/controlplane/copilot-login.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=75678b1aebd886426d39b6519e676e0d) The default username is *admin*, and the password is what you entered in the CloudFormation template. ## Required Aviatrix License and Marketplace Offers Three Marketplace subscriptions are required to deploy the Aviatrix platform. You will subscribe to these offerings by accessing the [Launch Aviatrix guided experience](https://launch.aviatrix.com). | **License or Offer** | **Description** | | - | - | | Aviatrix Enterprise Contract | The Aviatrix Enterprise Contract listing provides the license (customer-id) required to active the Aviatrix Cloud Network Controller and Aviatrix Cloud Network CoPilot. A free trial is available with this AWS offer. When the free trial expires, the customer must upgrade to one of the paid options in the listing. | | Aviatrix Cloud Network Controller | This Bring Your Own License (BYOL) offer integrates with the Aviatrix license. | | Aviatrix Cloud Network CoPilot | This Bring Your Own License (BYOL) offer integrates with the Aviatrix license. | If you subscribe to the free trial license, you receive notification emails 14, 7, and 1 day before the free trial expires and billing begins. ## Launch Step Details The Aviatrix launch experience guides you through a quick configuration process to get the Aviatrix Cloud Networking Platform up and running in your AWS environment. You can access the Aviatrix launch experience from [launch.aviatrix.com](https://launch.aviatrix.com). > [!TIP] > > Do not click the browser refresh or back buttons while using Launch Aviatrix. > To go back to a previous page, click on a Step in the Launch navigation. ### Step 1: Register an Aviatrix License 1. In the Launch Aviatrix licensing window, do one of the following: * If you need a license, click **AWS Marketplace** to open the Marketplace listing in a separate tab and continue to the next step. * If you already have a license, enter the key in the **License Key** field and skip to [Platform Subscription](#step-2-platform-subscription). You received the key in an email when you originally registered. The key is not retained anywhere else. If you do not have the key, contact your account manager. 2. On the Marketplace page, click **Try for free**. ![AWS subscription page](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/controlplane/aws-cloud-network-license-subscribe.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=c66f34732e4d972f5f47294455aa78cf) 3. On **Configure your Software Contract**, under Purchase click **Subscribe to free trial**, and then click **Set up your account**. ![AWS subscription page](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/controlplane/aws-cloud-network-license-subscribe-2.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=b7201a4b77d28484b29adf65cf773483) You will be redirected back to the Launch Aviatrix window. 4. Enter your business email address and click **Verify Email**. An email is sent to you from [launch@email.aviatrix.com](mailto:launch@email.aviatrix.com) with the subject line "Verify Your Identity," that contains a verification code. 5. Copy the code from the email, enter it in the field in Launch Aviatrix, and then click **Next**. A success page displays and shows your unique license key. This key and subscription ID are also emailed to you. > [!NOTE] > > This license key is not available anywhere else. Save the key and > subscription ID in a safe place. You need them in the future. 6. Click **Next**. ### Step 2: Platform Subscription Subscribe to Aviatrix Cloud Network Controller and Aviatrix Cloud Network CoPilot. 1. Click **Subscribe to Controller**. The AWS Marketplace listing opens on a separate tab. 2. On the Marketplace page, click **Continue to Subscribe**. 3. On the subscription page, click **Accept Terms**. Your subscription is submitted and *Pending* displays in the table on this page. You can continue to the next step while the subscription process completes, which takes about a minute. > [!NOTE] > > **Do *not* click** Continue to Configuration in AWS Marketplace. 4. Return to Launch Aviatrix and click **Subscribe to CoPilot**. 5. On the Marketplace page, click **Continue to Subscribe**. 6. On the subscription page, click **Accept Terms**. Your subscription is submitted and *Pending* displays in the table on this page. You can continue to the next step while the subscription process completes, which takes about a minute. > [!NOTE] > > **Do *not* click** Continue to Configuration. 7. Return to Launch Aviatrix and check both boxes indicating you subscribed to Controller and CoPilot. 8. Click **Next**. ### Step 3: Configuration 1. Select the AWS region for deployment. 2. Select whether an existing VPC or a new VPC should be used for deployment. If you choose an existing VPC, then during deployment you will be able to select which VPC to use. If you choose a new VPC, the VPC is automatically created for you during deployment. 3. Click **Next**. > [!NOTE] > > An IP address populates automatically based on your client IP. The **/32** is appended to the address to ensure that only the IP that initiated provisioning can initially access the Controller and CoPilot UIs. > > You can change the IP address after provisioning is complete, by modifying AWS > Security Groups. ### Step 4: Deployment 1. Select the configuration method for the deployment, either [CloudFormation](#launch-the-stack-with-cloudformation) or [Terraform](#deploy-with-terraform). The automated CloudFormation method is recommended over the manual Terraform method. #### Launch the Stack with CloudFormation You can subscribe to the Aviatrix platform and launch the Aviatrix stack directly from the Aviatrix launch experience at [launch.aviatrix.com](https://launch.aviatrix.com). It is recommended you use the default settings in the CloudFormation template. You can modify other settings after deployment. Only the following input is required: * VPC CIDR, if you are using an existing VPC * Subnet Availability Zone and CIDR, if you are using an existing VPC * Administrator password > [!WARNING] > > Ensure you have at least 1 VPC, 1 gateway, and two elastic IPs (EIPs) > available in the AWS region you are deploying to. If these are not available, > the deployment will fail. > [!NOTE] > > The IP address pre-populated in the IPv4 CIDR field, under Connection > Configuration, is the IP you are currently using. The **/32** is appended to > the address to ensure that only this user can initially access the Controller. > You can add more users later in CoPilot. To deploy by using CloudFormation, perform the following steps in Launch Aviatrix. 1. In **Step 4: Deployment** in Launch Aviatrix, click **Execute CloudFormation** to open the AWS **Quick create stack** template. 2. If you chose to use an existing VPC for deployment, do the following: 1. Scroll to the **VPC CIDR** field and enter a VPC name. You can get this information from your AWS VPC dashboard. 2. Choose a **Subnet Availability Zone** and enter a **Subnet CIDR** within the selected VPC. 3. Scroll to **Administrator Password** and **Confirm Administrator Password** fields and enter a password. ![Administrator password](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/controlplane/simple-launch-deployment-enter-pwd.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=e195541703f0fba5e4199e4bfa769a36) **Password requirements**: minimum 8 characters with at least 1 upper case, 1 lower case, 1 special character, and 1 number. This is the password you will use to log in to the CoPilot and Controller UIs. The default username is *admin*. 4. Scroll to the **Capabilities** section at the bottom of the Template and check the box, "I acknowledge that AWS CloudFormation might create IAM resources." 5. Click **Create Stack** to run CloudFormation. A browser tab opens to the AWS **CloudFormation > Stacks** page for your stack. > [!NOTE] > > If nothing happens when you click Create Stack, scroll through the AWS > CloudFormation template to check for error messages. 6. To monitor the progress of the stack creation in AWS, see [Monitor the CloudFormation Stack](#monitor-the-cloudformation-stack). ![Acknowledge and create stack](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/controlplane/simple-launch-deployment-ack-create-stack.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=da0d3846d005e2ea6155fed9f10a2b7d) > [!NOTE] > > It takes about 15 minutes for the stack creation to complete. After the CloudFormation stack is created, you can [access your CoPilot instance](#access-the-copilot-ui). #### Deploy with Terraform You can deploy Aviatrix by using Terraform, directly from the Aviatrix launch experience at [launch.aviatrix.com](https://launch.aviatrix.com). > [!WARNING] > > Ensure you have at least 1 VPC, 1 gateway, and two elastic IPs (EIPs) > available in the AWS region you are deploying to. If these are not available, > the deployment will fail. You should be familiar with Terraform modules to use this method. If you choose to deploy with Terraform, you must meet the following prerequisites: * Install [Boto3](https://boto3.amazonaws.com/v1/documentation/api/latest/index.html). * Configure [authentication for AWS](https://docs.aws.amazon.com/cli/v1/userguide/cli-authentication-user.html). * Install a minimum version of [Terraform 0.13](https://www.terraform.io/) and [Python3](https://www.python.org/). To deploy by using Terraform, perform the following steps in Launch Aviatrix. 1. In **Step 4: Deployment** of Launch Aviatrix, click **Deploy with Terraform**. 2. Ensure you have met all the prerequisites. 3. Click **Download Terraform Configuration File** and save the file to an appropriate directory. A Terraform file named *avx-launch.tf* downloads. It is recommended that you install the Terraform modules in a separate directory from the rest of your Terraform pipeline. 4. In the directory where you saved the TF file, run the following commands: ```bash theme={null} terraform init terraform plan terraform apply ``` Enter a vpc\_id from an existing or new VPC. Enter a public subnet\_id within the selected VPC. You can get this information from your AWS VPC dashboard. 5. To monitor the progress of the stack creation, see [Monitor the CloudFormation Stack](#monitor-the-cloudformation-stack). After the CloudFormation stack is created, you can [access your CoPilot instance](#access-the-copilot-ui). Advanced Terraform configurations can be implemented using the examples provided in the [GitHub repository](https://github.com/AviatrixSystems/terraform-aviatrix-aws-controlplane). #### Monitor the CloudFormation Stack You can monitor CloudFormation progress from the AWS CloudFormation console. The CloudFormation page in AWS should automatically launch after you submit the CloudFormation or Terraform templates. 1. If not logged into AWS, log into your account in the target region and go to **CloudFormation > Stacks**. 2. Select the name of the stack being created and then click the **Resources** tab. The default stack name is **AviatrixControlplane**. > [!TIP] > > Use the Refresh button to show more frequent updates to the table. 3. Scroll through the Resources table to locate the **AviatrixPlatformInitStepFunction** entry and click on the associated Physical ID link. It takes about two minutes for the step function entry to display in the table. ![Step function](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/controlplane/simple-launch-step-fn-click.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=c7c0292ce3d111cf4c75fecf509d5bde) 4. On the step functions page, click the **Executions** tab, and then click the link in the table for the running task. 5. Under **Graph View**, you can watch the progress of each step. **Wait until all steps turn green.** ![Step function progress 1](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/controlplane/simple-launch-step-fn-1.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=8a3859d932b3e2ec9fb96b25f14662ae) ![Step function progress 2](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/controlplane/simple-launch-step-fn-2.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=9d16393779588448778c707bf6d05bf1) When all steps are **green**, the process is complete. This typically takes about 15 minutes. You can now [access your CoPilot instance](#access-the-copilot-ui). ## Access the CoPilot UI **Congratulations!** You have successfully launched your Aviatrix platform. You can obtain the CoPilot UI URL in AWS. 1. Go to **CloudFormation > Stacks >** *\* and click the **Outputs** tab. The default stack name is *AviatrixControlplane*. 2. Scroll down to locate **AviatrixCoPilotURL** in the Key column and click the associated link under **Value**. The address displayed in the table is the one you will use to access CoPilot. ![CFT output 2](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/controlplane/simple-launch-cft-output.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=85321e07b0717f75392b8b78b640f8a4) If the browser displays a *Your connection is not private* message, click **Advanced** and **Proceed**. 3. Enter the username and the password you defined in CloudFormation and click **Log In** to open the CoPilot UI. > [!NOTE] > > The default username for the UI is *admin*, and the password is what you > entered in the CloudFormation Quick Create template. 4. In CoPilot, you can access Controller by clicking the **Application Information** icon at the top right and selecting **Aviatrix Controller**. You can use the CoPilot credentials to log in to Controller. ## Access the Controller UI You can access Aviatrix Controller from the CoPilot UI. By default, the credentials are the same for CoPilot and Controller. 1. Log into your Aviatrix CoPilot account. 2. Click the **Application Information** icon in the top right. 3. Select the **Aviatrix Controller** IP address. 4. Enter your username and password to log into the Controller. ## Troubleshooting if Stack Creation Fails * In AWS CloudFormation > Stacks, select the stack, select the Events tab, and examine the *Status reason* column for a possible cause. * Ensure that you are subscribed to [required marketplace subscriptions](#required-aviatrix-license-and-marketplace-offers). * If you have run the stack more than once, ensure that a unique new name is provided for the stack name. The default is AviatrixControlplane. * Ensure your AWS region has at least 1 VPC, 1 gateway, and 2 elastic IPs (EIPs) available for deployment. ## Optional Post-Deployment Steps Complete the following optional post-deployment steps if they meet your requirements. ### Modifying Accidental Termination for AWS Instances You can enable or disable the AWS option to protect against accidental termination of your EC2 instances. You cannot terminate an instance if termination protection is enabled. 1. In your AWS account, go to the **EC2 > Instances** page. 2. Locate and mark the checkbox for your Aviatrix Controller or CoPilot instance. You can only modify the Instance Settings for one instance at a time. If you select multiple instances, the menu items under Instance Settings are not available. 3. Go to **Actions > Instance Settings > Change termination protection**. 4. Check **Enable** or **Disable** and click **Save**. 5. Repeat these steps for other instances. ### Disabling Security Group Management Access Security Group Management is enabled by default. This feature ensures that newly created gateways can access the Controller. You can disable Security Group Management access for your Controller, if needed. > [!NOTE] > > When this feature is enabled, keep each Controller Security Group's outbound > rules at their default, open to Internet or All. This allows your Controller's > IP address to access the Internet. 1. In Aviatrix CoPilot, go to **Settings > Configuration**. 2. Under **Security**, locate the **Controller Security Group Management** card. 3. Select the Controller **Account**. 4. Set the toggle to **Off**. # Azure Getting Started Guide Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/controlplane/getting-started-guide-azure Aviatrix provides a product launch experience with automation and a simple UI guided flow to make it easy for you to begin using the product. Aviatrix provides a product launch experience with automation and a simple UI guided flow to make it easy for you to begin using the product. ## Prerequisites Ensure you have met the prerequisites for an Azure deployment. See [Preparing for Your Azure Deployment](https://legacy.docs.aviatrix.com/documentation/latest/getting-started/planning-azure-deployment.html) for pre-deployment requirements and considerations. ## Azure Simple Product Launch Overview The launch experience is currently available for Azure at [launch.aviatrix.com](https://launch.aviatrix.com). The launch experience automates the following: * Creation of a VNet for Controller and CoPilot (or you can select an existing VNet) * Deployment and configuration of IAM * Deployment of Controller and CoPilot into an existing or newly created VNet * Creation of network security groups securing platform access to only your IP * Initialization of Controller and CoPilot to current version * Configuration and integration between Controller and CoPilot > [!NOTE] > > The entire process completes in about 20 minutes when orchestrated with > Terraform. The guided flow involves four simple steps. The details for each step are outlined in [Launch Step Details](#launch-step-details). **Step 1: Aviatrix License** ![Step 1](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/controlplane/simple-launch-step-1-azure.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=c28e6b25fe60aa9bf0cb313614df23ca) **Step 2: Platform Subscription** ![Step 2](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/controlplane/simple-launch-step-2-azure.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=abea45c7649ae3b8d691604bba1b4b56) **Step 3: Configuration** ![Step 3](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/controlplane/simple-launch-step-3-azure.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=a35494245f98e25afae2a94f8fa8394f) **Step 4: Deployment** ![Step 4](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/controlplane/simple-launch-step-4-existing-vnet-azure.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=ca3d358fbea20e8b39abd3f14945135e) **Log in to CoPilot** ![CoPilot login screen](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/controlplane/copilot-login.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=75678b1aebd886426d39b6519e676e0d) The default username is *admin*, and the password is what you entered during the Terraform implementation. ## Required Aviatrix License and Marketplace Offers Three Azure Marketplace subscriptions are required to deploy the Aviatrix platform. You will subscribe to these offerings by accessing the [Launch Aviatrix guided experience](https://launch.aviatrix.com). For new deployments, you will subscribe to the following: | **License or Offer** | **Description** | | - | - | | **Aviatrix Enterprise Contract** | The Aviatrix Enterprise Contract provides the license (customer-id) needed to access the Aviatrix Cloud Network Controller and Aviatrix Cloud Network CoPilot. This service also calculates Aviatrix bills based on usage. | | **Aviatrix Cloud Network Controller** | This Bring Your Own License (BYOL) offer integrates with the Aviatrix license. | | **Aviatrix Cloud Network CoPilot** | This Bring Your Own License (BYOL) offer integrates with the Aviatrix license. | > [!NOTE] > > If you subscribe to the free trial license, you receive notification emails > 14, 7, and 1 day before the free trial expires and billing begins. ## Launch Step Details The Aviatrix launch experience guides you through a quick configuration process to get the Aviatrix Cloud Networking Platform up and running in your Azure environment. You can access the Aviatrix launch experience from [launch.aviatrix.com](https://launch.aviatrix.com). > [!TIP] > > Do not click the browser refresh or back buttons while using Launch Aviatrix. > To go back to a previous page, click on a Step in the Launch navigation. ### Step 1: Register an Aviatrix License 1. In the Launch Aviatrix licensing window, do one of the following: * If you need a license, click your preferred **CSP Marketplace** to open the Marketplace listing in a separate tab and continue to the next step. * If you already have a license, enter the key in the **License Key** field and skip to [Platform Subscription](#step-2-platform-subscription). You received the key in an email when you originally registered. The key is not retained anywhere else. If you do not have the key, contact your account manager. 2. On the Aviatrix Enterprise Contract page, review the Plans + Pricing and click **Get It Now**. ![Step 4](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/controlplane/azure-marketplace-listing-1.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=f9f5558f041d05274b75254861cf293e) All plans start with a 30-day free trial. 3. Read and accept the acknowledgment notice and click **Continue**. 4. On the Aviatrix Enterprise Contract page, select the **Plan** you want and click **Subscribe**. 5. On Subscribe To Aviatrix Enterprise Contract, enter the name for a new resource group and verify the resource group location. 6. Enter a descriptive name for the SaaS subscription and click **Review + Subscribe**. 7. Review the subscription details and click **Subscribe**. 8. When the subscription process completes, click **Configure account now**. You will be redirected back to Launch Aviatrix. 9. Enter your business email address and click **Verify Email**. An email is sent to you from [launch@email.aviatrix.com](mailto:launch@email.aviatrix.com) that contains a verification code. 10. Copy the code from the email, enter it in the field in Launch Aviatrix, and then click **Next**. A success page displays and shows your unique license key. This key and a subscription ID are also emailed to you. > [!WARNING] > > This license key is not available anywhere else. Save the key and > subscription ID in a safe place. You might need them in the future. 11. Click **Next**. ### Step 2: Platform Subscription Review the information about Aviatrix Cloud Network Controller and Aviatrix Cloud Network CoPilot. They will be onboarded in a later step. 1. Click **Next**. ### Step 3: Configuration Provide configuration details for the Terraform template. 1. Select the Azure region for deployment. 2. Confirm that the IP address you are currently using is displayed in the IP field. Your IP address populates automatically based on your client IP. The **/32** is appended to the address to ensure that only the IP that initiated provisioning can initially access the Controller and CoPilot UIs. If needed, you can change the IP address after provisioning is complete. 3. Click **Next**. 4. Select whether an existing VNet or a new VNet should be used for deployment. If you choose an existing VNet, then during deployment you will be able to select which VNet and subnet to use. If you choose a new VNet, the VNet and Resource Group are automatically created for you during deployment. ### Step 4: Deployment You can deploy Aviatrix Platform by using Terraform. You do not need to be familiar with Terraform modules to use this method. However, to deploy with Terraform, you must have done the following: * Installed the [Azure CLI](https://learn.microsoft.com/en-us/cli/azure/install-azure-cli). * Installed a minimum version of [Python3](https://www.python.org/). * Installed a minimum version of [Terraform 0.13](https://www.terraform.io/). If using an existing VNet you need the following information available: * Name of the target resource group in your target region * Name of the VNet in the resource group * Name of the subnet in the VNet * Subnet ID of the subnet > [!TIP] > > If you need assistance locating the required VNet information in the [Azure > portal](https://portal.azure.com/#browse/Microsoft.Network%2FvirtualNetworks), > see [Information Required for Azure Deployment to an Existing > VNet](https://legacy.docs.aviatrix.com/documentation/latest/getting-started/planning-azure-deployment.html#azure-details-existing-vnet). You can monitor provisioning details in the terminal where you run the Terraform commands. To see a list of the resources created by Terraform, go to [Resources Created in Azure](https://legacy.docs.aviatrix.com/documentation/latest/getting-started/planning-azure-deployment.html#azure-resources-terraform). Perform the following steps in Launch Aviatrix. 1. Click **Download Terraform Configuration File** to download *aviatrix-launch.tf*. 2. Modify the following lines in the *aviatrix-launch.tf* file and save the file: ``` resource_group_name = "CHANGEME" # Put your resource group name vnet_name = "CHANGEME" # Put your VNet name subnet_name = "CHANGEME" # Put your Subnet name subnet_id = "CHANGEME" # Put your subnet ID ``` For steps to get this information in the [Azure portal](https://portal.azure.com/#browse/Microsoft.Network%2FvirtualNetworks), see [Information Required for Azure Deployment to an Existing VNet](https://legacy.docs.aviatrix.com/documentation/latest/getting-started/planning-azure-deployment.html#azure-details-existing-vnet). 3. In a terminal window, run the following commands in the *directory in which you saved the TF file*: 1. Log in to Azure. ``` az login ``` In the terminal output, you will need to select the Azure subscription to associate with the Aviatrix account. 2. Initialize Terraform. ``` terraform init ``` 3. Preview a list of the changes (execution plan) Terraform will implement. ``` terraform plan ``` If you get a message about the *-out option*, you can safely ignore it and continue. 4. Apply the changes and enter **Yes** when requested. ``` terraform apply ``` > [!TIP] > > The *terraform apply* command previews the execution plan again. You can > use the *--auto-approve* option to apply the execution plan without a > second confirmation. > [!NOTE] > > It takes about 14 minutes for the Terraform process to complete. 4. Copy and save the output that results from the Terraform execution. *Example:* `"Copilot_management_login" = <"https://172.31.255.255/">` `"password" = <";a1wkE:P#mad3,*f">` `"username" = "admin"` You will use this information to log in to the Aviatrix Platform. > [!TIP] > > If you want to view the newly created resources in the Azure portal, go to > **Resource groups** and select the new resource group. After the Aviatrix Controller and CoPilot are configured, you can [access your CoPilot instance](#access-the-copilot-ui). ## Access the CoPilot UI **Congratulations!** You have successfully launched your Aviatrix platform. 1. In a browser window, enter the IP address for **Copilot\_management\_login** that you copied from the Terraform output. *Example*: "Copilot\*management\_login" = \*\<"[https://172.31.255.255/"\\>](https://172.31.255.255/"\\>)\_ 2. In the CoPilot login window, enter your credentials: * Username: The default is *admin*. * Password: Use the password from the Terraform output. *Example*: "password" = *\<";a1wkE:P#mad3,\*f">* ### Next Steps Check the [Post Deployment](https://legacy.docs.aviatrix.com/documentation/latest/getting-started/post-deployment.html) documentation for any tasks you might want to perform after launching CoPilot. # Deploy Aviatrix Controller through CSP Consoles Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/controlplane/manual-launch-controller-csp This guide provides step-by-step instructions for manually launching the Aviatrix Controller in AWS, Azure, and GCP using their respective cloud consoles or CLI tools. This guide provides step-by-step instructions for manually launching the Aviatrix Controller in AWS, Azure, and GCP using their respective cloud consoles or CLI tools. * [AWS Deployment](#aws-deployment) * [Azure Deployment](#azure-deployment) * [GCP Deployment](#gcp-deployment) * [Post-Deployment Setup](#post-deployment-tasks) ## Prerequisites * An active AWS, Azure, or GCP account with the required permissions * IAM roles and policies * Security group or firewall rules allowing required ports ## Deploy Aviatrix Controller in AWS Console ### Create IAM Role 1. Sign in to the **AWS IAM Console**. 2. Navigate to *Roles* → *Create Role*. 3. Select **EC2** as the trusted entity. 4. Attach the Aviatrix IAM Policy. See [AWS IAM Policies](/docs/enterprise/10.1/reference/general/iam-role). ### Launch EC2 Instance from AMI 1. Go to the AWS EC2 Console. 2. Click *AMIs* on the left-side menu. 3. Search for `avx-controller-g5` from the *Public images* tab. 4. Select the latest image. 5. Click *Launch instance from AMI*. ### Configure Instance Details Follow the steps below to configure the instance details: 1. Enter an instance name. 2. Choose an instance type (must be `t3.large` or larger). 3. Select or create a key pair. Download and store the `.pem` or `.ppk` file securely. 4. Network settings: 1. Choose the appropriate VPC and subnet with a route to the Internet. 2. Enable auto-assign Public IP. 3. Create or select a security group allowing inbound traffic on ports 22 and 443. 5. Storage configuration as needed. 6. Under *Advanced Details*: 1. Set IAM Role to `aviatrix-role-ec2`. 7. Review and launch the instance. Wait until its status is `running`. ### (Optional) Associate Elastic IP with your Instance It is recommended to associate an Elastic IP with your instance for a stable public IP address. 1. Navigate to the **Elastic IPs** section in the EC2 Console. 2. Click *Allocate Elastic IP address*. 3. Select the instance and click *Associate Elastic IP address*. 4. Select the EC2 instance you launched and click *Associate*. ## Deploy Aviatrix Controller in Azure Portal This section provides instructions for deploying the Aviatrix Controller in Azure using the Azure Portal. ### Launch from Azure Marketplace 1. Sign in to the **Azure Portal**. 2. Click *Create a resource*. 3. Search for `Aviatrix Controller` and select **Aviatrix Cloud Network Controller**. 4. Click *Create > Aviatrix Controller G5*. ### Configure VM Settings 1. Choose an existing Resource Group or create a new one. 2. Enter a VM Name. 3. Choose a Region (such as, `East US`). 4. Choose an availability zone. 5. Choose a security type (such as, `Standard`). 6. Choose a VM size (must be `Standard_D2s_v3` or higher). 7. Select Authentication type and SSH key. 8. For the inbound port rules, allow: * HTTPS (443) * SSH (22) 9. Under *Networking*, choose Virtual Network, Subnet and Public IP. 1. Ensure the subnet has a route to the internet (NAT Gateway or Internet Gateway). 2. For the public IP, you can either create a new one or use an existing one. 3. For the inbound port rules, ensure the following ports are allowed: * SSH (22) * HTTPS (443) 10. Click *Review + Create* to complete the deployment. ## Deploy Aviatrix Controller in GCP CLI This section provides instructions for deploying the Aviatrix Controller in GCP using the GCP CLI. > [!NOTE] > > Due to recent changes in the Google Cloud Platform (GCP) portal, deploying the > Aviatrix Controller through the portal UI is currently not supported. This > section provides instructions for CLI-based deployment as an alternative. ### Prerequisites * A Google Cloud project with billing enabled * IAM permissions (Compute Admin and Service Account Admin) * An SSH key pair * Network and firewall rules allowing access ### Install Google Cloud CLI Make sure you have the Google Cloud SDK (gcloud CLI) installed on your local machine. * Official installation guide: [https://cloud.google.com/sdk/docs/install](https://cloud.google.com/sdk/docs/install) To verify that the gcloud CLI has been installed, run the following command in your terminal: ```bash theme={null} gcloud --version ``` ### Initialize Google Cloud CLI After installing, authenticate and set your default project: ```bash theme={null} gcloud init ``` This command will: * Open a browser to authenticate your Google account. * Let you choose your Google Cloud project. * Optionally set default region/zone. ### Review the VM Image Starting in 10.1.0, the Aviatrix Controller image is published to the public `aviatrix-public` Google Cloud project. To list the available Aviatrix Controller images, run: ```bash theme={null} gcloud compute images list --project=aviatrix-public ``` Choose the image you want to deploy and note its full resource path. It will resemble this format: ``` projects/aviatrix-public/global/images/avx-controller-g5-202608212335 ``` ### Deploy via gcloud CLI Use this command to deploy a VM with the Aviatrix Controller image. Replace ``, ``, and `` with your actual values. ```bash theme={null} gcloud compute instances create \ --zone= \ --machine-type= \ --image=projects/aviatrix-public/global/images/avx-controller-g5-202608212335 ``` Example: ```bash theme={null} gcloud compute instances create avx-controller \ --zone=us-central1-c \ --machine-type=n2-standard-4 \ --image=projects/aviatrix-public/global/images/avx-controller-g5-202608212335 \ --metadata=user-data=' #cloud-config avx-controller: avx-controller-version: 10.1.0 ' ``` ### Create Firewall Rules Open the required ports to access your VM. #### Allow SSH (port 22) Run the following command to create a firewall rule that allows SSH access, where `` is your Google Cloud project ID. ```bash theme={null} gcloud compute firewall-rules create allow-ssh \ --network=default \ --allow=tcp:22 \ --source-ranges=0.0.0.0/0 \ --project= ``` #### Allow HTTPS (port 443) Run the following command to create a firewall rule that allows HTTPS access, where `` is your Google Cloud project ID. ```bash theme={null} gcloud compute firewall-rules create allow-https \ --network=default \ --allow=tcp:443 \ --source-ranges=0.0.0.0/0 \ --project= ``` #### Optional: Internal Traffic Run the following command to allow internal traffic within your GCP network. This is useful for communication between VMs in the same network, where `` is your Google Cloud project ID. ```bash theme={null} gcloud compute firewall-rules create allow-internal \ --network=default \ --allow=tcp,udp,icmp \ --source-ranges=10.128.0.0/9 \ --project= ``` ### (Optional) Set Up the GCP Service Account After deploying your Controller VM, create a Service Account for Aviatrix to manage your GCP environment. #### Create Service Account You can use an existing Service Account or create a new one specifically for the Aviatrix Controller. If you choose to create a new one, follow these steps: 1. Go to the **Google Cloud Console**. 2. Go to *IAM & Admin > Service Accounts* in the Cloud Console. 3. Click **+ CREATE SERVICE ACCOUNT**. 4. Fill in: * Service account name * Service Account ID * Description: Service Account for Aviatrix Controller 5. Click **CREATE AND CONTINUE**. #### Assign Required Roles Grant the following roles: * Compute Admin * Service Account User Click **DONE** to finish. #### Create and Download JSON Key 1. Click the service account you created. 2. Go to the *Manage Keys* tab. 3. Click *ADD KEY > Create new key*. 4. Choose **JSON**. 5. Click **CREATE**. This will download the JSON key file to your computer. Store it securely. ## Post-Deployment Setup After deploying the Aviatrix Controller, you need to complete the initial setup to configure it for your environment. ### Initial Setup of Aviatrix Controller Follow these steps to complete the setup of your Aviatrix Controller after deployment: 1. Enter the public IP of the instance in your browser to access the Controller UI. 2. Login the Controller UI using the default credentials: * Username: `admin` * Password: `` 3. Follow the setup wizard to change the Controller UI login password. 4. Enter email address for password recovery. 5. Enter your Customer ID. 6. Click **RUN** to complete the Initial Setup. ### Onboard Cloud Accounts After the Controller is set up, you can onboard your cloud accounts to manage them through the Aviatrix Controller. Follow the specific onboarding guides for each cloud provider. #### Onboard your AWS Account in Aviatrix Controller After the instance is running, you need to onboard your AWS account in the Aviatrix Controller. See [Onboard AWS Account](/docs/enterprise/10.1/guides/controlplane/onboard-account-controller-aws) for detailed instructions. #### Onboard your Azure Account in Aviatrix Controller After the VM is created, you need to onboard your Azure account in the Aviatrix Controller. See [Onboard Azure Account](/docs/enterprise/10.1/guides/controlplane/getting-started-guide-azure) for detailed instructions. #### Onboard your GCP Account in Aviatrix Controller Access your Aviatrix Controller Web UI via its External IP: `https://` 1. Go to **Cloud Accounts**. 2. Click **Add New Account**. 3. Select **GCP** as the Cloud Type. 4. Upload the **JSON Key File** you downloaded. 5. Name the account and save. The Controller now has permissions to manage GCP resources in your project. See [Onboard GCP Account](/docs/enterprise/10.1/guides/platform-administration/gcp-account-onboarding) for more detailed instructions. # Aviatrix MCP for Docs Source: https://docs.aviatrix.com/docs/enterprise/10.1/solutions/ai-agent-integrations/mcp-for-docs Connect an MCP-compatible AI agent to the Aviatrix documentation MCP server to search product documentation without leaving your IDE, terminal, or AI assistant. The Aviatrix documentation website is built for reading and learning: browse a topic, read the page, follow links to related concepts. The Aviatrix MCP for Docs server is built for a different moment: when you are already in your IDE, terminal, or AI assistant and need an answer without switching context. ## What This Is Aviatrix hosts an MCP server at `docs.aviatrix.com/mcp` that exposes the Aviatrix Documentation knowledge base to any MCP-compatible AI agent. Connect your agent once, and it can search product documentation, generate configuration, and answer product questions directly inside your current workflow, without opening a browser tab. > [!NOTE] > > MCP (Model Context Protocol) is a Linux Foundation open standard for AI agent > tool access. Once you configure the server endpoint in your MCP client, the > agent discovers and invokes available tools automatically. ## When to Use MCP vs. the Docs Website | | Docs website | Aviatrix MCP for Docs | | - | - | - | | Where you use it | Browser | Your IDE, terminal, or AI assistant | | Workflow | Leave your work to open docs, then return | Stay in your current task | | Context awareness | None — you copy answers back manually | The agent can combine your question with your code or task | | Best for | Reading and learning a topic in depth | Doing work: generating configuration, troubleshooting, finding commands | If you are exploring a new feature or learning how a concept fits together, the docs website remains the better tool. If you are mid-task and need a fast, specific answer, connect your agent to the MCP server instead. ## Available Tool The server exposes one tool: | Tool | Parameters | Returns | | - | - | - | | `search_aviatrix_documentation` | `query` (required), `version` (optional, for example `8.2`) | Up to 10 results, each with title, content excerpt, and link | Ask your agent a question in natural language. The agent calls `search_aviatrix_documentation` on your behalf and incorporates the results into its response. ## Connect Your Agent **1. Choose your MCP client** The Aviatrix MCP for Docs server works with any MCP-compatible client, including Claude Code, Claude Desktop, and Cursor. **2. Add the remote server endpoint** Configure your client to connect to `https://docs.aviatrix.com/mcp` using the MCP Streamable HTTP transport. In Claude Code, run: ```bash theme={null} claude mcp add --transport http aviatrix-docs https://docs.aviatrix.com/mcp ``` > [!NOTE] > > Connect to the remote hosted endpoint directly. Local MCP server > packages for this integration are not supported. **3. Restart or refresh your client** Restart your MCP client, or refresh the connectors list, so the new server is picked up. **4. Ask a question** Ask your agent a product question, for example "What are the Distributed Cloud Firewall group limits?" The agent searches Aviatrix documentation and responds with a cited answer. ## Example Questions * "Is the Onboard All Route Tables feature documented?" * "What are the latest Controller and CoPilot versions?" * "How do I configure Transit Egress?" * "Show me the API reference for version 8.2." # Aviatrix MCP for Enterprise Fabrics Source: https://docs.aviatrix.com/docs/enterprise/10.1/solutions/ai-agent-integrations/mcp-for-enterprise-fabrics Connect an MCP-compatible AI agent to the Aviatrix MCP Server for authenticated, read-only access to Controller and CoPilot data: topology, policy, flow intelligence, and diagnostics. Controller and CoPilot already hold the answer to most network and security questions your team asks: what does this SmartGroup cover, is there an active IPS alert, why is this S2C tunnel down. Getting that answer today means opening a dashboard, navigating to the right view, and cross-referencing policy against topology by hand. The Aviatrix MCP for Enterprise Fabrics server gives your AI agent direct, authenticated access to that same data, so it can answer the question in a single turn instead of sending you back to the UI. ## What This Is The Aviatrix MCP Server is an Aviatrix-hosted MCP server at `platform.mcp.aviatrix.com` that gives AI agents authenticated access to your Controller and CoPilot data across topology, policy, flow intelligence, and diagnostics. It calls the same Controller and CoPilot APIs the UI uses, so there are no gateway changes, no Terraform changes, and no new version requirement to adopt it. > [!NOTE] > > This server is generally available. Aviatrix continues to expand its tool > coverage (see [What's Next](#whats-next) for the current roadmap). ## What Your Agent Can Do The server exposes 51 tools across four domains: | Domain | Tools | Representative capabilities | | - | - | - | | Inventory and topology | 8 | Gateway list and detail, cloud workloads, S2C connections, FireNet inspection, AWS TGW attachments and domains | | Policy and security | 15 | DCF groups and policies, firewall log search and aggregation, IPS alerts, audit trail, policy hit stats, enforcement state, gateway performance, gateway throughput (bps) | | Flow intelligence | 6 | FlowIQ top talkers, spoke egress bandwidth, top egress destinations, IP classification, fabric CIDR enumeration, IP ownership lookup | | Diagnostics and operations | 22 | FlightPath (3 variants + instance discovery), ping, traceroute, BGP diagnostics (async), S2C diagnostics and logs, syslogs (sync and async), controller logs, stateful sessions, connectivity testing | Ask your agent a question in natural language, for example "What DCF policies cover the production VPCs, and are there any IPS alerts in the last 24 hours?" The agent queries groups, policies, and intrusion data in a single turn and can follow up with an action like running FlightPath between two spokes. ## Permission Scoping API keys are issued with an explicit scope at creation time, and scope is enforced server-side, not by agent instruction: * **`controller:read`** (default): 50 read-only tools covering inventory, policy, flow intelligence, and diagnostics. * **`controller:write`**: adds one operational tool, `aviatrix_reset_s2c_connection`. A `controller:read` key cannot invoke `aviatrix_reset_s2c_connection` regardless of what the agent requests. > [!WARNING] > > Only issue a `controller:write` key to an agent or user that needs to reset > S2C connections. Default to `controller:read` for all other use cases. > [!NOTE] > > The set of tools gated behind `controller:write` is expected to grow as > Aviatrix adds more operational capabilities to the server. Review the tool > list for any key with write scope after an MCP server update, rather than > assuming its capabilities are fixed at issuance time. ## Security * **Transport:** All communication between your MCP client and the Aviatrix MCP Server uses TLS over HTTPS (Streamable HTTP transport). * **Authentication:** OAuth 2.1 with PKCE. Your agent authenticates on every session. * **Credential storage:** Your Controller and CoPilot credentials are encrypted inside the Bearer token itself using AES-256. They are never stored in a database. The plaintext exists only in memory for the duration of an API call. * **API keys:** Encrypted at rest using AES-256. Revoke a key at any time from the login portal. * **Scope enforcement:** Unauthorized tools are absent from the tool list, not marked as "access denied." A `controller:read` key cannot discover write-only tools. * **Session lifecycle:** CoPilot sessions expire after approximately one hour. OAuth-capable clients (Claude Code, Claude Desktop) re-authenticate automatically with no user action required. * **Tool-call logging:** Each tool call writes one structured JSON line to CloudWatch in Aviatrix's AWS account. The line holds the tool name, timing, outcome, the CoPilot username used to sign in, the Controller hostname, and your organization's name where Aviatrix has it on record. Tool arguments are not logged. The line holds a hash of them instead. Error messages returned by your Controller are logged and can include gateway names or IP addresses. Sign-in events record the CoPilot URL, username, and source IP. Logs are kept for 90 days, and customers do not have access to them. ## Network Requirements The Aviatrix MCP Server connects to your Controller and CoPilot from a static IP address. Allow inbound HTTPS (port 443) from the following address on both your Controller and CoPilot network security groups: | Environment | Source IP | | - | - | | Production | `3.134.16.45` (`us-east-2`) | > [!NOTE] > > Both Controller and CoPilot must allow this IP. The MCP Server calls CoPilot > for policy, flow, and topology data, and calls Controller directly for > diagnostics such as syslog retrieval, ping, and traceroute. ## Connect Your Agent **1. Request access** Contact your Aviatrix account team to enable the Aviatrix MCP Server for your Controller and CoPilot. Aviatrix allowlists your environment before any API key can be issued. **2. Provision an API key** Once your environment is enabled, go to the Aviatrix MCP login portal and create an API key. Choose `controller:read` unless your use case requires resetting S2C connections or running BGP diagnostics. **3. Configure your MCP client** Add the Aviatrix MCP Server endpoint to your MCP client configuration. In Claude Code, run: ```bash theme={null} claude mcp add --transport http aviatrix-fabric https://platform.mcp.aviatrix.com/mcp ``` Any MCP-compatible client works, including Claude Code, Claude Desktop, and Cursor. **4. Authenticate** Your agent completes an OAuth 2.1 with PKCE authentication flow on connection. Approve the request when prompted. **5. Ask a question** Ask your agent about your environment, for example "What is the current Blast Radius for the production AI workload VPCs?" The agent selects and calls the relevant tools and returns a structured answer. ## What's Next | Phase | Status | Scope | | - | - | - | | 1 | Shipped | DCF visibility: groups, policies, firewall logs, IPS alerts, audit trail | | 2 | Shipped | Operational depth: diagnostics, S2C suite, BGP, FlightPath, syslogs, egress bandwidth | | 3 | Shipped | Flow intelligence: FlowIQ analysis, IP classification, fabric CIDR enumeration, egress annotation | | 4 | Planned | Write operations with safety gates, RBAC, multi-tenant isolation | ## Security FAQ > **Does the AI model (Claude, GPT, etc.) see my Controller password?** > > No. Your credentials are encrypted inside the Bearer token and decrypted > only by the Aviatrix MCP Server at runtime. The AI model receives > structured tool results (gateway lists, policy data, diagnostic output), > never credentials. > **Where are my credentials stored?** > > Inside the encrypted Bearer token itself, not in a database. The token uses > AES-256 encryption. The plaintext password exists only in Lambda memory for > the duration of an API call (typically under one second), then is discarded. > **Can the AI model modify my network configuration?** > > With a `controller:read` key (the default), no. Read-only keys expose 50 tools > that can query data but cannot change state. The only state-changing tool > (`aviatrix_reset_s2c_connection`) requires an explicit `controller:write` key > and is hidden from read-only sessions. > **What happens when an API key is revoked?** > > The key is invalidated immediately. Any in-flight or subsequent requests using > that Bearer token will fail authentication. No grace period. > **Does the MCP Server store conversation history or query results?** > > No. The MCP Server does not store conversation content, and tool results are > not written to its logs. Two asynchronous tools are the exception: the BGP > diagnostic and the gateway syslog download store their result in a database > so your client can collect it when it is ready. That record expires after one > hour and is deleted automatically, typically within a few days of expiring. > Each tool call also writes a log line with the tool name, timing, and > outcome, kept for 90 days. > **What network access does the MCP Server have to my environment?** > > The MCP Server connects to your Controller and CoPilot on port 443 only, from > a single static IP (`3.134.16.45`). All other outbound ports are blocked by > policy. The server cannot reach any other hosts in your environment. > > Outside your environment, the server calls one third-party service, > ipinfo.io, over HTTPS to identify who owns an IP address. It sends the public > source IP of each client that connects to the MCP Server, and, from the flow > analysis and IP lookup tools, IP addresses outside the RFC 1918 private > ranges. If any of your internal networks use addresses outside RFC 1918, > those can be included. No gateway names, credentials, or other data are > sent. > **What happens to data retrieved from my environment?** > > Tool results (gateway lists, firewall logs, diagnostic output) are > returned to the MCP client over TLS in a single response, and the server > does not index them. Most results exist only in Lambda memory for the > duration of the request. The asynchronous BGP diagnostic and gateway syslog > download are the exception: they store their result in an encrypted > database so your client can collect it, and that record expires after one > hour, as described above. Error messages > returned by your Controller are written to the server's logs and can > include gateway names or IP addresses. ## Related Reading For guidance on securing the MCP servers your own team deploys (not the Aviatrix MCP Server itself), see [AI Agent Workload Containment](/docs/enterprise/10.1/solutions/security-for-ai/ai-agent-workload-security) and [MCP Server Security Through Containment Architecture](/docs/enterprise/10.1/solutions/security-for-ai/mcp-server-egress-security) . # Suggested Use Cases for the Aviatrix Enterprise MCP Server Source: https://docs.aviatrix.com/docs/enterprise/10.1/solutions/ai-agent-integrations/mcp-suggested-use-cases Validated prompts and example outputs for using an AI assistant with the Aviatrix MCP server, covering health checks, security audits, CVE exposure analysis, upgrade planning, incident response, and infrastructure-as-code. The Aviatrix MCP server gives an AI assistant read access to your Controller and CoPilot: gateways, tunnels, BGP, Distributed Cloud Firewall (DCF) policies, SmartGroups, firewall and intrusion logs, audit trails, traffic analytics, and network diagnostics such as ping, traceroute, and FlightPath. This page collects prompts that have been validated against a live Aviatrix fabric, organized by the role most likely to run them, with abridged example outputs. > [!NOTE] > > All example outputs on this page come from a demo fabric: 25 gateways across > AWS, Azure, GCP, and OCI, with 5 full-mesh transits, 3 edge sites, 47 > SmartGroups, and a 30-policy DCF chain. Threat indicators shown, including > domains such as `malware.net` and exploit source IPs, are synthetic data > seeded into the demo environment. Your results reflect your own fabric. > [!NOTE] > > The depth and formatting of responses varies with your MCP client and the > model behind it. The prompts below describe what the MCP server returns; how > your assistant assembles, summarizes, and renders that data depends on the > client you use. ## Start here: fabric health sweep The best first prompt after connecting the MCP server. It exercises gateway inventory, tunnel state, Site-to-Cloud (S2C) connections, BGP diagnostics, controller logs, and DCF in a single pass, and gives you an immediate baseline for everything else on this page. ```text theme={null} Give me the current health of my network. Which gateways have issues right now, any tunnels or site-to-cloud connections that are down or flapping, and anything that needs my attention today? ``` After the summary lands, drill into any finding with a short follow-up such as "expand on the egress finding" or simply the finding number if your assistant numbered them. > **Example output (abridged, demo fabric)** > > **Fabric health: 25/25 gateways up, 5/5 S2C tunnels active. Three findings need > attention.** > > 1. **Egress to a flagged domain on one spoke.** `operations-aws-spoke-k8s` > permits traffic to `malware.net` (synthetic demo indicator) while the > identical SNI is denied on `marketing-azure-spoke-all` and > `accounting-aws-spoke-dev`. Root cause: the strict SNI WebGroup is attached > on two spokes but missing on `operations-aws-spoke-k8s`. This is policy > drift, not a detection gap. > 2. **IPS is in monitor-only mode.** Roughly 2,000 alerts over 7 days, 0 blocks. > Detection without enforcement. > 3. **BGP session reset 6 days ago** on `sv-equinix` (145 prefixes in / 158 out). > Session is currently established; worth correlating with any change that day. > > Everything else checked clean: no gateway CPU or memory distress, no tunnel > flaps in the current log window, default-deny DCF rule active and terminal. ## Security ### Security posture audit and auditor evidence Ask for a full posture sweep when you need to know what is inspected, what bypasses inspection, and what you could show an auditor. This is one of the heaviest prompts on the page; expect the assistant to make dozens of MCP calls. ```text theme={null} What is the security posture of my network? Which spokes are inspected or not, what is bypassing inspection, where are my policy gaps, and what would I show an auditor to prove segmentation and egress control are enforced? ``` The MCP server does not care which language you ask in. The same prompt in French, German, or Japanese runs the same tools and returns the same evidence. > **Example output (abridged, demo fabric)** > > **Inspection coverage: 13 of 14 spokes covered, 1 blind.** Ten spokes enforce > DCF locally and log continuously; the accounting spokes are inspected by the > vendor firewall on `transit-aws-us-east-1` (FireNet). > `operations-oci-spoke-shared` has no DCF logs and is not in the FireNet > inspection list: it moved 680 MB in 7 days with zero policy visibility. That is > your single blind spot. > > **Bypassing inspection:** a leftover `test-lambda-egress` rule (priority 50) > permits any protocol and port from the Lambda group to the internet; HTTPS > egress carries no IDS because the combined http+https IDS policy at priority 999 > has never matched (0 hits in 7 days, a dead rule). > > **Correlation finding:** the number 2 top talker by volume (13.5 GB in 7 days, > synthetic demo source IP) is also the source of a permitted HeartBleed exploit > attempt against `marketing-app-prod`. That is not a coincidence, that is a > pattern. > > **Auditor evidence pack:** 30-policy DCF export ending in an explicit > default-deny-all that fired 7,033,384 times in 7 days with zero allows, plus > timestamped cross-segment deny samples (marketing to accounting on 8443: Denied; > marketing to enterprise-data on 1433: Denied) alongside same-segment permits. ### CVE exposure analysis When a scanner flags a package on your gateways, exposure depends on reachability and traffic direction, not just package presence. The assistant can reason about both because it can read your DCF policy chain and security posture. ```text theme={null} CVE-XXXX-XXXXX (libssh2) was flagged on some of my spoke gateways. Which gateways have this package, is it actually reachable or exploitable given my security groups and how these gateways are configured, and what is my exposure? Give me a remediation summary I can hand to my SOC. ``` > **Example output (abridged, demo fabric)** > > **Reframing the question.** This CVE targets the SSH client side, so "is port 22 > reachable on the gateway" is the wrong question. The right question is "what on > these gateways initiates outbound SSH to untrusted endpoints", and the answer > is: nothing in the observed traffic or configuration. > > **Policy evidence:** exactly one TCP/22 permit exists in the 30-policy DCF > chain, scoped to a management SmartGroup. No gateway-initiated outbound SSH > appears in 7 days of flow data. > > **Verdict:** the scanner undersold the footprint (all 14 spokes share the same > base image, not just the flagged ones) and oversold the risk (no viable attack > path under current policy). Recommended remediation: patch on the normal cycle > as P3, not emergency P1; add a detection for unexpected outbound SSH as a > compensating control. > > A SOC-ready handoff table with per-gateway package status and a P1-P4 action > plan follows. ### Kubernetes SmartGroup and DCF coverage If you run Kubernetes behind Aviatrix, ask the assistant to map your SmartGroup selectors against your cluster reality. The mental model is close to a network policy label selector, enforced at the cloud network layer instead of the node. ```text theme={null} Can you break down the Kubernetes SmartGroups in DCF? Which namespaces and selectors are covered, and where are the gaps? ``` > **Example output (abridged, demo fabric)** > > **47 SmartGroups total, 9 with native Kubernetes selectors.** Three findings: > > 1. The `gcp-demo` namespace selector is over-broad: it matches every pod in the > namespace rather than the intended app labels. > 2. `azure-aks` is Kubernetes in name only: membership is CIDR-based with no > label selectors, so pod churn silently breaks the mapping. > 3. The qa environment has zero Kubernetes SmartGroup coverage. > > **A note on the follow-up "which pods are covered":** the MCP server has no pod > inventory tool. Kubernetes SmartGroups are dynamic; the selector is the > definition. To verify actual membership, compare the selector against > `kubectl get pods --show-labels` output, or check DCF logs for traffic > attributed to the SmartGroup. ## Operations ### Pre-upgrade risk assessment Before a controller or gateway upgrade, ask what tends to break. The assistant checks for the known risk patterns (custom SNAT, non-standard routing, missing HA, legacy features) with configuration evidence rather than folklore. ```text theme={null} I'm planning a controller upgrade. Which of my gateways are at risk: anything running custom SNAT policies, non-standard routing, single gateways with no HA, or config that tends to break on upgrade? Flag what I should check first. ``` The assistant also checks your premise. In one validated run, a request for a "7.2 to 8.0 pre-upgrade check" was refused because every gateway already reported 9.0: the assistant flagged the version mismatch instead of performing a meaningless pre-flight. > **Example output (abridged, demo fabric)** > > **Cleared risks:** no custom SNAT found (`snat_target` empty, `enable_nat=no` > across inspected gateways); no customized CIDRs, BGP over LAN, or PBR; at 25 > gateways, scale is a non-issue. > > **Risk register, ranked:** > > 1. **Zero HA fabric-wide.** No gateway has an HA peer, so every image upgrade is > a hard data-plane outage for that VPC: 25 sequential micro-outages. Plan > waves (dev, qa, prod) with transits in a dedicated window. > 2. **`transit-aws-us-east-1` is a single point of failure.** FireNet, S2C > termination, 4 transit peerings, 8 attachments, no HA peer. Verify vendor > firewall re-registration behavior before and after. > 3. **An apparently orphaned legacy FQDN gateway** with zero tags attached. > Legacy code paths are the most common upgrade casualties; identify the owner > or decommission before upgrading. > 4. **A Public Subnet Filtering gateway on a t3.micro.** Niche legacy feature on > a burstable instance; confirm filter behavior immediately post-upgrade. ### On-call incident walk When an application is reported down, one prompt can chain IP classification, path computation, gateway health, and policy verdicts. ```text theme={null} Pretend I'm on call and just got paged that an app is down. Walk the whole path for me: classify the source and destination IPs, run flightpath, check the gateways in between, and check for any matching firewall drops or intrusions. ``` > **Example output (abridged, demo fabric)** > > **Path walk for 10.2.2.100 to 10.1.2.10:8443:** > > 1. **Classification:** source is `marketing-app-prod` (marketing segment, > Azure); destination is `accounting-backend-api-prod` (accounting segment, > AWS). > 2. **FlightPath:** route exists end to end; security group and NACL checks pass > on both sides. The cloud path is clean. > 3. **Gateway health:** all gateways on the path up, no resource distress, > tunnels established. > 4. **Policy verdict:** DCF logs show the flow hitting `default-deny-all`. > Marketing to accounting is not covered by any permit rule. This is > segmentation working as designed, not an outage. If this flow is newly > required, it needs an explicit policy, not a ticket to networking. ### Investigating a log delivery gap When a log type stops appearing in CoPilot, the fastest diagnosis is a binary search over log counts, then eliminating causes layer by layer. This method generalizes to any telemetry gap. ```text theme={null} DCF firewall logs stopped appearing in CoPilot at some point in the last few months. Find exactly when they stopped, verify whether policies are still enforcing and gateways are still healthy, rule out configuration changes, and narrow down where in the delivery path the logs are being lost. ``` > **Example output (abridged, demo fabric)** > > **When:** log counts over successive time ranges pin the cutoff to a single > timestamp (in the validated run, 792,393 entries before the cutoff, zero after). > > **What still works:** enforcement is intact. The last logged entries show both > Permit and Deny with `is_enforced: True`, all producing gateways are up and > passing traffic, and other log types (gateway syslogs, NetFlow) continue to > flow. So the gap is in DCF log delivery, not in the firewall and not in the > gateways. > > **What was ruled out, with evidence:** no policy changes (audit timeline empty > around the cutoff), per-rule logging still enabled on all 29 policies, syslog > profile enabled and pointed at CoPilot, no external syslog redirect configured. > > **Conclusion:** every layer the API can verify is healthy, which isolates the > fault to the log delivery pathway between controller and CoPilot. The cutoff > coincides with a controller upgrade. Recommended next step: open a support > ticket citing the exact cutoff timestamp, the build number, and the elimination > evidence above. That evidence chain typically saves days of back-and-forth. ## Network engineering and IaC ### Topology diagram generation Ask for a diagram of the fabric the MCP server can see. This use case depends on your client's ability to write files; the MCP server supplies the topology data (gateways, peerings, attachments, edge sites), and the client renders it. ```text theme={null} Build a draw.io diagram of my fabric topology. ``` > **Example output (abridged, demo fabric)** > > An editable `.drawio` file reflecting the live fabric: 5 transit gateways in > full mesh across AWS, Azure, GCP, and OCI, 10 transit peerings, 14 spokes > grouped by segment, 3 edge sites, and the FireNet transit marked with its > inspection role. Because the data comes from the live Controller, the diagram > matches reality rather than the last time someone updated a slide. ### Terraform generation with live verification The assistant can generate Aviatrix Terraform grounded in your actual fabric: real VPC IDs, real transit names, real account names, pulled through the MCP server instead of guessed. ```text theme={null} Build Terraform to provision an Aviatrix spoke gateway in one of the VPCs. ``` > **Example output (abridged, demo fabric)** > > The assistant first asks clarifying questions (which cloud, which VPC, HA or > not, attach to which transit), then produces a module using the current provider > schema with real values from the fabric: the target VPC ID, region, account > name, and transit gateway name for attachment. Validate with > `terraform validate` and review before applying, as with any generated IaC. ## Capability boundaries The MCP server is honest about what it cannot answer, and you should be aware of these limits before relying on it: * **Gateway syslog retention is short**, on the order of 24 hours. A question like "how many times did this tunnel flap in the past 6 months" is unanswerable from gateway syslogs, and the assistant should say so rather than guess. For long-horizon history, ship logs to a SIEM. * **There is no pod inventory.** Kubernetes SmartGroups are defined by their selectors; verifying actual pod membership requires cluster-side tooling. * **Some data is not exportable via API in all builds**, for example WebGroup domain contents in certain versions. The assistant will tell you when a value must be read from the CoPilot UI instead. * **Scheduled rekeys are not flaps.** IPsec IKE reauthentication appears periodically in logs as make-before-break events; a good response distinguishes these from real tunnel instability. A refusal backed by a stated limitation is more useful than a fabricated answer. If your assistant confidently reports six months of tunnel history, be suspicious. ## Prompting tips * **Drill down instead of re-asking.** After a sweep, reply with the finding number or a short phrase ("expand on finding 2"). The assistant reuses the data it already fetched. * **State your premise and let it be checked.** Include your assumed versions, topology, or timeline in the prompt. Validated runs show the assistant correcting a wrong upgrade premise rather than executing it. * **Phrase time windows within retention.** Ask about "the last 24 hours" or "the current log window" for gateway syslogs; use CoPilot-backed data (DCF logs, flow data) for multi-day windows. * **Ask for a deliverable, not just an answer.** "Give me a remediation summary I can hand to my SOC" or "what would I show an auditor" produces a structured artifact you can forward, not a chat reply you have to rewrite. * **Any language works.** The tools are language-agnostic; ask in the language your team works in. # AI Agent Workload Containment Source: https://docs.aviatrix.com/docs/enterprise/10.1/solutions/security-for-ai/ai-agent-workload-security How Aviatrix DCF closes the network security gap for AI agent workloads: threat model, east-west containment, selective TLS inspection, and the business case for default-deny enforcement across any agent platform or framework. AI agents are autonomous by design. They decide which APIs to call, which tools to invoke, and what data to pass between them. They do all of this with unrestricted outbound internet access unless a network control explicitly limits it. Whether agents run on a managed cloud platform, a self-hosted orchestration framework, or a containerized custom runtime, that default-allow posture is the starting state. Closing it is the operator's responsibility, not the platform's. **Aviatrix AgentGuard** is the network enforcement plane for AI agent workloads. It applies DCF default-deny containment, SmartGroup East-West isolation, and WebGroup egress allow-lists consistently across every agent platform and framework, closing the network security gap that managed platforms and self-hosted runtimes leave open by default. ## Business Challenge AI agents operate differently from traditional application workloads. A conventional service calls a fixed set of upstream dependencies; its outbound traffic is deterministic and auditable. An AI agent's outbound traffic is decided at runtime by the model, based on input the model received. That input can be crafted by an attacker. Prompt injection, compromised tool dependencies, and supply chain attacks against model providers or MCP servers all exploit the same weakness: the agent's autonomous decision-making gets redirected to serve the attacker's intent, not the operator's. The network security gap is structural, and it applies across every agent deployment model. Managed platforms route agent egress through provider-controlled infrastructure with no in-line enforcement point the enterprise security stack can observe. Self-hosted frameworks running on Kubernetes or VMs start with the default-allow posture of the environment. In both cases, the shared responsibility model covers the platform or runtime. It does not cover what the agent inside it chooses to call. An agent that executes a prompt injection and attempts to exfiltrate data to an attacker-controlled endpoint is operating within the platform's design. The platform forwards that traffic without restriction. > [!NOTE] > > **March 2026: LiteLLM supply chain compromise.** The [LiteLLM supply chain > attack](https://www.theregister.com/2026/03/24/trivy_compromise_litellm/) > compromised four widely deployed AI tooling packages across five package > registries in twelve days. LiteLLM operates under the same trust model as AI > agents: trusted code making real outbound API calls in trusted pipelines. One > engineer added four command-and-control IPs to a DCF Global IP Blocklist. That > single update reached every gateway in the environment before the compromised > packages could communicate. Zero credentials were exfiltrated. Without a > network enforcement point, the outcome is a detection problem after the fact, > not a prevention problem before it. Two distinct threat vectors apply to AI agent workloads, regardless of how or where they are deployed. **Egress to the internet.** An agent making tool calls, invoking MCP servers, or running a code interpreter does so over outbound connections that are indistinguishable from legitimate traffic at the application layer. A compromised dependency or an injected prompt redirects those connections to attacker infrastructure. Without an enforcement point, the first signal is a detection event after exfiltration has already occurred. **Lateral movement to adjacent workloads.** AI agent spokes in an enterprise transit fabric sit alongside other workload spokes: internal APIs, data platforms, finance and HR systems. A compromised agent that can reach the open internet can also reach adjacent spokes unless East-West policy explicitly prevents it. The blast radius extends not just outward to the internet, but inward to the rest of the enterprise. Supply chain attacks add a third dimension that simple FQDN filtering cannot address. A compromised dependency can redirect the agent to a malicious path on a sanctioned host. Both `raw.githubusercontent.com/legitimate-path` and `raw.githubusercontent.com/attacker-path` present the same TLS endpoint; SNI-based filtering cannot tell them apart. Catching this class of attack requires decrypting the connection to inspect the URL path, which means selective TLS decryption scoped to the supply-chain risk surface. ## Business Case for AI Agent Containment ### Risk Mitigation Without an enforcement point in the agent's egress path, the blast radius of any compromise extends to every destination the agent can reach (the entire internet by default). Containment inverts this: the default is deny, and only explicitly declared destinations are reachable. When prevention fails at the application layer, the network boundary determines whether a compromised agent exfiltrates data or fails to open a socket. The egress risk does not exist in isolation. AI agent spokes in an enterprise transit fabric sit alongside other workload spokes: internal APIs, data platforms, finance and HR systems. A compromised agent that can reach the open internet can also pivot to those adjacent workloads unless East-West policy explicitly prevents it. SmartGroup deny rules between the agent spoke and every other spoke in the fabric contain the lateral movement risk to the agent boundary. The blast radius stops at the spoke, not at the edge of the enterprise. DCF selective TLS decryption closes the supply-chain gap directly. Tool-call egress is decrypted and inspected at the URL-path level. Platform control-plane traffic is explicitly excluded from decryption to preserve runtime compatibility. The decryption surface is scoped to the highest-risk traffic only: not a blanket intercept, but a targeted capability applied where passive filtering falls short. ### Regulatory and Compliance AI agents handling regulated data create a concrete compliance exposure when their egress is unrestricted. The same audit requirements that apply to other workloads apply here, with the added complexity that agent traffic is harder to characterize than traditional application traffic. | Framework | Requirement | How Aviatrix addresses it | | - | - | - | | **SOC 2 Type II** | CC6.6, CC6.7: logical access controls restricting communication to authorized destinations | Default-deny containment enforces allow-lists at the network layer; per-connection logs in CoPilot provide the continuous audit evidence the control is operating | | **HIPAA** | §164.312(e)(1): transmission security for ePHI | Workload-scoped containment limits which destinations an agent handling health data can reach; TLS inspection provides payload-level visibility on tool-call traffic | | **PCI-DSS v4** | Requirement 1.3: network access controls between trusted and untrusted networks | Spoke gateway enforcement at the VPC/VNet boundary; per-connection logs covering all agent egress, inbound and outbound | | **FedRAMP** | SC-7: boundary protection with continuous monitoring | DCF enforces boundary protection at the spoke gateway; CoPilot provides the continuous monitoring and per-connection log evidence required for annual authorization | ### Operational Excellence **Policy as code, deployed with the agent.** Aviatrix DCF policy is Terraform-native. The WebGroup allow-list for an agent's permitted tool destinations lives in the same repository as the agent configuration and deploys in the same pipeline run. When an agent adds a new tool endpoint or changes its model provider, the network policy update is a pull request, not a separate change ticket to a different team. **East-West containment across the transit fabric.** SmartGroups define identity-aware East-West policy between the agent spoke and adjacent workload spokes. A compromised agent cannot pivot to internal APIs, data platforms, or adjacent cloud workloads unless those paths are explicitly declared permitted. The default posture between the agent spoke and every other spoke is mutual deny. **Selective TLS decryption without operational risk.** DCF applies decryption per-rule, scoped to a specific destination SmartGroup. Platform control-plane endpoints are explicitly excluded from decryption, preserving certificate-pinning behavior and runtime compatibility regardless of which platform or framework the agent runs on. The decryption surface covers only the external tool-call traffic where supply-chain risk is highest. **Unified visibility across platforms.** The same CoPilot dashboard governs agent egress whether agents run on managed cloud platforms, self-hosted orchestration frameworks, or containerized custom runtimes. Security teams can answer "what can any of our agents reach?" from a single console, without maintaining separate policy engines per platform. **Rapid policy propagation.** When a new indicator of compromise emerges, an updated containment policy reaches every gateway in the environment before the threat can communicate, with no per-region rollout and no per-platform change window. ### Competitive Differentiation Enterprises do not end up running a single agent on a single platform. The operational reality is concurrent deployment across managed platforms such as AWS Bedrock AgentCore and Microsoft Foundry, self-hosted orchestration frameworks like LangGraph, AutoGen, and CrewAI, and new platforms that will be adopted next quarter. Each has its own insertion model, service endpoint requirements, and policy syntax. Managing agent network security with per-platform firewalls means a different policy engine for every agent type, no unified audit trail, and no consistent enforcement primitive across the estate. Aviatrix brings the same unification to AI agent network security that it brought to multicloud networking: one control plane, one policy model, and one audit log across every agent platform and framework. SmartGroups and WebGroups apply consistently whether the agent runs on a managed cloud platform or a self-hosted runtime in a Kubernetes cluster. [Gartner projects that 40% of enterprise software will embed agentic AI by 2028](https://www.gartner.com/en/newsroom/press-releases/2024-10-21-gartner-says-more-than-40-percent-of-agentic-ai-will-be-embedded-in-enterprise-software-tools-by-2028). Organizations that standardize on Aviatrix as the agent enforcement plane now are positioned to extend that policy consistently as the agent estate grows, rather than re-architecting security for each new platform. ## Key Components > **[Distributed Cloud Firewall](/docs/enterprise/10.1/concepts-architectures/components/dcf/dcf-overview)** > > Default-deny enforcement at the spoke gateway in the agent VPC or VNet. All > agent egress (tool calls, MCP server connections, model provider traffic, > code interpreter outbound) is intercepted and evaluated against declared > policy before any packet leaves the network boundary. > **[SmartGroups](/docs/enterprise/10.1/concepts-architectures/components/dcf/smartgroups-about)** > > Identity-aware source and destination matching keyed to agent workload > primitives: subnet CIDRs for agent instances, cloud provider Service Tags > for platform endpoints, and FQDN patterns. SmartGroups define the egress > source and the East-West boundaries between the agent spoke and adjacent > workload spokes in the transit fabric. > **[WebGroups](/docs/enterprise/10.1/concepts-architectures/components/dcf/webgroups-about)** > > FQDN-based destination filters for permitted tool-call targets: model > provider endpoints, sanctioned MCP servers, and approved REST APIs. > WebGroups enforce the agent's declared egress allow-list, and serve as the > target scope for DCF selective TLS decryption for URL-path enforcement on > supply-chain-risk hosts. > **[CoPilot Security Visibility](/docs/enterprise/10.1/concepts-architectures/components/controlplane/copilot-overview)** > > Centralized dashboard for all agent containment policies, real-time traffic > flows, and per-connection logs across every agent spoke, cloud, and > platform. Security teams get continuous visibility into what every agent can > reach and has actually reached, without querying individual VPCs or > correlating logs across cloud consoles. ## Implementation Guides The containment pattern is the same regardless of the agent platform: deploy a spoke gateway in the agent VPC or VNet, route agent egress through it, and enforce default-deny DCF policy scoped to that workload. * [AWS Bedrock AgentCore](/docs/enterprise/10.1/guides/security/egress/ai-agent-containment-aws-bedrock): spoke gateway insertion in the AgentCore VPC, AWS route table enforcement, IAM guardrails preventing PUBLIC-mode runtimes, and DCF policy with selective TLS decryption for supply-chain risk scenarios. * [Microsoft Foundry: With Transit](/docs/enterprise/10.1/guides/security/egress/ai-agent-containment-azure-foundry-transit): Foundry BYOVNet spoke attached to an existing Aviatrix Enterprise Transit, with UDR enforcement, ACA runtime exclusions, selective TLS inspection, and East-West containment across the transit fabric. * [Microsoft Foundry: Standalone](/docs/enterprise/10.1/guides/security/egress/ai-agent-containment-azure-foundry-standalone): minimum viable AgentGuard footprint for Foundry: a standalone spoke gateway in the Foundry VNet with no Aviatrix Transit required. A natural on-ramp that can be extended to the Transit topology later. * [Self-Hosted Agents on Kubernetes](/docs/enterprise/10.1/guides/security/egress/ai-agent-containment-self-hosted): pod-scoped containment policy for LangGraph, AutoGen, CrewAI, and custom agent runtimes deployed as Kubernetes workloads. # MCP Server Security Through Containment Architecture Source: https://docs.aviatrix.com/docs/enterprise/10.1/solutions/security-for-ai/mcp-server-egress-security How Aviatrix containment architecture closes the egress security gap for MCP server workloads: the business case, deployment approaches, compliance impact, and key capabilities. Every MCP server an enterprise deploys starts with unrestricted outbound internet access. The server that only needs to reach `api.github.com` can, by default, communicate with any destination. Containment policy, if it is configured at all, follows on a separate timeline, managed by a different team, in a different tool. When an AI model's decisions become real-world API calls, the gap between those two timelines is the attack surface. ## Business Challenge MCP servers are the execution layer for AI agent tools. They make autonomous outbound API calls on behalf of large language models, connecting to GitHub, Slack, Salesforce, Jira, internal databases, and cloud provider APIs. These are trusted tools executing in trusted pipelines, which is precisely what makes them a high-value target. A compromised dependency, a poisoned tool description, or a supply chain attack against a server image arrives through channels that look legitimate. The security gap is structural: egress containment for Kubernetes workloads sits outside the scope of any MCP gateway or hosting platform, regardless of how capable that platform is at the protocol layer. Protocol-layer controls govern what tool is called and what it returns. They do not govern what the server pod can reach at the OS level, below the application, on connections the protocol layer never sees. A compromised MCP server dependency that opens a direct connection to an exfiltration endpoint bypasses every protocol-layer control in place. This is not a theoretical risk. > [!NOTE] > > **September 2025: Postmark-MCP.** A rogue npm package (a near-identical clone of the legitimate Postmark MCP server) was [discovered silently forwarding every email it processed to the attacker's external server](https://thehackernews.com/2025/09/first-malicious-mcp-server-found.html). The MCP gateway saw normal, authenticated email operations. The exfiltration happened on an outbound connection the gateway never saw. Workload-level egress containment would have blocked that connection: the server was not declared to reach any destination other than the Postmark API, so every connection to the attacker's endpoint would have been denied and logged. > [!NOTE] > > **March 2026: TeamPCP supply chain attack.** The [TeamPCP cascading supply chain attack](https://www.theregister.com/2026/03/24/trivy_compromise_litellm/) compromised four widely deployed AI and security tooling packages (Trivy, Checkmarx KICS, LiteLLM, and the Telnyx Python SDK) across five package registries in twelve days. LiteLLM operates under the same trust model as MCP servers: trusted code, making real outbound API calls, executed in trusted pipelines. The attack moved through trust, not around defenses. The detection apparatus documented it after the fact. The consequences are concrete. Without egress containment, the blast radius of any MCP server compromise extends to the entire internet. A server handling GitHub integrations, clinical records lookups, or financial data queries has unrestricted access to every destination, not because that access was granted intentionally, but because no one configured a control that denied it. For organizations subject to SOC 2, HIPAA, PCI-DSS, or FedRAMP requirements, an MCP server with unrestricted egress is a documented compliance gap the moment it handles regulated data. ## How Aviatrix Closes the Gap Aviatrix DCF enforces egress containment at the VPC boundary, specifically at the spoke gateway where Kubernetes pod traffic exits the cluster via node NAT gateways. This enforcement point is below the application layer, transparent to the workload, and impossible to bypass from within the pod. Every MCP server starts denied. Only explicitly declared egress destinations are reachable. A server with no declared policy fails closed, not open. Containment policy targets individual MCP server pods by Kubernetes label, not by ephemeral IP address. WebGroups filter outbound connections by FQDN, which is essential for MCP servers communicating with SaaS APIs on CDN-backed or rotating addresses that IP-based controls cannot handle reliably. When a supply chain compromise attempts to open a connection to an attacker-controlled endpoint, there is no route; the attempt is blocked at the gateway and logged in CoPilot. ![Attack scenario: dependency compromise attempts exfiltration; Aviatrix DCF intercepts and blocks at the VPC boundary](https://mintcdn.com/aviatrix-14b37c43/RHxJw5yLDD5FKP-9/images/solutions/security-for-ai/mcp-supply-chain-kill-chain.png?fit=max\&auto=format\&n=RHxJw5yLDD5FKP-9\&q=85\&s=51d0400ee902f143402f845cb010ad52) For Obot-managed deployments, containment is lifecycle-coupled: Obot declares each server's permitted egress destinations, and the Aviatrix controller picks up that declaration and enforces it at the network boundary. When the server is deleted, the policy is deleted. For self-managed deployments, the same policy model applies via Kubernetes CRDs or Terraform. Either way, the enforcement primitive is the same: a per-server default-deny perimeter with an FQDN-based allow-list, enforced at the VPC boundary, visible in CoPilot across every cluster and cloud. ![Two-layer enforcement: Obot governs which MCP servers run and their declared egress policy; Aviatrix DCF enforces that declaration at the network boundary](https://mintcdn.com/aviatrix-14b37c43/RHxJw5yLDD5FKP-9/images/solutions/security-for-ai/obot-enforcement-architecture-conceptual.png?fit=max\&auto=format\&n=RHxJw5yLDD5FKP-9\&q=85\&s=5a8d4f671d07bdecf46551434a4237bf) ## Why Existing Controls Do Not Close This Gap Several categories of tooling address different parts of the AI agent security problem. Understanding where each operates clarifies why workload-level egress containment requires a dedicated approach. | Approach | What it governs | Why it does not close the containment gap | | - | - | - | | **Kubernetes NetworkPolicy** | IP address and port, intra-cluster | Cannot filter by domain name; no lifecycle coupling to MCP deployments; no multicloud visibility | | **Chokepoint inspection** (SASE, transit firewalls) | Traffic traversing a central inspection point | Kubernetes pod egress exits via node NAT gateways and never reaches the chokepoint | | **Application-layer AI security** (prompt injection, DLP) | Model and prompt layer | Cannot prevent unauthorized egress connections that originate below the application layer | | **MCP gateway platforms** (Obot, Kong, Smithery) | MCP protocol layer: auth, rate limiting, tool-call logging | Do not govern what the MCP server pod can reach on the network | Aviatrix DCF enforces at the VPC boundary via spoke gateways, at the point where Kubernetes egress actually exits the cluster, transparent to the workload and impossible to bypass from within the pod. Domain-based filtering via WebGroups, lifecycle coupling via the Obot integration or DIY CRDs, per-connection logging, and consistent containment policy across EKS, AKS, GKE, and self-managed clusters address each gap in the table above. ## The Business Case for MCP Server Egress Containment ### Risk Mitigation **Compromised dependency exposure.** MCP servers execute in trusted pipelines. A supply chain attack against a server image, a poisoned tool description, or a compromised transitive dependency arrives through channels that look legitimate. The September 2025 `postmark-mcp` incident is the clearest example: a malicious package clone forwarded every email it processed to an external server, undetected at the protocol layer, visible only as a network egress event. With unrestricted egress, the blast radius of any such compromise extends to the entire internet. With containment architecture, the blast radius is bounded by the declared allow-list for that specific server; a compromised server that attempts to reach an unauthorized destination finds no path, because one was never created. **The gap between deployed and secured.** In most environments, an MCP server starts taking traffic the moment it is scheduled. Containment policy follows on a separate timeline, run by a different team, often in a different tool. During that gap, a compromised or malicious MCP server can reach any destination on the internet. Containment architecture closes this gap by coupling the security policy to the deployment object itself: when the server is created, the policy is created; when the server is deleted, the policy is deleted. **Unrestricted egress as the structural problem.** Traditional security configurations permit by default and block by exception. A new MCP server deployed without an explicit containment policy inherits the default-allow posture of the environment. Aviatrix DCF inverts this: every server starts denied, and only declared destinations are reachable. A server that has not yet been given an explicit policy fails closed, not open. **Scoped blast radius, not complete prevention.** Egress containment reduces the attack surface materially without claiming to eliminate all risk. It does not prevent prompt injection through allowed API calls, and it does not prevent exfiltration via an approved SaaS endpoint. What it does is eliminate the arbitrary-destination egress surface: a compromised server that can only reach `api.github.com` cannot also reach an attacker-controlled endpoint on an arbitrary domain. ### Regulatory and Compliance Alignment Aviatrix DCF satisfies the workload-level egress controls that compliance frameworks require for AI agent systems: controls that must prevent unauthorized connections, not merely record them. Per-connection logging in CoPilot then generates the continuous audit evidence that proves those controls are operating. | Framework | Requirement | How Aviatrix DCF satisfies it | | - | - | - | | **SOC 2 Type II** | Logical access controls that restrict communication to authorized destinations (CC6.6, CC6.7) | Default-deny-per-server containment enforces allow-lists at the network layer, below the application; unauthorized egress is blocked, not just logged; CoPilot provides the continuous queryable record that the control is operating | | **HIPAA** | Minimum necessary standard: workload isolation for systems that can access PHI pathways | Unauthorized connections from MCP server pods are blocked at the VPC boundary before they can exfiltrate PHI; the containment layer operates below the application and cannot be bypassed by the workload | | **PCI-DSS Req. 1.3** | Restrict outbound traffic to only what is necessary for business operations | Domain-based filtering via WebGroups blocks every destination not on the declared allow-list, with FQDN-level granularity that IP-based controls cannot achieve; the firewall policy is the control, not a record of it | | **FedRAMP** | SC-7 boundary protection: restrict communications to authorized traffic only, with continuous monitoring | Aviatrix DCF enforces boundary protection at the spoke gateway, blocking unauthorized egress, while CoPilot provides the continuous monitoring and per-connection log evidence required for annual review | ### Operational Excellence **Lifecycle-coupled containment eliminates drift.** When containment policy is coupled to the workload, policy drift becomes structurally impossible. The policy is created when the server is created and removed when the server is deleted, automatically, without manual cleanup steps. Environments with frequent MCP server turnover stop accumulating orphaned policies that correspond to no running workload. **Policy as code, in the deployment pipeline.** Allowed egress destinations are declared alongside server configuration and travel with the workload through version control and GitOps pipelines. Containment policy is reviewed as part of the same change as the server definition and rolled back the same way, with no separate workflow and no separate team to coordinate with. **Propagation in seconds.** When a new threat is identified in your MCP server supply chain, updated containment policy reaches every workload before the compromised code can communicate, with no per-device pushes and no propagation delays across clusters. **Multi-persona policy layering.** Security admins, platform teams, and MCP server owners each manage their own containment layer independently. A security admin defines global threat rules without touching per-server policies. A platform team sets namespace-level egress baselines. Server owners declare specific allowed destinations. Each layer operates independently; changes in one layer require no coordination with others, and no team needs access to another team's controls to do their job. **Centralized visibility across the fleet.** Aviatrix CoPilot provides a single dashboard for all MCP server containment policies, real-time traffic flows, and per-connection logs across every cluster and cloud in the environment. Security teams get continuous visibility into what every MCP server can reach, without querying individual clusters or correlating logs manually. **Consistent containment across every cloud.** The same containment policy model and the same CoPilot dashboard govern MCP server egress across EKS, AKS, GKE, and self-managed Kubernetes clusters. Organizations running MCP servers across multiple cloud providers apply a single containment model and audit a single policy inventory, not cloud-specific configurations that diverge over time. ### Competitive Differentiation **The metric that matters changes.** In a detection-era posture, the organizing question is "did we detect the attack?" In a containment posture, it becomes "for every MCP server in our environment, what can it communicate with?" Aviatrix CoPilot answers that question continuously: every allowed and denied connection logged, every server's egress footprint visible, across every cluster and cloud. Security teams shift from reactive investigation after compromise to proactive posture management before it. Organizations that can demonstrate workload-level egress containment for their MCP servers are ahead of both their competitors and their regulators on AI agent security. [Gartner projects that 40% of enterprise applications will embed task-specific AI agents by end of 2026, up from less than 5% in 2025.](https://www.gartner.com/en/newsroom/press-releases/2025-08-26-gartner-predicts-40-percent-of-enterprise-apps-will-feature-task-specific-ai-agents-by-2026-up-from-less-than-5-percent-in-2025) The [Arkose Labs 2026 Agentic AI Security Report](https://www.arkoselabs.com/blog/agentic-ai-security-risk-enterprise-readiness/) found that 97% of enterprise security leaders expect a material AI-agent-driven security incident within the next 12 months. Organizations that close the containment gap now will absorb that incident without catastrophic impact; those that treat the protocol layer as sufficient will not. In regulated procurement contexts (healthcare, financial services, insurance), workload-level containment evidence for AI agent workloads is now a vendor qualification requirement in regulated procurement. Aviatrix DCF provides the containment evidence layer that buyers, auditors, and procurement teams are beginning to require alongside protocol-layer audit trails. ## Industry Success Stories > **Large Healthcare System: HIPAA workload isolation for Obot-managed MCP servers** > > **Challenge:** A major healthcare provider deploying Obot to manage clinical AI tools needed to demonstrate workload-level isolation for every MCP server accessing clinical APIs, as a prerequisite for HIPAA compliance review. Obot's protocol-layer audit trail satisfied tool-call logging requirements, but auditors required containment evidence at the workload level. > > **Solution:** Adding Aviatrix DCF to the Obot deployment meant that every MCP server deployment automatically generated a containment policy alongside its Kubernetes workload objects. Allowed egress destinations (declared per server in the Obot manifest) were pushed to the cluster as a workload-scoped policy. The Aviatrix controller picked up that policy and enforced it at the network boundary, with default-deny egress as the baseline. When a server was removed, its containment policy was removed with it. Per-connection logs in CoPilot provided the egress-layer audit trail alongside Obot's existing protocol-layer record. > > **Results:** > > * Workload isolation evidence satisfied for HIPAA compliance review, without restructuring the existing Obot deployment or Kubernetes architecture > * Zero orphaned containment policies across the MCP server fleet; policy lifecycle tied directly to the server deployment, not managed separately > * Per-connection logs available in CoPilot, covering the egress layer that protocol-layer logging does not reach > * Compliance review timelines reduced by eliminating manual egress documentation for each AI tool deployment > **Regional Insurance Provider: PCI-DSS compliance for self-deployed MCP servers** > > **Challenge:** An insurance group running self-deployed MCP servers for claims processing and customer workflow automation faced a PCI-DSS audit finding that servers handling payment-adjacent data had unrestricted internet egress. The team managed their own Kubernetes deployments and had no existing MCP gateway in place. > > **Solution:** The security team authored containment policies directly for each MCP server and committed them alongside server definitions in the existing GitOps repository. Each policy was pushed to the cluster as part of the normal deployment pipeline; the Aviatrix controller picked it up and enforced it at the network boundary, with no changes to the server code or cluster architecture. > > **Results:** > > * PCI-DSS Requirement 1.3 satisfied for all MCP workloads, with egress allow-lists scoped to confirmed API destinations > * Containment policy committed to version control alongside server definitions, reviewed in the same PR process as application changes > * Audit evidence generated automatically from CoPilot, replacing manually produced egress documentation > * No changes to the MCP server code, container images, or cluster configuration > **Enterprise AI Platform Team: Consistent containment across mixed deployment environments** > > **Challenge:** A financial services firm's platform engineering team running a mixed environment (some MCP servers deployed via Obot, others self-managed) needed consistent containment policy across both deployment paths, with independent controls for security admins, platform engineers, and individual server owners. > > **Solution:** Aviatrix DCF's multi-persona policy layering provided consistent enforcement across Obot-managed and self-deployed servers. Security admin global policies, platform-level namespace baselines, and per-server containment policies each pushed to the cluster independently; the Aviatrix controller enforced all layers simultaneously at the network boundary, with no coordination required between teams. > > **Results:** > > * Consistent containment posture across Obot-managed and self-deployed MCP servers, visible in a single CoPilot dashboard > * Security admin policy changes propagated across all clusters without involving the platform team or requiring deployment windows > * Per-server containment scoped to the MCP server owner's responsibility boundary, with separation of duty across personas > * Full cross-cluster visibility in CoPilot, without requiring per-team access grants to individual clusters ## ROI and Business Impact **Cost avoidance.** The [IBM Cost of a Data Breach Report 2025](https://www.ibm.com/reports/data-breach) puts the average global breach cost at \$4.44M. A single MCP server compromise that exfiltrates customer data carries the same regulatory and remediation cost as any other breach, plus the reputational cost of an AI-specific incident in a market still establishing trust in enterprise AI. Containment eliminates the open-destination egress attack surface. The cost of adding Aviatrix DCF to an existing Kubernetes deployment is substantially below the cost of a single incident. **Operational efficiency.** Lifecycle-coupled containment eliminates the manual policy management overhead associated with MCP server deployments. For platform teams managing dozens of MCP servers, removing manual policy creation, review, and cleanup from each deployment and deletion cycle reduces operational load meaningfully. **Compliance acceleration.** Organizations subject to SOC 2, HIPAA, PCI-DSS, or FedRAMP that can provide automated containment audit trails for MCP workloads reduce compliance review cycles. Audit preparation time decreases when CoPilot provides continuous, queryable connection history: containment evidence that complements protocol-layer audit trails to cover the full audit requirement for AI agent workloads. **Strategic value.** The [agentic AI market reached \$9B in 2026](https://www.fortunebusinessinsights.com/agentic-ai-market-114233). Organizations that can demonstrate workload-level containment alongside protocol-layer governance have a complete, defensible security posture for AI agent workloads, which is the combination that regulated buyers, auditors, and procurement teams are beginning to require. ## Key Components > **[Distributed Cloud Firewall for Kubernetes](/docs/enterprise/10.1/concepts-architectures/components/dcf/kubernetes-overview)** > > Enforces workload-scoped containment policy at the VPC boundary, where Kubernetes egress actually exits the cluster. Containment policies are declared alongside server definitions, pushed to the cluster, and picked up by the Aviatrix controller for immediate enforcement. > **[SmartGroups](/docs/enterprise/10.1/concepts-architectures/components/dcf/smartgroups-about)** > > Identity-based workload selectors that target containment policy to specific MCP server pods by Kubernetes labels, not by ephemeral IP addresses. This ensures that policy follows the workload as it scales and reschedules across nodes. > **[WebGroups](/docs/enterprise/10.1/concepts-architectures/components/dcf/webgroups-about)** > > Domain-based egress filters that match traffic by FQDN rather than IP address, essential for MCP servers communicating with SaaS APIs on rotating or CDN-backed addresses. WebGroups are the mechanism that makes domain-based allow-lists enforceable. > **[CoPilot Security Visibility](/docs/enterprise/10.1/concepts-architectures/components/controlplane/copilot-overview)** > > Centralized dashboard for all MCP server containment policies, real-time traffic flows, and per-connection logs. Provides continuous visibility into egress posture across every cluster and cloud, for both Obot-managed and self-deployed MCP servers. ## Implementation Guides * [Obot Integration: AKS](/docs/enterprise/10.1/guides/security/egress/mcp-egress-containment-obot-azure): lifecycle-coupled containment for Obot-managed MCP servers on Azure Kubernetes Service. * [Obot Integration: EKS](/docs/enterprise/10.1/guides/security/egress/mcp-egress-containment-obot-aws): lifecycle-coupled containment for Obot-managed MCP servers on Amazon EKS. * [Self-Hosted MCP Servers](/docs/enterprise/10.1/guides/security/egress/mcp-egress-containment-self-hosted): containment policy for servers deployed directly to Kubernetes via Helm, ArgoCD, Kustomize, or kubectl. # Transit Gateway BGP Over LAN in AWS Workflow Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/routing/transit-externaldevice-aws-bgpolan Steps to build an Aviatrix Transit Gateway to External Device connection using BGP over LAN in AWS, enabling communication without tunneling protocols like IPsec or GRE. This document describes steps to build an Aviatrix Transit Gateway to External Device using BGP over LAN in AWS. Refer to the following sections for the BGP over LAN workflows in other clouds: * [Azure Multicloud Transit BGP over LAN Workflow](/docs/enterprise/10.1/guides/connectivity/routing/transit-externaldevice-azure-bgpolan) * [Multicloud Transit BGP over LAN in GCP Workflow](/docs/enterprise/10.1/guides/connectivity/routing/transit-externaldevice-gcp-multipeer-bgpolan) ## Introduction Transit BGP to LAN allows Aviatrix Transit Gateways to communicate with a pair of instances in the same VPC in AWS without running any tunneling protocol such as IPsec or GRE. One use case is to interoperate with third-party virtual appliances such as SD-WAN cloud instances that do not have the capability to support BGP over any tunneling protocols. For example, integrating with SD-WAN gateways can be deployed as below, where an Aviatrix Multicloud Transit Gateway connects to a third-party cloud instance in the same VPC in AWS. ![sd\_wan\_integ\_aws](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/routing/sd-wan-integ-aws.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=11edc7cbea02ebf5c22ecd38082c8c8d) *Configuration Overview* The following is an overview on the steps to configure the BGP over LAN solution: 1. [Deploy Aviatrix Transit Solution](#deploying-the-aviatrix-multicloud-transit-solution) 2. [Launch third-party cloud instances](#launching-third-party-cloud-instances) 3. [Build BGP over LAN](#building-bgp-over-lan) For more information about Multicloud Transit Network and External Device, see: * [Multicloud Global Transit FAQ](/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/transit-network-faq) * [Global Transit Network Workflow Instructions (AWS/Azure/GCP/OCI)](https://legacy.docs.aviatrix.com/documentation/latest/network/gateway-transit-create.html) * [Aviatrix Transit Gateway to External Devices](https://legacy.docs.aviatrix.com/documentation/latest/network/transit-to-externaldevice.html) * [Transit Network Design Patterns](/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/multi-cloud-design-patterns) > [!NOTE] > > * This solution supports only [ActiveMesh 2.0](/docs/enterprise/10.1/concepts-architectures/components/connectivity/activemesh-about), check this doc [How to migrate to ActiveMesh 2.0](https://legacy.docs.aviatrix.com/documentation/latest/network/activemesh-migration.html) for migration detail. > * This solution is available to AWS and Azure. The AWS workflow in this document is an example. Adjust the topology depending on your requirements. > * **Controller 8.2 or later**: An instance size that supports at least 2 interfaces is required, such as t3.small and t2.medium in AWS. > * **Controller 6.3 to 8.1**: An instance size that supports at least 5 interfaces is required, such as c4.4xlarge, c5.4xlarge, and c5n.4xlarge in AWS. > * LAN interfaces for Aviatrix Transit Primary and third-party cloud instance must be in the same Availability Zone. ## The key ideas for this solution are: * A BGP session is established between a third-party cloud instance and an Aviatrix Transit Gateway via each LAN interface in the same VPC. * Data plane traffic also runs between a third-party cloud instance and Aviatrix Transit Gateway via each LAN interface without a tunnel protocol such as IPsec and GRE. ## Prerequisites * [Upgrade](/docs/enterprise/10.1/guides/maintenance/upgrade-overview) Aviatrix Controller to at least version 6.3 for the instance with a minimum of 5 interfaces and Controller version to 8.2 for the instance with a minimum of 2 interfaces. * In this example, you are going to deploy the below VPCs in AWS: * Transit VPC (for example, 10.1.0.0/16) by utilizing Aviatrix feature [Create a VPC](/docs/enterprise/10.1/reference/ui/cloud-resources/cloud-networks#create-vpcvnet) with Aviatrix FireNet VPC option enabled. * Spoke VPCs (for example, 192.168.1.0/24 and 192.168.2.0/24) by utilizing Aviatrix feature [Create a VPC](/docs/enterprise/10.1/reference/ui/cloud-resources/cloud-networks#create-vpcvnet) as the previous step or manually deploying it in each cloud portal. Moreover, feel free to use your existing cloud network. * Third-party cloud instance supports high throughput. ## Deploying the Aviatrix Multicloud Transit Solution Refer to [Global Transit Network Workflow Instructions](https://legacy.docs.aviatrix.com/documentation/latest/network/gateway-transit-create.html) for the below steps. Adjust the topology depending on your requirements. 1. Deploy the [Aviatrix Multicloud Transit Gateway and HA](https://legacy.docs.aviatrix.com/documentation/latest/network/gateway-transit-create.html) with High Performance Encryption Mode enabled. In this example, size c5n.4xlarge is selected to benchmark [performance](#performance-benchmark). 2. Deploy a [Spoke Gateway and HA](https://legacy.docs.aviatrix.com/documentation/latest/network/gateway-spoke-create.html) to launch Aviatrix Spoke gateway and enable HA with High Performance Encryption Mode enabled in the Spoke VPC. In this example, size c5n.4xlarge is selected to benchmark [performance](#performance-benchmark). 3. To resize an existing Aviatrix Gateway to t3.small or t2.medium (2 interfaces), only for BGP over LAN, ensure FireNet option is disabled, see [Resizing Aviatrix Gateway Instances](https://legacy.docs.aviatrix.com/documentation/latest/network/gateway-resize.html?expand=true) 4. Attach [Spoke Gateways to Transit Network](https://legacy.docs.aviatrix.com/documentation/latest/network/spoke-attach-transit.html). ## Launching Third-Party Cloud Instances Deploy third-party cloud instances in the same VPC where the Aviatrix Transit Gateways are located. 1. Create a third-party cloud instance and put the MGMT interface in the public gateway subnet. 2. Create a new public WAN subnet and a dedicated routing table for the WAN interface if needed. 3. Create a new private LAN subnet and a dedicated routing table (optional) for the LAN interface. 4. Make sure the function "Source/Dest check on third-party cloud instance's interfaces" is disabled. > [!NOTE] > > The primary Aviatrix Transit Gateway must be deployed in the same Available > Zone (AZ) as the first third-party cloud instance. The HA Transit Gateway if > deployed must reside in the same AZ as the second cloud instance. ## Building BGP over LAN Configure BGP over LAN on the Aviatrix Transit Gateway. **CoPilot UI** 1. In CoPilot, go to Networking > Connectivity > External Connections (S2C). 2. Click *External Connection*. 3. In the Add External Connection dialog, select *External Device* and *BGP over LAN*. 4. Enter the following information in the fields. | Setting | Value | | - | - | | Transit VPC Name | Select the Transit VPC ID where Transit GW was launched | | Connection Name | Provide a unique name to identify the connection to external device | | Aviatrix Transit Gateway BGP ASN | Configure a BGP AS number that the Transit GW will use to exchange routes with external device | | Primary Aviatrix Transit Gateway | Select the Transit GW | | Enable Remote Gateway HA | Check this option in this example to connect two external devices | | Remote BGP AS Number | Configure a BGP AS number that third-party cloud primary instance will use to exchange routes with Aviatrix Transit Primary | | Remote LAN IP | Use the private IP of the LAN interface of the third-party cloud primary instance | | Local LAN IP | Leave it blank and the Controller will assign an IP in the same subnet as the Remote LAN IP. Optionally configure an IP of your choosing within the same subnet of the Remote LAN IP. | | Remote BGP AS Number (Backup) | Configure a BGP AS number that the third-party cloud HA instance will use to exchange routes with Aviatrix Transit HA. | | Remote LAN IP (Backup) | Use the private IP of the LAN interface of the third-party cloud HA instance. | | Local LAN IP (Backup) | Leave it blank and the Controller will assign an IP in the same subnet as the Remote LAN IP (Backup). Optionally configure an IP of your choosing within the same subnet as the Remote LAN IP (Backup). | 5. Click *Save* to generate BGP session over LAN. **Terraform** Build BGP over LAN with the [aviatrix\_transit\_external\_device\_conn](https://registry.terraform.io/providers/AviatrixSystems/aviatrix/latest/docs/resources/aviatrix_transit_external_device_conn) resource. The referenced Transit Gateway must already have BGP over LAN interfaces provisioned (`bgp_lan_interfaces_count` on `aviatrix_transit_gateway`). **1. Create the BGP over LAN external device connection** Add the following `aviatrix_transit_external_device_conn` resource block to your Terraform configuration file: ```hcl theme={null} resource "aviatrix_transit_external_device_conn" "aws_bgp_over_lan" { vpc_id = "vpc-abcdef" connection_name = "aws-bgp-over-lan-conn" gw_name = "transit-gw-01" connection_type = "bgp" tunnel_protocol = "LAN" bgp_local_as_num = "65001" bgp_remote_as_num = "65002" remote_lan_ip = "10.1.4.10" ha_enabled = true backup_bgp_remote_as_num = "65003" backup_remote_lan_ip = "10.1.5.10" } ``` > [!NOTE] > > Leave `local_lan_ip` (and `backup_local_lan_ip`) unset to let the > Controller assign an IP in the same subnet as `remote_lan_ip`, matching > the CoPilot UI behavior. **2. Verify and Apply the Configuration** Run `terraform plan`, review the planned changes, and run `terraform apply`. ## Related Resources * [aviatrix\_transit\_external\_device\_conn Resource](https://registry.terraform.io/providers/AviatrixSystems/aviatrix/latest/docs/resources/aviatrix_transit_external_device_conn) ## Downloading the BGP over LAN configuration 1. On the External Connections (S2C) tab, click the vertical ellipsis ![20](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/routing/threedot.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=516b20b89d4d7e5676f53271547074a2) and select *Download Configuration*. 2. Select the Vendor type, Platform, and Software. 3. Click *Download*. ## Configuring BGP over LAN on Third-Party Cloud Instance 1. Open the downloaded BGP over LAN configuration file. 2. Configure the related BGP and LAN settings in the third-party cloud instance. ### Verifying the LAN Connection Status You can check the tunnel status by going to Diagnostics > Cloud Routes > External Connections and checking the Tunnel Status column for the external connection. ### Verifying the BGP Session Status You can check the BGP Session status by going to Diagnostics > Cloud Routes > BGP Info. The Status should be Established. If some external connections for the selected Transit Gateway are Not Established, the overall BGP Status for the Transit Gateway is Partially Established. ## Ready to Go At this point, run connectivity and performance tests to ensure everything is working correctly. You can do this with the [Diagnostic Tools](/docs/enterprise/10.1/reference/ui/diagnostics/diagnostic-tools) . ## Performance Benchmark ### End-to-End traffic via Aviatrix - Aviatrix The performance test is done with a pair of Aviatrix Transit Gateways as the third-party cloud instances, as shown below. #### Multiple flows result by using iperf3 tool with TCP 128 connections | Aviatrix Gateway size | Throughput (Gbps) | | - | - | | C5n.4xlarge | 23 - 24 | # Transit Gateway BGP Over LAN in Azure Workflow Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/routing/transit-externaldevice-azure-bgpolan Transit BGP to LAN allows Aviatrix Transit Gateways to communicate with a pair of instances in different VNets in Azure without running any tunneling protocol such as IPsec or GRE. ## Introduction Transit BGP to LAN allows Aviatrix Transit Gateways to communicate with a pair of instances in different VNets in Azure without running any tunneling protocol such as IPsec or GRE. One use case is to interoperate with third-party virtual appliances such as SD-WAN cloud instances that do not have the capability to support BGP over any tunneling protocols. For example, integrating with SD-WAN gateways can be deployed as below where an Aviatrix Multicloud Transit Gateway connects to a third-party cloud instance in different VNets in Azure. ![sd\_wan\_inte\_azure](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/routing/sd-wan-inte-azure.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=c12c586aeb7482e2c9408ee3303b805d) This document describes step-by-step instructions on how to build an Aviatrix Transit Gateway to External Device using BGP over LAN in Azure: 1. [Deploy Aviatrix Multicloud Transit Solution](#deploying-aviatrix-multicloud-transit-solution) 2. [Launch Third-Party Cloud Instances](#launch-third-party-cloud-instances) 3. [Build BGP over LAN](#building-bgp-over-lan) For other BGP over LAN workflows, see: * [Multicloud Transit BGP over LAN in AWS Workflow](/docs/enterprise/10.1/guides/connectivity/routing/transit-externaldevice-aws-bgpolan) * [Multicloud Transit BGP over LAN in GCP Workflow](/docs/enterprise/10.1/guides/connectivity/routing/transit-externaldevice-gcp-multipeer-bgpolan) For more information about Multicloud Transit Network and External Device, see: * [Multicloud Transit Network Workflow Instructions (AWS/Azure/GCP/OCI)](/docs/enterprise/10.1/guides/connectivity/transit/transitvpc-workflow) * [Aviatrix Transit Gateway to External Devices](/docs/enterprise/10.1/guides/connectivity/vpn/transit-externaldevice-workflow) * [Transit Network Design Patterns](/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/planning-azure) > [!NOTE] > > * ActiveMesh 2.0 is required. To migrate to ActiveMesh 2.0, see [Migrating from Classic Aviatrix Encrypted Transit Network to Aviatrix ActiveMesh Transit Network](https://legacy.docs.aviatrix.com/documentation/latest/network/activemesh-migration.html). > * This solution is available in AWS and Azure. To configure this solution for AWS, see [Multicloud Transit BGP over LAN in AWS Workflow](/docs/enterprise/10.1/guides/connectivity/routing/transit-externaldevice-aws-bgpolan). Adjust the topology depending on your requirements. > * LAN interfaces for Aviatrix Transit Primary and third-party cloud instance must be in the different VNets. ### The key ideas for this solution are: * A BGP session establishes between a third-party cloud instance and Aviatrix Transit Gateway via each LAN interface in different VNets. * Data plane traffic also runs between a third-party cloud instance and Aviatrix Transit Gateway via each LAN interface without a tunnel protocol such as IPsec and GRE. ### Prerequisites * This feature is available for 6.3 and later. [Upgrade](/docs/enterprise/10.1/guides/maintenance/controller-upgrade-workflow) Aviatrix Controller to at least version 6.3. * In this example, you are going to deploy the below VNets in Azure: * Transit VNets (for example, 10.1.0.0/16 and 10.2.0.0/16) by [creating a VNet](/docs/enterprise/10.1/guides/platform-administration/vpc-vnet-create) with the VPC Function Aviatrix FireNet VNet option enabled. * Spoke VNets (for example, 192.168.11.0/24 and 192.168.21.0/24) by [creating a VNet](/docs/enterprise/10.1/guides/platform-administration/vpc-vnet-create) as per the previous step or manually deploying it in each cloud portal. You can use your existing cloud network. * Third-party cloud instance supports high throughput. ## Deploying Aviatrix Multicloud Transit Solution Refer to [Global Transit Network Workflow Instructions](/docs/enterprise/10.1/guides/connectivity/transit/transitvpc-workflow) for the below steps. Adjust the topology depending on your requirements. 1. Deploy [Aviatrix Multicloud Transit Gateway and HA](https://legacy.docs.aviatrix.com/documentation/latest/network/gateway-transit-create.html) with High Performance Encryption Mode enabled in Transit VNet. > [!NOTE] > > Select the *BGP Over LAN* checkbox to enable that function. See [Performance Benchmarks](#performance-benchmarks) for more information about Gateway size and benchmark performance. 2. Deploy [Spoke Gateway and HA](https://legacy.docs.aviatrix.com/documentation/latest/network/gateway-spoke-create.html) to launch Aviatrix Spoke gateway and enable HA with High Performance Encryption enabled in Spoke VNet. 3. Attach [Spoke Gateways to Transit Network](https://legacy.docs.aviatrix.com/documentation/latest/network/spoke-gateway-attachments.html). 4. (Optional) You can [attach Azure Spoke VNet via native peering](#creating-azure-vnet-peering-between-aviatrix-transit-vnet-and-third-party-cloud-instance-transit-vnet) if you prefer not to encrypt the traffic between the Transit VNet and the Spoke VNet. In this example, this approach is selected to benchmark: [Performance Benchmarks](#performance-benchmarks). ### Launch Third-Party Cloud Instances Deploy third-party cloud instances in a separate Transit VNet. 1. Create a third-party cloud instance and put the MGMT interface in the public gateway subnet. 2. Create a new public WAN subnet and a dedicated routing table for the WAN interface if needed. 3. Create a new private LAN subnet and a dedicated routing table for the LAN interface. 4. Make sure the IP forwarding function on third-party cloud instance interfaces is enabled. > [!NOTE] > > An Aviatrix Transit Gateway and third-party cloud instance CANNOT be deployed > in the same Transit VNet. ### Building BGP over LAN #### Creating Azure VNet Peering Between Aviatrix Transit VNet and Third-Party Cloud Instance Transit VNet 1. In CoPilot, navigate to Networking > Connectivity > Native Peering. 2. Click *+Native Peering*. 3. In the Create Native Peering dialog, select Azure Cloud. 4. Select the VNet where the Aviatrix Transit gateway is located as Peer1. 5. Select the VNet where the third-party cloud instance is located as Peer2. 6. You can also select the route tables for each VNet by sliding Select Route Tables to *On*. 7. Click *Save*. #### Configuring BGP over LAN on Aviatrix Transit Gateway **CoPilot UI** 1. In CoPilot, navigate to Networking > Connectivity > External Connections (S2C). 2. Click *+External Connection*. 3. In the Add External Connection dialog, select *Connect Public Cloud to External Device*. 4. Select *BGP over LAN* from the drop-down. 5. Enter the following information in the fields provided. | Setting | Value | | - | - | | Name | Unique name to identify the external device connection. | | Local Gateway | Select the Transit VPC ID where the Transit gateway was launched (BGP over LAN must have been enabled for this Transit gateway). | | Local ASN | The local BGP AS number (for on-site equipment) the Spoke gateway will use to exchange routes with the external device. | | Remote ASN | Enter the BGP AS number of the cloud location the external device will use to exchange routes with the Transit Gateway. | | BGP ActiveMesh | Enable full mesh BGP connections to the external devices. | | Learned CIDR Approval | Set to On by default if selected for the Local Gateway. Otherwise, it is set to Off. | | Remote LAN IP | The private IP of the LAN interface of the third-party cloud primary instance. | | Local LAN IP | Aviatrix detects the Local LAN IP automatically. | 6. To create the BGP connection, click *Save*. The Transit Gateway route table is automatically updated with the user-defined route 0.0.0.0/0 (default route) pointing to the SDWAN LAN IP. **Terraform** Build BGP over LAN with the [aviatrix\_transit\_external\_device\_conn](https://registry.terraform.io/providers/AviatrixSystems/aviatrix/latest/docs/resources/aviatrix_transit_external_device_conn) resource. The referenced Transit Gateway must already have `enable_bgp_over_lan` set to `true`. > [!NOTE] > > The Azure VNet peering between the Aviatrix Transit VNet and the third-party > cloud instance VNet (created in > [Creating Azure VNet Peering Between Aviatrix Transit VNet and Third-Party > Cloud Instance Transit VNet](#azure-vnet-peering)) must exist before applying > this resource. Creating the connection without it fails with > `[AVXERR-TRANSIT-0172] Please create Azure VNET peering`. Use the CoPilot UI > (Networking > Connectivity > Native Peering) or your own Azure tooling to > create the peering; testing did not find an Aviatrix Terraform resource that > reliably satisfies this precondition. **1. Create the BGP over LAN external device connection** Add the following `aviatrix_transit_external_device_conn` resource block to your Terraform configuration file: ```hcl theme={null} resource "aviatrix_transit_external_device_conn" "azure_bgp_over_lan" { vpc_id = "transit-vnet:transit-rg:11111111-1111-1111-1111-111111111111" connection_name = "azure-bgp-over-lan-conn" gw_name = "transit-gw-azure-01" connection_type = "bgp" tunnel_protocol = "LAN" bgp_local_as_num = "65001" bgp_remote_as_num = "65002" remote_lan_ip = "10.2.4.10" remote_vpc_name = "sdwan-vnet:sdwan-rg:11111111-1111-1111-1111-111111111111" } ``` > [!NOTE] > > `remote_vpc_name` is required for Azure BGP over LAN connections and > uses the format `"VNet_Name:Resource_Group_Name:Subscription_ID"` for > the VNet where the third-party cloud instance is deployed. Leave > `local_lan_ip` unset to let the Controller auto-detect it. **2. Verify and Apply the Configuration** Run `terraform plan`, review the planned changes, and run `terraform apply`. ## Related Resources * [aviatrix\_transit\_external\_device\_conn Resource](https://registry.terraform.io/providers/AviatrixSystems/aviatrix/latest/docs/resources/aviatrix_transit_external_device_conn) #### (Optional) Downloading the BGP over LAN configuration sample from Aviatrix Controller 1. On the External Connections (S2C) tab, click the vertical ellipsis and then select *Download Configuration*. 2. In the Download Configuration dialog, select the following: * Vendor: select the device you are using (any device that is capable of running IPsec and BGP). * Platform: select the applicable platform for the chosen device. * Software: automatically selected based on the Vendor/Platform you select. 3. Click *Download*. #### Configuring BGP over LAN on Third-Party Cloud Instance 1. Log in to the Azure portal. 2. Create a user-defined routing table with default route (0.0.0.0/0) pointing nexthop to the Aviatrix Primary Transit LAN IP for the subnet where the third-party cloud primary instance LAN interface is located. 3. Create a user-defined routing table with default route (0.0.0.0/0) pointing nexthop to the Aviatrix HA Transit LAN IP for the subnet where third-party cloud HA instance's LAN interface is located, for HA deployment. 4. (Optional) Open the downloaded BGP over LAN configuration file. 5. Log in to the third-party cloud instance. 6. Program route to send traffic to the Aviatrix Transit LAN IP through the third-party cloud instance LAN interface. 7. Configure the related BGP and LAN information in the third-party cloud instance. 8. Check if the function 'eBGP multi-hop' is enabled, if the BGP session is not established. 9. Repeat these steps for HA deployment. > [!NOTE] > > You must create a default route 0.0.0.0/0 in the third-party cloud instance > LAN route table to point to the Aviatrix Transit LAN IP over VNET peering in > Azure. #### Verifying LAN and Transit Status * LAN status: Navigate to Diagnostics > Cloud Routes > External Connections. The Status and Tunnel Status columns indicate if the tunnel is Up or Down. * Transit gateway status: Navigate to Diagnostics > Cloud Routes > Gateway Routes. The Gateway Status and Tunnel Status columns indicate if the gateway and tunnel are Up or Down. #### Verifying BGP Session Status Navigate to Diagnostics > Cloud Routes > BGP Info to verify the BGP routes. The Status should be Established. If some external connections for the selected Transit Gateway are Not Established, the overall BGP Status for the Transit Gateway is Partially Established. ### Ready to Go At this point, run connectivity and performance test to ensure everything is working correctly. ### Performance Benchmarks #### End-to-End traffic via Native Spoke VNet - Aviatrix - Aviatrix - Native Spoke VNet The performance test is done with a pair of Aviatrix Transit Gateways as the third-party cloud instances, as shown below. ##### Multiple flows result by using iperf3 tool with TCP 128 connections | Aviatrix Gateway size | Throughput (Gbps) | | - | - | | Standard\_D5\_v2 | 22 - 23 | # Transit Gateway Multi-Peer BGP Over LAN with Azure Route Server Workflow Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/routing/transit-externaldevice-azure-bgpolan-ars The Aviatrix Platform allows Azure Route Server (ARS) integration for on-premises connectivity using Azure ExpressRoute with no overlay. Azure Route Server is a managed service with high availability. The Aviatrix Platform allows Azure Route Server (ARS) integration for on-premises connectivity using Azure ExpressRoute with no overlay. Azure Route Server is a managed service with high availability. It provides a mechanism for exchanging routes between Azure Software Defined Networking (SDN) and Network Virtual Appliances (NVAs) dynamically through Border Gateway Protocol (BGP). You can achieve full-mesh high availability by running two BGP peering endpoints. Aviatrix integrates with Azure Route Server by treating the Azure Route Server as a BGP over LAN peer and exchanging routes using BGP. This enables Azure cloud networks to connect to on-premises or branch locations and provides connectivity across hybrid environments. Customers who use high-speed Azure ExpressRoute connectivity with no encryption for hybrid environments can exchange routes between the Aviatrix Transit Gateways and the on-premises network connected via ExpressRoute. This solution provides you with an enterprise-grade transit network. The diagram below shows Azure Route Server integration with Aviatrix Transit Gateways. Full mesh is enabled so that both Transit Gateways peer with the two Azure Route Server IP endpoints in the Azure Route Server. ![Azure Route Server integration with Aviatrix Transit Gateways showing full mesh peering](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/routing/ars-high-level.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=df3fc082c8fcc0e60a592074c2bb50d8) This document provides instructions on how to build an Aviatrix Transit Gateway to External Device (Azure Route Server) using BGP over LAN in Azure. > [!WARNING] > > Currently, only 1 integration with either Azure Route Server or vWAN Edge is > supported in a Transit Gateway when using BGP over LAN. So only 1 BGP over LAN > External with ActiveMesh connection is supported per gateway. 1. [Perform the prerequisite steps](#prerequisites). 2. [Configure the VNet Peering Between Transit and Azure Route Server VNets](#configure-the-vnet-peering-between-transit-and-azure-route-server-vnets). 3. [Configure the BGP peering Between Azure Route Server and Transit Gateways](#configure-the-bgp-peering-between-azure-route-server-and-transit-gateways). 4. [Configure the BGP over LAN External Connection to Azure Route Server](#configure-the-bgp-over-lan-external-connection-to-azure-route-server). ## Prerequisites Ensure the following prerequisites are met: * The Aviatrix Controller is updated to software version 6.8 or above. * Aviatrix Transit Gateways are deployed with High Performance Encryption (HPE) and with BGP over LAN enabled. You only need to configure one BGP over LAN interface to peer with both Azure Route Server instances. ![Transit Gateway HPE mode with BGP over LAN enabled](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/routing/prereq-hpe-mode-bgpolan.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=6f5188db23ee43665ed0a3e7904f0a1f) * A BGP ASN is assigned to the Transit Gateways (configure by going to **CoPilot** > **Cloud Fabric** > **Gateways** > **Transit Gateways** tab, select the Transit Gateway and go to **Settings** > **Border Gateway Protocol (BGP)** > **Local AS Number**). * Complete the following tasks in Azure: 1. Create a VNet to deploy the Azure Route Server. 2. Deploy the Azure Route Server in this VNet by referring to the applicable Azure documentation. 3. Go to your Route Server > Overview and record the ASN and the private IP addresses of the Azure Route Server endpoints you created. You will use these later in the Aviatrix CoPilot configuration. ![Azure Route Server ASN and private IP addresses](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/routing/prereq-asn-private-ip.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=b23b4648499455ee4bccc1819f52affa) ## Configure the VNet Peering Between Transit and Azure Route Server VNets You need to configure the VNet peering between the VNet that is hosting the Azure Route Server and the Aviatrix Transit VNet. Follow these steps to configure the peering parameters from the Azure Route Server VNet to the Aviatrix Transit VNet. > [!TIP] > > If you are using Terraform, make sure to explicitly set the argument > "allow\_forwarded\_traffic" to "True" for both VNet peerings. 1. Launch the Azure Portal. 2. Go to Virtual networks and select your Route Server VNet. 1. Under Settings, click **Peerings**. 2. On the Peerings page, click **Add**. 3. On the Add peering page, select the following options: ![Add peering configuration options](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/routing/add-peering.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=0507eebd9da1293d99cf0ab0ebe574d3) 3. Click **Add**. ## Configure the BGP peering Between Azure Route Server and Transit Gateways 1. In Aviatrix CoPilot, go to **Cloud Fabric** > **Gateways**, select your primary Transit Gateway and click **Gateway Diagnostics**. Then, click **Interface Stats** and record the IP address assigned to the BGP over LAN interface. Repeat this step for the HA Transit Gateway. ![Primary Transit Gateway interface information](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/routing/gateway-interface-info.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=817536c674513c305a1ed672f78c5672) ![HA Transit Gateway interface information](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/routing/gateway-interface-info-hagw.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=c90f4f9ba7deb7638c8c02bb47d279a1) 2. In your Azure portal, go to **Route Servers** > select your Azure Route Server > **Peers**, and click **Add**. 3. Configure the Azure Route Server peering to both remote Aviatrix Transit Gateways in the Transit VNet by specifying the ASN you configured for your Aviatrix Transit Gateways and the IP address of the BGP over LAN network interface on each Transit Gateway. Refer to the [Prerequisites](#prerequisites) section to find the ASN number. ![Azure Route Server peers configuration](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/routing/ars-peers.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=8e151443b06b4a6c8a2b100df754b951) 4. Go to **Route Servers** > select your Azure Route Server > **Configuration**. 5. Next to Branch-to-branch, select **Enabled**. This option allows the Azure Virtual Network Gateways to propagate the routes the Azure Route Server has learned from the Aviatrix Transit Gateways. It is disabled by default. ![Enable branch-to-branch on Azure Route Server](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/routing/enable-branch-to-branch.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=91689ef5f2915be4e890810fe2ec0e5d) ## Configure the BGP over LAN External Connection to Azure Route Server **CoPilot UI** In Aviatrix CoPilot: 1. Go to **Networking** > **Connectivity** > **External Connections (S2C)** tab. 2. Click **+ External Connection**. 3. Provide the following information. | **Parameter** | **Description** | | - | - | | Name | A name for this connection. | | Connect Local Gateway To | Select the External Device radio button. Click the dropdown menu and select **BGP over LAN**. | | Local Gateway | Transit Gateway to connect to Azure Route Server. | | Local ASN | Transit gateway's Local AS Number. | | Remote ASN | 65515. Azure Route Server always resides in ASN 65515 and cannot be changed. | | Remote vnet:rg:sub | Route Server VNet. | | BGP ActiveMesh | Set this toggle to **On**. When you select BGP Activemesh, Aviatrix Controller creates two peers from each Transit Gateway to both instances of Azure Route Server. This is required for the correct operation of Azure Route Server. | | Learned CIDR Approval | This is Off and disabled by default unless the Local Gateway you select has [Learned CIDR Approval](https://legacy.docs.aviatrix.com/documentation/latest/network/bgp-connection-settings.html?expand=true#gateway-learned-cidr-approval) turned On; the Connection option selected, and the BGP connection selected. Then it is On by default (not editable). When this setting is On, an approval process is set up for gateway learned CIDRs for this BGP external connection. This approval process improves security for your network. If an unapproved CIDR address attempts to access the connection, CoPilot sends an approval email to the CoPilot admin so that the admin can approve or block access. | | Remote LAN IP | Azure Route Server instance 0 IP Address. Refer to the [Prerequisites](#prerequisites) section to find Azure Route Server IP addresses. | | Local LAN IP | Primary Transit Gateway BGP over LAN IP Address. | 4. Click **+ Connection** to add a new connection. | **Parameter** | **Description** | | - | - | | Remote LAN IP | Azure Route Server instance 1 IP Address. | | Local LAN IP | HA Transit Gateway LAN IP Address. | 5. Click **Save**. **Terraform** Build the BGP over LAN connection to Azure Route Server with the [aviatrix\_transit\_external\_device\_conn](https://registry.terraform.io/providers/AviatrixSystems/aviatrix/latest/docs/resources/aviatrix_transit_external_device_conn) resource, using `bgp_activemesh` to peer with both Azure Route Server instances from both the primary and HA Transit Gateway. **1. Create the BGP over LAN external device connection to Azure Route Server** Add the following `aviatrix_transit_external_device_conn` resource block to your Terraform configuration file: ```hcl theme={null} resource "aviatrix_transit_external_device_conn" "azure_route_server_conn" { vpc_id = "transit-vnet:transit-rg:11111111-1111-1111-1111-111111111111" connection_name = "azure-route-server-conn" gw_name = "transit-gw-azure-01" connection_type = "bgp" tunnel_protocol = "LAN" bgp_local_as_num = "65001" bgp_remote_as_num = "65515" remote_vpc_name = "route-server-vnet:route-server-rg:11111111-1111-1111-1111-111111111111" remote_lan_ip = "10.4.0.4" backup_remote_lan_ip = "10.4.0.5" backup_bgp_remote_as_num = "65515" enable_bgp_lan_activemesh = true ha_enabled = true } ``` > [!NOTE] > > `bgp_remote_as_num` is always `65515` for Azure Route Server and cannot > be changed. `remote_lan_ip`/`backup_remote_lan_ip` are the two Azure > Route Server instance IP addresses recorded in > [Prerequisites](#prerequisites). Only one integration with Azure Route > Server or vWAN Edge per Transit Gateway is supported when using BGP over > LAN. **2. Verify and Apply the Configuration** Run `terraform plan`, review the planned changes, and run `terraform apply`. ## Related Resources * [aviatrix\_transit\_external\_device\_conn Resource](https://registry.terraform.io/providers/AviatrixSystems/aviatrix/latest/docs/resources/aviatrix_transit_external_device_conn) ## Verifying the LAN Connection Status You can check the tunnel status by going to **Diagnostics** > **Cloud Routes** > **External Connections**. Confirm that Tunnel Status is Up and that HA Status (Activemesh for the Route Servers) and BGP Status is enabled. ![LAN tunnel status showing Up status](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/routing/lan-tunnel-status.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=a6e22b6b5dedcd560a93c6c9715c6e19) ## Verifying the BGP Session Status You can check the BGP session status by going to **Diagnostics** > **Cloud Routes** > **BGP Info**. The Status should be Established. If some external connections for the selected Transit Gateway are Not Established, the overall BGP Status for the Transit Gateway is Partially Established. ![BGP session status showing Established](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/routing/check-status.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=3ac992e0eeb754868fa11a4c86e38fa6) You can also check the BGP Learned and Advertised routes. ![CoPilot BGP info showing learned and advertised routes](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/routing/copilot-bgp-info.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=bbf0dd7e59657cac8fd2cc5cc77a6090) ## Related Topics * [Transit Gateway BGP over LAN in Azure Workflow](/docs/enterprise/10.1/guides/connectivity/routing/transit-externaldevice-azure-bgpolan) * [Transit Gateway BGP over LAN in AWS Workflow](/docs/enterprise/10.1/guides/connectivity/routing/transit-externaldevice-aws-bgpolan) * [Transit Gateway Multi-Peer BGP over LAN in GCP Workflow](/docs/enterprise/10.1/guides/connectivity/routing/transit-externaldevice-gcp-multipeer-bgpolan) # Transit Gateway Multi-Peer BGP Over LAN in GCP Workflow Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/routing/transit-externaldevice-gcp-multipeer-bgpolan Transit BGP to LAN allows Aviatrix Transit Gateways to communicate with multiple instances in the same VPC in GCP without running any tunneling protocol such as IPsec or GRE (traffic needs to be sent to the subnet gateway for forwarding). ## Introduction Transit BGP to LAN allows Aviatrix Transit Gateways to communicate with multiple instances in the same VPC in GCP without running any tunneling protocol such as IPsec or GRE (traffic needs to be sent to the subnet gateway for forwarding). One use case is to interoperate with third-party virtual appliances such as SD-WAN cloud instances that do not have the capability to support BGP over any tunneling protocols. For example, integrating with SD-WAN gateways can be deployed as below, where Aviatrix Multicloud Transit Gateways connect to third-party cloud instances in the same VPC in GCP: ![SD-WAN Integration in GCP](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/routing/sd-wan-integ-gcp.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=308a9e68b9b1ec6751c8e28788c4a6f9) This document describes a step-by-step instruction on how to build Aviatrix Transit Gateway to External Device using BGP over LAN. In this Tech Note, you will learn the following: 1. [Deploy Aviatrix Transit Solution](#deploying-aviatrix-multicloud-transit-solution) 2. [Launch third-party cloud instances](#launching-third-party-cloud-instances) 3. [Build BGP over LAN](#building-bgp-over-lan) For other BGP over LAN workflows, see: * [Multicloud Transit BGP over LAN in AWS Workflow](/docs/enterprise/10.1/guides/connectivity/routing/transit-externaldevice-aws-bgpolan) * [Transit Gateway BGP over LAN in Azure Workflow](/docs/enterprise/10.1/guides/connectivity/routing/transit-externaldevice-azure-bgpolan) For more information about Multicloud Transit Network and External Device, see: * [Multicloud Global Transit FAQ](/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/transit-network-faq) * [Global Transit Network Workflow Instructions (AWS/Azure/GCP/OCI)](/docs/enterprise/10.1/guides/connectivity/transit/transitvpc-workflow) * [Aviatrix Transit Gateway to External Devices](/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/transit-externaldevice-usecase) * [Transit Network Design Patterns](/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/multi-cloud-design-patterns) > [!NOTE] > > * ActiveMesh 2.0 is required. To migrate to ActiveMesh 2.0, see [Migrating > from Classic Aviatrix Encrypted Transit Network to Aviatrix ActiveMesh Transit > Network](https://legacy.docs.aviatrix.com/documentation/latest/network/activemesh-migration.html). > * This solution is available in Azure when connecting to a single BGP peer. > Multi-peer BGP is supported in GCP and AWS. The workflow with GCP here is an > example. Adjust the topology depending on your requirements. \* GCP does not > allow interfaces to be added to an instance after deployment. Verify the > design before creating the instances to make sure they have all the required > interfaces. ### The key ideas for this solution are: * A BGP session establishes between third-party cloud instances and Aviatrix Transit Gateways via each LAN interface in the same VPC. * Data plane traffic also runs between third-party cloud instances and Aviatrix Transit Gateways via each LAN interface without a tunnel protocol such as IPsec or GRE. ### Prerequisites * [Upgrade](/docs/enterprise/10.1/guides/maintenance/upgrade-overview) Aviatrix Controller to at least version 6.6. * Third-party cloud instance supports high throughput. ### Deploying Aviatrix Multicloud Transit Solution Refer to [Global Transit Network Workflow Instructions](/docs/enterprise/10.1/guides/connectivity/transit/transitvpc-workflow) for the below steps. Adjust the topology depending on your requirements. 1. Deploy [Aviatrix Multicloud Transit Gateway and HA](https://legacy.docs.aviatrix.com/documentation/latest/network/transit-gateway-about.html?expand=true) with High Performance Encryption Mode encryption enabled in Transit VPC. Ensure that you create the required number of BGP over LAN connections. 2. Deploy [Spoke Gateway and HA](https://legacy.docs.aviatrix.com/documentation/latest/network/spoke-gateway-about.html?expand=true) with High Performance Encryption Mode encryption enabled in Spoke VPC(s). 3. [Attach Spoke Gateways to Transit Network](https://legacy.docs.aviatrix.com/documentation/latest/network/spoke-gateway-attachments.html?expand=true). ### Launching Third-Party Cloud Instances Deploy third-party cloud instances with an interface in the same VPC as the Aviatrix Transit Gateway. 1. Create a third-party cloud instance and put the MGMT interface in public gateway subnet. 2. Create a new WAN subnet and dedicated routing table for the WAN interface if needed. 3. Create a new LAN subnet and a dedicated routing table for the LAN interface. 4. Make sure the IP forwarding function is enabled on the third-party cloud instances. > [!NOTE] > > GCP allows a maximum of eight interfaces per instance, and the maximum limit > depends on the number of vCPUs. Due to this limitation, the solution supports > seven BGP peers without FireNet enabled and six BGP peers with FireNet > enabled. ### Building BGP over LAN **CoPilot UI** 1. In CoPilot, navigate to Networking > Connectivity > External Connections (S2C). 2. Click *+External Connection*. 3. In the Add External Connection dialog, select *Connect Public Cloud to External Device*. 4. Select *BGP over LAN* from the drop-down. 5. Enter the following information in the fields provided. | Setting | Value | | - | - | | Name | Unique name to identify the external device connection. | | Local Gateway | Select the Transit VPC ID where the Transit gateway was launched (BGP over LAN must have been enabled for this Transit gateway). | | Local ASN | The local BGP AS number (for on-site equipment) the Spoke gateway will use to exchange routes with the external device. | | Remote ASN | Enter the BGP AS number of the cloud location the external device will use to exchange routes with the Transit Gateway. | | BGP ActiveMesh | Enable full mesh BGP connections to the external devices. | | Learned CIDR Approval | Set to On by default if selected for the Local Gateway. Otherwise, it is set to Off. | | Remote LAN IP | The private IP of the LAN interface of the third-party cloud primary instance. | | Local LAN IP | Aviatrix detects the Local LAN IP automatically and assigns an IP in the same subnet as the Remote LAN IP. Optionally, you can configure a specific IP within the same subnet as the Remote LAN IP. | 6. Click *Connect* to generate the BGP sessions. 7. Create an external connection for each BGP peer. **Terraform** Build BGP over LAN with the [aviatrix\_transit\_external\_device\_conn](https://registry.terraform.io/providers/AviatrixSystems/aviatrix/latest/docs/resources/aviatrix_transit_external_device_conn) resource. The referenced Transit Gateway must already have BGP over LAN interfaces provisioned. Add one resource block per BGP peer. **1. Create a BGP over LAN external device connection for each peer** Add an `aviatrix_transit_external_device_conn` resource block per third-party cloud instance to your Terraform configuration file: ```hcl theme={null} resource "aviatrix_transit_external_device_conn" "gcp_bgp_over_lan_peer1" { vpc_id = "transit-gcp-vpc~-~my-gcp-project" connection_name = "gcp-bgp-over-lan-conn-1" gw_name = "transit-gw-gcp-01" connection_type = "bgp" tunnel_protocol = "LAN" bgp_local_as_num = "65001" bgp_remote_as_num = "65011" remote_lan_ip = "10.3.4.10" local_lan_ip = "10.3.4.9" } resource "aviatrix_transit_external_device_conn" "gcp_bgp_over_lan_peer2" { vpc_id = "transit-gcp-vpc~-~my-gcp-project" connection_name = "gcp-bgp-over-lan-conn-2" gw_name = "transit-gw-gcp-01" connection_type = "bgp" tunnel_protocol = "LAN" bgp_local_as_num = "65001" bgp_remote_as_num = "65012" remote_lan_ip = "10.3.5.10" local_lan_ip = "10.3.5.9" } ``` > [!NOTE] > > For GCP, `vpc_id` uses the format `"vpc_name~-~project_name"`. Unlike > the CoPilot UI, which can auto-detect and assign the Local LAN IP, > Terraform requires `local_lan_ip` to be set explicitly for each > connection. Omitting it fails with > `[AVXERR-TRANSIT-0208] Missing Local(HA) LAN IP parameter`. Set > `local_lan_ip` to one of the IP addresses already assigned to the > Transit Gateway's BGP over LAN interfaces (visible in the > `bgp_lan_ip_list` attribute of the > [aviatrix\_transit\_gateway](https://registry.terraform.io/providers/AviatrixSystems/aviatrix/latest/docs/resources/aviatrix_transit_gateway) > resource, or in CoPilot). **2. Verify and Apply the Configuration** Run `terraform plan`, review the planned changes, and run `terraform apply`. ## Related Resources * [aviatrix\_transit\_external\_device\_conn Resource](https://registry.terraform.io/providers/AviatrixSystems/aviatrix/latest/docs/resources/aviatrix_transit_external_device_conn) #### (Optional) Downloading the BGP over LAN configuration sample from Aviatrix Controller 1. On the External Connections (S2C) tab, click the vertical ellipsis and then select *Download Configuration*. 2. In the Download Configuration dialog, select the following: * Vendor: select the device you are using (any device that is capable of running IPsec and BGP). * Platform: select the applicable platform for the chosen device. * Software: automatically selected based on the Vendor/Platform you select. 3. Click *Download*. #### Configuring BGP over LAN on the Third-Party Cloud Instance(s) 1. (Optional) Open the downloaded BGP over LAN configuration file. 2. Configure the relevant BGP over LAN information on the third-party cloud instance(s). #### Verifying the Connection Status Navigate to Diagnostics > Cloud Routes > Gateway Routes to check the gateway and tunnel status. #### Verifying the BGP session status on Aviatrix Controller Navigate to Diagnostics > Cloud Routes > BGP Info and expand a Gateway Name to check the Neighbor Status. The Status should be Established. If some external connections for the selected Transit Gateway are Not Established, the overall BGP Status for the Transit Gateway is Partially Established. ### Ready to Go At this point, run connectivity and performance tests to ensure everything is working correctly. You can use the [Diagnostic Tools](/docs/enterprise/10.1/guides/security/firenet/transit-firenet-diagnostics) for this. # Configure VRF End-to-End Propagation Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/routing/vrf-configuration Step-by-step instructions for enabling VRF end-to-end propagation across spoke-transit and transit-transit peerings in Aviatrix CoPilot, and for verifying that segment identity and traffic isolation carry through the transit layer. Virtual Routing and Forwarding (VRF) end-to-end propagation extends an existing first-hop VRF configuration so that a spoke's segment identity carries through the transit layer to the remote side of a spoke-transit or transit-transit peering, instead of terminating at the first transit hop. This guide covers enabling that propagation in Aviatrix CoPilot, verifying that segment identity propagated successfully, and validating that traffic between different VRF segments remains isolated. ## Prerequisites Before you enable VRF end-to-end propagation, confirm the following. * Controller version: Aviatrix Controller is running version 10.1 or later. * Gateway software version: Both gateways in each peering you plan to configure (the spoke or Edge Spoke gateway and its peer Transit Gateway, or the two Transit Gateways in a transit-transit peering) must be running gateway software 10.1 or later. * Existing first-hop VRF configuration: A [first-hop VRF configuration](/docs/enterprise/10.1/guides/security/network-segmentation-secured#segmentenable) from Controller 9.0 must already be in place: VRF support enabled at the controller level, with spoke gateways attached to their segmentation domains. * Supported cloud service providers: AWS, Azure, and GCP are supported. > [!NOTE] > > On Azure, VRF end-to-end propagation does not support spoke VPCs or VNets > that have overlapping CIDR ranges. * Existing spoke-transit or transit-transit peering: At least one peering already established between the gateways you want to propagate VRF end-to-end across. * Administrative access: Access to **Networking** > **Network Segmentation** > **Settings** in CoPilot. ## Enable VRF End-to-End Propagation ![CoPilot Network Segmentation Settings page showing the Support for VRF card with the toggle, Transit Peering Up-to-date and Stale counts, and the Rollout Changes to Transit Peerings button](https://mintcdn.com/aviatrix-14b37c43/8ugLIgiKPrIvpbRt/images/guides/connectivity/routing/enable-vrf-end-to-end-propagation.png?fit=max\&auto=format\&n=8ugLIgiKPrIvpbRt\&q=85\&s=ec547aedd9fdde7498a0a39d59413eb2) **1. Open Network Segmentation settings** In Aviatrix CoPilot, go to **Networking** > **Network Segmentation** > **Settings**. **2. Confirm Support for VRF is enabled** Locate the **Support for VRF** card and confirm the toggle switch is set to **On**. This reflects the first-hop VRF configuration from the [Prerequisites](#prerequisites). If the toggle is **Off**, VRF has not been enabled at the controller level, and you must complete [that configuration](/docs/enterprise/10.1/guides/security/network-segmentation-secured#segmentenable) before continuing. **3. Review peering attachment status** In the **Support for VRF** card, review the **Transit Peering Up-to-date** and **Transit Peering Stale** counts. A peering counted as stale has not yet had VRF end-to-end propagation applied. **4. Open the rollout dialog** Click **Rollout Changes to Transit Peerings**. The **Rollout Support for VRF** dialog opens, listing every eligible peering with its **Transit/Edge Spoke Gateway**, **Peering Type** (Intra-Group, Transit-Transit, Edge Spoke-Transit, or BGP Spoke-Transit), and **Transit Gateway Instance**. **5. Select the peerings to propagate** Select the checkbox next to each spoke-transit (Edge Spoke-Transit or BGP Spoke-Transit) or transit-transit peering where you want to enable VRF end-to-end propagation. Clear the checkbox for any peering where you want to disable it. **6. Roll out the change** Click **Rollout**. CoPilot applies the change to every selected peering. > [!NOTE] > > Disabling **Support for VRF** entirely (turning the card's main toggle > **Off**) affects every peering at once and causes all transit peerings to > become stale. Do not turn off the main toggle unless you intend to remove VRF > end-to-end propagation across your entire environment. ## Verify Segment Identity in CoPilot After you roll out the change, confirm that segment identity is propagating end-to-end across the peerings you selected. **1. Recheck the peering attachment status** Return to **Networking** > **Network Segmentation** > **Settings** and open the **Support for VRF** card. Confirm that the peerings you enabled now count toward **Transit Peering Up-to-date** rather than **Transit Peering Stale**. **2. Confirm the segmentation domain mapping** Confirm that each spoke's segmentation domain (its VRF identity) is still correctly reflected on **Networking** > **Network Segmentation** > **Network Domains** after propagation. ## Validate Traffic Isolation VRF end-to-end propagation is only correctly configured if traffic within a segment still reaches its intended destinations and traffic across different segments remains blocked. **1. Confirm same-segment connectivity** From a workload in one spoke, confirm connectivity to a workload in another spoke that shares the same VRF segment across the peering you configured. Connectivity should succeed. **2. Confirm cross-segment isolation** From the same workload, attempt connectivity to a workload in a spoke that belongs to a different VRF segment across the same peering. The connection should fail, confirming that traffic isolation between segments is preserved end-to-end. **3. Inspect VRF-tagged tunnel traffic (optional)** To inspect the underlying tunnel traffic instead of only its connectivity outcome, go to **Diagnostics** > **Diagnostic Tools** > **Gateway Diagnostics** > **Packet Capture** and select the gateway whose tunnel you want to inspect. If both the Controller and the selected gateway are on 10.1 or later, a **Skip Aviatrix Headers** toggle appears; enabling it strips the Aviatrix VRF tunnel headers from the capture so filters and saved captures show the original inner packets instead of the VRF-tagged outer headers. ## Related Resources * [VRF Multi-Tenancy](/docs/enterprise/10.1/concepts-architectures/components/connectivity/vrf-overview) * [VRF Design Patterns](/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/vrf-design-patterns) # Migrating a CSR Transit to AWS Transit Gateway (TGW) Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/tgw/tgw-csr-migrate This document assumes that you have deployed a CSR Transit solution with Transit hub CSR instances and VGWs in Spoke VPCs. The steps below provide instructions to migrate a live CSR deployment to Aviatrix with the Transit Gateway Orchestrator. This document assumes that you have deployed a [CSR Transit solution](https://aws.amazon.com/answers/networking/aws-global-transit-network/) with Transit hub CSR instances and VGWs in Spoke VPCs. The steps below provide instructions to migrate a live CSR deployment to Aviatrix with the Transit Gateway Orchestrator. The objectives here are: * No change to any on-prem network. * No change to the connectivity between AWS VGW and on-prem. (either over DX or over Internet or both) * Re-use AWS VGW deployed in CSR based Transit hub VPC if possible. * No change to existing VPC infrastructure. * Minimum operation downtime. There are a couple of patterns during the migration phase, consider the one that meets your requirements. > [!NOTE] > > This document assumes you have already launched an Aviatrix Controller. Before the migration process starts, plan out what network domains you need to create and which network domains should connect other domains. If you are not sure and need to transition, proceed with no worries. The network domains can be added and modified at any time. 1. [Launch a Transit Gateway](https://legacy.docs.aviatrix.com/documentation/latest/network/tgw-create.html). 2. If you have plans for custom network domains, follow [these instructions](/docs/enterprise/10.1/guides/security/network-segmentation-secured) to create them. Then, build connection policies. If you do not intend to build custom network domains, skip this section. 3. [Launch an Aviatrix Transit GW and enable HA in the Transit hub VPC](https://legacy.docs.aviatrix.com/documentation/latest/network/tgw-vpc-attachment-create.html). As a best practice, create a new Transit hub VPC to deploy the Aviatrix Transit GW. 4. This step has two options: * Option A: [Connect Aviatrix Transit GW to VGW](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-create-aws-vgw.html). At this point, VGW starts to advertise to the Aviatrix Transit GW. Make sure you specify a different "AS" number for the BGP session of the Aviatrix Transit GW connection to the VGW. Also note that if the Transit GW and the VGW are in the same account and same VPC, VGW must be detached from the VPC. A diagram for this migration path is shown below: ![tgw\_csr\_migrate\_pattern1](https://mintcdn.com/aviatrix-14b37c43/0QYX2iFaLoliwUis/images/guides/connectivity/tgw/tgw-csr-migrate-pattern1.png?fit=max\&auto=format\&n=0QYX2iFaLoliwUis\&q=85\&s=0b46c6e9928ecf4be00d62ac452f7577) * Option B: Connect Aviatrix Transit GW to CSR. There are certain situations where you need to keep the CSR as the connection point to on-prem (for example, you need to use CSR route summarization feature to control routes to VGW to be under 100). In such a scenario, use [External Device](https://legacy.docs.aviatrix.com/documentation/latest/network/transit-externaldevice-connect.html) option in [Transit VPC workflow](/docs/enterprise/10.1/guides/connectivity/transit/transitvpc-workflow) to create an IPSec and BGP connection to CSR, as shown in the diagram below. After all Spoke VPCs are migrated, delete the connection to CSR, connect the Aviatrix Transit GW to VGW. ![tgw\_csr\_migrate\_pattern2](https://mintcdn.com/aviatrix-14b37c43/0QYX2iFaLoliwUis/images/guides/connectivity/tgw/tgw-csr-migrate-pattern2.png?fit=max\&auto=format\&n=0QYX2iFaLoliwUis\&q=85\&s=64a1691a628b9a57a46b0fdbebd38f8f) 5. Remove a Spoke VPC. Select one Spoke VPC that has VGW deployed. Remove the VPC Transit Network tag. This will effectively detach the Spoke VPC from the CSR Transit Network. Make sure the above Spoke VPC CIDR route entry has been removed from the Transit Network. 6. [Attach a Spoke VPC](https://legacy.docs.aviatrix.com/documentation/latest/network/tgw-vpc-attachment-create.html) to the corresponding network domain. 7. Repeat steps 5 and step 6 for the remaining Spoke VPCs. 8. Remove the Transit Hub VGW CSR tag after all Spoke VPCs have been migrated to Aviatrix Transit GW. This effectively detaches the VGW from CSR. The effective operation downtime for each Spoke VPC is the time between the Transit Network tag being removed for the Spoke VPC and the Spoke VPC being attached to Aviatrix Transit GW. It should be a few minutes. # Migrating a DIY TGW to Aviatrix Managed TGW Deployment Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/tgw/tgw-migrate-aviatrix-tgw Use this document to migrate to an Aviatrix-managed TGW deployment from an existing AWS Transit Gateway (TGW) deployment. Use this document to migrate to an Aviatrix-managed TGW deployment from an existing AWS Transit Gateway (TGW) deployment. The objectives here are: * Minimum downtime during migration. * No change to existing VPC infrastructure. * Minimum change to on-prem network. > [!NOTE] > > This document assumes you have already > [deployed Aviatrix CoPilot](/docs/enterprise/10.1/guides/controlplane/copilot-deployment) > . Before the migration process starts, plan out what network domains you need to create and which network domains should connect other domains. If you are not sure and need to transition, proceed with no worries. The network domains can be added and modified at any time. ## The Solution There are multiple ways to migrate. For example, you can simply detach a spoke VPC from the existing TGW and attach it to Aviatrix-managed TGW and then build hybrid connection if necessary. In this implementation, the migrated spoke VPCs can communicate with the not yet migrated VPCs during migration process, and in addition the migrated spoke VPCs can communicate with on-prem network, thus reducing the downtime, as shown in the migration architecture below. ![migration\_architecture](https://mintcdn.com/aviatrix-14b37c43/0QYX2iFaLoliwUis/images/guides/connectivity/tgw/migration-architecture.png?fit=max\&auto=format\&n=0QYX2iFaLoliwUis\&q=85\&s=ef74f274057fe5fc860fec078c2ed754) The key idea is to build an IPsec tunnel between TGW VPN and Aviatrix Transit Gateway, so that migrated VPC can communicate with not yet migrated VPCs and also to on-prem. 1. [Launch a new AWS Transit Gateway](https://legacy.docs.aviatrix.com/documentation/latest/network/tgw-create.html). 2. If you have plans for custom network domains, [create new network domains](/docs/enterprise/10.1/guides/security/network-segmentation-secured) . Then, build your domain connection policies. If you do not intend to build custom network domains, skip this section. 3. [Launch an Aviatrix Transit GW and enable HA in the Transit Hub VPC](https://legacy.docs.aviatrix.com/documentation/latest/network/tgw-vpc-attachment-create.html). As a best practice, create a new Transit Hub VPC to deploy the Aviatrix Transit GW. Create a TGW VPN Attachment using the steps below. ## Creating TGW VPN Attachment 1. [Create](https://legacy.docs.aviatrix.com/documentation/latest/network/tgw-vpn-connection-create.html) a VPN attachment. 2. After the attachment is created, download your VPN file. Make sure you select Vendor *Generic* and download the configuration text file. ## Creating VPN on Aviatrix Transit Gateway This step is to create the other end of the VPN tunnel that terminates on the Aviatrix Transit GW. 1. [Create an external connection](/docs/enterprise/10.1/guides/connectivity/vpn/transit-externaldevice-workflow) using the information from the downloaded VPN file. 2. Select BGP as the connection type. 3. Use the downloaded VPN file to enter the following information: | Information from Downloaded VPN File | Field in External Connection | | - | - | | Customer Gateway ASN | Aviatrix Transit Gateway BGP ASN | | BGP Remote ASN Number | Virtual Private Gateway ASN | | Virtual Private Gateway under *Outside* IP Addresses | Remote Device IP | | Pre-shared Key | Pre-shared Key | | Customer Gateway | Local Tunnel IP | | Virtual Private Gateway under *Inside* IP Addresses | Remote Tunnel IP | ## Starting to Migrate VPCs [Detach](https://legacy.docs.aviatrix.com/documentation/latest/network/tgw-vpc-attachment-detach.html) your VPCs from the existing TGW [attach](https://legacy.docs.aviatrix.com/documentation/latest/network/tgw-vpc-attachment-create.html) them to an Aviatrix-managed TGW. Before or after you detach a VPC, you may need to clean up the VPC route tables entries so that they do not have conflict routes entries when later attaching it to Aviatrix-managed TGW. Repeat this step to migrate all VPCs. ## Building the Hybrid Connectivity Once all VPCs have migrated to Aviatrix-managed TGW deployment, the migrated VPCs communicate with on-prem via Aviatrix Transit GW to the existing TGW and then to on-prem. At this point, you can move existing TGW Direct Connect to Aviatrix Transit GW or to Aviatrix managed TGW directly. ## Deleting DIY TGW After all VPCs and hybrid connectivity if any are all removed, you can [delete](https://legacy.docs.aviatrix.com/documentation/latest/network/tgw-detach.html) the existing TGW. # Overview of AWS Transit Gateway Orchestrator Features Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/tgw/tgw-overview The Aviatrix CoPilot > Networking > Connectivity > AWS TGW tab provides a list of AWS Transit Gateways. Select any AWS TGW on this page to review and create Attachments, Network Domains, Connection Policies, TGW Routes, or Approvals for any AWS TGW. The **Aviatrix CoPilot > Networking > Connectivity > AWS TGW tab** provides a list of AWS Transit Gateways. Select any AWS TGW on this page to review and create Attachments, Network Domains, Connection Policies, TGW Routes, or Approvals for any AWS TGW. For background information, refer to the [TGW Orchestrator FAQ](/docs/enterprise/10.1/concepts-architectures/components/tgw/tgw-orchestrator-faq) . On this page, you can: * [Create](https://legacy.docs.aviatrix.com/documentation/latest/network/tgw-create.html) an AWS TGW. * [Edit](#audit-settings) audit settings. * Create AWS TGW [attachments](#attachments-tab), including attachments to Transit Gateways, VPCs, VPNs, Direct Connects, AWS TGW Peerings, and TGW Connections. * Create TGW-based [Network Domains](#creating-an-aws-tgw-network-domain). * Review [connection policies](#connection-policies-tab). * Review [TGW Routes](#tgw-routes-tab). * Review [TGW Approvals](#approval-tab). ## Creating an AWS TGW To use the AWS TGW (Transit Gateway) feature, you must first create an AWS Transit Gateway. This step creates an AWS Transit Gateway in a specified region with a specified AWS account. Aviatrix CoPilot also automatically creates the **Default\_Domain**, the **Shared\_Service\_Domain** and the **Aviatrix\_Edge\_Domain** and the corresponding AWS Transit Gateway route tables. The three domains are connected. If you attach a VPC to the Default Domain or Shared Service Domain, the VPCs can communicate with each other and can access on-premises environments through the Aviatrix Edge Domain. The three domains are connected, implying that if you attach a VPC to the Default Domain or Shared Service Domain, the VPCs can communicate with each other and can access on-premises through the Aviatrix Edge Domain. **CoPilot UI** To create an AWS Transit Gateway: 1. Go to **Aviatrix CoPilot > Networking > Connectivity > select the AWS TGW tab**. 2. Click **+ AWS TGW**. ![create\_tgw](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/tgw/create-tgw.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=ab976e7a262786708d653caced1e129e) | Setting | Value | | - | - | | Account Name | An [Aviatrix account](/docs/enterprise/10.1/guides/platform-administration/aviatrix-account) that corresponds to an IAM role or account in AWS. | | Region | One of the AWS regions. | | TGW Name | The name of the AWS Transit Gateway. | | AWS Side AS Number | TGW ASN number. The default AS number is 64512. | | **Advanced Settings** | | | FireNet Inspection Mode | Select either mode: **Domain-Based** - This mode allows you to specify a Spoke VPC/VNet that needs inspection by defining a connection policy of the Spoke VPC/VNet's Security Domain to the Firewall Domain. **Connection-Based** - This mode allows you to inspect traffic going across a specific pair of Security Domains. This inspection mode reduces the amount of traffic being inspected and reduces the instances size requirements on both FireNet Gateways and firewalls. | | TGW CIDR(s) | Enter the TGW CIDR ranges. | 3. Click **Save**. The AWS TGW is created. If for some reason it was not created, you can go to Monitor > Notifications > Tasks and check what errors occurred during creation. **Terraform** Create an AWS Transit Gateway with the [aviatrix\_aws\_tgw](https://registry.terraform.io/providers/AviatrixSystems/aviatrix/latest/docs/resources/aviatrix_aws_tgw) resource. The three default domains (Default\_Domain, Shared\_Service\_Domain, and Aviatrix\_Edge\_Domain) and their connections are created automatically, matching the CoPilot UI behavior. **1. Create the AWS TGW** Add the following `aviatrix_aws_tgw` resource block to your Terraform configuration file: ```hcl theme={null} resource "aviatrix_aws_tgw" "tgw" { account_name = "devops" tgw_name = "aws-tgw-01" region = "us-east-1" aws_side_as_number = "64512" } ``` **2. Verify and Apply the Configuration** Run `terraform plan`, review the planned changes, and run `terraform apply`. ## Audit Settings To edit audit settings: 1. Go to **Aviatrix CoPilot > Networking > Connectivity > AWS TGW tab**. 2. Click **Audit Settings**. 3. Turn the **Auto Edit (Every Night)** setting on to set up an audit every night. This setting is Off by default. When it is Off, only manual auditing occurs. 4. Click **Save**. Your edits are saved. ## Attachments tab The Attachments tab lists the TGW attachments. To review attachments: 1. Go to **Aviatrix CoPilot > Networking > Connectivity > select the AWS TGW tab > select an existing AWS TGW**. 2. Select the **Attachments** tab. Use the tabs under Attachments to review and create attachments to: * [Transit Gateways](https://legacy.docs.aviatrix.com/documentation/latest/network/tgw-attach-transit-gw-to-tgw.html) * [VPCs](https://legacy.docs.aviatrix.com/documentation/latest/network/tgw-vpc-attachment-create.html) * [VPNs](https://legacy.docs.aviatrix.com/documentation/latest/network/tgw-vpn-connection-create.html) * [Direct Connects](https://legacy.docs.aviatrix.com/documentation/latest/network/tgw-direct-connect.html) * [AWS TGW Peerings](https://legacy.docs.aviatrix.com/documentation/latest/network/tgw-peering-attachment-create.html) * [TGW Connections](https://legacy.docs.aviatrix.com/documentation/latest/network/tgw-connect.html) ## Network Domains tab The Network Domains tab displays the TGW route table entries. You can also add an AWS TGW network domain here. ### Creating an AWS TGW Network Domain To create network domains in an AWS TGW: 1. Go to Networking > Connectivity > AWS TGW. 2. Click an existing AWS TGW and go to the Network Domains tab. 3. Click **+ Network Domain**. 4. Enter a name for the network domain. 5. Select the network domain type: * User Domain * Aviatrix Firewall Domain: Select if this domain will be used in a Transit FireNet configuration. You can then select a previously created AWS TGW FireNet (optional). > [!NOTE] > > If you select an AWS TGW FireNet, it must be in the same region as the AWS > TGW (for example, `us-east-1`). * Native Firewall Domain: Select if this domain will be used for non-Aviatrix Transit FireNet based firewall traffic inspection (only supports an active-standby firewall deployment). * Native Egress Domain: Select if this domain will be used for non-Aviatrix Transit FireNet based Internet-bound traffic (only supports an active-standby firewall deployment). 6. Click **Save**. If you associate one of the above AWS TGW network domains to the network domain, now any resources (regardless of whether they are AWS TGW or Aviatrix Transit) that share the network domain can communicate with each other. The following is now also possible: * AWS TGW native Edge connections can route to Spokes attached to an Aviatrix Transit network. * Can peer an AVX Transit to an AWS TGW that is connected to on-premises. The network domains in both of these domains will work across Transit gateway peering and vice versa (but not AWS TGW peering). * Any two Aviatrix network domains that are connected are able to route to TGW Shared Services (for example, if Transit domain 1 is connected to Transit domain 2, and TGW domain A is associated with Transit domain 1, then TGW domain A can communicate with Spoke gateways in Transit domain 2). * TGW domains associated with any two connected Aviatrix network domains can communicate with each other (as long as they are not in the same TGW). > [!NOTE] > > VPCs in the same AWS TGW but in different domains cannot communicate with each > other unless you configure a connection policy. > > For a VPC attached to the TGW domains that are connected to an Edge domain, > its VPC CIDR is propagated to the main route table of the Transit gateway and > its connected Transit/Spoke gateways. ### Enabling Intra Domain Inspection Intra domain inspection allows traffic between AWS TGW VPC/VNets in the same Network Domain to be redirected to a Firewall Domain for inspection before reaching the destination. By default, traffic between VPCs in the same Network Domain does not get inspected by firewalls in the FireNet deployment. > [!NOTE] > > You can only enable Intra Domain Inspection if the selected AWS TGW has one of > its Firewall Domains selected in an AWS TGW FireNet. To enable intra domain inspection: 1. Go to **Aviatrix CoPilot > Networking > Connectivity > select the AWS TGW tab > select the AWS TGW > select the Network Domains tab**. 2. Find the Shared\_Service\_Domain in the table and click the three dots icon in its row. Select **Enable Intra Domain Inspection**. 3. In the dialog, select a Firewall Domain. 4. Click **Enable**. ### Enabling Egress Inspection This option applies to connection-based inspection mode. When connection-based inspection is enabled, use this option to enable Egress inspection for a specific domain. To enable intra domain inspection: 1. Go to **Aviatrix CoPilot > Networking > Connectivity > select the AWS TGW tab > select the AWS TGW > select the Network Domains tab**. 2. Find the Network Domain in the table and click the three dots icon in its row. Select **Enable Egress Domain Inspection**. ## Connection Policies tab To view Connection Policies: 1. Go to **Aviatrix CoPilot > Networking > Connectivity > select the AWS TGW tab > select an existing AWS TGW**. 2. Select the **Connection Policies** tab. 3. Use the Network Domain dropdown menu to view all gateways in that domain that are available for connection to this AWS TGW. ## Inspecting Inter Region Traffic The Network Domain associated with each TGW Peering attachment is available for user. The Network Domain has the name **peering\_\**. For example, for the TGW with name tgw-1, the peering Network Domain is **peering\_tgw-1**. You can specify a FireNet inspection policy on this Network Domain. When you do so, it implies that any cross-region traffic is inspected. To connect the peering domain with FireNet Domain: 1. Go to **Aviatrix CoPilot > Networking > Connectivity > select the AWS TGW tab > select an existing AWS TGW**. 2. Select the **Connection Policies** tab. 3. Find the Network Domain in the table. In the Connection Policy column, click the toggle switch to enable the connection. 4. Click **Commit**. > [!NOTE] > > To avoid double inspections by two FireNet gateways associated with each TGW, > configure the connection policy between peering domain and FireNet domain on > only one TGW. ## TGW Routes tab The TGW Routes tab displays the routes in each network domain. To review TGW Routes: 1. Go to **Aviatrix CoPilot > Networking > Connectivity > select the AWS TGW tab > select an existing AWS TGW**. 2. Select the **TGW Routes** tab. 3. Click the **Network Domains** dropdown menu to select a different domain to review. ## Approval tab ### TGW Approval ![tgw\_approval](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/tgw/tgw-approval.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=84affecb28bebdd0d341be92cfe90f68) TGW VPN and TGW Direct Connect Gateway (TGW DXGW) dynamically learns BGP routes from remote peer. Aviatrix CoPilot periodically pulls the TGW route table and propagate these routes to Spoke VPCs route table that have connection policy to the VPN. There are scenarios where you require an approval process before these learned CIDRs propagation take place. For example, a specific TGW VPN may be connected to a partner network and you need to make sure undesirable routes, such as the default route (0.0.0.0/0) are not propagated into your own network and accidentally bring down the network. Approval is enabled on per TGW VPN and TGW DXGW bases. When Approval is enabled on a TGW VPN, dynamically learned routes trigger an email to the CoPilot admin. To review Approvals: 1. Go to **Networking > Connectivity > select the AWS TGW tab > select the AWS TGW > select the Approval tab**. 2. Click the **AWS TGW Attachment** dropdown to select the attachment to review. 3. Make sure **Learned CIDR Approval** is enabled. 4. All routes appear, both unapproved and already approved. 5. To approve an unapproved route, click **Approve** in its row. Now, the route can be propagated. 6. To disapprove an approved route, click **Remove** in its row. ### How the Approval Feature Works When Learned CIDR Approval is enabled, TGW route table route propagation to the connected Network Domain is turned off. That is, the TGW VPN/DXGW learned routes are statically programmed into the TGW route table of connected Network Domains after the routes are approved. This is illustrated in the following two examples. #### Example 1: Two TGW VPN/DXGW in the same domain ![tgw\_two\_vpn\_approval](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/tgw/tgw-two-vpn-approval.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=28c85d98c3627ac718290a4df42c5eae) In the example above, two identical VPN CIDRs 10.10.1.0/24 are advertised to two TGW VPNs but are in the same domain. Both have Approval enabled. Whichever VPN attachment learns the CIDR first and is approved, its attachment is programmed into Spoke associated TGW route table, in this case, VPN1 attachment is approved first and is programmed into the Spoke associated TGW route table. VPN2 CIDR should continue to remain in pending list. If VPN1 withdraw route 10.10.1.0/24, you can initiate approval by moving the VPN2 pending CIDR to the approved panel, and this time it should be programmed. #### Example 2: One TGW VPN requires approval and another one does not ![tgw\_vpn\_different\_domains](https://mintcdn.com/aviatrix-14b37c43/DtLW-mqMRi95qZN9/images/guides/connectivity/tgw/tgw-vpn-different-domains.png?fit=max\&auto=format\&n=DtLW-mqMRi95qZN9\&q=85\&s=b60255346e19289e6bbed1429c0cd9f6) In the second example, TGW VPN2 link 10.10.9.0/24 is in a different domain and does not require approval. Its route is propagated to the Spoke TGW route table, while TGW VPN1 link 10.10.1.0/24 is statically programmed to Spoke TGW route table after approval is initiated by the customer. Note in the second example, if TGW VPN2 link advertises the same network CIDR 10.10.1.0/24, this CIDR will be propagated first and TGW VPN1 approval request will be rejected and the CIDR 10.10.1.0/24 from TGW VPN1 remains in the approval pending list. # AWS Transit Gateway Route Limit Test Validation Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/tgw/tgw-route-limit While an AWS Transit Gateway (TGW) carries thousands of routes in the TGW route table, a TGW VPN has the same 100 route limit as the VGW VPN. ## Introduction While an AWS Transit Gateway (TGW) carries thousands of routes in the TGW route table, a TGW VPN has the same 100 route limit as the VGW VPN. ## Test Validation In the following setup, we launch a Cisco CSR 1000v as the Customer Gateway and use it to attach a VPN connection to the TGW. The TGW and the CSR also run BGP between themselves. The detailed steps are [here](https://docs.aws.amazon.com/vpc/latest/tgw/tgw-vpn-attachments.html). ![tgw-route-limit](https://mintcdn.com/aviatrix-14b37c43/0QYX2iFaLoliwUis/images/guides/connectivity/tgw/tgw-route-limit.png?fit=max\&auto=format\&n=0QYX2iFaLoliwUis\&q=85\&s=8af4537b69b56329808ae7af5f71dcc2) After the VPN tunnel is created between TGW and CSR, run the following command at CSR to verify the BGP peer connection, where 169.254.11.233 is the TGW's BGP ID: ``` ip-10-10-0-106#show ip bgp neighbors 169.254.11.233 | include BGP state BGP state = Established ``` Turn on BGP debug on CSR and then configure CSR to advertise 101 routes to TGW through BGP. CSR shows the following BGP debug messages: ``` *Dec 18 00:29:28.213: %BGP-3-NOTIFICATION: received from neighbor 169.254.11.233 6/1 (Maximum Number of Prefixes Reached) 7 bytes 00010100 000064 *Dec 18 00:29:28.213: BGP: ses global 169.254.11.233 (0x7F36A1AF6C60:1) Receive NOTIFICATION 6/1 (Maximum Number of Prefixes Reached) 7 bytes 00010100 000064 *Dec 18 00:29:28.213: %BGP-5-NBR_RESET: Neighbor 169.254.11.233 reset (BGP Notification received) *Dec 18 00:29:28.213: BGP: ses global 169.254.11.233 (0x7F36A1AF6C60:1) Reset (BGP Notification received). *Dec 18 00:29:28.213: BGP: 169.254.11.233 went from Established to Closing ``` The above messages show that when the TGW exceeds its route limit (100), it sends a BGP reset message to CSR. The BGP connection becomes idle after these messages. ``` ip-10-10-0-106#show ip bgp neighbors 169.254.11.233 | include BGP state BGP state = Idle ``` This test indicates that TGW has the same BGP total prefix limitation (100) as a VGW. ## Summary TGW VPN has a total 100 BGP prefix. In addition, a TGW cannot summarize Spoke VPC routes. # Explicit Azure Route Table Selection for Single IP SNAT Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/transit/azure-route-table-selection-snat Pick which Azure route tables receive the 0/0 default route when Single IP SNAT is enabled on an Aviatrix spoke gateway, replacing the previous auto-detection behavior. Starting March 31, 2026, Microsoft is retiring default outbound internet access for Azure virtual machines. Subnets in new VNets that rely on Azure-provided outbound access lose connectivity unless an explicit outbound method (NAT gateway, public IP, or load balancer rule) is configured. Before 9.0, Aviatrix identified Azure private route tables by the presence of a `0/0 → None` route and programmed those tables with `0/0 → avx-gateway`. This did not work for brownfield deployments where existing route tables had no default route, or used an outbound mechanism such as a NAT gateway. In 9.0, you select which Azure route tables receive `0/0 → avx-gateway` when Single IP SNAT is enabled on an Aviatrix spoke gateway. The Controller saves the original default route for each selected table and restores it when SNAT is disabled or the table is deselected. ## Supported Configurations | Configuration | Supported | | - | - | | Azure spoke gateway (Single IP SNAT) | Yes | | Azure ARM spoke gateway | Yes | | Azure transit gateway | Yes | | Gateway groups | Yes | | Centralized egress (FireNet) | No | ## Prerequisites * Aviatrix Controller 9.0 or later. * An Azure VNet with one or more route tables already created and associated to subnets. * Azure RBAC permissions to modify route tables in the target resource group. ## Migration from 8.2 Existing Azure spoke gateways that had Single IP SNAT enabled before 9.0 keep working. The Controller migrates the previously auto-detected private route tables and surfaces them as explicitly selected, so traffic continues to flow without disruption. For Terraform deployments, the migration is not automatic. After upgrading the Aviatrix provider to 9.0, `terraform plan` shows a diff because the new `private_route_table_config` attribute is unset. Update your Terraform configuration to declare the route tables that should be programmed; if you apply the plan without updating the configuration, Terraform deselects the route tables that previously received `0/0 → avx-gateway`. ## Behavior Notes * Original route preservation: when Aviatrix programs `0/0 → avx-gateway` on a route table that already had a default route, the original is saved. When SNAT is disabled or the route table is deselected, the original is restored. * Route tables without a default route: Aviatrix adds `0/0 → avx-gateway` and removes it when SNAT is disabled. * No selection, no default route: if no route tables are selected, no default route is programmed on the CSP. This matches the egress FireNet and transit-propagated default route behavior. * Backward compatibility: explicit selection and the previous auto-detection method coexist. Existing implicit selections are migrated on upgrade. * Gateway failover: route table programming is preserved across gateway restart and failover. ## Terraform Configuration The `private_route_table_config` attribute is available on these resources in the 9.0 provider: * `aviatrix_spoke_gateway` * `aviatrix_transit_gateway` * `aviatrix_spoke_group` * `aviatrix_transit_group` * `aviatrix_azure_spoke_native_peering` ### Identifier Format Route tables are specified using a combined identifier: ``` route_table_name:resource_group_name ``` For example: `my-route-table:my-resource-group` ### Example ```hcl theme={null} resource "aviatrix_spoke_gateway" "azure_spoke" { # ... existing configuration ... snat_enabled = true private_route_table_config = ["rtb1:resource-group-1", "rtb2:resource-group-2"] } ``` Each list entry follows the `route_table_name:resource_group_name` format. ## Limitations * Route tables must already exist in Azure. Aviatrix does not create route tables. * Azure subnets without an associated route table cannot be managed by this feature. * Centralized egress (FireNet) is not supported. * Existing Terraform configurations are not migrated automatically. After the provider upgrade to 9.0, update your configuration to declare `private_route_table_config` and avoid plan drift. ## Related Topics * [Spoke Gateway SNAT and DNAT](/docs/enterprise/10.1/guides/connectivity/vpn/spoke-gateway-snat-dnat) # Single-Region Multicloud Transit Network Workflow Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/transit/transitvpc-workflow This document provides instructions to build a hub-and-spoke network model across multiple clouds in a single-region with Aviatrix Spoke and Transit Gateways. This document provides instructions to build a hub-and-spoke network model across multiple clouds in a single-region with Aviatrix Spoke and Transit Gateways. The hub-and-spoke model consists of an Aviatrix Transit Gateway and a set of Spoke Gateways, as shown in the diagram below. The network traffic flows between the Spoke VPCs through the Transit VPC. ![Single-Region Multicloud Transit Network](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/transit/srmc.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=725502206096cd7ce5c41609ed3e2b08) ## Prerequisites 1. If you have not launched an Aviatrix Controller, refer to the following Guides for your Cloud Service Provider: * [AWS Getting Started Guide](/docs/enterprise/10.1/guides/controlplane/getting-started-guide-aws) * [Azure Getting Started Guide](/docs/enterprise/10.1/guides/controlplane/getting-started-guide-azure) > [!NOTE] > > If this is your first time using Aviatrix, make sure to go through the > Controller onboarding process and create an Aviatrix account that corresponds > to an IAM role. 2. Identify a VPC or VNet in a region where you want to launch the Transit Gateway, and name it *Transit VPC* or VNet. > [!TIP] > > Aviatrix recommends using the [*Create a VPC*](/docs/enterprise/10.1/reference/ui/cloud-resources/cloud-networks#create-vpc-vnet) tool with the option *Transit + FireNet* to create a Transit VPC/VNet that has all the necessary subnets and route tables fully populated. For a Transit network in AWS, see [Prerequisites for a Transit Network in AWS](https://legacy.docs.aviatrix.com/documentation/latest/network/transit-spoke-aws-requirements.html). 3. Create a VGW or reuse an existing VGW. The VGW should not be attached to the Transit VPC or VNet if you plan to launch Transit Gateway in the same VPC or VNet. This VGW can be attached to a different VPC or VNet if this VPC or VNet CIDR is different from the Transit VPC or VNet where the Transit Gateway is launched or in a different region and account. This VGW should be connected to on-premises either over Direct Connect or over the Internet. ## Configuration Workflow While the instructions below reference AWS, the workflow applies to any public cloud in which Aviatrix Transit Network is supported. **CoPilot UI** Follow these steps to build a single-region multicloud Transit Network: 1. [Create a Primary and Highly Available (HA) Transit Gateway](#creating-a-transit-gateway). 2. [Create a Primary and Highly Available (HA) Spoke Gateway](#creating-a-spoke-gateway). 3. [Attach the Spoke Gateway to the Transit Gateway](#creating-a-spoke-to-transit-attachment). ### 1. Creating a Transit Gateway To create primary and Highly Available (HA) Transit Gateway instances, provide the Transit Gateway general details, then assign the subnet and IP addresses for the gateway instances. **1. Navigate to the Transit Gateways** In Aviatrix CoPilot, go to **Cloud Fabric** > **Gateways** > **Transit Gateways**. For quick access, at the top of the navigation panel, type *Transit Gateways* in the search bar. **2. Create a Transit Gateway** Click **+ Transit Gateway** and provide the following general information. **General Information** | Parameter | Description | | - | - | | Name | A name for the Transit gateway. The name must begin with a letter, may include only letters, numbers, underscores, and dashes (excluding special characters or spaces), and must not exceed 50 characters in length. | | Cloud | The Cloud Service Provider (CSP) where to create the Transit Gateway. When you select AWS and Azure, you can use the dropdown menu to select Standard or Global, China, or GovCloud. | | Account | The cloud access account for creating the Transit Gateway. | | Region | The cloud region in which to create the Transit Gateway. | | VPC/VNet | The VPC or VNet in the selected region in which to create the Transit Gateway. If the selected Transit gateway will be used in a Transit FireNet workflow, selecting a VPC/VNet that has the *Transit + FireNet* function enabled means that a particular set of /28 subnets has been created across two availability zones. This function is enabled when the VPC/VNet is created. | | Instance Size | The gateway instance size. When selecting Transit Gateway instance size, choose a t2 series for Proof of Concept (POC) or prototyping only. Transit Gateway of t2 series instance type has a random packet drop of 3% for packet size less than 150 bytes when interoperating with VGW. This packet drop does not apply to Spoke Gateway. When selecting the gateway size, note that the size you select affects your IPsec performance. You can change the Transit Gateway size later. | | High Performance Encryption | To enable High Performance Encryption (HPE) for the Transit Gateway, set this toggle to *On*. HPE enables 10Gbps and higher IPsec performance between two single Aviatrix Gateway instances or between a single Aviatrix Gateway instance and an on-premises Aviatrix appliance. You cannot set High Performance Encryption to *On* or *Off* after the Transit Gateway is created. | | Support for IPv6 | To enable IPv6 addressing and routing for the Transit Gateway, set this toggle to *On*. Fill the Account, Region, and VPC/VNet fields before enabling IPv6 support. | | Attach to Transit Gateways | The Transit Gateway(s) to peer with this Transit Gateway. | **3. Advanced Settings** In the **Advanced Settings** section, set the advanced gateway settings that may apply. **Advanced Settings** | Parameter | Description | | - | - | | Transit Egress Capability (all clouds except OCI and Alibaba) | To add Transit Egress Capability to this Transit Gateway, set this toggle to *On*. Transit Gateways that have Transit Egress Capability enabled can have attachments added in the Transit FireNet workflow, or be selectable in the Transit Egress workflow. For Azure and GCP, selecting *Transit Egress Capability* must occur when the gateway is created. Otherwise, it will not display as an available Transit Gateway in the Transit FireNet or Transit Egress workflows. | | Gateway Load Balancer (AWS only) | For AWS, if you enable Transit Egress Capability you can also enable the Gateway Load Balancer option, which creates a load balancer within the selected VPC. If this option is enabled here it will show as On and disabled if you add Transit FireNet to this gateway as part of the Transit FireNet workflow. | | Subnet (GCP only) | For GCP, if you enable Transit Egress Capability you must also select a subnet. | | BGP over LAN (Azure) | To enable BGP over LAN connections for this Transit Gateway, set this toggle to *On*. For Azure, also enter the number of BGP over LAN interfaces you need (maximum is eight). | | BGP over LAN (GCP) | To enable BGP over LAN connections for this Transit Gateway, set this toggle to *On*. For GCP, select the subnet on which to apply the BGP over LAN connection. For GCP, you cannot set BGP over LAN to *On* after the Transit Gateway is created. | **4. High Availability Instances** In the **Instances** section, assign the subnets and IP addresses for the Transit Gateway instances. **High Availability Instances Settings** | Parameter | Description | | - | - | | Attach to Subnet | The subnet in which to create the gateway instance. For best practice, select a different subnet in a different availability zone from the other Transit gateway instance. | | Public IP | The public IP address of the gateway instance. (AWS only) To allocate a new EIP, leave Public IP as *Allocate New Static Public IP*. | To create a highly available (HA) gateway instance, click **+ Instance** and assign the subnet and IP address. A Transit gateway supports up to 15 HA gateway instances. The gateway instances are created in active-active mode. **5. Resource Tags** In the **Resource Tags** section, you can add custom resource tags for the Transit Gateway. 1. Click **+ Resource Tag**. 2. Enter a key to identify the resource and a value for the Key. The Key must be unique. 3. Click the checkmark icon to add the key:value pair to the resource tags. 4. Click the Tick button to save it. **6. Save the Transit Gateway** Click **Save**. You can monitor the progress of the gateway creation by going to **CoPilot** > **Monitor** > **Notifications** > **Tasks** tab. Click the expand arrow next to the gateway name to see the progress. ### 2. Creating a Spoke Gateway To create primary and Highly Available (HA) Spoke Gateway instances, provide the Spoke Gateway general details, then assign the subnet and IP addresses for the gateway instances. ![spoke vpc](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/transit/spoke-vpc.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=95ba7453b9e78c423f2498a3f6659ce8) **1. Navigate to the Spoke Gateways** In Aviatrix CoPilot, go to **Cloud Fabric** > **Gateways** > **Spoke Gateways**. For quick access, at the top of the navigation panel, type *Spoke Gateways* in the search bar. **2. Create a Spoke Gateway** Click **+ Spoke Gateway** and provide the following general information. **General Information** | Parameter | Description | | - | - | | Name | A name for the Spoke Gateway. The name must begin with a letter, may include only letters, numbers, underscores, and dashes (excluding special characters or spaces), and must not exceed 50 characters in length. | | Cloud | The Cloud Service Provider (CSP) where to create the Spoke Gateway. For AWS and Azure, you can use the dropdown menu to select Standard or Global, China, or GovCloud. | | Account | The cloud access account for creating the Spoke Gateway. | | Region | The cloud region in which to create the Spoke Gateway. | | VPC/VNet | The VPC or VNet in the selected region in which to create the Spoke Gateway. | | Instance Size | The gateway instance size. When selecting the gateway size, note that the size you select affects your IPsec performance. | | High Performance Encryption | To enable High Performance Encryption (HPE) for the Spoke Gateway, set this toggle to *On*. HPE enables 10Gbps and higher IPsec performance between two single Aviatrix Gateway instances or between a single Aviatrix Gateway instance and an on-premises Aviatrix appliance. You cannot turn High Performance Encryption *On* or *Off* after the Spoke Gateway is created. | | Support for IPv6 | To enable IPv6 addressing and routing for the Transit Gateway, set this toggle to *On*. Fill the Account, Region, and VPC/VNet fields before enabling IPv6 support. | | Attach to Transit Gateway | Select the Transit Gateway to attach to the Spoke Gateway. | | Attach to Egress Transit FireNet | Select the Egress Transit FireNet to attach to the Spoke Gateway. This option is only available if an Egress Transit FireNet has been created. | **3. Advanced Settings** In the **Advanced Settings** section, set the advanced gateway settings that may apply. **Advanced Settings** | Parameter | Description | | - | - | | BGP (all clouds) | To enable the Spoke Gateway to run BGP connection to external routers and dynamically exchange routes, set this toggle to *On*. | | BGP over LAN (Azure) | To enable BGP connection over LAN: Set this toggle to *On*. You must set both *BGP* and *BGP over LAN* settings to *On* to enable BGP over LAN connection on the Spoke Gateway. Enter the number of LAN interfaces you need (maximum is eight) for the BGP connection. You cannot delete an interface after the Transit Gateway is created. | | Global VPC (GCP) | To connect the Spoke Gateway to a global VPC, set this toggle to *On*. | **4. High Availability Instances** In the **Instances** section, assign the subnet and IP address for the Spoke Gateway instances. **High Availability Instances Settings** | Parameter | Description | | - | - | | Attach to Subnet | The subnet in which to create the Spoke gateway instance. For best practice, select a different subnet in a different availability zone from the other Spoke gateway instances. | | Public IP | The public IP address of the gateway instance. (AWS only) To allocate a new EIP, leave Public IP as *Allocate New Static Public IP*. | To create a highly available (HA) gateway instance, click **+ Instance** and designate the subnet and IP address of the gateway instance. A Spoke Gateway can have up to 15 HA gateway instances. All gateway instances are created in active-active mode. A BGP-enabled Spoke Gateway can have only two HA gateway instances. A Spoke Gateway with Site2Cloud or customized SNAT and DNAT enabled can have only two HA gateway instances. **5. Resource Tags** In the **Resource Tags** section, you can add custom resource tags for the Spoke Gateway. 1. Click **+ Resource Tag**. 2. Enter a *key* to identify the resource and a *value* for the Key. The Key must be unique. 3. Click the checkmark icon to add the *key:value* pair to the resource tags. 4. Click **Save**. **6. Save the Spoke Gateway** Click **Save**. You can monitor the progress of the gateway creation by going to **CoPilot** > **Monitor** > **Notifications** > **Tasks** tab. Click the expand arrow next to the gateway name to see the progress. ### 3. Creating a Spoke to Transit Attachment To attach a Spoke Gateway to a Transit Gateway: **1. Navigate to the Spoke Gateways** In Aviatrix CoPilot, go to **Cloud Fabric** > **Gateways** > **Spoke Gateways**. **2. Manage Gateway Attachment Dialog** In the table, locate the Spoke Gateway you want to attach and click the **Manage Gateway Attachment** icon on the right side of its row. **3. Create a Spoke to Transit Attachment** In the Manage Gateway Attachment dialog, click the **Transit Gateway** tab and then click **+ Attachment** and provide the following information. **Spoke to Transit Attachment Settings** | Parameter | Description | | - | - | | Transit Gateway | The Transit Gateway to attach to the Spoke Gateway. | In the *Advanced* section, set the advanced settings that may apply. | Parameter | Description | | - | - | | Customize Route Table Attachment | Enables custom route tables when set to *On*. From the *Select Route Tables* dropdown menu, select the custom route table(s) to attach to this Spoke Gateway. | | Max Performance | Creates the maximum number of High Performance Encryption (HPE) tunnels for the Transit-to-Spoke attachment when set to *On*. The number of tunnels that are created depends on the gateway instance sizes. Max Performance option is valid only when both the Spoke and Transit Gateways are launched with HPE enabled and are in the same cloud type. When the Spoke and Transit Gateways are launched in different cloud types, only one tunnel is created (even when HPE is enabled for both the Spoke and Transit Gateway). If Max Performance is *Off*, only one tunnel is created (even when HPE is enabled for both the Spoke and Transit Gateway). To switch between multiple tunnels or one tunnel, detach and reattach the Spoke Gateway to the Transit Gateway. | > [!NOTE] > > Controller 10.1.0 adds an **Early Access** capability to > edit a spoke attachment's managed route tables in place, without > detaching and reattaching the gateway. From CoPilot, open the spoke > gateway's **Managed Route Tables** card to add, remove, or > change the attached route tables. This is a new alternative to setting > route tables through the > **Customize Route Table Attachment** option above. > > This capability is **Early Access** in Controller 10.1.0, > available on AWS and Azure. To request Early Access, contact your > Aviatrix account team. To attach this Spoke Gateway to another Transit Gateway, click **+ Attachment** again and provide the attachment details. **4. Save the Spoke to Transit Attachment** Click **Save**. A notification appears indicating that the Spoke Gateway is attached to the Transit Gateway. **Terraform** Use the [Aviatrix Terraform provider](https://registry.terraform.io/providers/AviatrixSystems/aviatrix/latest) to build a single-region hub-and-spoke Transit Network. > [!NOTE] > > Module examples pin `terraform-aviatrix-modules/mc-transit` and `mc-spoke` to > version `{"10.0.1"}`. Check the > [Terraform Registry](https://registry.terraform.io/namespaces/terraform-aviatrix-modules) > for newer patch releases before you pin a specific version in your > configuration. Complete the [Prerequisites](#prerequisites) before you begin. The steps to create a single-region hub-and-spoke network are as follows: **1. Get the Existing VPC Information** Retrieve the details of the existing Transit and Spoke VPCs with the `aviatrix_vpc` data source block. ```hcl theme={null} data "aviatrix_vpc" "transit_vpc_aws" { name = "transit-vpc-aws" # your Transit VPC name for AWS } data "aviatrix_vpc" "spoke_vpc_aws" { name = "spoke-vpc-aws" # your Spoke VPC name for AWS } data "aviatrix_vpc" "spoke_vpc_azure" { name = "spoke-vpc-azure" # your Spoke VPC name for Azure } data "aviatrix_vpc" "spoke_vpc_gcp" { name = "spoke-vpc-gcp" # your Spoke VPC name for GCP } ``` **2. Create the Transit Gateway** Create primary and Highly Available (HA) Transit Gateway instances with the `mc-transit` module. Use the VPC data source block retrieved in the previous step to set the `region`, `cidr`, `name`, `vpc_id`, `gw_subnet`, and `hagw_subnet` fields. The following code snippet is for AWS (`cloud = "AWS"`). For Azure, GCP, and OCI, set the arguments as per the [Aviatrix Terraform provider documentation](https://registry.terraform.io/providers/AviatrixSystems/aviatrix/latest/docs). ```hcl theme={null} module "mc-transit" { source = "terraform-aviatrix-modules/mc-transit/aviatrix" version = "10.0.1" cloud = "AWS" region = data.aviatrix_vpc.transit_vpc_aws.region cidr = data.aviatrix_vpc.transit_vpc_aws.cidr account = "aws-account" name = data.aviatrix_vpc.transit_vpc_aws.name gw_name = "transit-gw-aws" use_existing_vpc = true vpc_id = data.aviatrix_vpc.transit_vpc_aws.vpc_id gw_subnet = data.aviatrix_vpc.transit_vpc_aws.public_subnets[0].cidr hagw_subnet = data.aviatrix_vpc.transit_vpc_aws.public_subnets[1].cidr } ``` **3. Create Spoke Gateway and Attach to Transit Gateway** Create primary and HA AWS Spoke Gateway instances with the `mc-spoke` module. Use the VPC data source block retrieved in the previous step to set the `cloud`, `name`, `region`, `account`, `transit_gw`, `use_existing_vpc`, `vpc_id`, `gw_subnet`, and `hagw_subnet` fields. Attach the Spoke Gateway to the Transit Gateway with the `transit_gw` argument. ```hcl theme={null} module "spoke_aws_1" { source = "terraform-aviatrix-modules/mc-spoke/aviatrix" version = "10.0.1" cloud = "AWS" name = data.aviatrix_vpc.spoke_vpc_aws.name region = data.aviatrix_vpc.spoke_vpc_aws.region account = "aws-account" transit_gw = module.mc-transit.transit_gateway.gw_name use_existing_vpc = true vpc_id = data.aviatrix_vpc.spoke_vpc_aws.vpc_id gw_subnet = data.aviatrix_vpc.spoke_vpc_aws.public_subnets[0].cidr hagw_subnet = data.aviatrix_vpc.spoke_vpc_aws.public_subnets[1].cidr } ``` **4. Reuse the Code Snippet for Other Clouds** To create Spoke Gateways and attach them to the Transit Gateway for other clouds, reuse the same code snippet by replacing the AWS specific fields with the other clouds specific fields. For example, for Azure: ```hcl theme={null} module "spoke_azure_1" { source = "terraform-aviatrix-modules/mc-spoke/aviatrix" version = "10.0.1" cloud = "Azure" name = data.aviatrix_vpc.spoke_vpc_azure.name region = data.aviatrix_vpc.spoke_vpc_azure.region account = "azure-account" transit_gw = module.mc-transit.transit_gateway.gw_name use_existing_vpc = true vpc_id = data.aviatrix_vpc.spoke_vpc_azure.vpc_id gw_subnet = data.aviatrix_vpc.spoke_vpc_azure.subnets[0].cidr hagw_subnet = data.aviatrix_vpc.spoke_vpc_azure.subnets[1].cidr } ``` For GCP: ```hcl theme={null} module "spoke_gcp_1" { source = "terraform-aviatrix-modules/mc-spoke/aviatrix" version = "10.0.1" cloud = "GCP" name = data.aviatrix_vpc.spoke_vpc_gcp.name region = data.aviatrix_vpc.spoke_vpc_gcp.region account = "gcp-account" transit_gw = module.mc-transit.transit_gateway.gw_name use_existing_vpc = true vpc_id = data.aviatrix_vpc.spoke_vpc_gcp.vpc_id gw_subnet = data.aviatrix_vpc.spoke_vpc_gcp.subnets[0].cidr hagw_subnet = data.aviatrix_vpc.spoke_vpc_gcp.subnets[0].cidr } ``` **5. Verify and Apply the Configuration** Run `terraform plan`, review the planned changes, and run `terraform apply`. ### Examples The following are a few examples of creating a single-region multicloud Transit Network using Terraform. The following examples are for Azure, AWS, and GCP. For OCI and Alibaba, reuse the same code snippet by replacing the Azure, AWS, and GCP specific fields with the OCI and Alibaba specific fields. > **Multicloud Transit with Azure Hub and Workload-Aligned Spokes in AWS and GCP** > > This example reflects a common multicloud pattern: a European business runs its line-of-business app on Azure, keeps regulated customer data on AWS, and uses GCP for analytics. An Azure Transit Gateway in `West Europe` acts as the hub, and one HA Spoke per cloud carries the workload that best fits that cloud. Aviatrix peers the underlying CSPs transparently, so the three Spokes route to each other through the single Transit Gateway. > > The CIDRs are aligned on a `10.x.0.0/22` plan so each Spoke can be summarized cleanly toward the on-premises network later. Common tags are applied to every gateway for cost allocation, with a per-cloud tag layered on each Spoke. > > ```hcl theme={null} > module "azure_transit" { > > source = "terraform-aviatrix-modules/mc-transit/aviatrix" > version = "10.0.1" > > cloud = "Azure" > name = "transit-azure-weu" > region = "West Europe" > account = "azure-prod" > cidr = "10.10.0.0/23" > instance_size = "Standard_D3_v2" > tags = { > Owner = "platform-networking" > CostCenter = "2042" > Application = "customer-portal" > } > } > > module "spoke_azure_app" { > source = "terraform-aviatrix-modules/mc-spoke/aviatrix" > version = "10.0.1" > > cloud = "Azure" > name = "spoke-azure-app" > region = "West Europe" > account = "azure-prod" > cidr = "10.20.0.0/22" > transit_gw = module.azure_transit.transit_gateway.gw_name > instance_size = "Standard_D3_v2" > tags = { > Owner = "platform-networking" > CostCenter = "2042" > Application = "customer-portal" > Tier = "app" > } > } > > module "spoke_aws_data" { > source = "terraform-aviatrix-modules/mc-spoke/aviatrix" > version = "10.0.1" > > cloud = "AWS" > name = "spoke-aws-data" > region = "eu-west-1" > account = "aws-prod" > cidr = "10.30.0.0/22" > transit_gw = module.azure_transit.transit_gateway.gw_name > tags = { > Owner = "platform-networking" > CostCenter = "2042" > Application = "customer-portal" > Tier = "data" > } > } > > module "spoke_gcp_analytics" { > source = "terraform-aviatrix-modules/mc-spoke/aviatrix" > version = "10.0.1" > > cloud = "GCP" > name = "spoke-gcp-analytics" > region = "europe-west1" > account = "gcp-prod" > cidr = "10.40.0.0/22" > transit_gw = module.azure_transit.transit_gateway.gw_name > } > ``` > > > [!TIP] > > > > For OCI or Alibaba, copy any Spoke block and replace the cloud-specific fields (`cloud`, `region`, `account`). All other arguments behave the same way across providers. > **Enterprise AWS Hub with Production, Non-Production, and Shared-Services Spokes** > > This example reflects a common enterprise starting point: a single Aviatrix Transit Gateway in AWS's N. Virginia region acts as the hub for a payments workload. Production and Non-Production environments live in separate Spoke VPCs so blast radius and change windows can be managed independently. A third Spoke holds shared services such as logging, identity, and CI/CD runners that must be reachable from both environments. > > All gateways are deployed in HA. High Performance Encryption (`insane_mode`) is enabled on the Transit Gateway and on the Production Spoke for line-rate encrypted east-west traffic. Common tags are applied to every gateway for cost allocation, and per-environment tags are layered on top of the Spokes. > > A `locals` block centralizes the account, region, and base tag set so the same values are not repeated in every module call. > > ```hcl theme={null} > > module "transit_aws" { > source = "terraform-aviatrix-modules/mc-transit/aviatrix" > version = "10.0.1" > > cloud = "AWS" > name = "transit-aws-use1" > region = "us-east-1" > account = "aws-prod" > cidr = "10.100.0.0/23" > insane_mode = true > tags = { > Owner = "platform-networking" > CostCenter = "1001" > Application = "payments" > } > } > > module "spoke_prod_payments" { > source = "terraform-aviatrix-modules/mc-spoke/aviatrix" > version = "10.0.1" > > cloud = "AWS" > name = "spoke-prod-payments" > region = "us-east-1" > account = "aws-prod" > cidr = "10.110.0.0/22" > transit_gw = module.transit_aws.transit_gateway.gw_name > insane_mode = true > tags = { > Owner = "platform-networking" > CostCenter = "1001" > Application = "payments" > Environment = "prod" > } > } > > module "spoke_nonprod_payments" { > source = "terraform-aviatrix-modules/mc-spoke/aviatrix" > version = "10.0.1" > > cloud = "AWS" > name = "spoke-nonprod-payments" > region = "us-east-1" > account = "aws-prod" > cidr = "10.120.0.0/22" > transit_gw = module.transit_aws.transit_gateway.gw_name > tags = { > Owner = "platform-networking" > CostCenter = "1001" > Application = "payments" > Environment = "nonprod" > } > } > > module "spoke_shared_services" { > source = "terraform-aviatrix-modules/mc-spoke/aviatrix" > version = "10.0.1" > > cloud = "AWS" > name = "spoke-shared-services" > region = "us-east-1" > account = "aws-prod" > cidr = "10.130.0.0/22" > transit_gw = module.transit_aws.transit_gateway.gw_name > tags = { > Owner = "platform-networking" > CostCenter = "1001" > Application = "payments" > Environment = "shared" > } > } > ``` > > > [!TIP] > > > > To enforce traffic isolation between Production, Non-Production, and Shared-Services Spokes at the Transit, set `enable_segmentation = true` on the `mc-transit` module and place each Spoke in its own Network Domain. See [Network Segmentation Secured](/docs/enterprise/10.1/guides/security/network-segmentation-secured). *** ## View the Network Topology You can view the network topology in CoPilot by going to **Cloud Fabric** > **Topology**. For more information about using the topology map, see [Gaining Visibility into Your Network Topology](/docs/enterprise/10.1/guides/monitoring/topology-visibility). ## View the Gateway's Connections and Route Tables You can view the Spoke and Transit Gateway connections and route table information. **1. Navigate to the Gateways** In Aviatrix CoPilot, go to **Cloud Fabric** > **Gateways**, and select the Spoke or Transit Gateway to open the gateway's page. **2. View the Connections of the Gateway** Click the **Attachments** tab to view the gateway's connections. **3. View the Route Tables of the Gateway** Click the **VPC/VNet Route Tables** tab, then from the **Route Table** dropdown menu, select a route table to view the routes. ## Related Topics * You can peer Transit Gateways to expand your Aviatrix Transit Network across multiple regions and clouds. See [Multi-Region Multicloud Transit Gateway Peering Workflow](/docs/enterprise/10.1/guides/connectivity/peering/transit-gateway-peering-workflow). * You can segment your Aviatrix Transit Network to create network isolation and enhance security. To learn more, see [Network Segmentation Secured](/docs/enterprise/10.1/guides/security/network-segmentation-secured). * You can connect the Transit Gateway to external devices such as on-premises firewalls and routers to connect to your datacenter. See [Connecting the Transit Network to On-Premises](https://legacy.docs.aviatrix.com/documentation/latest/network/transit-to-externaldevice.html). * You can connect the Transit Gateway to an Edge Gateway to extend your Aviatrix Transit Network to the network edge. See [Hybrid Cloud Connectivity](https://legacy.docs.aviatrix.com/documentation/latest/network/edge-connectivity.html). * You can automate your multicloud Transit Network configuration by using Terraform. To learn more, see [Aviatrix Terraform Namespace](https://registry.terraform.io/namespaces/terraform-aviatrix-modules). * You can add your cloud accounts to your CoPilot. To learn more, see [Cloud Account](/docs/enterprise/10.1/reference/ui/cloud-resources/cloud-account#add-cloud-account). # Site2Cloud Solution for Encryption over Direct Connect/ExpressRoute Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/vpn/aws-azure-encryption AWS Direct Connect and Azure ExpressRoute provide a private routed circuit between an AWS VPC and an Azure VNet. AWS Direct Connect and Azure ExpressRoute provide a private routed circuit between an AWS VPC and an Azure VNet. The Aviatrix Site2Cloud feature provides encryption over Direct Connect or ExpressRoute. This document describes how to implement the feature over Express Route. The same method applies to AWS. The VNet VPN gateway that terminates the ExpressRoute connects VNet virtual machines with the on-premises servers in a traditional routing domain. While Azure ExpressRoute provides a private link between a on-premises network and an Azure VNet without going through the Internet, packets between on-premises edge and VNet travel through exchange points and third party provider networks and are not encrypted. ## Aviatrix Solution for Encryption over ExpressRoute The Aviatrix Site2Cloud solution can be applied to encrypt traffic over ExpressRoute, as shown below. ![Topology Express Route](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/topology-expressroute.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=b32ab7b03378405e90123a2566127673) In the diagram above, an encrypted IPsec tunnel is established between an Aviatrix gateway and the edge router. An Aviatrix gateway is deployed in a separate subnet from the subnets where the user virtual machines are launched. (The Controller is not drawn.) This is necessary as the Aviatrix gateway is the router for user subnets to reach the enterprise data center. An Aviatrix gateway can be deployed in high availability (HA) where a backup gateway is ready to take over should the primary IPsec tunnel go down. ## Configuration Workflow Before you begin: * Make sure your Controller is [upgraded to the latest version](/docs/enterprise/10.1/guides/maintenance/controller-gateway-upgrade-steps). * Decide if you want to enable HA for the gateway. The configuration workflow is as follows, with major steps highlighted. 1. Create an Aviatrix gateway in a VNet where you would like to connect to the enterprise datacenter. Make sure the gateway is launched in a different subnet from the user subnets. * Create a Spoke Gateway * Create a Specialty Gateway > [!NOTE] > > If enabling HA, when creating the primary gateway, add a second gateway > Instance in the same VNet. The second Instance (for HA) should use a > different subnet from primary gateway and user subnets. 2. Create a [Static Route-Based external connection](/docs/enterprise/10.1/guides/connectivity/vpn/transit-externaldevice-workflow) using the following information: | Field | Value | | - | - | | Local Gateway | Select the Gateway launched earlier as the primary gateway. | | Real Local Subnet CIDR(s) | Specify a list of the source network CIDRs that will be encrypted. If left blank, the full CIDR is used. If you enter a value, make sure you include the VPC/VNet as well. These Local Subnets are advertised to Remote Subnets that the connection can reach. | | Virtual Local Subnet CIDR(s) | A virtual local network CIDR that maps to the real local subnet | | Remote Device Type | Select *Generic*. | | Real Remote Subnet CIDR(s) | Enter the network CIDR of the Enterprise data center. If there are multiple subnets separate them with commas. | | Virtual Remote Subnet CIDR(s) | A virtual remote network CIDR that maps to the real remote subnet | | Pre-Shared Key | Optional (auto-generated if not entered) | | Attach Over | Select *Private Network*. | | Remote Device Destination IP | Enter the private IP address of the edge router for the Enterprise data center. | 3. If you added an HA entry to the Aviatrix Gateway created above, you can add High Availability for this external connection. Click *+Connection* in the Add External Connection dialog to add another row and enter the Remote Device IP, Local Gateway Instance, Local Tunnel IP (optional), and Remote Tunnel IP (optional) for the HA gateway. 4. Click *Save*. ## Download an External Connection Configuration If you are connecting an Aviatrix gateway and an on-premises router or firewall, Aviatrix can generate a configuration file that you can apply to your remote router or firewall. The configuration file contains the Aviatrix gateway tunnel details, such as the Public IPv4 and IPv6 addresses, VPC/VNet CIDR, pre-shared key, and encryption algorithm. You can download the configuration file and then import the details to your remote router or firewall to configure the other end of the VPN tunnel. After creating an external connection, to download an external connection configuration: 1. In Aviatrix CoPilot, go to **Networking > Connectivity > External Connections (S2C)** tab. 2. On the External Connections (S2C) tab, locate the connection you created and click the vertical ellipsis ![vertical ellipsis icon](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/connectivity/vpn/vertical-ellipsis-icon.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=dfa275cca2e54747ed314060d27d437d) icon on the right side of the row. 3. Select the following values: * **Vendor**: Select your remote site device. * Select **Generic** for anything that is not an Aviatrix gateway. * Select **Aviatrix**, if you are connecting two Aviatrix gateways. * **Platform and Software**: * If you selected a Generic vendor, the Platform field is populated as Generic, and the Software field is populated with Vendor Independent. * If you selected the Aviatrix vendor, the Platform field is populated with UCC, and the Software version is 1.0. * If you selected a specific hardware vendor (such as Cisco), select from the available platforms belonging to that vendor are displayed in the Platform field (ISR, ASR, and CSR are for Cisco routers), and the Software field is populated with the related software version. 4. Click **Download**. At the enterprise data center or remote site, configure encryption on the edge device. Make sure your peer network is Subnet2 and Subnet3, as shown in this example. ## Related Topics * [Externaldevice Settings](https://legacy.docs.aviatrix.com/documentation/latest/network/externaldevice-settings.html) # Using Aviatrix Site2Cloud Tunnels to Access VPC Endpoints in Different Regions Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/vpn/aws-site2cloud-tunnels VPC Endpoints in AWS allow you to expose services to customers and partners over AWS PrivateLink. In situations where allowing resources to be accessed directly from the Internet is undesirable, VPC Endpoints can enable internal VPC connectivity to resources in other accounts. [VPC Endpoints](https://docs.aws.amazon.com/vpc/latest/userguide/vpce-interface.html) in AWS allow you to expose services to customers and partners over AWS PrivateLink. In situations where allowing resources to be accessed directly from the Internet is undesirable, VPC Endpoints can enable internal VPC connectivity to resources in other accounts. One limitation of Endpoints is that it is a regional construct, meaning you can't use it to provide connectivity to resources across regions. In some cases it's not possible to move these workloads. This is where Aviatrix can help overcome that limitation. The end design will look similar to the diagram below. ![VPC Endpoints Diagram](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/connectivity/vpn/vpc-endpoints-diagram.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=3596579ca4e9b5a764355d6ad11dd484) ## Environment Requirements In this example there are: * Two VPCs in US-East-1. One customer/partner VPC (10.10.10.0/24) with an Endpoint, and our VPC (10.10.11.0/24) with an Endpoint Service tied to an internal Load Balancer. * One VPC (10.10.12.0/24) in US-East-2 that hosts our workload. * A set of Aviatrix Gateways: two in Aviatrix VPC in US-East-1, and two in the workload VPC in US-East-2. Deploying a set of HA Gateways is documented [here](/docs/enterprise/10.1/guides/gateways/gateway-ha-about). Once deployed, a set of Site2Cloud tunnels will be built. Documentation for building a tunnel between Aviatrix Gateways is [here](/docs/enterprise/10.1/guides/connectivity/vpn/s2c-aviatrixgw-to-aviatrixgw). They should be built in an active-passive manner to avoid asymmetric routing in AWS. ## Deploy an Internal Load Balancer in AWS > [!NOTE] > > Before beginning, make sure you have private subnets in your Availability > Zones. You must select at least two Availability Zones in the following > procedure that traffic will be routed to, and the subnets in these > Availability Zones must be private to prevent the receipt of Internet traffic. 1. From the EC2 section in the AWS console, choose Load Balancers. 2. Click **Create Network Load Balancer**. ![Create Load Balancer](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/create-load-balancer.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=12fd9c2387da152bc4ba2c5c91c67e4d) ![Select Balancer](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/select-balancer.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=f09b732b8c487e62e6620d91b119a3a8) 3. Give the load balancer a name. 4. Select the **Internal** Scheme. 5. Select the **IPv4** IP address type. 6. In the Network mapping area, select all the Availability Zones in the US-East-1 VPC. Remember that these Availability Zones must contain private subnets. 7. In the Listeners and routing area, select Protocol **TCP** and Port **80**. 8. Also in this area, create a new target group using port 80 (this opens a new tab in your browser). The target type is 'instance'. Health Checks will be TCP-based. ![Config Routing](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/config-routing.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=72cf537cb75c0911d57477f4839c14ce) 9. Click **Next**. 10. On the Register targets page, select the Aviatrix Gateways in our US-East-1 VPC and move them to Registered Targets. 11. Go back to the Load Balancer creation browser tab. In the Listeners and routing area, select the target group you created above. ![AWS Add Target Group](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/aws-add-targetgroup.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=55febb15b5e02c9a17c4bc5b81f2d8d2) 12. Click **Create load balancer**. On the next tab you can view your load balancer. ![NLB Created](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/nlb-created.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=2d2b39665735f5d6c5c93897913f338f) ### Attach an Endpoint Service to Load Balancer 1. From the VPC dashboard area of the AWS console, click Endpoint Services. 2. Click **Create endpoint service**. The new Load Balancer will be in the list as an available Network Load Balancer. ![Create Endpoint](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/create-endpoint.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=fb20f8352b4bb8fffc1aba8a11e16deb) 3. Enter a name for the endpoint. 4. Ensure that the **Network** Load balancer type is selected. 5. Under Available load balancers, select the load balancer you created. 6. Under Additional settings, select the **Acceptance required** checkbox. 7. Ensure that the **IPv4** supported IP address type is selected. 8. Click **Create**. The Service ARN will be what our customer uses to register a service in their VPC. ![Endpoint Service](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/endpoint-service.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=782e789ccd4793490ab0f5d0a4b4a33f) ### Create Endpoint in Customer VPC 1. In the VPC area of the AWS console, create a new Endpoint. 2. Enter the ARN from the last step, and select the Customer VPC to expose an endpoint in. Once built, the Endpoint DNS names can be used to route traffic. ![Endpoint VPC](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/endpoints-vpc.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=2607d38d0c848681918b8f565197eb80) ## Configure Destination NAT rules on Aviatrix Gateway A Destination NAT (DNAT) rule sends traffic from our VPC in US-East-1 to the workload VPC in US-East-2. 1. In Aviatrix CoPilot, click the name of the gateway associated with the VPC you created in US-East-1. 2. On the Settings tab, expand the Network Address Translation (NAT) area. 3. Turn **On** Destination NAT. 4. Select the instance (not the HA instance). 5. Add a new rule with the following values: * Src (Source) CIDR: 10.10.11.0/24 (source of US-East-1 VPC) * Dst (Destination) CIDR: private IP of primary gateway * Dst (Destination) Port: 80 * Protocol: TCP * Connection: None * DNAT IP: 10.10.12.69 (workload VPC available via Site2Cloud tunnel) * DNAT port: 80 6. Click **Save**. 7. Turn **On** Apply Route Entry to commit the rule. 8. Select the HA instance and repeat steps 4-6 to create a second rule for updating the Destination CIDR to point to the private IP of the HA gateway. ## Test Connections Ensure health checks on your internal Load Balancer are healthy. Network Security Groups on your workload VPC (10.10.12.0/24) allow traffic from the Aviatrix VPC in US-East-1 (10.10.11.0/24). Only one tunnel will be active in our scenario, and Aviatrix will update the route tables to point to the active tunnel. A simple way to test connectivity is to edit the /etc/hosts file on a Linux instance to point to one of the DNS entries from the Endpoint in the Customer VPC. # Accessing a Virtual IP Address Instance via Aviatrix Transit Network Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/vpn/firewall-onprem-virtualip This document addresses the scenario where a customer on-prem firewall device needs to route encrypted traffic to a partner network in the cloud (AWS/Azure/GCP), but due to concerns for overlapping CIDR blocks to the customer network, the customer side enforces a policy that the destination IP... This document addresses the scenario where a customer on-prem firewall device needs to route encrypted traffic to a partner network in the cloud (AWS/Azure/GCP), but due to concerns for overlapping CIDR blocks to the customer network, the customer side enforces a policy that the destination IP address must be a public or a virtual IP address regardless of whether the partner network is in the RFC 1918 range. For example, the VPC instance IP address that the on-prem machine should send data to is 172.123.4.5, but the on-prem machine must instead send data to a virtual IP address 54.189.117.94 (or even 100.100.100.100). Normally this problem can be solved by combining the [Site2Cloud](/docs/enterprise/10.1/guides/connectivity/vpn/s2c-configuration-workflow) feature and [DNAT](https://legacy.docs.aviatrix.com/documentation/latest/network/snat-dnat-settings.html) feature. There are situations where there are multiple applications in different VPCs, and it is desirable to access different virtual addresses without building multiple IPSEC tunnels to the cloud networks. This can be accomplished by building an Aviatrix Transit Network where Spoke VPCs host these different applications, as shown in the diagram below. ![transit-publicIP](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/transit-pubip.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=5f7ca10190e79f0b61b816a401c95f95) ## Determine the Virtual IP address As this virtual IP address is what the on-prem host sees, it should not change. There are a couple of options for a virtual IP address: * You can allocate an EIP in the VPC for this virtual IP address. Do not associate this EIP to any instance. * Alternatively, if the EC2 instance that on-prem hosts need to send data to has an EIP, you can use that EIP. You can also try a reserved public IP address range (for example, 100.100.x.x range). ## Launch a Spoke Gateway Create a [Spoke Gateway](/docs/enterprise/10.1/reference/ui/cloud-fabric/gateways#view-spoke-gateways) in VPC 172.32.0.0/16. ## Customize Spoke Gateway Advertised Routes 1. After creating the Spoke Gateway, click the name of the Gateway and then click the Settings tab. 2. Expand the Routing area. 3. In the Customize Spoke Advertised VPC/VNet CIDRs area, enter 54.189.117.94/32. With this customization, the Spoke gateway advertises 54.189.117.94/32 to the Transit Gateway and subsequently to on-prem. ## Attach the Spoke Gateway You can attach the Spoke Gateway to Transit by clicking the edit ![edit icon](https://mintcdn.com/aviatrix-14b37c43/VY6W0iTAMUQkrobS/images/security/edit-icon.png?fit=max\&auto=format\&n=VY6W0iTAMUQkrobS\&q=85\&s=d37797a7c48eefc52ed6e66ccf9915bf) icon next to the Spoke Gateway and selecting the Transit VPC (based on the example diagram at the beginning of this document) from the Attach to Transit Gateway dropdown. ## Configure DNAT on the Spoke Gateway This step is to configure the Spoke gateway to translate the destination virtual IP address 54.189.117.94 to the real private IP address 172.123.4.5. 1. On the Gateway page, highlight the Spoke gateway and click **Edit**. 2. Scroll down to Destination NAT. Follow the instructions in [Destination NAT](https://legacy.docs.aviatrix.com/documentation/latest/network/snat-dnat-settings.html) to configure, as shown below. Use the "Connection" field to specify the Site2Cloud connection name configured in Step 3. ![dnat-config](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/dnat-config.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=82d1eb00b9d7e48da415fa482d2e4712) ## Test Test connectivity from the on-prem host to the EC2 instance. For example, ping the virtual IP address 54.189.117.94 from an on-prem host machine. The ping should reach 172.123.4.5. # Networks with Overlapping CIDRs Scenarios Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/vpn/overlapping-network-connectivity-solutions This document describes a few scenarios of overlapping networking CIDRs and their solutions. The solution uses the Mapped option of the Aviatrix Site2Cloud feature when building IPsec tunnels. This document describes a few scenarios of overlapping networking CIDRs and their solutions. The solution uses the Mapped option of the Aviatrix [Site2Cloud](/docs/enterprise/10.1/guides/connectivity/vpn/s2c-configuration-workflow) feature when building IPsec tunnels. Using Mapped Site2Cloud provides the advantage of not having to configure individual SNAT/DNAT rules, as all virtual and physical network addresses are 1-1 translated. This document does not go into specifics of the actual configurations. For this information, see [Configuring Overlapping Networks with Network Mapped IPsec](/docs/enterprise/10.1/guides/connectivity/vpn/overlapping-networks-mapped-ipsec) . ## Scenario 1: On-prem Overlaps with Spoke in Aviatrix Transit Deployment In this scenario, the Aviatrix Transit solution is deployed, and a resource on the on-prem site overlaps with a Spoke CIDR it needs to communicate with, as shown in the diagram below. ![Overlap On-Prem Aviatrix Transit](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/overlap-onprem-aviatrix-transit.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=6d60353f14196c1ba795447614d37b3f) This scenario is made possible by the *Forward Traffic to Transit Gateway* option that you can enable after configuring your Site2Cloud connection. See [Forwarding Traffic to Transit Gateway](https://legacy.docs.aviatrix.com/documentation/latest/network/externaldevice-settings.html#forward-traffic-transit) for more information. ## Scenario 2: Multi-Sites Overlap in Aviatrix Transit Deployment This scenario extends the previous solution to include multi-sites, as shown in the diagram below. ![Overlap On-Prem Aviatrix Transit](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/overlap-onprem-aviatrix-transit.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=6d60353f14196c1ba795447614d37b3f) This scenario is made possible by the *Forward Traffic to Transit Gateway* option that you can enable after configuring your Site2Cloud connection. See [Forwarding Traffic to Transit Gateway](https://legacy.docs.aviatrix.com/documentation/latest/network/externaldevice-settings.html#forward-traffic-transit) for more information. Spoke 2 in this scenario is a landing Spoke. When the Site2Cloud Forwarding option referenced above is enabled, NAT occurs on the landing Spoke and ensures that bidirectional traffic flow is possible between on-prem routers and local Spoke and Transit gateways. Either side can now initiate traffic (locally or remotely), as per what you configured in your Site2Cloud connection. If you select only one of these, you cannot initiate from the other direction and NAT translation will not occur. Enabling the [Auto Advertise Spoke Site2Cloud CIDRs](https://legacy.docs.aviatrix.com/documentation/latest/network/spoke-gateway-route-settings.html?expand=true#auto-advertise-cidrs) option (select a Spoke Gateway and select this option from the Settings tab) in this scenario ensures that the other Spokes in the scenario are aware of the virtual CIDRs on which the landing Spoke is going to perform NAT (Spoke 2). If you select this Auto Advertise option ensure that you do not advertise more than the CSP-allowed limit of routes per route table. For example, for AWS the routes per route table limit is described [here](https://docs.aws.amazon.com/vpc/latest/userguide/amazon-vpc-limits.html). ## Scenario 3: On-prem Overlaps with Spoke VPC/VNet in TGW Deployment In this scenario, on-prem site-1 overlaps with Spoke-1 VPC/VNet CIDR. They are both 172.32.0.0/16 and want to communicate with each other. The solution is to deploy an Aviatrix Gateway in Spoke-2 VPC/VNet and build an IPsec tunnel between Spoke-2 gateway and the on-prem. In the deployment, both Spoke-1 and Spoke-2 are attached to TGW and are in the same Network Domain. The diagram shown below illustrates how to use Spoke VPC Advertised Routes to build a more complex network. You can launch an Aviatrix Gateway in Spoke-1 directly and build the IPsec tunnel. > [!TIP] > > VPC Spoke-1 is for illustration purposes. The destination network that overlaps with the on-prem site may be an on-prem network that connects with AWS TGW via AWS Direct Connect or VPN. Similarly, the on-prem network could be a VPC/VNet in the cloud. ![Overlap On-Prem TGW](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/overlap-onprem-tgw.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=1df6883afff3817e1c2ff69752ee9f43) Following are the steps to set up the above networks. 1. [Attach VPC/VNet Spoke-2 to TGW](/docs/enterprise/10.1/reference/ui/networking/connectivity#edit-aws-tgw) . 2. [Launch Aviatrix gateway in Spoke-2](/docs/enterprise/10.1/reference/ui/cloud-fabric/gateways#view-spoke-gateways) . You can add an instance when creating the Spoke Gateway if you want to use HA. 3. Create a [Static Route-Based (Mapped) connection](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-create-static-route-based-mapped.html) from Spoke-2 to Site-1 using these values: | Parameter | Value | | - | - | | Local Subnet (real) | 172.32.0.0/16 | | Local Subnet (virtual) | 192.168.0.0/16 | | Remote Subnet (real) | 172.32.0.0/16 | | Remote Subnet (virtual) | 100.100.0.0/16 | 4. Configure the on-premises site-1 IPsec. Key parameters are: * Local Subnet: 172.32.0.0/16 * Remote Subnet: 192.168.0.0/16 You can [download the configuration from step 4](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-download-configuration.html) to use in the on-prem configuration. 5. Check the tunnel status at Diagnostics > Cloud Routes > External Connections. 6. Advertise 100.100.0.0/16 to TGW from Spoke-2 VPC/VNet: 1. On the Networking > Connectivity > AWS TGW tab, click the name of your AWS TGW. 2. Click the Attachments sub-tab and then click VPC. 3. Attach the Spoke VPC you created in Step 2 above and ensure that the Customize Spoke VPC Routes field contains 172.34.0.0/100, 100.100.0.0/16 (where 172.34.0.0/16 is the Spoke-2 VPC/VNet CIDR and 100.100.0.0/16 is the virtual network CIDR of on-prem site-1). 7. Test connectivity. From on-prem site-1 to ping an instance in Spoke-1 using the Spoke-1 virtual network CIDR with the real host portion of its IP address. For example, if the instance in Spoke-1 is 172.32.10.15, then site-1 should ping 192.168.10.15. ## Scenario 4: Multi-Sites Overlap in TGW Deployment Scenario 1 can be extended to on-prem multi sites that have overlapping or identical network addresses, as shown in the diagram below. ![Overlap Multi On-Prem TGW](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/overlap-multi-onprem-tgw.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=a376e64fba1415e4c486dd3fc9a5bc40) 1. [Attach VPC/VNet Spoke-2 to TGW](/docs/enterprise/10.1/reference/ui/networking/connectivity#edit-aws-tgw) . 2. [Launch Aviatrix gateway in Spoke-2](/docs/enterprise/10.1/reference/ui/cloud-fabric/gateways#view-spoke-gateways) . You can add an instance when creating the Spoke Gateway if you want to use HA. 3. Create a [Static Route-Based (Mapped) connection](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-create-static-route-based-mapped.html) from Spoke2 to Site-1 using these values: | Parameter | Value | | - | - | | Local Subnet (real) | 172.32.0.0/16 | | Local Subnet (virtual) | 192.168.0.0/16 | | Remote Subnet (real) | 172.32.0.0/16 | | Remote Subnet (virtual) | 100.100.0.0/16 | 4. Create an on-premises site-1 to Spoke-2 Gateway IPsec connection with an on-premises router or firewall: * Route Based VPN * Local Subnet: 172.32.0.0/16 * Remote Subnet: 192.168.0.0/16 5. Check the tunnel status at Diagnostics > Cloud Routes > External Connections. 6. Create a [Static Route-Based (Mapped) connection](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-create-static-route-based-mapped.html) from Spoke-2 to Site-2 using these values: | Parameter | Value | | - | - | | Local Subnet (real) | 172.32.0.0/16 | | Local Subnet (virtual) | 192.168.0.0/16 | | Remote Subnet (real) | 172.32.0.0/16 | | Remote Subnet (virtual) | 100.200.0.0/16 | 7. Create an on-prem site-2 to Spoke-2 gateway IPsec connection with an on-prem router or firewall. Key parameters: * Route Based VPN * Local Subnet: 172.32.0.0/16 * Remote Subnet: 192.168.0.0/16 8. Advertise 100.100.0.0/16 100.200.0.0/16 to TGW from Spoke-2 VPC/VNet: 1. On the Networking > Connectivity > AWS TGW tab, click the name of your AWS TGW. 2. Click the Attachments sub-tab and then click *VPC*. 3. Attach the Spoke-2 VPC if it is not attached already. 4. Ensure that the Customize Spoke VPC Routes field contains 172.34.0.0/100, 100.100.0.0/16, 100.200.0.0/16 (where 172.34.0.0/16 is Spoke-2 VPC/VNet CIDR; 100.100.0.0/16 is the virtual network CIDR of on-prem site-1; and 100.200.0.0/16 is the virtual network CIDR of on-prem site-2). 9. Test connectivity for on-prem site-1 by pinging an instance in Spoke-1 using the Spoke-1 virtual network CIDR with the real host portion of its IP address. For example, if the instance in Spoke-1 is 172.32.10.15, then site-1 should ping 192.168.10.15. 10. Test connectivity for on-prem site-2 by pinging an instance in Spoke-1 using the Spoke-1 virtual network CIDR with the real host portion of its IP address. For example, if the instance in Spoke-1 is 172.32.10.15, then site-2 should ping 192.168.10.15. ## Solving Overlapping IP Addresses Real-world use cases sometimes require a combination of [Site2Cloud](/docs/enterprise/10.1/guides/connectivity/vpn/s2c-configuration-workflow) and other features, such as [SNAT](https://legacy.docs.aviatrix.com/documentation/latest/network/snat-dnat-settings.html) and [DNAT](https://legacy.docs.aviatrix.com/documentation/latest/network/snat-dnat-settings.html). To learn more, see [Configuring Overlapping Networks with Customized SNAT and DNAT](https://legacy.docs.aviatrix.com/documentation/latest/network/s2c-overlapping-ip.html). # Configuring Overlapping Networks with Network Mapped IPsec Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/vpn/overlapping-networks-mapped-ipsec This document illustrates an example solution to a specific use case. In this AWS use case, a customer needs to connect certain on-prem hosts to certain EC2 instances in a VPC over an IPsec tunnel over the Internet, but the on-prem network range overlaps with the VPC CIDR range, and the... ## The Scenario This document illustrates an example solution to a specific use case. In this AWS use case, a customer needs to connect certain on-prem hosts to certain EC2 instances in a VPC over an IPsec tunnel over the Internet, but the on-prem network range overlaps with the VPC CIDR range, and the requirement from the customer is that no NAT function will be performed on the customer side. In addition, traffic can be initiated from either side. The scenario is described in the following diagram: ![overlap\_rbi](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/overlap-rbi.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=882ee79c28a9d6d7f5acaaf03f53ff6e) ``` VPC CIDR = 10.20.0.0/20, instance-1 in VPC-1 has an IP address 10.24.1.4. On-Prem CIDR = 10.20.0.0/20, host-1 in On-Prem has an IP address 10.24.7.101. ``` The traditional solution is to build an IPsec tunnel between the two networks and use SNAT/DNAT rules to translate each addresses, as demonstrated in this [example](/docs/enterprise/10.1/guides/connectivity/vpn/s2c-cidrs-overlap-connect). Such a solution requires a potentially large number of SNAT/DNAT rules which is difficult to configure and maintain. ## The Solution This solution uses the "network mapped" feature in Site2Cloud which removes the need to configure individual SNAT/DNAT rules. This solution uses a Site2Cloud route-based IPsec tunnel using Virtual Tunnel Interface (VTI) between VPC and On-Prem Router. The packet flow is demonstrated as below: 1. instance-1 sends a packet to host-1 with a virtual destination IP address, for example 192.24.7.101. From instance-1's point of view, the destination instance is a virtual address - 192.24.7.101. 2. When the packet arrives at the VPC-1 gateway, the gateway performs DNAT on the packet to translate the virtual destination IP address to 10.24.7.101 which is the host-1 physical IP address. 3. The gateway at VPC then translates the packet source IP address (10.24.1.4) to a virtual source IP address (172.24.1.4). 4. The packet then arrives at the on-prem Cisco IOS Router with a destination IP address of 10.24.7.101 and a source IP address of 172.24.1.4. From host-1's point of view, instance-1's address is a virtual IP address - 172.24.1.4. 5. When host-1 sends a packet to instance-1, the destination is the virtual IP address 172.24.1.4. 6. When the packet arrives at the VPC-1 gateway over the IPSEC tunnel, the VPC gateway translates its destination IP address from virtual address 172.24.1.4 to 10.24.1.4. 7. The VPC gateway then translates the source IP address of the packet from 10.24.7.101 to virtual address 192.24.7.101. ## The Configuration Steps ### Following the Site2Cloud Workflow to Launch Gateways In Aviatrix CoPilot, [create a Transit Gateway](https://legacy.docs.aviatrix.com/documentation/latest/network/transit-gateway-about.html?expand=true). For the above example, a gateway is also launched in VPC/VNet-2 to emulate an on-prem environment. ## Creating a Site2Cloud Tunnel In CoPilot, go to *Networking > Connectivity > External Connections (S2C)* tab, and create [Static Route-Based Unmapped connection](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-create-static-policy-based-unmapped.html). ### VPC/VNet-1 gateway-1 side For the VPC/VNet-1 gateway side, the Local Subnet field should be 192.168.0.43/32, and the Remote Subnet field should be 10.17.7.81/32, as shown below. ![500](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/connectivity/vpn/vpc1-to-vpc2-ipsec.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=25aebe59bc9e2335b7d8d849e6c256cc) ### Configure On-Prem Cisco Router On the Networking > Connectivity > External Connections (S2C) tab, click the vertical ellipsis ![25](https://mintcdn.com/aviatrix-14b37c43/mOZ0lmP7pRPIryx3/images/guides/platform-administration/threedot.png?fit=max\&auto=format\&n=mOZ0lmP7pRPIryx3\&q=85\&s=6ed1c085429177f16d2735e2406d15c3) and click *Download Configuration*. In the Download Configuration dialog, select: * *Cisco* from the Vendor dropdown. * *ISR, ASR, or CSR* from the Platform dropdown. * *IOS(XE)* from the Software dropdown. Click *Download*. Save the configuration file as a reference for configuring your Cisco IOS router. The following is a sample configuration based on the Site2Cloud configuration above. ![ios\_config\_template](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/ios-config-template.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=c424dedc6f5ae387d7895df69ab3cbb8) Either ssh into the Cisco router or connect to it directly through its console port. Apply the following IOS configuration to your router: ```text theme={null} ! Aviatrix Site2Cloud configuration template ! ! You need to populate these values throughout the config based on your setup: ! : the isakmp policy number ! : the IPsec tunnel interface number ! : the source interface of tunnel packets ! : any un-used IPv4 address for the tunnel interface ! when static routing is used ! ! -------------------------------------------------------------------------------- ! IPsec Tunnel ! -------------------------------------------------------------------------------- ! #1: Internet Key Exchange (IKE) Configuration ! A policy is established for the supported ISAKMP encryption, ! authentication, Diffie-Hellman, lifetime, and key parameters. ! crypto keyring 52.40.45.197-20.42.145.156 pre-shared-key address 20.42.145.156 key ! crypto isakmp policy 1 encryption aes 256 hash sha256 authentication pre-share group 14 lifetime 28800 crypto isakmp keepalive 10 3 periodic crypto isakmp profile 52.40.45.197-20.42.145.156 keyring 52.40.45.197-20.42.145.156 self-identity address match identity address 20.42.145.156 255.255.255.255 ! !--------------------------------------------------------------------------------- ! #2: IPsec Configuration ! The IPsec transform set defines the encryption, authentication, and IPsec ! mode parameters. ! crypto ipsec transform-set 52.40.45.197-20.42.145.156 esp-aes 256 esp-sha256-hmac mode tunnel crypto ipsec df-bit clear ! crypto ipsec profile 52.40.45.197-20.42.145.156 set security-association lifetime seconds 3600 set transform-set 52.40.45.197-20.42.145.156 set pfs group14 set isakmp-profile 52.40.45.197-20.42.145.156 ! !--------------------------------------------------------------------------------------- ! #3: Tunnel Interface Configuration ! The virtual tunnel interface is used to communicate with the remote IPsec endpoint ! to establish the IPsec tunnel. ! interface Tunnel1 ip address 10.10.10.10 255.255.255.255 ip mtu 1436 ip tcp adjust-mss 1387 tunnel source GigabitEthernet1 tunnel mode ipsec ipv4 tunnel destination 20.42.145.156 tunnel protection ipsec profile 52.40.45.197-20.42.145.156 ip virtual-reassembly ! !--------------------------------------------------------------------------------------- ! #4: Static Routing Configuration ! The static route directs the traffic to the Aviatrix remote subnets via the tunnel ! interface. ! ip route 172.24.0.0 255.255.240.0 Tunnel1 !--------------------------------------------------------------------------------------- ``` You can check the Tunnel Status at Diagnostics > Cloud Routes > External Connections. ## Testing the Site2Cloud Connection Make sure your instance's Security Groups inbound rules are configured properly. From instance-1, you should be able to ping host-1 by "ping 192.24.7.101". From host-1, you should be able to ping instance-1 by "ping 172.24.1.4" # Proxy-ID for Routed IPSec Connections Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/vpn/proxy-id-routed-ipsec-connections This document describes how to enable IKE phase 2 Proxy-ID traffic selectors on route-based Site-to-Cloud IPSec connections so Aviatrix gateways interoperate with policy-based IPSec peers while remaining route-based on the Aviatrix side. This document describes how to enable Proxy-ID negotiation on route-based IPSec Site-to-Cloud connections in Aviatrix Controller version 9.0 or later. ## Introduction Partner or third-party networks often terminate IPSec as **policy-based** VPN. Those peers expect IKE phase 2 **traffic selectors** (Proxy-IDs) scoped to specific source and destination CIDRs. Historically, Aviatrix route-based IPSec negotiated a broad selector (for example `0.0.0.0/0`), which does not match peers that require CIDR-specific selectors. When **Proxy-ID for Routed Connections** is enabled on a route-based connection, the gateway derives traffic selector pairs from the connection's configured local and remote CIDR ranges and negotiates those selectors during IKE. The Aviatrix gateway continues to use route mode with a virtual tunnel interface; the remote peer can remain policy-based. **Distributed Cloud Firewall (DCF)** continues to enforce policy on the Site-to-Cloud tunnel interface. For more information about Site2Cloud and external device connectivity, refer to: * [Site2Cloud External Connection Workflow](/docs/enterprise/10.1/guides/connectivity/vpn/s2c-configuration-workflow) * [Transit Gateway to External Devices Workflow](/docs/enterprise/10.1/guides/connectivity/vpn/transit-externaldevice-workflow) * [External Connection Settings](/docs/enterprise/10.1/guides/connectivity/gateways/external-connection-settings) ## Prerequisites * Needs Aviatrix Controller version 9.0 or later. * Needs a **route-based IPsec** Site-to-Cloud external connection (**Type** set to **Static Routing over IPsec** in CoPilot). ## Supported Connection Subtypes Proxy-ID applies to the following Site-to-Cloud cases. In CoPilot, these correspond to **Static Routing Type** options on the External Device wizard: * **Unmapped NAT** — selectors follow the connection's real local and remote CIDRs. * **Mapped NAT** — selectors follow virtual local CIDRs and real remote CIDRs. * **Active Mesh** — Transit or Spoke external device connections. * **Custom Mapped** — selectors follow the configured local and remote virtual and real CIDR pairs. CoPilot UI for this subtype is not yet available; use API or CLI. ## Configure Proxy-ID on a Route-Based Site-to-Cloud Connection To configure Proxy-ID on a route-based Site-to-Cloud connection: 1. In CoPilot, go to **Networking** > **Connectivity** > **External Connections (S2C)**. 2. Click **+ External Connection** and select **External Device**. 3. Set **Type** to **Static Routing over IPsec**. Proxy-ID is supported only on this Type; it is not available on **BGP over IPsec**, **BGP over GRE**, or **BGP over LAN**. 4. Set **Static Routing Type** to **Unmapped NAT**, **Mapped NAT**, or **Active Mesh** (depending on the Site-to-Cloud subtype you need). 5. Select the **Local Gateway** and configure the **Remote Subnet CIDR(s)**. Local CIDRs are derived from the selected gateway. 6. Toggle **Proxy-ID** to **On**. Traffic selector pairs are derived automatically from the connection's CIDR ranges. 7. Enter the remaining connection parameters and click **Save**. To enable Proxy-ID on an existing connection: 1. Go to **Networking** > **Connectivity** > **External Connections (S2C)**. 2. Select the route-based IPsec connection and click **Edit**. 3. Toggle **Proxy-ID** to **On**. 4. Click **Save**. ## Configure Proxy-ID on ActiveMesh Transit or Spoke to External Device To configure Proxy-ID on an ActiveMesh Transit or Spoke external device connection: 1. In CoPilot, go to **Networking** > **Connectivity** > **External Connections (S2C)**. 2. Click **+ External Connection** and select **External Device**. See [Transit Gateway to External Devices Workflow](/docs/enterprise/10.1/guides/connectivity/vpn/transit-externaldevice-workflow) for the broader workflow context. 3. Set **Type** to **Static Routing over IPsec**. 4. Set **Static Routing Type** to **Active Mesh**. 5. Select the Transit or Spoke Gateway as the **Local Gateway** and configure the **Remote Subnet CIDR(s)**. 6. Toggle **Proxy-ID** to **On**. 7. Click **Save**. > [!NOTE] > > Proxy-ID is not supported on **BGP** external connections or on tunnel > protocols **other than IPSec**. The Controller returns an error if Proxy-ID is > enabled on an unsupported combination. ## Verifying Proxy-ID on the Connection To verify Proxy-ID on the connection: 1. Open the external connection detail for the Site2Cloud or External Device connection. 2. Confirm **Proxy-ID Enabled** is **true** and review **Proxy-ID Pairs** for the expected local and remote CIDR pairs. 3. In CoPilot, go to **Diagnostics** > **Cloud Routes** > **External Connections** and confirm tunnel status for the connection. 4. If IKE negotiation fails, use **Diagnostics** > **Diagnostic Tools** > **Connectivity Diagnostics** or see [Troubleshooting IPsec VPN Connection with IKEv2](/docs/enterprise/10.1/guides/connectivity/vpn/troubleshooting-ipsec-vpn-ikev2) . For general diagnostics, see [Diagnostic Tools](/docs/enterprise/10.1/reference/ui/diagnostics/diagnostic-tools) . ## Proxy-ID Limitations * Proxy-ID applies to route-based IPSec Site-to-Cloud connections only. Policy-based connections already use explicit policies for traffic selection. * For ActiveMesh Transit or Spoke to External Device, Proxy-ID requires **Static** routing and **IPSec**. **BGP** and **non-IPSec** tunnel protocols are not supported with Proxy-ID. * Traffic selectors are derived **automatically** from configured CIDR ranges. You cannot specify selectors that do not align with those CIDRs. * If no valid local or remote CIDR pairs can be derived, enabling Proxy-ID may **log an error** without blocking connection creation in all cases. * CoPilot UI support may vary by Site2Cloud subtype; use **API** or **CLI** to configure Proxy-ID on **Mapped** or **Custom Mapped** workflows if the control is not yet shown in the UI. ## Related topics * [Site2Cloud External Connection Workflow](/docs/enterprise/10.1/guides/connectivity/vpn/s2c-configuration-workflow) * [Transit Gateway to External Devices Workflow](/docs/enterprise/10.1/guides/connectivity/vpn/transit-externaldevice-workflow) * [External Connection Settings](/docs/enterprise/10.1/guides/connectivity/gateways/external-connection-settings) * [Troubleshooting IPsec VPN Connection with IKEv2](/docs/enterprise/10.1/guides/connectivity/vpn/troubleshooting-ipsec-vpn-ikev2) # Aviatrix Gateway to Aviatrix Gateway Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/vpn/s2c-aviatrixgw-to-aviatrixgw This section describes how to configure an IPsec tunnel between an Aviatrix Gateway and another Aviatrix Gateway using Aviatrix Site2Cloud. This section describes how to configure an IPsec tunnel between an Aviatrix Gateway and another Aviatrix Gateway using Aviatrix Site2Cloud. > [!NOTE] > > There are only a couple of reasons to use Site2Cloud when connecting two Aviatrix Gateways: > > 1. You have overlapping CIDR blocks but need to peer two VPC/VNets. > 2. The two Aviatrix Gateways are not part of the same Controller (i.e., one is > at your customer and the other one is in your environment). ## Deployment There are two Aviatrix Gateways in this scenario. Since you are using the Site2Cloud feature, you must configure each side of the tunnel individually. In this procedure the gateways are referred to as **Gateway A** and **Gateway B**. You can pick either gateway to be **Gateway A** or **Gateway B**. ### Configure Tunnel from Gateway A to Gateway B 1. Follow the steps in [Site2Cloud Configuration Workflow](/docs/enterprise/10.1/guides/connectivity/vpn/s2c-configuration-workflow) to create a Mapped or Unmapped external (Site2Cloud) connection. Use this table for specific field values. | Field | Description | | - | - | | Connect Public Cloud to | External Device: Static Route-Based/Static Route-Based (Mapped) | | Local Gateway | Select **Gateway A** from the dropdown. | | Remote Device Type | Aviatrix | | Remote Device IP | Enter the public IP address of **Gateway B**. | | Pre-shared Key | Leave blank and Aviatrix will generate | 2. Click **Save**. 3. Once complete, click the vertical ellipsis ![25](/docs/enterprise/10.1/guides/connectivity/vpn/images/threedot.png) menu next to the new external connection and select **Download Configuration**. 4. Select **Aviatrix** for Vendor, **UCC** for Platform and **1.0** for Software. 5. Click **Download**. You can use the information in this file to create the other side of the tunnel. ### Configure Tunnel from Gateway B to Gateway A 1. Go to Networking > Connectivity > External Connections (S2C). 2. Create a new external connection using the information from the downloaded configuration file. Use this table for specific values: | Field | Description | | - | - | | Connect Public Cloud to | External Device: Static Route-Based/Static Route-Based (Mapped). | | Local Gateway | Select **Gateway B** from the dropdown. | | Remote Device Type | Aviatrix | | Remote Device IP | Enter the public IP address of **Gateway A**. | | Pre-shared Key | Leave blank and Aviatrix will generate | 3. Click **Save**. ## Test Once complete, you can check tunnel status at Diagnostics > Cloud Routes > External Connections. ## Troubleshoot Wait 2-3 minutes for the tunnel to come up. If it does not come up within that time, check the IP addresses to confirm they are accurate. Additional troubleshooting is available at Diagnostics > Diagnostic Tools. ## Related Topics * [About External Connection Settings](https://legacy.docs.aviatrix.com/documentation/latest/network/externaldevice-settings.html) # Aviatrix Gateway to AWS Virtual Private Gateway (VGW) Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/vpn/s2c-aviatrixgw-to-aws-vgw This section describes how to configure an IPsec tunnel between an Aviatrix Gateway and an AWS VGW. This section describes how to configure an IPsec tunnel between an Aviatrix Gateway and an AWS VGW. ![gw2vgw](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/gw-to-vgw.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=ef4d3ee9e5471bb8e01c2f2560e6c819) For this use case, you will first configure the AWS VGW VPN connection, and then download the configuration from AWS and import it into Aviatrix. ## Create the Customer Gateway > [!NOTE] > > **Prerequisites** > > 1. You have a VGW created and attached to a VPC. > 2. You have an Aviatrix Gateway provisioned in a different VPC. You will need > this gateway's public IP address for the steps below. > > You have the option of adding a High Availability (HA) instance when you create > your Aviatrix Gateway. 1. Log in to your [VPC Dashboard](https://console.aws.amazon.com/vpc/home) in the region where your VGW is located. 2. Create a new [Customer Gateway](https://console.aws.amazon.com/vpc/home#CreateCustomerGateway). In the IP address field, enter the public IP address of the Aviatrix Gateway. ![awscg](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/aws-cg.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=b8efab83f3881ef1da2ac791a93705c5) ## Create a VPN Connection 1. Create a [VPN Connection](https://console.aws.amazon.com/vpc/home#CreateVpnConnection:). ![aws vpn connection](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/aws-vpn.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=62744f4b9462f3910b3a64d96dd10957) | Field | Description | | - | - | | Name | Enter any name here | | Target gateway type | Select Virtual private gateway | | Customer Gateway | Select **Existing** and then select the gateway you created in the previous procedure. | | Routing Options | Select **Static** | | Static IP Prefixes | Enter the CIDR(s) of the VPC where the Aviatrix Gateway resides. | 2. You can leave the Tunnel 1 and 2 options as their defaults. Click **Create VPN connection**. 3. Select the VPN you just created and click **Download Configuration**. ![download vpn configuration](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/aws-download-vpn-config1.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=71c6652e82e4de69882ec18a38b457a0) 4. At the dialog, select **Generic** for the Vendor, **Generic** for the Platform and **Vendor Agnostic** for the Software. 5. Click **Download**. You will use this file to create the other side of the tunnel. ![download vpn configuration dialog](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/aws-download-vpn-config.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=e4bd4e7bcd7d79ad79cd37a60c8d7d7f) ## Configure the External Connection 1. Log in to CoPilot and go to Networking > Connectivity > External Connection. 2. Click **+External Connection**. 3. Follow the steps in one of these documents to create an Unmapped connection to an AWS TGW Remote Gateway: * [Static Route-Based](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-create-static-route-based-unmapped.html) * [Static Policy-Based](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-create-static-policy-based-unmapped.html) > [!NOTE] > > When creating this external connection, do the following: > > * Ensure that the Algorithms checkbox is selected. > * Select the **Generic** Remote Device Type. Any other Remote Devices listed > here are only valid with Controller version 6.7 or lower. 4. Enter the following values when configuring the external connection (make sure you have the downloaded configuration file from the previous section available): * Remote Subnet CIDR(s): This is usually the the Spoke VPC that on-prem wants to reach (in this example, 10.20.0.0/20) * Pre-Shared Key: Enter the value that matches the value from #1: Internet Key Exchange Configuration > Pre-Shared Key from the downloaded configuration file * Remote Device IP: Enter the value that matches #3: Tunnel Interface Configuration > Outside IP Addresses > Virtual Private Gateway from the downloaded file (in this example, 3.212.183.205) ![tunnel configuration](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/tunnel-config.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=5aad6a946cf7f19cd324b7e004a1d0f7) ## Testing the Connection Once complete, you can check the tunnel status at Diagnostics > Cloud Routes > External Connections. ## Troubleshooting If the tunnel does not come up within the expected time frame, you can check the IP addresses to confirm that they are accurate. Additional troubleshooting is available under Diagnostics > Diagnostic Tools. ## Appendix: Enable HA You can enable HA for Aviatrix Site2Cloud connection to AWS VGW. Please add the following extra steps to the configuration. ### Creating an Aviatrix HA Gateway Before creating a Site2Cloud connection, make sure that you [add an HA instance when creating your Aviatrix Gateway](/docs/enterprise/10.1/guides/gateways/gateway-ha-about#create-ha-gateway-instance) . ### Creating a VPN Connection Between VGW and the Aviatrix HA Gateway Follow the steps from [Create the Customer Gateway](#create-the-customer-gateway) above. The only differences are: * When you create the Customer Gateway, enter the Aviatrix HA gateway public IP * When you create the VPN connection: * Select the same VGW used for the primary VPN connection * Select the customer gateway you just created for HA * Enter the CIDR(s) of the VPC where the HA Aviatrix Gateway resides under Static IP Prefixes Download the configuration for this new VPN connection just like you did earlier for the primary VPN connection. ### Create Aviatrix Site2Cloud Connection with HA In CoPilot, go to Networking > Connectivity > External Connections (S2C) and add a new external connection. Add an instance for HA. Additional fields are displayed when checked. > [!NOTE] > > You need the information from the downloaded VPN configuration file to > configure the external connection between the AWS VPN and the Aviatrix HA > Gateway. Follow the same steps you did for the primary connection. | Field | Description | | - | - | | Backup Gateway | Select the Aviatrix HA Gateway you just created | | Remote Device IP Address (Backup) | Enter the value that matches the value from #3: Tunnel Interface Configuration > Outside IP Addresses > Virtual Private Gateway | | Pre-shared Key (Backup) | Enter the value that matches the value #1: Internet Key Exchange Configuration > Pre-Shared Key | Other fields should be filled as instructed in above section [Configure the External Connection](#configure-the-external-connection). ## Related Topics * [External Device Settings](https://legacy.docs.aviatrix.com/documentation/latest/network/externaldevice-settings.html) # Aviatrix Gateway to Azure VPN Gateway Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/vpn/s2c-aviatrixgw-to-azure-vpn This document describes how to configure Site2Cloud IPsec tunnels between an Aviatrix Gateway and an Azure Virtual Network Gateway (VNG). This document describes how to configure Site2Cloud IPsec tunnels between an Aviatrix Gateway and an Azure Virtual Network Gateway (VNG). ## Configuration Workflow Before you begin, ensure you have the latest Controller software. 1. In Aviatrix CoPilot, go to **Cloud Fabric > Gateways > Specialty Gateways** tab. From **+ Gateway** dropdown menu, select **Other**, then create a gateway. 2. At the Azure portal, go to the Virtual network gateways page. Fill in the following information to create a new Virtual Network Gateway: | Name | Description | | - | - | | Name | Enter a name for the Azure VPN gateway (for example, Azure-VPN-GW). | | Gateway type | VPN | | VPN type | Policy-based | | SKU | Basic | | Location | Select a desired location. | | Virtual network | Select a desired VNet. | > [!NOTE] > > Azure is [deprecating the Basic Load Balancer](https://learn.microsoft.com/en-us/answers/questions/1033471/retirement-announcement-basic-load-balancer-will-b) in September 2025. You can [migrate to the Standard SKU Load Balancer](/docs/enterprise/10.1/guides/uservpn/migrate-lb-sku). 3. Once the virtual network gateway is provisioned, record its Public IP address. 4. In Aviatrix CoPilot, create a [Static Policy-Based (Unmapped) external connection](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-create-static-policy-based-unmapped.html). 5. Once the Site2Cloud connection is created, locate the same connection on the **External Connections (S2C)** page. 6. Click the vertical ellipsis ![25](https://mintcdn.com/aviatrix-14b37c43/VY6W0iTAMUQkrobS/images/security/threedot.png?fit=max\&auto=format\&n=VY6W0iTAMUQkrobS\&q=85\&s=9498dd71f99faa81563f9d3f0baf4463) icon and select **Download Configuration**. 7. The Download Configuration dialog displays. Select the following values for each specific field: * Vendor: Generic * Platform: Generic * Software: Vendor Independent 8. Click **Download**. 9. Collect the following information from the downloaded configuration template: | Pre-Shared Key from #1 | Internet Key Exchange Configuration | | - | - | | Aviatrix Gateway Public IP from #3 | Tunnel Interface Configuration | | Cloud Network(s) from the Subnets section of #3 | Tunnel Interface Configuration | 10. At the Azure portal, go to the **Local network gateways** page. Enter the following information to create a local network gateway: | Name | Description | | - | - | | Name | Enter a local gateway name (e.g. AVX-GW) | | IP address | Enter the Aviatrix Gateway's public IP collected at Step 6 | | Address space | Enter the "Cloud Network" CIDR collected at Step 6 | | Configure BGP settings | Unmark this checkbox | 11. At Azure portal, go to **Virtual network gateways** page and select the gateway created at Step 2. 12. Select **Connections** from **Settings**. Enter the following information to create a connection: | Name | Description | | - | - | | Name | Enter a VPN connection name (e.g. Azure-AVX-S2C) | | Connection type | Select **Site-to-site (IPsec)** | | Virtual network gateway | Select the VPN gateway created at Step 2 | | Local network gateway | Select the local gateway created at Step 7 | | Shared key (PSK) | Enter the pre-shared key collected at Step 6 | ## Troubleshooting To check a tunnel state, go to the **Networking > Connectivity > External Connections (S2C)** tab. There should be a green dot next to the name of the external connection. To troubleshoot a tunnel state, go to [Diagnostics > Diagnostic Tools > Connectivity Diagnostics](/docs/enterprise/10.1/reference/ui/diagnostics/diagnostic-tools). ## Related Topics * [About External Connection Settings](/docs/enterprise/10.1/guides/connectivity/gateways/external-connection-settings) * [Migrating to Standard Load Balancer SKU](/docs/enterprise/10.1/guides/uservpn/migrate-lb-sku) # Aviatrix Gateway to Check Point(R77.30) Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/vpn/s2c-aviatrixgw-to-checkpoint-r77 This document describes how to build an IPsec tunnel-based Site2Cloud connection between Aviatrix Gateway and Check Point Firewall. To simulate an on-prem Check Point Firewall, we use a Check Point CloudGuard IaaS firewall VM at AWS VPC. This document describes how to build an IPsec tunnel-based Site2Cloud connection between Aviatrix Gateway and Check Point Firewall. To simulate an on-prem Check Point Firewall, we use a Check Point CloudGuard IaaS firewall VM at AWS VPC. > [!NOTE] > > If you do not have access to AWS, you can simulate an on-prem Firewall by > deploying the Palo Alto Firewall in any other cloud (such as Microsoft Azure, > Google Cloud Platform, or Oracle Cloud Infrastructure). The network setup is as follows: **VPC1 (with Aviatrix Gateway)** *VPC1 CIDR: 10.0.0.0/16* *VPC1 Public Subnet CIDR: 10.0.1.0/24* *VPC1 Private Subnet CIDR: 10.0.2.0/24* **VPC2 (with Check Point Security Gateway)** *VPC2 CIDR: 10.10.0.0/16* *VPC2 Public Subnet CIDR: 10.10.0.0/24* *VPC2 Private Subnet CIDR: 10.10.1.0/24* ## Launching Check Point Security Gateway VM Refer to the [Check Point vSEC for AWS Lab Setup Guide](https://supportcenter.checkpoint.com/supportcenter/portal?eventSubmit_doGoviewsolutiondetails=\&solutionid=sk111013) to launch a CheckPoint VM with at least two network interfaces. One interface serves as a WAN port and is in VPC2's public subnet. The other interface serves as a LAN port and is in VPC2's private subnet. Collect the public IP address of the WAN port. ## Creating an External Connection 1. In Aviatrix CoPilot, launch an [Aviatrix Transit Gateway](/docs/enterprise/10.1/reference/ui/cloud-fabric/gateways#view-transit-gateways) in VPC1's public subnet. Collect both the public and private IP address of the Gateway. 2. Navigate to Networking > Connectivity > External Connections (S2C) and click **Add New** to create a Site2Cloud connection (selecting Pre-Shared Key) using the values for one of the below options: * [Static Route-Based](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-create-static-route-based-unmapped.html) * [Static Policy-Based](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-create-static-policy-based-unmapped.html) Substitute the following values: * Remote Device Type: Generic * Algorithms: turn Off * Remote Device IP: Public IP of the Check Point VM WAN port * Remote Subnet CIDR(s): 10.10.1.0/24 (VPC2 private subnet) * Local Subnet CIDR(s): 10.0.2.0/24 (VPC1 private subnet) 3. After the connection is created, select the vertical ellipsis ![threedot](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/threedot.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=36963cd7f65aa469c6d5fe2021fa71a7) menu for that connection and select **Download Configuration**. 4. Select **Generic** from the **Vendor** dropdown list and click the **Download** to download the external (S2C) configuration. Use this configuration file to configure the tunnels and interfaces in your Check Point VM firewall. ## Downloading and Installing SmartConsole 1. Using a browser, connect to the Gaia Portal of the CheckPoint-VM at `https:/CheckPoint-VM_Public-IP`: ![image0](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/download-smart-console1.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=90e866fa6d54d54e4e41cd082f11d1e4) 2. Click **Overview** at the left navigation bar, and then click **Download Now!** to download SmartConsole. 3. Install SmartConsole at your local machine and launch SmartDashboard. ### Creating Network Objects at SmartConsole 1. At the Check Point SmartDashboard window, select the **Desktop** tab. Right click the **Networks** folder at the left navigation bar and select **Network**. 2. Create one network for private subnet of VPC2 (Check Point VPC). ![image1](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/network-checkpoint.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=41093302add3b8eb85cd56abd34e290b) | **Field** | **Value** | | - | - | | Name | Arbitrary (e.g. CP-Private-Subnet) | | IPv4 Network Address | VPC2 private subnet CIDR | | IPv4 Net mask | VPC2 private subnet mask | 3. Create one network for private subnet of VPC1 (Aviatrix Gateway VPC). ![image2](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/network-private.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=d8a86ab9402243425e723ed9e8cb2a4b) | **Field** | **Value** | | - | - | | Name | Arbitrary (e.g. AVX-Private-Subnet) | | IPv4 Network Address | VPC1 private subnet CIDR | | IPv4 Net mask | VPC1 private subnet mask | ### Configuring Check Point Security Gateway with VPN 1. At the SmartDashboard window, select the **Desktop** tab and expand the **Check Point** folder at the left navigation bar. Note that your gateway VM with the name format "gw-xxxxxx" is automatically created. ![image3](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/desktop-gw-config1.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=09bfb005ea9fe27d6e2697f3bcc801b4) 2. Right-click the gateway name and select **Edit** from the menu. 3. At the Check Point Gateway > General Properties window: ![image4](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/edit-gw-one.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=15d542d2fabc806678bb9d22b61f60c9) | **Field** | **Value** | | - | - | | IPv4 Address | Private IP of CheckPoint VM WAN port | | Test SIC Status | Make sure the status is "communicating" | | Network Security | Select **IPsec VPN** | 4. At **Check Point Gateway - Topology** window, select **Manually defined** for VPN Domain. Select the network created when you created a network for private subnet of VPC2 (Check Point VPC). ![image5](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/edit-gw-two.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=0c9f1434a94dd50b4fa0895b29148561) 5. At **Check Point Gateway - Topology** window, double-click "eth0" (Check Point WAN port). Select **External (leads out to the Internet)**. ![image6](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/edit-gw-three.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=da7a44617da09e1c5f13ddae7c4074eb) 6. At **Check Point Gateway - Topology** window, double click "eth1" (Check Point LAN port). Select **Internal (leads to the local network)**. ![image7](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/edit-gw-four.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=826df33fb9ba745c8c93d6ae08b6f84d) 7. At the **Check Point Gateway - IPsec VPN - Link Selection** window, configure the parameters as follows: ![image8](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/edit-gw-five.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=a077469a7c045dba88e5c4cea7c56ebe) | **Field** | **Value** | | - | - | | Statically NATed IP | Public IP of Check Point WAN port | | Selected address from topology table | Private IP of Check Point WAN port | 8. At the **Check Point Gateway - IPsec VPN - VPN Advanced** window, configure the parameters as follows: ![image9](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/edit-gw-six.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=2266754c5c9980099c0b4656a18d7ba0) ### Configuring an Interoperable Device to Represent Aviatrix Gateway 1. At Check Point SmartDashboard window, select the **Desktop** tab. Right-click the **Networks** folder at the left navigation bar to create a new interoperable device. 2. At the Interoperable Device - General Properties window: ![image10](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/interoperable-one.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=398617748e7776289ab35dd128c61976) | **Field** | **Value** | | - | - | | Name | Arbitrary (e.g. AVX-GW) | | IPv4 Address | Public IP of Aviatrix Gateway | 3. At the **Interopable Device - Topology** window, select **Manually defined** for VPN Domain. Select the network private subnet of VPC1 (Aviatrix Gateway VPC) you created above. ![image11](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/interoperable-two.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=e3df7cdf3e40a63b902d08fbb62d9750) 4. At the **Interopable Device - IPsec VPN - Link Selection** window, select **Always use this IP address > Main Address**. ![image12](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/interoperable-three.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=1582b2d532fbaf6574bbb6a3f7f9c22b) 5. At the **Interopable Device - IPsec VPN - VPN Advanced** window, select **Use the community settings**. ![image13](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/interoperable-four.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=a19f9af4276fbbe7e2c6f8246c33b895) ### Creating a VPN Community 1. At SmartDashboard **IPsec VPN** tab, select **Overview** from left navigation bar. Click **New** to create a Meshed Community. ![image14](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/community-one.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=f42d960a824950054cafb7301bdd38f4) 2. At **Meshed Community Properties - General** window, create one community with a name (e.g. Site2Cloud-avx). ![image15](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/community-two.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=2a4a94758b4dc63c4c53629ef3fe1280) 3. At **Meshed Community Properties - Participating Gateways** window, add both Check Point Security Gateway (e.g. gw-fe024c) and the interopable device created when you configured an interoperable device to represent the Aviatrix Gateway (e.g. AVX-GW) to this community. ![image16](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/community-seven.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=51997d3d591baafe2eae0568f17bbdbe) 4. At **Meshed Community Properties - Encryption** window, select the options according to the Site2Cloud configuration for configuring CheckPoint-VM you saved and downloaded above. ![image17](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/community-three.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=37f01bfce0f351a7f3c6bed16c70669a) 5. At **Meshed Community Properties - Tunnel Management** window, select **One VPN tunnel per Gateway pair** for **VPN Tunnel Sharing**. ![image18](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/community-four.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=23faeffcf67fe8440e3ce8148eddd0e7) 6. At the **Meshed Community Properties - Advanced Settings - Shared Secret** window, enter **Shared Secret** by copying the **Pre-Shared Key** from the Site2Cloud configuration downloaded above. ![image19](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/community-five.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=a0def91553a8ac6e32bfc43ea60a5de9) 7. At the **Meshed Community Properties - Advanced Settings - Advanced VPN Properties** window, enter the Phase1 and Phase2 parameters according to the Site2Cloud configuration downloaded above. ![image20](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/community-six.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=fdd6f1ee62402f0f6939ab5eff17839c) ### Creating Firewall Rule for VPN Traffic 1. At SmartDashboard window, select the **Firewall** tab. 2. Select **Policy** to add a new rule. ![image21](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/firewall-one.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=6dd5691649cfe95813d3ca6d4920cce7) | **Field** | **Value** | | - | - | | VPN | Select the Meshed VPN Community created above | | Install On | Select Check Point Security Gateway | 3. Click **Install Policy** to push the firewall policy to the Check Point Security Gateway. ![image22](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/firewall-two.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=daf5f73437c67f22208776536234dc85) ### Troubleshooting and Verifying at Check Point Security Gateway 1. At SmartDashboard window, from **SmartConsole** dropdown list, select **SmartView Monitor**. ![image23](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/cpmonitor-one.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=d95068087231f43aea506e7701519f16) 2. At the SmartView Monitor window, select **VPNs** from **Gateway Status** and verify **Encrypted Traffic**. ![image24](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/cpmonitor-two.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=05d3a79ea252048fdaf8583d6ee6b5b7) ### Verifying the Connection Status In CoPilot, go to the Diagnostics > Cloud Routes > External Connections tab to confirm the Status and Tunnel Status of the external connection. ### Troubleshooting You can use the Diagnostic Tools (under Diagnostics in the left navigation menu) to troubleshoot any connection issues. For support, please open a support ticket at [Aviatrix Support Portal](https://support.aviatrix.com). ## Related Topics * [About External Connection Settings](https://legacy.docs.aviatrix.com/documentation/latest/network/externaldevice-settings.html) # Aviatrix Gateway to Check Point (R80.10) Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/vpn/s2c-aviatrixgw-to-checkpoint-r80 This document describes how to build an IPsec tunnel based Site2Cloud connection between Aviatrix Gateway and Check Point Firewall. To simulate an on-prem Check Point Firewall, we use a Check Point CloudGuard IaaS firewall VM at AWS VPC. This document describes how to build an IPsec tunnel based Site2Cloud connection between Aviatrix Gateway and Check Point Firewall. To simulate an on-prem Check Point Firewall, we use a Check Point CloudGuard IaaS firewall VM at AWS VPC. > [!NOTE] > > If you do not have access to AWS, you can simulate an on-prem Firewall by > deploying the Palo Alto Firewall in any other cloud (such as Microsoft Azure, > Google Cloud Platform, or Oracle Cloud Infrastructure). The network setup is as follows: **VPC1 (with Aviatrix Gateway)** *VPC1 CIDR: 10.12.0.0/16* *VPC1 Public Subnet CIDR: 10.12.0.0/23* *VPC1 Private Subnet CIDR: 10.12.2.0/23* **VPC2 (with Check Point Security Gateway)** *VPC2 CIDR: 10.24.0.0/16* *VPC2 Public Subnet CIDR: 10.24.0.0/23* *VPC2 Private Subnet CIDR: 10.24.2.0/23* ## Launching Check Point Security Gateway VM Launch a CheckPoint VM with at least two network interfaces. One interface serves as a WAN port and is in VPC2's public subnet. The other interface serves as a LAN port and is in VPC2's private subnet. Collect the public IP address of the WAN port. ## Creating an External Connection 1. In Aviatrix CoPilot, launch an [Aviatrix Transit Gateway](/docs/enterprise/10.1/reference/ui/cloud-fabric/gateways#view-transit-gateways) in VPC1's public subnet. Collect both the public and private IP address of the Gateway. 2. Navigate to Networking > Connectivity > External Connections (S2C) and click **Add New** to create a Site2Cloud connection (selecting Pre-Shared Key) using the values for one of the below options: * [Static Route-Based](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-create-static-route-based-unmapped.html) * [Static Policy-Based](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-create-static-policy-based-unmapped.html) 3. Substitute the following values: * Remote Device Type: Generic * Algorithms: turn Off * Remote Device IP: Public IP of the Check Point VM WAN port * Remote Subnet CIDR(s): 10.24.2.0/23 (VPC2 private subnet) * Local Subnet CIDR(s): 10.12.2.0/23 (VPC1 private subnet) 4. After the connection is created, select the vertical ellipsis ![vertical ellipsis icon](https://mintcdn.com/aviatrix-14b37c43/VY6W0iTAMUQkrobS/images/security/threedot.png?fit=max\&auto=format\&n=VY6W0iTAMUQkrobS\&q=85\&s=9498dd71f99faa81563f9d3f0baf4463) menu for that connection and select **Download Configuration**. 5. Select **Generic** from the **Vendor** dropdown list and click the **Download** to download the external (S2C) configuration. Use this configuration file to configure the tunnels and interfaces in your Check Point VM firewall. ## Downloading and Installing SmartConsole 1. Using a browser, connect to the Gaia Portal of the CheckPoint-VM at `https:/CheckPoint-VM_Public-IP:` 2. Click **Download Now!** as shown below to download SmartConsole. ![Download SmartConsole from Gaia Portal](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/checkpoint-r80-image1.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=4a02a037aa2387d6cb3d0b34a2aee9c7) 3. Install SmartConsole at your local machine and launch SmartDashboard. ### Creating Network Objects at SmartConsole 1. At the Check Point SmartDashboard window, go to New > Network and create two objects. ![Create new network object in SmartDashboard](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/checkpoint-r80-image2.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=14679c8ee036a3558fc3d770dce015d2) 2. Create one network for private subnet of VPC2 (Check Point VPC). ![Check Point VPC private subnet network object](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/checkpoint-r80-image3.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=f0e51bf7feca4ac9d47db9c1fb50ccd8) | **Field** | **Value** | | - | - | | Name | Arbitrary (e.g. CP-Private-Subnet) | | IPv4 Network Address | VPC2 private subnet CIDR | | IPv4 Net mask | VPC2 private subnet mask | 3. Create one network for private subnet of VPC1 (Aviatrix Gateway VPC). ![Aviatrix Gateway VPC private subnet network object](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/checkpoint-r80-image4.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=8ba79d0c27dd925dcf83cf70f8cc9d6a) | **Field** | **Value** | | - | - | | Name | Arbitrary (e.g. AVX-Private-Subnet) | | IPv4 Network Address | VPC1 private subnet CIDR | | IPv4 Net mask | VPC1 private subnet mask | ### Configuring Check Point Security Gateway with VPN 1. At the SmartDashboard window, go to **Gateways and services** > double-click on the gateway. ![Check Point gateway in SmartDashboard](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/checkpoint-r80-image5.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=9d05295ee549da4df324e6dd6791617b) 2. Enter the private IP of the Check Point VM WAN port in the IPv4 address field. 3. Under Network Security, select **IPsec VPN**. ![IPsec VPN checkbox selection](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/checkpoint-r80-image6.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=02d363f28d553102050ac286f4261744) 4. Go to Network management > **VPN domain** > click **Manually defined** and select the network created previously (see [Creating Network Objects at SmartConsole](#creating-network-objects-at-smartconsole) above). ![VPN domain manual definition](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/checkpoint-r80-image7.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=509438b0baf43843ecfec61d7c8500b2) 5. Go to Network management and double-click "eth0" (Check Point WAN port). 6. Select **External (leads out to the Internet)** and click **OK**. 7. Go to Network management and double-click "eth1" (Check Point LAN port). 8. Click on modify. 9. Select the following: * **Override** * **This network (internal)** * **Specific**, and select the network created previously (see [Creating Network Objects at SmartConsole](#creating-network-objects-at-smartconsole) above). 10. Click **OK**. 11. Double-click on gateway as shown in step 1 above. 12. Select IPsec VPN in the left menu and then select the **Statically NATed IP** radio button. 13. Next to this radio button, enter the public IP of the Check Point WAN port. 14. Double-click on the gateway as shown in step 1 above. 15. Select VPN Advanced in the left menu and ensure that **Use the community settings** is selected. 16. Ensure that the **Support NAT traversal** checkbox is selected. 17. Click **OK**. ### Configuring an Interoperable Device to Represent Aviatrix Gateway 1. Go to Gateways and services > New network objects > Interoperable devices. 2. Click **Add new** and then use the image below to create a new interoperable device to represent Aviatrix Gateway. ![Add new interoperable device](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/checkpoint-r80-image12.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=083c227e18b8b2284dd192fc08cef5c1) ![Interoperable device configuration](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/checkpoint-r80-image13.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=ded7ae78aa4f06453e52540945132d78) 3. Double-click on Interoperable device > avx-gwv (created in step 1 of this section) > General properties. The IPv4 address will be the public IP of the Aviatrix Gateway. ![Interoperable device general properties](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/checkpoint-r80-image-14.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=6691d5f0b185db0d8d71300372dce968) 4. Double-click on Interoperable device > avx-gwv (created in step 1 in this section). 5. Select Topology in the left menu and select the **Manually defined** radio button. 6. In the field next to the radio button, select the network for the private subnet of the VPC1 (Aviatrix Gateway VPC) network created above. 7. Click **OK**. 8. Double-click on Interoperable device > avx-gwv (created in step 1 of this section. 9. Click Link Selection under IPsec VPN in the left menu and then select the **Always use this IP address** and **Main address** radio buttons. 10. Click **OK**. 11. Double-click on Interoperable device > avx-gwv (created in step 1 of this section). 12. Select VPN Advanced under IPsec VPN in the left menu. 13. Select **Use the community settings** and click **OK**. ### Creating a VPN Community 1. Click on VPN communities on the smart console. Then, create a Star Community as shown below. ![VPN communities in SmartConsole](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/checkpoint-r80-image18.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=ca74bab94655d6a0be011ccc12c6d3ff) ![Create Star Community](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/checkpoint-r80-image19.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=f5d588953fe72766f16848fed1234418) 2. After creating the VPN community, double-click on the created VPN community > Gateway tab. 3. In the star-community dialog, select the gateway created above (see [Configuring Check Point Security Gateway with VPN](#configuring-check-point-security-gateway-with-vpn)). ![Star community gateway selection](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/checkpoint-r80-image20.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=48beeac89822094a211f9fd76f46f410) 4. Click **OK**. 5. Double-click on the created VPN community and then select Encryption in the left menu. 6. Select the options according to the Site2Cloud configuration downloaded previously (see [Creating an External Connection](#creating-an-external-connection) above). 7. Click **OK**. 8. Double-click on the created VPN community and then select Tunnel management in the left menu. 9. Select **Set Permanent Tunnels** / **On all tunnels in the community**. 10. Select the **One VPN tunnel per Gateway pair** radio button. 11. Click **OK**. 12. Double-click on created VPN community > VPN routing > select as shown in the image below. ![VPN routing configuration](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/checkpoint-r80-image23.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=1676db14a6c6cb68f529e669a41f3cd0) 13. Double-click on the created VPN community. In the left menu select Shared Secret. 14. Enter the Shared Secret by copying the Pre-Shared Key from the Site2Cloud configuration downloaded previously (see [Creating an External Connection](#creating-an-external-connection) above). ![Shared Secret configuration](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/checkpoint-r80-image24.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=ae7ad01702067fb83cba9fa87147c016) 15. Click **OK**. 16. Double-click on the created VPN community. 17. In the left menu select Advanced. 18. Enter the Phase1 and Phase2 parameters according to the Site2Cloud configuration downloaded previously (see [Creating an External Connection](#creating-an-external-connection) above). 19. Click **OK**. ### Creating Firewall Rule for VPN Traffic 1. Go to security and policies. Add a policy and click **Install Policy**. ![Firewall rule for VPN traffic](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/checkpoint-r80-image26.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=b07bbe20a667aaaa004027e5538f7e4b) ### Troubleshooting and Verifying at Check Point Security Gateway 1. Go to **Logs and monitor** and dd a new tab. 2. Click **Open Tunnel & User Monitoring**. ![Open Tunnel and User Monitoring](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/checkpoint-r80-image27.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=78d6371d414cf7bed196248c69b557be) 3. Click **IPsec VPN** to see the tunnel status. ![IPsec VPN tunnel status](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/checkpoint-r80-image28.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=4bc2e2ddcdf1d5e9d3f8e35a2e77ab03) ![IPsec VPN tunnel details](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/checkpoint-r80-image29.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=67d8e9151b3ac9d1a85ba6fc705a7c6d) ### Verifying the Connection Status In CoPilot, go to the Diagnostics > Cloud Routes > External Connections tab to confirm the Status and Tunnel Status of the external connection. ### Troubleshooting You can use the Diagnostic Tools (under Diagnostics in the left navigation menu) to troubleshoot any connection issues. Below is the sample output for ping from an instance in the Aviatrix private subnet to an instance in the Check Point private subnet. ![Ping test from Aviatrix subnet to Check Point subnet](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/checkpoint-r80-image32.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=2fd5f3520330176bc4b9b78925f015e1) ## Related Topics * [About External Connection Settings](/docs/enterprise/10.1/guides/connectivity/gateways/external-connection-settings) * [Aviatrix Gateway to Check Point(R77.30)](/docs/enterprise/10.1/guides/connectivity/vpn/s2c-aviatrixgw-to-checkpoint-r77) * [Aviatrix Gateway to Cisco ASA](/docs/enterprise/10.1/guides/connectivity/vpn/s2c-aviatrixgw-to-cisco-asa) # Aviatrix Gateway to Cisco ASA Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/vpn/s2c-aviatrixgw-to-cisco-asa This document describes how to build an IPsec tunnel based Site2Cloud connection between an Aviatrix Gateway and Cisco ASA Firewall. This document describes how to build an IPsec tunnel based Site2Cloud connection between an Aviatrix Gateway and Cisco ASA Firewall. The network setup is as follows: ![Network setup diagram](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/threedot.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=36963cd7f65aa469c6d5fe2021fa71a7) **VPC/VNet-AVX (with Aviatrix Gateway)** *VPC/VNet CIDR: 10.0.0.0/16* *VPC/VNet Public Subnet CIDR: 10.0.1.0/24* *VPC/VNet Private Subnet CIDR: 10.0.2.0/24* **On-Prem (with Cisco ASA Firewall)** *On-Prem Network CIDR: 10.10.0.0/16* ## Creating an External Connection 1. In Aviatrix CoPilot, launch an [Aviatrix Transit Gateway](https://legacy.docs.aviatrix.com/documentation/latest/network/transit-gateway-about.html?expand=true) at the subnet (public subnet for AWS, GCP, or OCI) of VPC/VNet-AVX. Collect the Gateway's public IP addresses (52.8.12.122 in this example). 2. Navigate to Networking > Connectivity > External Connections (S2C) to create a Site2Cloud connection using the values for one of the below options: * [Static Route-Based](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-create-static-route-based-unmapped.html) * [Static Policy-Based](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-create-static-policy-based-unmapped.html) Substitute the following values: * Remote Device Type: Generic * Algorithms: turn Off * Remote Device IP: Public IP of ASA WAN port * Remote Subnet CIDR(s): 10.10.0.0/16 (VPC2 private subnet) * Local Subnet CIDR(s): 10.0.2.0/24 (VPC1 private subnet) 3. After the connection is created, select the vertical ellipsis menu for that connection and select **Download Configuration**. 4. In the Download Configuration dialog, select **Cisco** from the **Vendor** dropdown menu. 5. Select **ASA 5500 Series** from the **Platform** dropdown menu. 6. Select the proper ASA Software version from **Software** drop down list depending on your ASA running OS. 7. Click **Download**. 8. Save the configuration file as a reference for configuring your ASA. The following is an excerpt of an ASA sample configuration based on the Site2Cloud configuration above. ![ASA sample configuration excerpt](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/document-cisco.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=77092e9b6bf210e5c49a569b6ba59805) ### Configuring Cisco ASA 1. Either SSH into the ASA or connect to it directly through its console port. 2. Issue the **configure terminal** command in privileged **EXEC** mode to start global configuration mode. The prompt changes to the following : ``` hostname(config)# ``` 3. Enter the CLIs as per the downloaded configuration. Note that you may need to modify these CLIs to fit your ASA configuration. ## Verifying the Connection Status In CoPilot, go to the Diagnostics > Cloud Routes > External Connections tab to confirm the Status and Tunnel Status of the external connection. ## Troubleshooting You can use the Diagnostic Tools (under Diagnostics in the left navigation menu) to troubleshoot any connection issues. For support, please open a support ticket at [Aviatrix Support Portal](https://support.aviatrix.com). ## Related Topics * [About External Connection Settings](https://legacy.docs.aviatrix.com/documentation/latest/network/externaldevice-settings.html) # Aviatrix Gateway to Cisco Ios Router Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/vpn/s2c-aviatrixgw-to-cisco-ios-router This document describes how to build an IPsec tunnel based Site2Cloud connection between an Aviatrix Gateway and a Cisco IOS router. This document describes how to build an IPsec tunnel based Site2Cloud connection between an Aviatrix Gateway and a Cisco IOS router. The network setup is as follows: **VPC/VNet-AVX (with Aviatrix Gateway)** *VPC/VNet CIDR: 10.100.0.0/24* **On-Prem (with Cisco IOS Router)** *On-Prem Network CIDR: 10.10.2.0/24* ## Creating a Site2Cloud Connection 1. In Aviatrix CoPilot, launch an [Aviatrix Transit Gateway](/docs/enterprise/10.1/reference/ui/cloud-fabric/gateways#view-transit-gateways) at the subnet (public subnet for AWS, GCP, or OCI) of VPC/VNet-AVX. Collect the Gateway's public IP addresses (52.4.65.172 in this example). 2. Navigate to Networking > Connectivity > External Connections (S2C) to create a Site2Cloud connection using the values for one of the below options: | Static Route-Based | Static Policy-Based | | - | - | | [Static Route-Based](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-create-static-route-based-unmapped.html) | [Static Policy-Based](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-create-static-policy-based-unmapped.html) | Substitute the following values: * Remote Device Type: Generic * Algorithms: turn Off * Remote Device IP: Public IP of IOS Router WAN port (52.9.196.2 in this example) * Remote Subnet CIDR(s): 10.10.2.0/24 (On-Prem Network CIDR) (VPC2 private subnet) * Local Subnet CIDR(s): 10.100.0.0/24 (VPC/VNet-AVX CIDR) (VPC1 private subnet) 3. After the connection is created, select the vertical ellipsis ![25](/docs/enterprise/10.1/guides/connectivity/vpn/images/threedot.png) menu for that connection and select **Download Configuration**. 4. In the Download Configuration dialog, select **Cisco** from the **Vendor** dropdown menu. 5. Select **ISR, ASR, or CSR** from the Platform dropdown menu. 6. Select the appropriate Software version. 7. Click **Download**. 8. Save the configuration file as a reference for configuring your Cisco IOS router. The following is a sample configuration based on the Site2Cloud configuration above. ![s2c sample config](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/s2c-sample-config.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=fa42ff57d49bae07b6a627e58d2376aa) ## Configuring Cisco IOS Router 1. Either ssh into the Cisco router or connect to it directly through its console port. 2. Apply the following IOS configuration to your router. > [!NOTE] > > From version 5.0, we use the gateway's public IP address as the identifier, so > the "match identity address" should use the public IP instead of the private > IP as shown below. ![image1](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/s2c-ios.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=5b1b79f39e434578eb51ac66f5ed2d1d) ## Verifying the Connection Status In CoPilot, go to the Diagnostics > Cloud Routes > External Connections tab to confirm the Status and Tunnel Status of the external connection. ## Troubleshooting You can use the Diagnostic Tools (under Diagnostics in the left navigation menu) to troubleshoot any connection issues. For support, please open a support ticket at [Aviatrix Support Portal](https://support.aviatrix.com). ## Appendix 1: Transit Connection to Cisco ISR/ASR Over the Internet The following is the topology used for the sample configuration below: ![External-Device-Internet](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/external-device-internet.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=cc25da31a725b586938eb4d1ffb005e1) For connection over the Internet, an Aviatrix Transit Gateway and Cisco ISR/ASR use each other's public IP to create an IPsec tunnel and establish a BGP connection. The following diagrams display mappings between a sample downloaded configuration and its corresponding Cisco ISR/ASR router configuration: ![500](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/transitgw-phase1.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=c8c7205765e8573610bfb334b2482533) ![500](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/transitgw-phase2.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=eba30f0250eb07b16a4952994bb73dee) ![500](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/transitgw-tunnel.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=91a0e9f97febb67a83f4b58439d6fd97) ![500](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/transitgw-bgp.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=e2be856a671ec1d0af6aeddfa939c481) ## Appendix 2: Transit Connection to Cisco ISR/ASR over Direct Connect The following is the topology used for the sample configuration below: ![500](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/external-device-dx.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=973fb902b33414e662b263a5b208fa09) For connection over Direct Connect, the Aviatrix Transit Gateway and Cisco ISR/ASR use each other's private IP to create an IPsec tunnel and establish BGP connection. > [!NOTE] > > The ASN number of the Aviatrix Transit Gateway created earlier in this > document (Local ASN) should be the same as the VGW's ASN number (7224 in this > example). Without it, the Transit VPC/VNet CIDR advertised from VGW to on-prem > ASR/ISR will be advertised by ASR/ISR back to the Aviatrix Transit Gateway. > With the same ASN number, the Aviatrix Transit Gateway will drop the route to > the Transit VPC/VNet CIDR. The following diagrams display mappings between a sample configuration from [Creating a Site2Cloud Connection](#creating-a-site2cloud-connection) and its corresponding Cisco ISR/ASR router configuration: ![500](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/transitgw-phase1-dx.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=6af56ea23d52f21807944893536edfd0) ![500](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/transitgw-phase2-dx.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=b3f8c59a08e73d37155364be7160b01d) ![500](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/transitgw-tunnel-dx.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=96ef17f0b2052734e6696ef8b0104c02) ![500](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/transitgw-bgp-dx.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=8d8c7d35dc4c983854a1d0f5bac683df) ## Related Topics * [Overview of Transit Gateway to External Devices](/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/transit-externaldevice-usecase) * [Externaldevice Settings](https://legacy.docs.aviatrix.com/documentation/latest/network/externaldevice-settings.html) # Aviatrix Gateway to FortiGate Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/vpn/s2c-aviatrixgw-to-fortigate This document describes how to configure an IPsec tunnel between an Aviatrix Gateway and a FortiGate firewall using Aviatrix Site2Cloud. This task is divided into two parts: This document describes how to configure an IPsec tunnel between an Aviatrix Gateway and a FortiGate firewall using Aviatrix Site2Cloud. This task is divided into two parts: 1. Configure a [Site2Cloud tunnel](#setting-up-external-s2c-connection) in Aviatrix CoPilot. 2. Configure a [VPN tunnel](#fortigate-configuration) and related components in the FortiGate Firewall. ## Setting up External (S2C) Connection 1. In Aviatrix CoPilot, launch an [Aviatrix Transit Gateway](/docs/enterprise/10.1/reference/ui/cloud-fabric/gateways#view-transit-gateways) . 2. Navigate to Networking > Connectivity > External Connections (S2C) and click *Add New* to create a Site2Cloud connection using the values for one of the below options (for either you can select either PSK or certificate-based authentication). * [Static Route-Based](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-create-static-route-based-unmapped.html) * [Static Policy-Based](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-create-static-policy-based-unmapped.html) 3. After the connection is created, select the vertical ellipsis ![Three dots menu](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/threedot.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=36963cd7f65aa469c6d5fe2021fa71a7) menu for that connection and select *Download Configuration*. 4. Select *Generic* from the *Vendor* dropdown list and click the *Download* to download the external (S2C) configuration. Use this configuration file to configure the tunnels and interfaces in your Fortinet FortiGate firewall. ## FortiGate Configuration The configuration and screenshots below make the following three assumptions: * There are two interfaces on the FortiGate: * Interface port1 is an externally facing interface. * Interface port2 is an internally facing interface. * You have a subnet in AWS, Azure, or GCP in a VPC/VNet that has an Aviatrix Gateway. This subnet is defined as "10.0.0.0/16" for the examples below but it can be any valid CIDR range. \[!NOTE] In the examples below this range is referred to as *AWS\_Cloud*. * You have a subnet behind your FortiGate firewall that will be accessible in the cloud. This subnet is defined as "172.16.0.0/20" in the examples below but it can be any valid CIDR range. > [!NOTE] > > In the examples below, this range is referred to as *Shared\_With\_AWS*. ### Configuring Named Address Ranges in FortiGate Access the FortiGate Dashboard. Under *Policy & Objects* > *Addresses*, create two new addresses: AWS\_Cloud and Shared\_With\_AWS. #### AWS\_Cloud | Field | Expected Value | | - | - | | Name | AWS\_Cloud | | Type | Subnet | | Subnet / IP Range | CIDR matching the range specified in tunnel configuration (remote to FortiGate) | | Interface | Any | | Show in Address List | Enabled | | Static Route Configuration | Enabled | ![AWS Cloud Configuration](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/aws-cloud-config.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=6f5889247ea063fc3e711f49311b05ad) #### Shared\_With\_AWS | Field | Expected Value | | - | - | | Name | Shared\_With\_AWS | | Type | Subnet | | Subnet / IP Range | CIDR matching the range specified in tunnel configuration (local to FortiGate) | | Interface | Any | | Show in Address List | Enabled | | Static Route Configuration | Enabled | ![Shared with AWS Configuration](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/shared-with-aws-config.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=00e22cc477d2df3d4a1f6d86abf70c31) ### Creating an IPsec Tunnel on FortiGate 1. Log in to the FortiGate and access the Dashboard. 2. In the VPN menu, select *IPsec Wizard*. 3. Change the Template Type to "Custom." 4. Enter any value as the Name. For this example, we are using "aviatrix-gatew." 5. Click *Next >*. 6. Fill out the Network fields as recommended below: | Field | Expected Value | | - | - | | Name | aviatrix-gatew (for example) | | Template Type | Custom | ![FortiGate Configuration](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/fortigate5.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=312ea4ebb33b9f492d25065d7e7d518a) ### New VPN Tunnel Tab Complete the Network fields on the New VPN Tunnel tab as follows: *Network section of New VPN Tunnel Tab* | Field | Expected Value | | - | - | | IP Version | IPv4 | | Remote Gateway | Static IP Address | | IP Address | Public IP address of Aviatrix Gateway | | Interface | Select the Appropriate Port/Interface | | Local Gateway | Disabled | | Mode Config | Unmark this checkbox | | NAT Traversal | Enable | | Keepalive Frequency | Any value | | Dead Peer Detection | On Demand | | Forward Error Correction | Unmark this checkbox | | Advanced Options | Disabled | ![FortiGate Network Configuration](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/fortigate6.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=dd2bbde1afe586bd54e2c9c1684f4816) *Authentication section of New VPN Tunnel Tab* | Field | Expected Value | | - | - | | Method | Pre-shared Key | | Pre-shared Key | Enter the value from the downloaded configuration or the value typed in to the field in Aviatrix Site2Cloud | | IKE Version | 1 | | IKE Mode | Main (ID protection) | ![Authentication Configuration](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/authentication.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=83f5f309bf3545aa2394cb96e61ac5d2) *Phase 1 Proposal section of New VPN Tunnel Tab* | Field | Expected Value | | - | - | | Encryption | Match value specified in Aviatrix S2C configuration (Phase 1 Encryption) | | Authentication | Match value specified in Aviatrix S2C configuration (Phase 1 Authentication) | | Diffie-Hellman Group | Match value specified in Aviatrix S2C configuration (Phase 1 DH Groups) | | Key Lifetime (seconds) | 28800 | | Local ID | Leave Blank | ![Phase 1 Proposal](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/phase1proposal.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=8421336736fc1270a30c4bea2e52500f) *XAUTH section of New VPN Tunnel Tab* | Field | Expected Value | | - | - | | Type | Disabled | ![XAUTH Configuration](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/connectivity/vpn/xauth.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=9034a4a0951cfad0b21015686bbdb805) *Phase 2 Selectors > New Phase 2 seciton of New VPN Tunnel Tab* | Field | Expected Value | | - | - | | Name | Any String Value | | Comments | Any String Value | | Local Address | Named Address - *Shared\_With\_AWS* | | Remote Address | Named Address - *AWS\_Cloud* | ![Phase 2 Selector](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/phase2selector.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=90bad2f576de5fae1fe85334de745262) *Advanced section of New VPN Tunnel Tab* > [!NOTE] > > Obtain the values from the downloaded configuration file. | Field | Expected Value | | - | - | | Encryption | Match value specified in Aviatrix S2C configuration (Phase 2 Encryption) | | Authentication | Match value specified in Aviatrix S2C configuration (Phase 2 Authentication) | | Diffie-Hellman Group | Match value specified in Aviatrix S2C configuration (Phase 2 DH Groups) | | Key Lifetime (seconds) | 3600 | ![Phase 2 Advanced](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/phase2advanced.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=f08161839d399af0138748928b682bbc) 7. Click *OK*. 8. Navigate to Network > Interfaces. 9. Click on the Tunnel created above (e.g. aviatrix-gatew) and assign the IP address from the downloaded configuration file. ![FortiGate Interface Configuration](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/fortigate14.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=0661ab1c8b47b2cc1a9ac6b19e18a609) ### Configure IPv4 Policy 1. Go to Policy & Objects > IPv4 DoS Policy. 2. Create two new IPv4 policies: * Outbound traffic from FortiGate (Shared\_With\_AWS) to Aviatrix (AWS\_Cloud) ![IPv4 Outbound Policy](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/ip4outboundpolicy.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=bf9a8f00d724fb8b08219e0fac4c6e9e) * Inbound traffic from Aviatrix (AWS\_Cloud) to FortiGate (Shared\_With\_AWS) ![IPv4 Inbound Policy](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/ip4inboundpolicy.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=0f8465e9b2a18c1c646ccd2aa34a905b) > [!NOTE] > > The reference to port2 in the screenshots should be replaced with your own > interface name that represents the internal facing interface. > [!NOTE] > > Be sure to select *ACCEPT* for "action" and select *ALL* for "service." ### Adding a Static Route In the FortiGate UI, go to Network > Static Routes and add a new static route for traffic destined to "AWS\_Cloud" to use the VPN tunnel. ![Static Route Configuration](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/static-route-fortigate.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=ac7ee1178f32effe18cde40d2a9be362) > [!NOTE] > > If Named Address is disabled, be sure that you enabled Static Route > Configuration on the Address configuration. > > ![Address Static Configuration](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/address-static-config.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=56851b63650d33421c7662a0dc1bd0bb) ### IPsec Monitor 1. In the Fortigate UI, navigate to Dashboard > Network and click the IPsec widget. 2. Select the Aviatrix tunnel, and click *Bring Up*. 3. You can then check the tunnel status in CoPilot under Diagnostics > Cloud Routes. ## Troubleshoot *Error Message* `failed to get valid proposal` `no suitable proposal found` *Solution* Check that the Phase 1 authentication, encryption, and Diffie-Hellman groups match on both sides. If you are experiencing low IPsec throughput, you may want to configure two commands on the Fortigate. ```cli theme={null} config system global set ipsec-asic-offload disable end configure system global set ipsec-hmac-offload disable end ``` ## Related Topics * [About External Connection Settings](/docs/enterprise/10.1/guides/connectivity/gateways/external-connection-settings) # Aviatrix Gateway to Juniper SRX Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/vpn/s2c-aviatrixgw-to-juniper-srx Aviatrix does not officially support Juniper SRX, but you can configure Juniper SRX to work with the Aviatrix software. Aviatrix does not actively test our software with Juniper SRX and you should rigorously test your configuration in a development environment before deploying in a production... > [!NOTE] > > Aviatrix does not officially support Juniper SRX, but you can configure > Juniper SRX to work with the Aviatrix software. Aviatrix does not actively > test our software with Juniper SRX and you should rigorously test your > configuration in a development environment before deploying in a production > system. This document describes how to build an IPsec tunnel-based Site2Cloud connection between an Aviatrix Gateway and a JuniperSRX Firewall. The network setup is as follows: **VPC/VNet-multicloudvpc1 (with Aviatrix Gateway)** *VPC/VNet CIDR: 10.1.1.0/16* *VPC/VNet Subnet CIDR (public in AWS, GCP, or OCI): 10.1.1.0/24* *VPC/VNet Private Subnet CIDR: 10.1.2.0/24* **On-Prem (with Juniper SRX Firewall)** *On-Prem Network CIDR: 10.0.0.0/16* *On-prem Public Network CIDR: 10.0.3.0/24* *On-prem Private Network CIDR: 10.0.2.0/24* ## Creating an External (Site2Cloud) Connection 1. In Aviatrix CoPilot, launch an [Aviatrix Transit Gateway](https://legacy.docs.aviatrix.com/documentation/latest/network/transit-gateway-about.html?expand=true) at the subnet of VPC/VNet-multicloudvpc1 (public subnet for AWS, GCP, or OCI). Collect the Gateway's public IP addresses (3.213.233.93 in this example). 2. Navigate to Networking > Connectivity > External Connections (S2C) and click **Add New** to create a Site2Cloud connection using the values for one of the below options (for either you can select either PSK or certificate-based authentication). * [Static Route-Based](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-create-static-route-based-unmapped.html) * [Static Policy-Based](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-create-static-policy-based-unmapped.html) Substitute the following values: * Remote Device Type: Generic * Algorithms: turn Off * Remote Device IP: Public IP of Juniper SRX WAN port (18.214.241.32 in this example) * Remote Subnet CIDR(s): 10.0.2.0/16 (On-Prem Private Network CIDR) * Local Subnet CIDR(s): 10.1.2.0/24 (VPC-multicloudvpc1 private subnet) 3. After the connection is created, select the vertical ellipsis ![25](https://mintcdn.com/aviatrix-14b37c43/mOZ0lmP7pRPIryx3/images/guides/platform-administration/threedot.png?fit=max\&auto=format\&n=mOZ0lmP7pRPIryx3\&q=85\&s=6ed1c085429177f16d2735e2406d15c3) menu for that connection and select **Download Configuration**. 4. Select **Generic** from the **Vendor** dropdown list and click the **Download** to download the external (S2C) configuration. Use this configuration file to configure the tunnels and interfaces in your Juniper SRX firewall. The following is an SRX sample configuration based on the Site2Cloud configuration above. ![image3](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/juniper-s2c3.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=3394caecba5d47f0f01047d11a8cf04b) ### Configuring JuniperSRX Apply the following configuration to your SRX: [https://s3-us-west-2.amazonaws.com/aviatrix-download/docs/srx\_site2cloud.txt](https://s3-us-west-2.amazonaws.com/aviatrix-download/docs/srx_site2cloud.txt) ### Verifying the Tunnel Status In Aviatrix CoPilot, go to the Diagnostics > Cloud Routes > External Connections tab to confirm the Status and Tunnel Status of the external connection. ### Troubleshooting To troubleshoot, in CoPilot go to Diagnostics > Diagnostic Tools. ## Related Topics * [About External Connection Settings](https://legacy.docs.aviatrix.com/documentation/latest/network/externaldevice-settings.html) * [Aviatrix Gateway to FortiGate](/docs/enterprise/10.1/guides/connectivity/vpn/s2c-aviatrixgw-to-fortigate) * [Site2Cloud Solution for Encryption over Direct Connect/ExpressRoute](https://legacy.docs.aviatrix.com/documentation/latest/network/s2c-case-study.html) # Aviatrix Integration with Microsoft's SSE Solution Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/vpn/s2c-aviatrixgw-to-microsoft-entra This document provides instructions on how to build an external connection from an Aviatrix Gateway to Microsoft's Security Service Edge (SSE) Solution. This document provides instructions on how to build an external connection from an Aviatrix Gateway to Microsoft's Security Service Edge (SSE) Solution. The Aviatrix Platform allows Microsoft's SSE Solution integration for remote sites and hybrid-cloud connectivity for secure access to cloud resources. ## What is Microsoft's SSE Solution? Microsoft's SSE solution is an identity and access management solution that allows organizations to implement Zero Trust internet security. Network traffic from remote sites, such as on-premises, datacenters, colocations, branch offices, and hybrid cloud environments is forwarded to Microsoft's SSE Solution cloud proxy for authentication and secure access to cloud resources. To learn more about Microsoft's SSE Solution, refer to [Microsoft documentation](https://learn.microsoft.com/en-us/entra/global-secure-access/overview-what-is-global-secure-access). ## What is Aviatrix integration with Microsoft's SSE Solution? Aviatrix integration with Microsoft's SSE Solution provides a holistic solution that encompass *data security* and *secure access* for connecting on-premises and hybrid cloud workloads to cloud resources. In this solution, Aviatrix Gateways provide the data encryption and High Performance Encryption (HPE) for high-bandwidth connections for *data security* from remote sites to cloud. Microsoft's SSE Solution allows to apply network security policies for *secure access* to the cloud resources. Aviatrix Gateways deployed on VPCs or VNets and connected to remote sites, for example datacenters, branch offices, or customer and partner sites, forwards network traffic from the remote sites to Microsoft's SSE Solution for network security inspections. Key benefits of Aviatrix integration with Microsoft Entra include: * **Procurement via Azure Marketplace**: Aviatrix is fully procurable via the Azure Marketplace. This provides simplified procurement experience for customers. * **Cloud service provider intergration**: Aviatrix has tight integration with cloud service providers for automated deployment and lifecycle management for simplified user experience. * **High Performance Encryption**: Aviatrix has a unique patented technology that allows high-bandwidth connections for customers connecting on-premises datacenters, large branches, or other cloud environments to Microsoft Entra SSE. * **Multiple form factors**: Aviatrix provides physical and virtual form-factors for deploying Aviatrix Gateways in on-premises locations. * **Automated deployments**: Aviatrix is an expert in Infrastructure as Code for accelerating and automating deployments. ### How does Microsoft's SSE Solution integration work? The Aviatrix Gateways run a BGP session over the internet to Microsoft's SSE Solution gateway to dynamically exchange routes and establish IPsec tunnels for forwarding the network traffic. ### What Aviatrix Gateways are Supported? The following Aviatrix Gateways support connectivity to Microsoft's SSE Solution. * **Transit Gateways** * **Spoke Gateways** ## Deployment Models You can set up centralized or distributed connectivity from Aviatrix Gateways to Microsoft's SSE solution. ### Centralized SSE Hub A centralized SSE hub is a hub-and-spoke architecture where remote sites and cloud workloads (spokes) are connected to Microsoft's SSE Solution via the Aviatrix Transit Gateway (hub). The Transit Gateway forwards network traffic destined for the cloud to Microsoft's SSE Solution for security inspections. Transit Gateways can be deployed across multiple clouds and cloud regions where Microsoft's SSE Solution access is required. A key advantage to a centralized hub architecture is that it allows you to connect new remote sites or cloud workloads and leverage the same connectivity from the Transit Gateway to Microsoft's SSE Solution for traffic forwarding without the need to configure additional IPsec tunnels. Transit Gateway supports redundant active-active peering connections to Microsoft's SSE Solution for high-availability. All network traffic is forwarded across both the primary and secondary HA active peering to Microsoft's SSE Solution. Transit Gateway provides high performance encrypted connectivity to Microsoft's SSE Solution across remote sites and clouds for high-bandwidth and higher aggregated throughput for large sites. ![microsoft-entra-centralized](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/microsoft-entra-centralized.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=edef11e06d61228d7f80cf806030b9ed) ### Distributed SSE Connectivity A distributed SSE architecture is where network traffic from the cloud and remote sites are forwarded to Microsoft's SSE Solution via the Aviatrix Spoke Gateway. Each Spoke Gateway establishes connectivity with Microsoft's SSE Solution.Spoke Gateways can be deployed across multiple clouds and cloud regions where Microsoft's SSE Solution access is required. In a distributed SSE architecture, to connect new remote sites or cloud workloads to Microsoft's SSE Solution, you need to configure new IPsec tunnels for traffic forwarding. ![microsoft-entra-distributed](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/microsoft-entra-distributed.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=1ee1ec7d1a4d8578f52a5840eac9165a) ## Connecting an Aviatrix Gateway to Microsoft's SSE Solution The following are the steps for connecting an Aviatrix Gateway to Microsoft's SSE Solution. ### Prerequisites Before you set up Aviatrix connectivity to Microsoft Entra SSE remote network, ensure the following prerequisites are met: 1. Ensure that you are running the latest version of Aviatrix CoPilot and Controller. Aviatrix integration with Microsoft's SSE Solution is supported with Aviatrix CoPilot 4.15 and Controller version 7.2.4820 and above. 2. Ensure the Aviatrix Gateway is deployed with BGP enabled. ### Configuring BGP over IPsec External Connection to Microsoft's SSE Solution The following are the high-level steps for building BGP over IPsec tunnel connection between an Aviatrix Gateway and Microsoft's SSE Solution. When building an IPsec tunnel, you configure both sides of the tunnel. 1. Configure one side of tunnel from the Aviatrix Gateway by [creating an external BGP over IPsec connection to Microsoft's SSE Solution](#step-1-configure-bgp-over-ipsec-tunnel-on-the-aviatrix-gateway). 2. Configure the other side of the tunnel [by creating a remote network in Microsoft's SSE Solution and adding a device link to the Aviatrix Gateway](#step-2-configure-bgp-over-ipsec-tunnel-on-microsofts-sse-solution). 3. [Update the Aviatrix Gateway tunnel configuration](#step-3-update-bgp-remote-configurations-on-the-aviatrix-gateway) with the remote IP address for Microsoft's SSE Solution. 4. [Verify the BGP session is up](#step-4-verify-the-bgp-session-status). #### Step 1: Configure BGP over IPsec tunnel on the Aviatrix Gateway Configure an IPsec tunnel on the Aviatrix Gateway by creating an external BGP over IPsec connection to Microsoft's SSE Solution. 1. In Aviatrix CoPilot, go to **Networking > Connectivity > External Connections (S2C)** tab. 2. From the **+ External Connection To** dropdown menu, select **Microsoft's SSE Solution**. 3. In Create External Connection to Microsoft's SSE Solution, provide the following information. | **Field** | **Description** | | - | - | | Name | A name to identify the connection. | | Local Gateway | The Aviatrix Gateway to connect to Microsoft's SSE Solution. | CoPilot shows the attributes of the gateway. Use these attributes to [create a remote network in Microsoft's SSE Solution and add a device link to the Aviatrix Gateway](#step-2-configure-bgp-over-ipsec-tunnel-on-microsofts-sse-solution). #### Step 2: Configure BGP over IPsec tunnel on Microsoft's SSE Solution In Microsoft Entra admin center, configure the other side of the tunnel by creating a remote network in Microsoft's SSE Solution and adding a device link to the Aviatrix Gateway. 1. Click **Launch Microsoft's SSE Solution** to launch Microsoft Entra admin center. 2. Use the attributes below (from [Step 1](#step-1-configure-bgp-over-ipsec-tunnel-on-the-aviatrix-gateway)) to create the Microsoft Entra remote network and add a device link to the Aviatrix Gateway. Refer to the [Microsoft documentation](https://learn.microsoft.com/en-us/entra/global-secure-access/how-to-create-remote-networks) for more information on how to create the Microsoft Entra remote network connection. | **Field** | **Description** | | - | - | | Region | The cloud region where the Aviatrix Gateway is deployed. | | Device IP Address | The Public IP address of the Aviatrix Gateway. | | Device BGP Address | The Private IP address of the Aviatrx Gateway. | | Device ASN | The BGP AS number assigned to the Aviatrix Gateway. | | Tunnel Redundancy | **Zone**: Indicates a secondary HA gateway is configured. CoPilot version 4.15 does not support HA gateway connection to Microsoft's SSE Solution. **None**: Indicates a secondary HA gateway is not configured. | #### Step 3: Update BGP Remote Configurations on the Aviatrix Gateway In Aviatrix CoPilot, update the Aviatrix Gateway tunnel configuration with the remote Public IP address and BGP AS Number of Microsoft's SSE Solution remote network. 1. In **IPsec Configuration**, provide the following information. | **Field** | **Description** | | - | - | | Algorithms | The encryption algorithm and protocol to use for authenticating the communication between the Aviatrix Gateway and Microsoft Entra remote network. **Default**: The default Aviatrix-supported encryption algorithms. **Custom**: Allows you to modify the default values. See [Algorithms](https://legacy.docs.aviatrix.com/documentation/latest/network/externaldevice-settings.html#algorithms). | | IKEv2 | The default protocol used for authentication and encryption of packets between the Aviatrix Gateway and Microsoft's SSE Solution. This setting can not be changed. See [IKEv2](https://legacy.docs.aviatrix.com/documentation/latest/network/externaldevice-settings.html#algorithms). | 2. In **Tunnel Configuration**, provide the following details of the remote network connectivity that you configured in Microsoft Entra admin center. > [!TIP] > > You can view the remote network configuration details in Microsoft Entra admin center by going to **Global Secure Access > Connect > Remote Network**. Locate the remote network and click **View configuration**. The following is an example of a remote network connection details in Microsoft Entra admin center. ![microsoft-entra-connection-details](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/microsoft-entra-connection-details.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=f410acc6d4d20a945bc27b68f38f5deb) > [!NOTE] > > CoPilot version 4.15 does not support ActiveMesh connections to Microsoft's > SSE Solution gateway. | **Field** | **Description** | | - | - | | Remote Device IP | The Public IP address of Microsoft's SSE Solution gateway. | | Remote ASN | The BGP AS Number that Microsoft's SSE Solution gateway will use to exchange routes with Aviatrix Gateway. | | Local Tunnel IP | The Public IP address of the Aviatrix Gateway, in the format *local\_tunnel\_ip/netmask* (for example, 10.2.2.2/30). | | Remote Tunnel IP | The BGP address of Microsoft's SSE Solution gateway, in the format *remote\_tunnel\_ip/netmask* (for example, 10.2.2.1/30). | | Pre-Shared Key | The pre-shared key you specified when adding the device link to the Aviatrix Gateway in Micosoft Entra admin center. The pre-shared key must be the same for the Aviatrix Gateway and Microsoft's SSE Solution. | 3. Click **Save**. #### Step 4: Verify the BGP Session Status To verify the connection between the Spoke Gateway and Microsoft Entra SSE remote network is established: 1. In CoPilot, go to **Diagnostics > Cloud Routes > External Connections** tab. 2. In the table, locate the connection that you created. The **Status** should be Established. If the connection is not established, you can run [diagnostics](/docs/enterprise/10.1/reference/ui/diagnostics/diagnostic-tools#run-connectivity-analysis) to perform an analysis or review logs. # Aviatrix Gateway to Oracle Drg Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/vpn/s2c-aviatrixgw-to-oracle-drg This document describes how to configure an IPsec tunnel between an Aviatrix Gateway and an Oracle Dynamic Routing Gateway (DRG). This document describes how to configure an IPsec tunnel between an Aviatrix Gateway and an Oracle Dynamic Routing Gateway (DRG). ![gw2drg](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/gw2drg.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=e09376d6914f5300e7da1d670a48968c) ## Deployment For this use case, you will create an IPsec connection from DRG first and then configure a Site2Cloud connection in Aviatrix CoPilot. ### Creating an IPsec Connection from DRG > [!NOTE] > > **Prerequisites** > > 1. You have a DRG created and attached to a VCN. > 2. You have an Aviatrix Gateway provisioned in a VPC/VNet. You will need this > gateway's public IP address and its VPC/VNet CIDR for the steps below. 3. Log in to your Oracle Cloud Console and create a route rule for the DRG. You need to modify the desired route table and create a route rule to take any traffic destined for the Aviatrix Gateway's VPC/VNet CIDR and route it to the DRG. 1. Under *Core Infrastructure*, go to *Networking* and click *Virtual Cloud Networks*. 2. Click your VCN. 3. Select the desired route table(s) for your VCN. 4. Click *Edit Route Rules*. 5. Create a new route rule as follows and save it. | Field | Description | | - | - | | Target Type | Dynamic Route Gateway | | Destination CIDR Block | Aviatrix GW's VPC/VNet CIDR (172.19.0.0/16 in this example) | | Target Dynamic Routing Gateway | Select the desired existing DRG | ![vcn\_route\_table](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/vcn-route-table.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=abbd0b6858dd6b5e884a27be0ee1cc60) 4. Log in to your Oracle Cloud Console and create security rules. Edit the security list associated with your VCN subnets by adding two new rules: one ingress rule for traffic coming from the Aviatrix Gateway's VPC/VNet and one egress rule for traffic going to the Aviatrix Gateway's VPC/VNet. 1. Under *Core Infrastructure*, go to *Networking* and click *Virtual Cloud Networks*. 2. Click your VCN. 3. Select the desired security list(s) associated with your subnets. 4. Click *Edit All Rules*. 5. In *Allowed Rule for Ingress* section, enter the following values to create a rule to allow incoming traffic from Aviatrix Gateway's VPC/VNet. | Field | Description | | - | - | | Source Type | CIDR | | Source CIDR | Aviatrix GW's VPC/VNet CIDR (172.19.0.0/16 in this example) | | IP Protocols | All Protocols | ![vcn-security-rule-ingress](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/vcn-security-rule-ingress.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=77be928d0203f7eb924e3fe9f8b38426) 6. In *Allowed Rule for Egress* section, enter the following values to create a rule to allow outgoing traffic to the Aviatrix Gateway's VPC/VNet. | Field | Description | | - | - | | Destination Type | CIDR | | Destination CIDR | Aviatrix GW's VPC/VNet CIDR (172.19.0.0/16 in this example) | | IP Protocols | All Protocols | ![vcn-security-rule-egress](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/vcn-security-rule-egress.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=47ff287cf6b25155ff6946d1d656aac0) 5. Create a CPE object. In this task, you create the CPE object, which is a logical representation of the Aviatrix Gateway. 1. Under *Core Infrastructure*, go to *Networking* and click *Customer-Premises Equipment*. 2. Click *Create Customer-Premises Equipment*. 3. Enter the following values and click *Create*. | Field | Description | | - | - | | Create in Compartment | Leave as is (the VCN's compartment) | | Name | A descriptive name for the CPE object | | IP Address | Public IP address of Aviatrix Gateway | | Tags | Optional | ![cpe](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/cpe.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=cae3ca5f0a2b43964aedb1d345665c8a) 6. From the DRG, create an IPsec connection to the CPE object. 1. Under *Core Infrastructure*, go to *Networking* and click *Dynamic Routing Gateways*. 2. Click the DRG created earlier. 3. Click *Create IPsec Connection*. 4. Enter the following values and click *Create IPsec Connection*. | Field | Description | | - | - | | Create in Compartment | Leave as is (the VCN's compartment) | | Name | A descriptive name for the IPsec connection | | Customer-Premises Equipment Compartment | Leave as is (the VCN's compartment) | | Customer-Premises Equipment | Select the CPE object created earlier | | Static Route CIDR | Aviatrix GW's VPC/VNet CIDR (172.19.0.0/16 in this example) | | Tags | Optional | ![ipsec-connection](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/ipsec-connection.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=3052c4ce1ea5902be1d7451457054eb8) 5. Once the IPsec connection enters the *Available* state, click the *Action* icon (three dots), and then click *Tunnel Information*. Copy the *IP Address* of the VPN headend and the *Shared Secret* for configuring an external (S2C) connection in Aviatrix CoPilot. ![ipsec-info](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/ipsec-info.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=4bc95a3ecb3d86fcb40894316f0823dd) 7. Log into Aviatrix CoPilot. 8. Create a [Transit Gateway](https://legacy.docs.aviatrix.com/documentation/latest/network/transit-gateway-about.html?expand=true). If you want to add HA to the Transit Gateway, you must [do it when you initially create the Transit Gateway](https://legacy.docs.aviatrix.com/documentation/latest/network/transit-gateway-about.html?expand=true#ha-transit-gateway). You must also [follow the steps here](#creating-a-second-ipsec-connection-between-the-same-drg-and-aviatrix-ha-gateway-optional) to add a new IPsec connection in Oracle. 7. Follow the steps in [Site2Cloud Configuration Workflow](/docs/enterprise/10.1/guides/connectivity/vpn/s2c-configuration-workflow). 8. Use one of these tables for specific field values (Unmapped): * [Static Policy-Based (Unmapped) External Connection Using Terraform](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-create-static-policy-based-unmapped.html) * [Static Route-Based External Connection (Unmapped)](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-create-static-route-based-unmapped.html) 9. Replace the following values as noted: * Remote Device Type: Oracle * Remote Subnet CIDR(s): Oracle VCN CIDR (10.1.1.0/24 in this example). * Remote Device IP: IP address copied from the Oracle IPsec connection. * Pre-Shared Key: Shared Secret copied from the Oracle IPsec connection. ## Test Once complete, test the communication using the tunnel by sending traffic between instances in the Aviatrix Gateway's VPC/VNet and Oracle VCN. 1. In CoPilot, go to Diagnostics > Cloud Routes > External Connections. 2. Verify that the external (S2C) connection Tunnel Status is Up. ## Troubleshoot Wait 2-3 minutes for the tunnel to come up. If it does not come up within that time, check the IP addresses to confirm they are accurate. Additional troubleshooting is available from Diagnostics > Diagnostic Tools. ## Creating a Second IPsec Connection Between the Same DRG and Aviatrix HA Gateway (Optional) From the Oracle Cloud console, create a second IPsec connection between the same DRG and the Aviatrix HA Gateway created [here](#deployment). 1. Create a new CPE at Oracle Cloud Console for the Aviatrix HA Gateway: | Field | Description | | - | - | | Create in Compartment | Leave as is (the VCN's compartment) | | Name | A descriptive name for the second CPE object | | IP Address | Public IP address of Aviatrix HA Gateway | | Tags | Optional | 2. Create a new IPsec connection at Oracle Cloud Console for the Aviatrix HA Gateway: | Field | Description | | - | - | | Create in Compartment | Leave as is (the VCN's compartment) | | Name | A descriptive name for the second IPsec connection | | Customer-Premises Equipment Compartment | Leave as is (the VCN's compartment) | | Customer-Premises Equipment | Select the second CPE object created earlier | | Static Route CIDR | Aviatrix GW's VPC/VNet CIDR (172.19.0.0/16 in this example) | | Tags | Optional | 3. Once the second IPsec connection enters the *Available* state, click the *Action* icon (three dots), and then click *Tunnel Information*. Please copy the *IP Address* of the VPN headend and the *Shared Secret*. ## Related Topics * [About External Connection Settings](https://legacy.docs.aviatrix.com/documentation/latest/network/externaldevice-settings.html) # Aviatrix Gateway to Palo Alto Firewall Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/vpn/s2c-aviatrixgw-to-paloalto-firewall This document describes how to build an IPsec tunnel-based Site2Cloud connection between an Aviatrix Gateway and a Palo Alto Networks Firewall. To simulate an on-prem Firewall, we use a VM-Series in an AWS VPC. This document describes how to build an IPsec tunnel-based Site2Cloud connection between an Aviatrix Gateway and a Palo Alto Networks Firewall. To simulate an on-prem Firewall, we use a VM-Series in an AWS VPC. > [!NOTE] > > If you do not have access to AWS, you can simulate an on-prem Firewall > by deploying the Palo Alto Firewall in any other cloud (such as > Microsoft Azure, Google Cloud Platform, or Oracle Cloud Infrastructure). The network setup is as follows: **VPC1 (with Aviatrix Gateway)** *VPC1 CIDR: 10.0.0.0/16* *VPC1 Public Subnet CIDR: 10.0.1.0/24* *VPC1 Private Subnet CIDR: 10.0.2.0/24* **VPC2 (with Palo Alto Networks VM-series)** *VPC2 CIDR: 10.13.0.0/16* *VPC2 Public Subnet CIDR: 10.13.0.0/24* *VPC2 Private Subnet CIDR: 10.13.1.0/24* ## Certificate-Based Authentication If you want to use certificate-based authentication when establishing a Site2Cloud connection with your Palo Alto VM-Series firewall, you must do the following: 1. Generate a certificate from your Palo Alto VM-Series firewall. 2. Export your certificate in PEM format. 3. In Aviatrix CoPilot, upload the CA certificate generated from your Palo Alto VM-Series firewall under Networking > Connectivity > Certificates. 4. Create your [Site2Cloud connection](/docs/enterprise/10.1/guides/connectivity/vpn/s2c-configuration-workflow). 5. After creating the Site2Cloud connection, download the resulting configuration. 6. Download the Aviatrix CA certificate from Networking > Connectivity > Certificates. 7. Upload the Aviatrix CA certificate to your on-prem Palo Alto VM-Series firewall. 8. In the Palo Alto VM-Series UI, use the information from the downloaded configuration file to configure your tunnels/interfaces. 9. In the Palo Alto VM-Series UI, configure the IKE Gateway depending on if you are using PSK or certificate-based authentication. ## Configuration Workflow If you are not using certificate-based authentication in your Site2Cloud connection with your Palo Alto firewall you can skip these sections: * Creating and Generating a Self-Signed Root Certificate * Importing the Aviatrix CA Certificate ### Creating and Generating a Self-Signed Root Certificate If you are creating a Site2Cloud connection between your Palo Alto VM-Series firewall and your Aviatrix gateway in the cloud and want to use certificate-based authentication, you must generate a CA certificate in the firewall UI that you will upload to Aviatrix CoPilot under Networking > Connectivity > Certificates. See [Create a Self-Signed Root CA Certificate](https://docs.paloaltonetworks.com/pan-os/9-1/pan-os-admin/certificate-management/obtain-certificates/create-a-self-signed-root-ca-certificate) for more information on certificates in Palo Alto. 1. In the Palo Alto VM-Series firewall UI, navigate to Device > Certificate Management > Certificates > Device Certificates. 2. Click **Generate** at the bottom of the window. The Generate Certificate dialog displays. ![generate cert](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/generate-cert.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=f393bbcebbce5ccb5d255d6960f010cc) 3. Fill out the following information: * Certificate Name: a name that makes sense to you, such as PAN-CA * Common Name: a descriptor that makes sense to you, such as pan-to-avx.com * Signed By: select Certificate Authority * Cryptographic Settings: * Select Elliptic Curve DSA (ECDSA); this is currently the only algorithm supported by Site2Cloud cert-based authentication. * Select 256 bits * Select sha256 digest 4. Click **Generate**. This creates the root CA certificate that will be used to sign the PAN to Aviatrix certificate you will create in the next step. 5. Under Device > Certificate Management > Certificates > Device Certificates, generate another certificate (signed by the PAN-CA root you created) and populate as follows: * Certificate Name: a name that makes sense to you, such as pan-to-avx-cert * Common Name: a name that makes sense to you, such as pan-device.com * Signed by: PAN-CA (created in above steps) * Cryptographic Settings: Elliptic Curve DSA algorithm; 256 bits; sha256 digest * Certificate Attributes: refer to the aforementioned Palo Alto Networks URL for information on attributes to use for device certificate creation 6. Click **Generate**. 7. Export the PAN-CA certificate for uploading to the Networking > Connectivity > Certificates page in CoPilot. 8. See the CA Certificate page for details on uploading this certificate. ### Setting up External (S2C) Connection 1. Launch a Palo Alto Networks VM-series with at least two network interfaces. One interface serves as a WAN port and is in VPC2's public subnet. The other interface serves as a LAN port and is in VPC2's private subnet. Collect the public IP address of the WAN port. 2. In Aviatrix CoPilot, launch an [Aviatrix Transit Gateway](https://legacy.docs.aviatrix.com/documentation/latest/network/transit-gateway-about.html?expand=true) in VPC1's public subnet. Collect both the public and private IP address of the Gateway. 3. Create a Site2Cloud connection using one of the below options (for PSK or certificate-based authentication). * [Static Route-Based](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-create-static-route-based-unmapped.html) * [Static Policy-Based](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-create-static-policy-based-unmapped.html) Substitute the following values: * Remote Device Type: Generic * Algorithms: turn Off * Remote Device IP: Public IP of Palo Alto Networks VM Series WAN port * Remote Subnet CIDR(s): 10.13.1.0/24 (VPC2 private subnet) * Local Subnet CIDR(s): 10.0.2.0/24 (VPC1 private subnet) 4. After the connection is created, select the vertical ellipsis ![25](https://mintcdn.com/aviatrix-14b37c43/VY6W0iTAMUQkrobS/images/security/threedot.png?fit=max\&auto=format\&n=VY6W0iTAMUQkrobS\&q=85\&s=9498dd71f99faa81563f9d3f0baf4463) menu for that connection and select **Download Configuration**. 5. Select **Generic** from the **Vendor** dropdown list and click the **Download** to download the external (S2C) configuration. Use this configuration file to configure the tunnels and interfaces in your Palo Alto Network VM-Series firewall. ### Importing the Aviatrix CA Certificate If you are creating an external (S2C) connection between your Palo Alto VM-Series firewall and your Aviatrix gateway, you must download the Aviatrix CA certificate as per the instructions on the CA Certificate page, and then upload it to your Palo Alto VM-Series UI as follows: 1. In the Palo Alto VM-Series UI, navigate to Device > Certificate Management > Certificates > Device Certificates. 2. At the bottom of the window, click **Import**. 3. In the Import Certificate dialog, enter the following information: * Certificate Name: a name that makes sense to you * Certificate File: click Browse to navigate to the location of the Aviatrix CA certificate * File Format: select Base64 Encoded Certificate (PEM). 4. Click **OK**. 5. Navigate to Device > Certificate Management > Certificate Profile. In the Certificate Profile dialog enter the following: * Name: enter a name for the profile (such as AVX-CA). * CA Certificates: click **Add** and select AVX-CERT (or whatever name you gave to the imported Aviatrix CA certificate) from the CA Certificate list. 6. Click **OK**. 7. Click **OK** again on the main Certificate Profile dialog. ### Adding a Tunnel Interface 1. Log into the Palo Alto Networks VM Series UI. 2. Navigate to Network > Interface > Tunnel and click **Add** to create a new tunnel interface and assign the following parameters. | **Field** | **Value** | | - | - | | Interface Name | tunnel.1 | | Virtual Router | Select the existing **default** virtual router | | Security Zone | Select the layer 3 internal zone from which traffic originates | ![create tunnel interface](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/create-tunnel-interface.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=d9679f630571c591e5ac50468661c18a) > [!NOTE] > > If the tunnel interface is in a zone different from the one where the > traffic will originate, a policy needs to be created to allow the > traffic to flow from the source zone to the zone containing the tunnel > interface. ### Setting up IKE Crypto Profile and IKE Gateways 1. In the Palo ALTO VM-Series UI, navigate to Network > Network Profiles > IKE Crypto, click **Add** and define the IKE Crypto profile (IKEv1 Phase-1) parameters. ![image1](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/ike-crypto-profile.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=db7008c200e4340ccc9726e95da5f9b4) 2. If using PSK (Pre-shared Key) for authentication with Site2Cloud, navigate to Network > Network Profiles > IKE Gateways to configure the IKE Phase-1 Gateway. These parameters should match the Site2Cloud configuration downloaded under [Setting up External (S2C) Connection](#setting-up-external-s2c-connection). > [!NOTE] > > According to the Palo Alto Networks official documents, it is not necessary to add the Peer Identification. However, Aviatrix recommends adding it, to make sure the tunnel is working. In the event > that the IPsec tunnel is up but traffic is not passing between the Cloud and > on-premises, you may want to enable NAT-T in Palo Alto Networks > Firewall. | Field | Value | | - | - | | Interface | Palo Alto Networks WAN port | | Peer IP Address | Aviatrix Gateway public IP | | Pre-shared Key | Key from Site2Cloud configuration downloaded in [Setting up External (S2C) Connection](#setting-up-external-s2c-connection) | | Peer Identification | Peer public IP address (if the Controller version is below 5.0, it should be the peer private IP) | ![image2](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/ike-gw-1.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=4f8dcf8bfb8042fb95541b9f8d19fa88) 3. Under Advanced Options, select the IKE Crypto profile created in step 1 of this procedure. ![ike gw 2](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/ike-gw-2.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=49998d24f5de8b70db55af175f19f547) 4. Under Network > Network Profiles > IPsec Crypto, click **Add** to create a new profile. Define the IPsec crypto profile (IKEv1 Phase-2). These parameters should match the Site2Cloud configuration downloaded during [Setting up External (S2C) Connection](#setting-up-external-s2c-connection). ![image4](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/ipsec-crypto-profile.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=e931466c0c5b09d39d0b4d1458c25941) ### Adding a New IPSec Tunnel 1. Under Network > IPsec Tunnels, click **Add** to create a new IPsec Tunnel. At the **General** window: | **Field** | **Value** | | - | - | | Tunnel Interface | Tunnel interface created earlier | | IKE Gateway | IKE gateway created earlier | | IPsec Crypto Profile | IPsec crypto profile created earlier | ![image5](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/ipsec-tunnel-one.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=4c9d2d5f6de1e6c27d73f0270935c4c0) 2. At **Proxy IDs** window: | **Field** | **Value** | | - | - | | Local | VPC2 private subnet CIDR | | Remote | VPC1 private subnet CIDR | | Protocol | Any | ![image6](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/ipsec-tunnel-two.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=31a312f2f57cfc61252088f1ed495e3a) 3. Under Network > Virtual Routers, click on the virtual router profile, then click Static Routes > default, and add a new route with the VPC1 private subnet as the destination. | **Field** | **Value** | | - | - | | Destination | VPC1 private subnet CIDR | | Interface | Tunnel interface created at Step 5.1 | ![image7](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/static-route.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=8d31eae8108bef6cbdc7bfe44c869158) 4. Commit the configuration. The IPsec tunnel status turns green. ![image10](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/ipsec-tunnel-status.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=8fce38b20791c28a2e9b6bc26b392198) 5. If using certificate-based authentication with Site2Cloud: 1. Go to Network > Network Profiles > IKE Gateways. These parameters should match the SIte2Cloud configuration downloaded under [Setting up External (S2C) Connection](#setting-up-external-s2c-connection). 2. In the IKE Gateway dialog enter the following: | Field | Value | | - | - | | Name | A name of your choosing | | Version | IKEv2 only mode | | Interface | ethernet 1/1 | | Local IP Address | IP address of on-prem | | Peer IP Address Type | IP | | Peer Address | IP address of cloud gateway | | Authentication | Certificate | | Local Certificate | The device certificate you created earlier | | Local Identification | FQDN (hostname) such as pan-device.com | | Peer Identification | FQDN (hostname) such as gw-spoke.aviatrix.network.com | | Peer ID Check | Exact | | Certificate Profile | Select the certificate profile you created in the previous section | 6. Click **OK**. 7. Navigate to Device > Certificate Management > Device Certificates > PAN-CA and export this certificate as a PEM file. 8. You must now import this certificate on the Networking > Connectivity > Certificates page in CoPilot, to use when setting up the external (Site2Cloud) connection between CoPilot and the Palo Alto VM-Series firewall. ### Finishing the Configuration 1. In the AWS portal, configure the VPC Route Table associated with the private subnet of VPC2. Add a route with a destination of VPC1's private subnet with the Palo Alto Networks VM LAN port as the gateway. 2. Send traffic between VPC1's and VPC2's private subnets. 3. In Aviatrix CoPilot, go to the Diagnostics > Cloud Routes > External Connections tab to confirm the Status and Tunnel Status of the external connection. To troubleshoot, in CoPilot go to Diagnostics > Diagnostic Tools. ## Related Topics * [External Device Settings](https://legacy.docs.aviatrix.com/documentation/latest/network/externaldevice-settings.html) # Aviatrix Gateway to Sonicwall Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/vpn/s2c-aviatrixgw-to-sonicwall This document describes how to build an IPsec tunnel based Site2Cloud connection between Aviatrix Gateway and Sonicwall. This document describes how to build an IPsec tunnel based Site2Cloud connection between Aviatrix Gateway and Sonicwall. The network setup is as follows: **VPC/VNet-AVX (with Aviatrix Gateway)** *VPC/VNet CIDR: 10.0.0.0/16* **On-Prem (with Sonicwall)** *On-Prem Network CIDR: 10.16.100.0/24* ## Creating a Site2Cloud Connection 1. Launch a [Spoke](/docs/enterprise/10.1/reference/ui/cloud-fabric/gateways#view-spoke-gateways) or [Transit Gateway](/docs/enterprise/10.1/reference/ui/cloud-fabric/gateways#view-transit-gateways) in the subnet of VPC/VNet-AVX (public subnet in AWS, GCP, or OCI). Make note of the Gateway's public IP address (35.161.77.0 in this example). 2. Navigate to Networking > Connectivity > External Connections (S2C) and click **Add New** to create a Site2Cloud connection using the values for one of the below options (for either you can select either PSK or certificate-based authentication). * [Static Route-Based](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-create-static-route-based-unmapped.html) * [Static Policy-Based](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-create-static-policy-based-unmapped.html) 3. Substitute the following values: * Local Gateway: select the Aviatrix Gateway created above * Remote Device Type: Generic (or **Sonicwall** if your version of Controller is lower than Controller version 6.7. If using a higher Controller version, only select **Generic** or **Aviatrix**). * Remote Device IP: Public IP of Sonicwall (66.7.242.225 in this example) * Remote Subnet CIDR(s): 10.16.100.0/24 (on-prem network CIDR) * Local Subnet CIDR(s): 10.0.0.0/16 * Pre-Shared Key is optional; it is auto-generated if not entered ## Creating Address Objects for the VPN subnets In the Sonicwall UI, go to Network > Address Objects > click **Add**. ### Creating an Address Object for the Local Network | **Field** | **Value** | | - | - | | Name | Arbitrary e.g. Site2Cloud-local | | Zone | LAN | | Type | Network | | Network | The LAN network range | | Network Mask/Prefix | e.g. 255.255.255.0 | ![Address Object for the Local Network](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/sw-lan-address-obj.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=a2c01189513f788b17460d04fd5f0662) ### Creating an Address Object for the Cloud Network | **Field** | **Value** | | - | - | | Name | Arbitrary e.g. site2cloud-cloud | | Zone | WAN | | Type | Network | | Network | The Cloud network range | | Network Mask/Prefix | e.g. 255.255.0.0 | ![Address Object for the Cloud Network](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/sw-wan-address-obj.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=4cc2f648668805b0d2195159de6d2662) #### Configuring the VPN Tunnel 1. Navigate to VPN > Settings > click **Add**. 2. On the **General** tab fill in the following fields: | **Field** | **Value** | | - | - | | Policy Type | Site to site | | Authentication Method | IKE using Preshared Secret | | Name | Arbitrary (e.g. Aviatrix-GW) | | IPsec Primary Gateway Address | The public IP of the Aviatrix Gateway | | IPsec Secondary Gateway Address | The public IP of the Aviatrix HA Gateway if configured | | Shared Secret | Arbitrary | | Confirm Shared Secret | Re-enter Shared Secret | | Local IKE ID | Leave blank | | Peer IKE ID | Leave blank | ![VPN Tunnel General Settings](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/sw-single-vpn.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=e411240c0c5de998af82b083762cd2b4) ### Assigning the Local and Remote Address Objects to the Tunnel 1. Select the **Network** tab and select the Address objects created above. 2. Choose local network from list: e.g. Site2Cloud-local. 3. Select the **Proposals** tab and set the IKE and IPsec values. | **Field** | **Value** | | - | - | | Exchange | Main Mode | | DH Group | Group2 | | Encryption | AES-256 | | Authentication | SHA1 | | Life Time (seconds) | 28800 | IPsec (Phase 2) Proposals | **Field** | **Value** | | - | - | | Protocol | ESP | | Encryption | AES-256 | | Authentication | SHA1 | | Enable Perfect Forward Secrecy | Mark this checkbox | | DH Group | Group 2 | | Life Time (seconds) | 3600 | ![VPN Policy Settings](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/connectivity/vpn/vpn-policy-sonicwall.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=4aea1957f3ea6787788cc49ca3395466) > [!NOTE] > > If the Secondary Peer IP is configured, then Peer IKE ID must be left blank or > else failover will not work properly. ![Failover VPN Settings](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/sw-failover-vpn.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=6c8ff6357d1e53ec36c1332d43843f65) ## Advanced Settings 1. Click the **Advance** tab. 2. Mark the **Enable Keep Alive** checkbox. 3. Click **OK** to save. ![Advanced VPN Settings](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/sw-prop-vpn.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=39ce8d2f4ef903d40f1e5fa50d1dabae) ## Related Topics * [About External Connection Settings](https://legacy.docs.aviatrix.com/documentation/latest/network/externaldevice-settings.html) # Site2Cloud Certificate-Based Authentication Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/vpn/s2c-cacert If you want to use certificate-based authentication when establishing a Site2Cloud (external) connection between your Aviatrix gateways, or between an Aviatrix gateway and an external device, you use the CA Certificate tab to: If you want to use certificate-based authentication when establishing a Site2Cloud (external) connection between your Aviatrix gateways, or between an Aviatrix gateway and an external device, you use the CA Certificate tab to: * Add external device CA certificates * Download the Aviatrix gateway CA certificate so that you can provide it to the external device The external device CA certificates must be available before you [configure the Site2Cloud (external) connection](/docs/enterprise/10.1/guides/connectivity/vpn/s2c-configuration-workflow) . On the external device side, you: * Obtain the IPsec VPN gateway device certificate * Export the Trusted Root CA certificate for use in the Site2Cloud configuration * Import the Aviatrix CA certificate (downloaded from the CA Certificate tab) * Use the information in the downloaded Site2Cloud configuration file to configure your tunnels/interfaces On the Aviatrix side, you: * Configure the Site2Cloud connection * Select the remote certificate (generated from the external device) when prompted * Enter the remote identifier when prompted (depends on the external device; typically the Remote Identifier is the value of the common name or subject field in the VPN gateway device certificate) * Export the Aviatrix CA certificate * Download the Site2Cloud configuration you created, to use when configuring tunnels/interfaces on your external device Currently only the [Palo Alto VM-Series firewall](/docs/enterprise/10.1/guides/connectivity/vpn/s2c-aviatrixgw-to-paloalto-firewall) is supported as an external device. For information on using certificate-based authentication between two Aviatrix gateways, see [Aviatrix Gateway to Aviatrix Gateway](/docs/enterprise/10.1/guides/connectivity/vpn/s2c-aviatrixgw-to-aviatrixgw). ## Adding a CA Certificate After you obtain the CA certificate from your external device, you must upload it on the CA Certificate tab before creating your Site2Cloud (external) connection. 1. In CoPilot, navigate to Networking > Connectivity > Certificates. 2. Click *+Certificate*. 3. In the Add Certificate dialog enter a unique name for the certificate. 4. Select the CA certificate to upload. 5. Click *Save*. If you have received an email notification that a CA certificate is about to expire, or one of the certificates is showing as Invalid on the CA Certificate list, you use the above procedure to add the new certificate. You must then delete the expired certificate. > [!NOTE] > > You cannot switch to another certificate after the Site2Cloud connection has > been created. ### Downloading the Aviatrix CA Certificate You must download the Aviatrix CA certificate and upload it to your external device (or Aviatrix gateway) for the Site2Cloud connection to work. 1. In CoPilot, navigate to Networking > Connectivity > Certificates. 2. Click *Download Aviatrix CA Certificate*. ### Deleting a Certificate You must delete a certificate if it has expired. These certificates show as 'invalid' in the CA Certificate table. > [!NOTE] > > You should not delete the certificate while it is in use; this will bring down > the Site2Cloud connection. Only admin users can delete certificates. 1. Navigate to Networking > Connectivity > Certificates. 2. Click the Delete ![delete icon](https://mintcdn.com/aviatrix-14b37c43/k65bKcXF8onWe87v/images/guides/platform-administration/delete-icon.png?fit=max\&auto=format\&n=k65bKcXF8onWe87v\&q=85\&s=4a9bea9e8ad5bd505064d4c0be159385) icon next to the certificate you want to delete. 3. (optional) On the Networking > Connectivity > External Connections (S2C) tab: 1. Select the appropriate gateway and then select *Connectivity Diagnostics* from the vertical ellipsis ![25](https://mintcdn.com/aviatrix-14b37c43/mOZ0lmP7pRPIryx3/images/guides/platform-administration/threedot.png?fit=max\&auto=format\&n=mOZ0lmP7pRPIryx3\&q=85\&s=6ed1c085429177f16d2735e2406d15c3) menu. 2. In the Connectivity Diagnostics Tools dialog, select *IPsec Service* and then click *Run*. This removes the deleted certificate from the gateway cache. ## Limitations * Only the Palo Alto VM-Series firewall is supported in this version of Site2Cloud cert-based authentication. * Only the Elliptic Curve DSA algorithm (256-bit) is supported in this version. * Only the PEM certificate file type is supported in this version. * Validation of PEM certificates that you upload is limited. If a certificate contains a malformed base64-encoded block, the upload does not detect or reject it. Validate your certificate, for example with OpenSSL, before you upload it. * You can only use one certificate group (all the certificates with the same tag name) per Site2Cloud connection. * You can only roll back the platform version if the previous version supports certificate-based authentication (not supported prior to 6.8). * The Aviatrix UserVPN® feature cannot be used in conjunction with Site2Cloud certificate-based authentication. # Connect Networks With Overlapping CIDRs Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/vpn/s2c-cidrs-overlap-connect This document illustrates an example solution to a use case where a customer needs to: ## Connect Networks With Overlapping CIDRs ## The Scenario This document illustrates an example solution to a use case where a customer needs to: * Connect certain on-prem hosts to certain virtual machine (EC2/GCE) instances in a VPC/VNet over an IPsec tunnel over the Internet * Their on-prem network range overlaps with the VPC/VNet CIDR range * No NAT function will be performed on the customer side * Traffic can be initiated from either side > [!NOTE] > > This solution works for specific hosts and virtual machine instances on each > side. The scenario is described in the following diagram, where VPC/VNet-2 represents an on-prem environment. ![overlap](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/overlap.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=98485c51ada98fb34bdbf134610bff32) * VPC/VNet-1 CIDR = 10.17.0.0/20; instance-1 in VPC/VNet-1 has an IP address 10.17.4.179. * VPC/VNet-2 CIDR = 10.17.0.0/20; instance-2 in VPC/VNet-2 has an IP address 10.17.7.81. ## The Solution The solution is to build a Site2Cloud IPsec tunnel between VPC/VNet-1 and VPC/VNet-2 and apply both source NAT (SNAT) and destination NAT (DNAT) on VPC/VNet-1 gateway. The packet flow is demonstrated as below: 1. instance-1 sends a packet to instance-2 with a virtual destination IP address, for example 172.16.0.43. From instance-1's point of view, the destination instance is a virtual address - 172.16.0.43. 2. When the packet arrives at the VPC/VNet-1 gateway, the gateway does DNAT on the packet to translate the virtual destination IP address to 10.17.7.81 which is the instance-2 physical IP address. 3. The gateway at VPC/VNet-1 then translates the packet source IP address (10.17.4.179) to a virtual source IP address (for example, 192.168.0.43). 4. The packet then arrives at VPC/VNet-2 with destination IP address 10.17.7.81 and source IP address 192.168.0.43. From instance-2's point of view, instance-1's address is a virtual IP address - 192.168.0.43. 5. When instance-2 sends a packet to instance-1, the destination is the virtual IP address 192.168.0.43. 6. When the packet arrives at the VPC/VNet-1 gateway over the IPsec tunnel, the VPC/VNet-1 gateway translates its destination IP address from virtual address 192.168.0.43 to 10.17.4.179. 7. The VPC/VNet-1 gateway then translates the source IP address of the packet from 10.17.7.81 to virtual address 172.16.0.43. ## The Configuration Steps ### Launch Gateways Create a [Spoke Gateway](/docs/enterprise/10.1/reference/ui/cloud-fabric/gateways#view-spoke-gateways) . Leave optional parameters unchecked. ### Creating External Connections 1. Create a [Static Route-Based (Mapped) external connection](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-create-static-route-based-mapped.html) for VPC-1/Gateway-1 using the following information: | Field | Value | | - | - | | Name | S2C-VPC-OnPrem (for example) | | Connect Public Cloud To | Static Route-Based (Mapped) | | Local Gateway | Select the Aviatrix Gateway created earlier in this document | | Real Local Subnet CIDR(s) | 10.24.0.0/20 (VPC-Cloud Network CIDR; subnet of VPC-1) | | Virtual Local Subnet CIDR(s) | Any/20 (VPC-Cloud Network Virtual CIDR) | | Remote Device Type | Generic | | Real Remote Subnet CIDR(s) | 10.24.0.0/20 (On-Prem Network CIDR; subnet of the on-prem router) | | Virtual Remote Subnet CIDR(s) | Any/20 (On-Prem Network Virtual CIDR) | | Pre-Shared Key | Optional (auto-generated if not entered) | | Remote Device IP | Public IP of IOS Router WAN port (52.40.45.197 in this example) | | Local Gateway Instance | Select the Gateway instance to associate with this IP (especially important if you are adding a second instance for HA) | Leave optional parameters Off (Over Private Network/IKEv2/Algorithms). You can add a secondary Connection (instance) for HA if required. > [!TIP] > > The Local & Remote Subnet (virtual) IP range can be anything but the subnet > should be the same as the Physical/Real subnet. 2. Click *Save*. 3. Create an Unmapped external connection for VPC/VNet-2 Gateway-2 using one of these options: * [Static Route-Based](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-create-static-route-based-unmapped.html) * [Static Policy-Based](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-create-static-policy-based-unmapped.html) | Field | Value | | - | - | | Name | VPC-2-to-VPC-1 (for example) | | Connect Public Cloud To | Static Route-Based (Unmapped) or Static Policy-Based (Unmapped) | | Local Gateway | Select the Aviatrix Gateway created earlier in this document | | Local Subnet CIDR(s) | 10.17.7.81/32 | | Remote Device Type | Generic | | Remote Subnet CIDR(s) | 192.168.0.43/32 | | Pre-Shared Key | Optional (auto-generated if not entered) | | Remote Device IP | 52.10.222.115 | | Local Gateway Instance | Select the Gateway instance to associate with this IP (especially important if you are adding a second instance for HA) | 4. Click *Save*. 5. Go to Diagnostics > Cloud Routes > External Connections tab to check the tunnel status. > [!NOTE] > > You do not need to download the configuration because both ends of the network are on the VPC/VNet. > > Make sure the pre-shared Keys are the same for both ends. In the above example, > "Aviatrix101#" can be used as the pre-shared key. ### Configuring DNAT on Gateway-1 You now configure the gateway to translate the destination IP address 172.16.0.43 to the real private IP address 10.17.7.81, before routing happens. 1. Go to Cloud Fabric > Gateways > Spoke Gateways. 2. Click the name of the gateway you created earlier in this document. 3. Click the Settings tab and expand the Network Address Translation (NAT) area. 4. Turn *On* Destination NAT. 5. Add rules as per the screenshot below. 6. Turn *On* Apply Route Entry for each rule to commit the rule(s). ![dnat](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/dnat.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=dca2d37c551893fde382fccd78985e9c) ### Configuring SNAT on Gateway-1 This step is to translate the packet source IP address after routing happens. In this example, the address is translated from 10.17.7.81 to 172.16.0.43 for packets going from on-prem (VPC/VNet-2) to VPC/VNet-1, and 10.17.4.179 to 192.168.4.43 for packets going from VPC/VNet-1 to on-prem (VPC/VNet-2). On the same VPC/VNet-1 gateway, configure SNAT as shown below (turn *On* Source NAT and then click *Customized SNAT*). You can enter "Dst CIDR" as qualifier to reduce the scope of the rule as a good practice. The reason that the address is 10.17.7.81/32 is that the destination has already been translated after the DNAT rule before routing happens. ![snat](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/snat.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=1645997a5947217c2e3a1607c505da64) ## Testing the Site2Cloud Connection Make sure your instance's Security Groups inbound rules are configured properly. From instance-1, you should be able to ping host-1 by "ping 192.24.7.101". From host-1, you should be able to ping instance-1 by "ping 172.24.1.4" # Site2Cloud External Connection Workflow Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/vpn/s2c-configuration-workflow This document describes how to establish connectivity between an Aviatrix gateway in AWS, Azure, or GCP and your on-premises router or firewall using Aviatrix Site2Cloud (S2C) external connection. This document describes how to establish connectivity between an Aviatrix gateway in AWS, Azure, or GCP and your on-premises router or firewall using Aviatrix Site2Cloud (S2C) external connection. Aviatrix S2C external connection builds an encrypted connection between two sites over the Internet in an easy-to-use and template-driven manner. On one end of the tunnel is an Aviatrix Gateway. The other end could be an on-premises router, firewall, or another public cloud VPC or VNet, that Aviatrix does not manage. ![site2cloud-new.png](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/site2cloud-new.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=c0af4cfe95878ba46ada5cd56119fdbc) The following Site2Cloud external connections are supported: * **Static Route-Based (Unmapped)**: connect to a remote site that supports route-based VPN tunneling. * **Static Route-Based (Mapped)**: connect overlapping networks between the cloud and on-premises. * **Static Policy-Based (Unmapped)**: connect to a remote site that supports policy-based VPN tunneling. ## Supported Gateways The following Aviatrix gateways are supported: * Spoke Gateway (not BGP enabled) in AWS, Azure, and GCP * Speciality Gateway ## Route-Based vs. Policy-Based Static Routing The Aviatrix Platform supports both route-based and policy-based static routing. The following Site2Cloud external connections are supported: * Static Route-Based (Unmapped)]: connect to a remote site that supports route-based VPN tunneling. * Static Route-Based (Mapped)]: connect overlapping networks between the cloud and on-premises. * Static Policy-Based (Unmapped)]: connect to a remote site that supports policy-based VPN tunneling. ### Route-Based Route-Based routing uses routing tables to direct network traffic through the VPN tunnel. Route-based tunnels leverage virtual tunnel interfaces (VTIs) to establish connectivity between endpoints, ensuring that any traffic matching the routing table criteria is transmitted through the VTI interface. It is typically deployed for cloud-to-cloud or cloud-to-on-premises connectivity where communication across multiple networks is required. Route-based tunnels offer seamless integration with dynamic routing protocols, such as BGP, and can handle large-scale, dynamic environments. The presence of VTI interfaces streamlines setup and troubleshooting, as all tunnel traffic flows through a predictable, logical interface and security policies can be applied more broadly. > [!NOTE] > > Starting with **Controller 9.0**, route-based IPSec Site-to-Cloud connections > can negotiate IKE phase 2 **Proxy-ID** selectors toward policy-based peers > when **Proxy-ID** is enabled. See > [Proxy-ID for Routed IPSec Connections](/docs/enterprise/10.1/guides/connectivity/vpn/proxy-id-routed-ipsec-connections) > . ### Policy-Based Policy-based tunnels, in contrast to route-based tunnels, do not use virtual tunnel interfaces (VTIs). Instead, all encrypted traffic traverses a single physical interface (such as eth0). Policy-based routing employs explicit policies, such as access control lists, to match and route traffic through the tunnel.These policies explicitly define which source and destination IP addresses, applications, or protocols are to be encrypted and tunneled. Only traffic matching the defined policies is allowed through the tunnel. This approach provides more restrictive and granular control over which data traverses the tunnel, making it suitable for use cases in multi-cloud architectures, where enforcing granular policies and segmenting traffic is critical for both security and performance optimization. ## Unmapped vs. Mapped The Aviatrix platform offers two types of static routing: Unmapped and Mapped. ### Unmapped Unmapped Static Routing refers to static routes configured on an Aviatrix gateway that are not associated with, nor propagated to, the underlying cloud provider's route tables—such as AWS VPC route tables or Azure VNet route tables. **Key Characteristics** * No Association with Cloud Provider Route Tables: Routes exist solely on the Aviatrix gateway and are invisible to the cloud provider's native routing infrastructure. * Interface and Tunnel Independence: These routes are not tied to specific tunnels or interfaces, allowing flexibility in the path traffic uses to reach its destination. * Direct Use of Local and Remote Subnets: The VPN configuration utilizes the actual (real) subnet CIDRs from both local and remote sites. * No NAT: Network Address Translation is not performed. The real subnet addresses remain unchanged as traffic traverses the VPN. * Best for Non-Overlapping CIDRs: This method is appropriate when there is no overlap between the local and remote networks' CIDR blocks. **Ideal Use Cases** * You want to keep routing simple and direct, with traffic reaching its destination via any available route. * There is no requirement to control which tunnel or interface traffic uses. * There are no overlapping subnets between connected networks, so address translation is unnecessary. ### Mapped Mapped Static Routing, in contrast, involves propagating the static routes into the cloud provider's route tables. This ensures that traffic originating from any resource within a VPC or VNet is directed toward the Aviatrix gateway for further processing. Mapped Static Routing provides the advantage of not needing to configure individual SNAT/DNAT rules. **Key Characteristics** * Propagated to Cloud Provider Route Tables: These routes are visible and actionable within the cloud provider's native route tables, enabling source-based traffic redirection. * Association with Tunnels/Interfaces: Mapped static routes are tied to specific VPN tunnels, gateways, or interfaces, ensuring traffic is routed over a defined path. * Use of NAT and Virtual CIDRs: Network Address Translation is employed to map real subnet CIDRs to virtual (translated) subnet CIDRs for the VPN tunnel. * Translation for Overlapping Networks: The local and/or remote subnets are translated as needed, making this approach ideal for environments with overlapping address spaces. It provides flexibility to build address translations for many scenarios (multi-site CIDR overlapping with on-premises CIDRs; multi-site CIDRs overlap and on-premises CIDRs do not overlap). **Ideal Use Cases** * When route control is needed at the source, and you must ensure traffic follows a specific path. * When there are overlapping subnet CIDRs between the local and remote sites, requiring address translation. * When you want to present different network views to remote sites for security or segmentation purposes. ### Custom Mapped The Custom Mapped option for the Static Route-Based Mapped routing provides more complex Site2Cloud external connection configuration options. You can have locally initiated traffic flows only; remotely initiated traffic flows only; and differing NAT translation based on these local and remote initiated flows. ![s2c-custom-mapped.png](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/s2c-custom-mapped.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=f881bfff01edf5e3d45dc9deb5df2e0b) ## Network Device Support The Aviatrix Platform supports [various on-premises firewall and router devices](https://legacy.docs.aviatrix.com/documentation/latest/network/s2c-configuration-other-devices.html) that terminate VPN connections for establishing Site2Cloud external connections. ## Create a Site2Cloud External Connection > [!NOTE] > > If you are planning to use certificate-based authentication for your > Site2Cloud connection, ensure that you have already > [generated and exported the external device CA certificate](/docs/enterprise/10.1/guides/connectivity/vpn/s2c-cacert) > . To establish a Site2Cloud external connection, you can create one of the following connections: * [Static Route-Based (Unmapped)](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-create-static-route-based-unmapped.html): connect to a remote site that supports route-based VPN from a Spoke Gateway. * [Static Route-Based (Mapped)](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-create-static-route-based-mapped.html): connect overlapping networks between the cloud and on-prem from a Spoke Gateway. * [Static Policy-Based (Unmapped)](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-create-static-policy-based-unmapped.html): connect to a remote site that supports policy-based VPN with static configuration from any local gateway (Unmapped). ## Create a Site2Cloud External Connection with High Availability To establish a High Availability (remote backup or failover) connection in the event that the primary remote connection becomes unavailable, you may configure an additional remote device to serve as the failover connection. ![external-connection-backup.png](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/external-connection-backup.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=8e92436ffbe3fc7d208416d07e747498) > [!NOTE] > > * If Mapped NAT is enabled, HA in the Site2Cloud (external) connection is not supported. > > * You can use the same Pre-Shared Key for the HA connection (if PSK-based > authentication is selected). ## Download an External Connection Configuration If you are connecting an Aviatrix gateway and an on-premises router or firewall, Aviatrix can generate a configuration file that you can apply to your remote router or firewall. The configuration file contains the Aviatrix gateway tunnel details, such as the Public IPv4 and IPv6 addresses, VPC/VNet CIDR, pre-shared key, and encryption algorithm. You can download the configuration file and then import the details to your remote router or firewall to configure the other end of the VPN tunnel. After creating an external connection, to download an external connection configuration: 1. In Aviatrix CoPilot, go to **Networking > Connectivity > External Connections (S2C)** tab. 2. On the External Connections (S2C) tab, locate the connection you created and click the vertical ellipsis icon on the right side of the row. 3. Select the following values: * **Vendor**: Select your remote site device. * Select **Generic** for anything that is not an Aviatrix gateway. * Select **Aviatrix**, if you are connecting two Aviatrix gateways. * **Platform and Software**: * If you selected a Generic vendor, the Platform field is populated as Generic, and the Software field is populated with Vendor Independent. * If you selected the Aviatrix vendor, the Platform field is populated with UCC, and the Software version is 1.0. * If you selected a specific hardware vendor (such as Cisco), select from the available platforms belonging to that vendor are displayed in the Platform field (ISR, ASR, and CSR are for Cisco routers), and the Software field is populated with the related software version. 4. Click **Download**. ## View an External Connection Details After an external connection is created, you can view and change its settings. To view the details of an external connection: 1. Go to **Networking** > **Connectivity** > **External Connections (S2C) tab**. 2. Select the external connection in the table. 3. Click the **Details** tab to review the general information about the external connection. 4. Click the **Settings** tab to change the settings. ## Checking Tunnel Status You can check the tunnel connection between the Aviatrix gateway and the remote device from **Diagnostics > Cloud Routes > External Connections**. Locate the external connection in the list and check that it's **Status** shows as **Established**. If the external connection is not established, you can troubleshoot network connection from **Diagnostics > Diagnostic Tools**. ## Related Topics * To learn about how Site2Cloud is used by the Distributed Cloud Firewall, see [External Connection (Site2Cloud) and Distributed Cloud Firewall](/docs/enterprise/10.1/reference/dcf/dcf-s2c) . * [Overview of Aviatrix Site2Cloud](/docs/enterprise/10.1/concepts-architectures/components/connectivity/s2c-overview) * [Aviatrix Integration with Microsoft's SSE Solution](/docs/enterprise/10.1/guides/connectivity/vpn/s2c-aviatrixgw-to-microsoft-entra) # Connect Overlapping VPC/VNet to On-prem Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/vpn/s2c-connect-overlap-vpc Organizations usually plan out their cloud network address ranges. But there are times when a VPC/VNet CIDR overlaps with an on-prem network address range, yet still requires connectivity to on-prem. Organizations usually plan out their cloud network address ranges. But there are times when a VPC/VNet CIDR overlaps with an on-prem network address range, yet still requires connectivity to on-prem. In this document, the scenario is such that traffic is always initiated from on-prem to VPC/VNet. The constraint is that there should be no source NAT nor destination NAT performed in the on-prem network. As shown in the diagram below, the on-prem network address range is 10.20.0.0/16. All other VPCs connect to on-prem via Aviatrix Transit solution. However, there is one VPC named spoke-vpc with an identical CIDR of 10.20.0.0/16. ![overlap\_cidr](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/overlap-cidr.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=3e95a539243f274a7d30e704abead904) Since the on-prem network does not perform any NAT functions, NAT must be performed in the cloud network. ## Allocate Mapped Virtual Address Spaces Allocate two 1-1 mapped corresponding virtual address spaces for the on-prem network and spoke-vpc/vnet. For example, allocate the virtual network 100.105.0.0/16 for the on-prem network, and 100.101.0.0/16 for the spoke-vpc/vnet virtual VPC/VNet CIDR. These two virtual address spaces must not overlap with any on-prem or cloud address spaces. ## Launch an Aviatrix Gateway [Launch an Aviatrix Gateway](/docs/enterprise/10.1/reference/ui/cloud-fabric/gateways#view-spoke-gateways) in the spoke-vpc/vnet. ## Build an IPsec Tunnel Build an IPsec tunnel between spoke-vpc/vnet and the VPN Gateway (VGW/VPN Connect). 1. In the CSP Console (AWS, Azure, GCP, or OCI) for the VPC/VNet service, create a VPN connection. Use the same VGW that is used for the Aviatrix Transit solution to create an IPsec tunnel to spoke-vpc/vnet with static routes 100.101.0.0/16 configured, as shown below (AWS Console example). ![VGW configuration](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/connectivity/vpn/vgw-config.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=a5d537fe03242eb908e155b8e534976d) 2. Save the VPN connection. 3. Download the VPN configuration file. You will use some of this information when creating the external connection. ## Create External Connection 1. In Aviatrix CoPilot, go to Networking > Connectivity > External Connections (S2C). 2. Create an Unmapped external connection for VPC/VNet-2 Gateway-2 using one of these options: * [Static Route-Based](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-create-static-route-based-unmapped.html) * [Static Policy-Based](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-create-static-policy-based-unmapped.html) 3. Use these values: | Field | Value | | - | - | | Name | spoke-vpc-to-to-tgw | | Connect Public Cloud To | Static Route-Based (Unmapped) or Static Policy-Based (Unmapped) | | Local Gateway | Select the Aviatrix Gateway created earlier in this document (in this example, the Aviatrix gateway for the spoke-vpc) | | Local Subnet CIDR(s) | 100.101.0.0/16 (spoke-vpc virtual address) | | Remote Device Type | Generic | | Remote Subnet CIDR(s) | 10.20.0.0/16 (on-prem CIDR) 100.105.0.0/16 (on-prem virtual CIDR) | | Pre-Shared Key | Optional (if needed, the tunnel #1 Pre-Shared Key from the downloaded VPN configuration file) | | Remote Device IP | 52.24.197.135 (the Virtual Private Gateway Public IP from the downloaded VPN configuration file) | | Local Gateway Instance | Select the Gateway instance to associate with this IP (especially important if you are adding a second instance for HA) | ![Site2Cloud configuration](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/site2cloud.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=f04769565b1e121f954cf4a7ff09c8a9) ## Perform both SNAT and DNAT Functions on the Aviatrix Gateway 1. In Aviatrix CoPilot, go to Cloud Fabric > Gateways > Spoke Gateways and select the 'spoke-vpc/vnet' gateway you created. 2. Click the Settings tab. 3. Expand the Network Address Translation area. 4. Turn **On** Source NAT. 5. Create a rule that matches the criteria in the following screenshot. 6. Turn **On** Destination NAT. 7. Create a rule that matches the criteria in the following screenshot. You are translating the cloud virtual destination address to its real address for each instance in the VPC/VNet. 1. Mark the session with a number that is easy to remember. In this example, it is 119. 2. Scroll up to find Source NAT. Translate the marked session to any on-prem virtual source address, as shown in the screenshot below. ![nat\_config](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/nat-config.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=8b67bb7a03b81e146e74cbcf81fa276a) Since the VPN Gateway (VGW/VPN Connect) normally runs a BGP session to on-prem for a Transit Network, the spoke-vpc/vnet virtual CIDR 100.101.0.0/16 should be propagated to on-prem. From on-prem, the destination IP address takes the range 100.101.0.0/16. # Site2Cloud With Customized SNAT Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/vpn/s2c-customized-snat This document demonstrates how to create a Site2Cloud connection between two VPCs by using a VGW and an Aviatrix gateway. The Aviatrix gateway also serves as a Source NAT device and translates source IPs of traffic initiated from a peering VPC to an IP address selected by users. This document demonstrates how to create a Site2Cloud connection between two VPCs by using a VGW and an Aviatrix gateway. The Aviatrix gateway also serves as a Source NAT device and translates source IPs of traffic initiated from a peering VPC to an IP address selected by users. ## Environment Requirements There are two VPCs as illustrated in the diagram below. The VPC-1 CIDR is 10.0.0.0/16 and the VPC-2 CIDR is 172.19.0.0/16. The Site2Cloud connection is between a VGW in VPC-1 and an Aviatrix gateway in VPC-2. ![s2c-snat](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/s2c-snat.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=31121917479f99d06835ad6f8f402ae9) You will also configure customized SNAT at the Aviatrix gateway, which translates the source IP of traffic initiated from VPC-1 (10.0.0.0/16) to a user selected IP address (192.168.1.10 in this example). This way, VPC-2 VMs will see all packets from VPC-1 with the same source IP address (192.168.1.10). ## Steps to Configure Site2Cloud Connection and SNAT 1. Install an Aviatrix gateway in VPC-2 by following [these instructions](https://legacy.docs.aviatrix.com/documentation/latest/network/spoke-gateway-about.html?expand=true). 2. Create an Unmapped external connection between the VGW in VPC-1 and the Aviatrix gateway in VPC-2. * [Static Route-Based](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-create-static-route-based-unmapped.html) * [Static Policy-Based](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-create-static-policy-based-unmapped.html) > [!NOTE] > > When creating the connection, select *Generic* for Remote Device Type. 3. Update VPC-1 Route Tables at AWS portal to ensure that traffic with VPC-2 (172.19.0.0/16) as the destination takes the VGW as "Target": | **Field** | **Value** | | - | - | | Destination | 172.19.0.0/16 | | Target | VGW ID | 4. Configure Customized SNAT at the Aviatrix gateway. a. In Aviatrix CoPilot, go to Cloud Fabric > Gateways > Spoke Gateways. b. Select the Aviatrix gateway created in VPC-2. c. On the Settings tab, expand *Network Address Translation (NAT)* section. d. Set Source NAT toggle *On*. e. Select *Customized SNAT*. f. Configure the following SNAT rule. | **Field** | **Value** | | - | - | | Src CIDR | VPC-1 CIDR (10.0.0.0/16) | | Src Port | Leave it blank | | Dst CIDR | VPC-2 CIDR (172.19.0.0/16) | | Dst Port | Leave it blank | | Protocol | all | | Connection | None | | Mark | Leave it blank | | SNAT IPs | User selected IP (192.168.1.10) | | SNAT Port | Leave it blank | ![s2c-snat2](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/s2c-snat2.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=ae29ff5ec9dfab3099a660e3371c2841) g. Click *Save*. The rule appears in the table. h. Set *Apply Route Entry* toggle *On* to commit the rule. ### Test Site2Cloud Connection and SNAT Use the following tools to test the connection and verify the source IP NAT to the IP selected. * Go to Diagnostics > Cloud Routes > External Connections to verify that the external connection status is Up. * Ping from an Ubuntu VM in VPC-1 to another Ubuntu VM in VPC-2. * Turn on "tcpdump icmp -n" at the Ubuntu VM in VPC-2. Verify the source IP of the pings is 192.168.1.10. ## Related Topics * [Using Customized SNAT and DNAT on Spoke Gateway](/docs/enterprise/10.1/guides/connectivity/vpn/spoke-gateway-snat-dnat) * [Site2Cloud with NAT to fix Overlapping VPC Subnets](/docs/enterprise/10.1/guides/connectivity/vpn/s2c-nat-overlapping) # Site2Cloud with NAT to fix Overlapping VPC Subnets Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/vpn/s2c-nat-overlapping This document describes how to reach VMs in two overlapping subnets within three different VPCs. This document describes how to reach VMs in two overlapping subnets within three different VPCs. ## Environment Description There are three VPCs as illustrated in the diagram below. * **VPC1**: VPC CIDR - 10.3.0.0/16 * **VPC2**: VPC CIDR - 172.29.0.0/16 * **VPC3**: VPC CIDR - 10.3.0.0/16 ![s2c-overlapping](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/s2c-overlapping.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=1dee0eb096c0148b7d81c72e2da840e5) * Both VPC1 and VPC3 have a subnet with the same CIDR (10.3.3.0/24). * VPC2 VMs need to access both VPC1 and VPC3 VMs in their 10.3.3.0/24 subnets. * Since VPC1 VMs may change their private IP addresses after some unplanned reboots, VPC2 VMs have to access them through DNS. * VPC2 VMs need to access VPC3 VMs through their private IP addresses. To solve this overlapping subnet issue, you create four Site2Cloud (S2C) connections: * Unmapped S2C connection between VPC1 and VPC2 * Unmapped S2C connection between VPC2 and VPC1 * Mapped S2C connection between VPC2 and VPC3 * Mapped S2C connection between VPC3 and VPC2 Detailed configuration steps are illustrated below. ## Configuring Site2Cloud Connections First, install Aviatrix gateways in VPC1, VPC2, and VPC3. Follow [these instructions](https://legacy.docs.aviatrix.com/documentation/latest/network/spoke-gateway-about.html?expand=true). > [!NOTE] > > Do not enable SNAT for the new gateways in these VPCs. ### Configuring Connections Between VPC1 and VPC2 Follow these steps to create the Unmapped S2C connections between VPC1 and VPC2: 1. Create a [Static Route-Based (Unmapped)](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-create-static-route-based-unmapped.html) or [Static Policy-Based (Unmapped)](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-create-static-policy-based-unmapped.html) connection between VPC1 and VPC2 by using the values below (other settings are Off by default). * Local Gateway: Select the Aviatrix Gateway in VPC2 * Remote Device Type: Aviatrix * Remote Device IP: Enter the public IP of the Aviatrix Gateway in VPC1 * Remote Subnet CIDR(s): Enter VPC1 CIDR (10.3.0.0/16 in this case) * Leave Pre-Shared Key blank; this value will be auto-generated 2. Create a second [Static Route-Based (Unmapped)](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-create-static-route-based-unmapped.html) or [Static Policy-Based (Unmapped)](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-create-static-policy-based-unmapped.html) connection between VPC2 and VPC1 by using the values below. * Local Gateway: Select the Aviatrix Gateway in VPC1 * Remote Device Type: Aviatrix * Remote Device IP: Enter the public IP of the Aviatrix Gateway in VPC2 * Remote Subnet CIDR(s): Enter VPC2 CIDR (172.29.0.0/16 in this case) * Leave Pre-Shared Key blank ### Configuring Connections Between VPC2 and VPC3 Follow these steps to create the Mapped S2C connections between VPC2 and VCP3: 1. Create a [Static Route Mapped connection](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-create-static-route-based-mapped.html) between VPC2 to VPC3 by using the values below (other settings are Off by default): * Local Gateway: Select the Aviatrix Gateway in VPC2 * Remote Device Type: Aviatrix * Remote Device IP: Enter the public IP of the Aviatrix Gateway in VPC3 * Real Local Subnet CIDR(s): Enter VPC2 CIDR (172.29.0.0/16 in this case) * Virtual Local Subnet CIDR(s): Enter VPC2 CIDR again (172.29.0.0/16 in this case) * Real Remote Subnet CIDR(s): Enter VPC3 CIDR (10.3.0.0/16 in this case) * Virtual Remote Subnet CIDR(s): Enter the virtual CIDR (10.49.0.0/16 in this example) * Leave Pre-Shared Key blank; this value will be auto-generated > [!NOTE] > > For Remote Subnet, you map the real subnet CIDR (10.3.0.0/16) to the virtual subnet CIDR (10.49.0.0/16). The masks of both real and virtual subnets have to be the same (/16 in this case). > > The IP addresses in real and virtual subnets are a one-to-one mapping by > translating 10.3.x.y to 10.49.x.y. For example, for VPC2 VM to reach 10.3.1.100 > in VPC3, the VPC2 VM needs to use IP address 10.49.1.100. > > For Local Subnet, you don't need to map the real subnet CIDR (172.29.0.0/16) to > a different virtual subnet CIDR because 172.29.0.0/16 in VPC2 doesn't conflict > with any subnet in VPC1 or VPC3. 172.29.0.0/16 is used for both Real Local > Subnet and Virtual Local Subnet. 2. Create a second [Static Route Mapped connection](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-create-static-route-based-mapped.html) between VPC2 to VPC3 by using the values below: * Local Gateway: select the Aviatrix Gateway in VPC3 * Remote Device Type: Aviatrix * Remote Device IP: Enter the public IP of the Aviatrix Gateway in VPC2 * Real Local Subnet CIDR(s): Enter VPC3 CIDR * Virtual Local Subnet CIDR(s): Enter VPC3 CIDR again * Real Remote Subnet CIDR(s): Enter VPC2 CIDR * Virtual Remote Subnet CIDR(s): Enter VPC2 CIDR ### Testing the Site2Cloud Connection You can verify the status of the Site2Cloud connections is *Up* from *Diagnostics > Cloud Routes > External Connections*. You can troubleshoot the connections from *Diagnostics > Diagnostic Tools*. # Site2Cloud to a Public IP Address Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/vpn/s2c-public-ip This document addresses the scenario where a customer on-prem firewall device needs to route encrypted traffic to a partner network in the cloud (AWS/Azure/GCP). This document addresses the scenario where a customer on-prem firewall device needs to route encrypted traffic to a partner network in the cloud (AWS/Azure/GCP). However due to concerns for overlapping CIDR blocks to the customer network, the customer side enforces a policy that the destination IP address must be a public IP address regardless of if the partner network is in the RFC 1918 range. For example, the VPC instance IP address that the on-prem machine should send data to is 172.32.0.243, but the on-prem machine must instead send data to a public IP address 53.34.19.23 (or even 100.100.100.100). The scenario is shown in the diagram below. ![site2cloud-publicIP](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/site2cloud-publicIP.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=e25faf8895cb962fd5c638cc0bfd28ef) This problem can be solved by combining the [Site2Cloud](https://legacy.docs.aviatrix.com/documentation/latest/network/site-to-cloud.html) feature and the [DNAT](https://legacy.docs.aviatrix.com/documentation/latest/network/snat-dnat-settings.html) feature. ## Determine the Public IP Address As this public IP address is what the on-prem host sees, it should not change. You can do one of the following: * Allocate an EIP in the VPC for this public IP address. Make sure you do not associate this EIP to any instance. * Alternatively, if the EC2 instance that on-prem hosts need to send data to has an EIP, you can use that EIP. You can also try a reserved public IP address range (for example, 100.100.x.x range). ## Launch a Spoke Gateway [Launch a Spoke gateway](/docs/enterprise/10.1/reference/ui/cloud-fabric/gateways#view-spoke-gateways) in VPC 172.32.0.0/16. In this example the gateway name is Spoke1. ## Create an External Connection 1. Navigate to Networking > Connectivity > External Connections (S2C) and click *Add New* to create a Site2Cloud connection using the values for one of the below options (for either you can select either PSK or certificate-based authentication). * [Static Route-Based](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-create-static-route-based-unmapped.html) * [Static Policy-Based](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-create-static-policy-based-unmapped.html) ![site2cloud-publicIP-config](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/site2cloud-publicIP-config.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=75cd19cc84aa7a0ffde1032c302e0956) * The Local Subnet CIDR(s) field is the real or virtual public IP address that the on-prem host sends to. If there are multiple instances in VPC that needs to be addressed, enter multiple IP addresses separated by commas. * The Remote Device IP is the on-prem firewall public IP. * The Remote Subnet CIDR(s) is the on-prem network range. ## Download the Configuration Template Click the vertical ellipsis ![vertical ellipsis icon](https://mintcdn.com/aviatrix-14b37c43/VY6W0iTAMUQkrobS/images/security/threedot.png?fit=max\&auto=format\&n=VY6W0iTAMUQkrobS\&q=85\&s=9498dd71f99faa81563f9d3f0baf4463) and select *Download Configuration* to download the template and send to the customer. Select the *Generic* Vendor Type. ## Configure DNAT This step is to configure the gateway to translate the destination IP address 53.34.19.23 to the real private IP address 172.32.0.242. 1. Navigate to Cloud Fabric > Gateways > Spoke Gateways and select the Spoke Gateway created earlier in this document. 2. Click the Settings tab and expand the Network Address Translation (NAT) area. 3. Turn On Destination NAT. 4. Add a rule using the following information from this example: * Source CIDR: 172.34.0.0/24 * Destination CIDR: 53.34.19.23/32 * Protocol: all * Connection: The Site2Cloud connection you configured earlier in this document * DNAT IPs: 172.32.0.243 ![dnat-config](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/dnat-config.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=82d1eb00b9d7e48da415fa482d2e4712) ## Test the Site2Cloud Connection and DNAT Go to the Diagnostics > Cloud Routes > External Connections tab in Aviatrix CoPilot and verify that the external connection status is "Up". Use ping to test connectivity from the on-prem host to the EC2 instance. For example, ping 53.34.19.23 from an on-prem host machine. The ping should reach 172.32.0.243. # Building Site to Site IPsec VPN Connection Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/vpn/site-to-site-vpn You can use Aviatrix gateways to connect one site to another. This solution requires one Aviatrix gateway in each location that needs to be connected. These on-premises gateways can be deployed as virtual machines on VMware, KVM or Hyper-V. You can use Aviatrix gateways to connect one site to another. This solution requires one Aviatrix gateway in each location that needs to be connected. These on-premises gateways can be deployed as virtual machines on VMware, KVM or Hyper-V. ## Environment Requirements An Aviatrix Site to Site IPsec tunnel is accomplished by one gateway initiating the session with the other gateway. For this to work at least one of the Aviatrix virtual appliances needs to be accessible via a public IP address. This can be accomplished by setting up the public IP address on the edge router in the on-premises network and configuring NAT from that public IP address to the Aviatrix VM with a 1-1 IP address NAT. The only ports that need to be forwarded from the edge router to the VM are UDP ports 500 and 4500. ![Environment Requirements Diagram 1](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/environment-requirments1.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=3e0d90ad18ac46c9b7a5868b1d817f31) On the other site, the second gateway does not need a public IP assigned to the Aviatrix gateway. This second gateway will reach outbound to the first Aviatrix GW (GW1). ![Environment Requirements Diagram 2](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/environment-requirments2.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=046c795880b64aa8c808787428f304ea) The last requirement is to configure static routes in the internal routers (default gateway of the Aviatrix VM) in both sites. This static route should send traffic destined for the other site to the Aviatrix GW as the next hop. ![Environment Requirements Diagram 3](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/environment-requirments3.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=355b82080567ff0dcb0595ff3311a0dd) ## Steps to Configure IPSec Connectivity 1. Install an Aviatrix gateway in each site. See [Creating a Spoke Gateway](https://legacy.docs.aviatrix.com/documentation/latest/network/gateway-spoke-create.html). 2. Configure an external connection (Site2Cloud) for Gateway 1. Aviatrix Site2Cloud feature builds an encrypted connection between the two sites over the Internet. a. In CoPilot, go to *Networking* > *Connectivity* > *External Connections (S2C)* and click *+External Connection*. b. Enter a name for the connection. c. Select the *External Device* radio button, then click on the dropdown menu and select *Static Route-Based* for route-based VPN connection or *Static Policy-Based* for policy-based VPN connection. d. Follow the instructions in [Static Route-Based External Connection (Unmapped)](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-create-static-route-based-unmapped.html) or [Static Policy-Based (Unmapped) External Connection Using Terraform](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-create-static-policy-based-unmapped.html) using the values in the table below: | Field | Description | | - | - | | Local Gateway | The name of Gateway 1 created above. | | Local Subnet CIDR(s) | The subnet CIDR range(s) for Gateway 1. | | Remote Device Type | Aviatrix | | Remote Subnet CIDR(s) | The subnet CIDR range(s) for Gateway 2. | | Remote Device IP | The public IP of Gateway 2. | e. Click *Save*. The connection is listed on the External Connections (S2C) tab. 3. [Download the configuration](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-download-configuration.html). 4. Log in to Gateway 2's CoPilot on the other site. 5. On the *Networking* > *Connectivity* > *External Connections (S2C)* tab, add a new connection using the downloaded configuration information above. This will start the IPsec negotiations between both gateways. You can check the status of the connection by going to *Diagnostics* > *Cloud Routes* > *External Connections*. # Troubleshooting Transit Gateway Connections Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/vpn/tgw-activemesh-troubleshoot To Troubleshoot Transit Gateway Connections: To Troubleshoot Transit Gateway Connections: 1. Go to **Aviatrix CoPilot > Diagnostics > Cloud Routes**. 2. **Check IPsec Tunnel**: For BGP learned routes, select the **BGP Info** tab, find the gateway, and check if the IPsec tunnel is in Up state. If it is not, select the Gateway Routes tab, find the gateway, and click the **Gateway Diagnostics** icon on the right. Since all BGP sessions run inside IPsec tunnel, this is the first thing you should check. 3. **Check BGP Session**: For BGP learned routes, check if BGP session is established. Go to the **BGP Info** tab, expand the gateway, and make sure the BGP session is in Established State. If it is not, go to the Gateway Routes tab, find the gateway, and click the **Gateway Diagnostics** icon on the right. Select the transit gateway, run commands, such as "show ip bgp". Other ways to troubleshoot: 1. **Check Route Database** For all routes, check if the CoPilot see all the learned routes from TGW, BGP, Transit Peering, and Static. 2. **Check Aviatrix Transit Gateway Programmed Routes** Review gateway routes for Transit Gateways. Make sure the route you are looking for is in the table and has a next hop with metric 100 or lower. 3. **Sync Routes** If for any reason the Route Database on the Aviatrix Platform become inconsistent with the Aviatrix Transit Gateway route table, [sync the routes](https://legacy.docs.aviatrix.com/documentation/latest/network/tgw-vpc-attachment-create.html) to force program the routes on the gateway again. If any of the above steps show failure, there is an error, please open a support ticket at [Aviatrix Support Portal](https://support.aviatrix.com) for more debugging assistance. If all above steps succeed, the connectivity issue lies somewhere else. Check Spoke VPC/VNet route table and TGW route table if applicable. If this is TGW based deployment, [run an audit](https://legacy.docs.aviatrix.com/documentation/latest/network/tgw-vpc-attachment-create.html#vpc-audit-routes). Any missing routes in either VPC/VNet route table or TGW route table should be discovered. # Transit Gateway GRE Tunneling in AWS Workflow Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/vpn/transit-externaldevice-aws-gre-tunneling Connecting to on-prem network over the GRE tunneling protocol in AWS is an alternative to IPsec. When GRE tunneling is used, Aviatrix Multicloud Transit Gateways interoperate directly with on-prem network devices over AWS Direct Connect. ## Introduction Connecting to on-prem network over the GRE tunneling protocol in AWS is an alternative to IPsec. When GRE tunneling is used, Aviatrix Multicloud Transit Gateways interoperate directly with on-prem network devices over AWS Direct Connect. ![transit\_gateway\_external\_device\_bgp\_over\_gre\_diagram](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/gre-bgp-diagram.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=7a14dab09541a9cbe81823e5640fcc0e) This diagram shows where Aviatrix Multicloud Transit Gateways connect to an on-prem edge router over Direct Connect. This document provides step-by-step instructions on how to build Aviatrix Transit Gateway to External Device using GRE over AWS Direct Connect: 1. [Building underlay connectivity with AWS Direct Connect 10 Gbps capacity](#building-underlay-connectivity-with-aws-direct-connect). 2. [Deploying Aviatrix Transit Solution](#deploying-the-aviatrix-multicloud-transit-solution). 3. [Establishing connectivity between edge router and Aviatrix Transit Gateway to form GRE tunnel](#building-connectivity-between-edge-router-and-aviatrix-transit-gateway). 4. [Building GRE tunnel and BGP over GRE](#confirming-that-edge-router-and-aviatrix-transit-gateway-can-reach-each-others-ip-for-gre-tunnel). 5. [Enabling ECMP Load Balancing to achieve high performance](#configuring-ecmp-load-balancing-for-high-performance). For more information about Multicloud Transit Network and External Device, refer to: * [Multi Cloud Global Transit FAQ](/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/transit-network-faq) * [Global Transit Network Workflow Instructions (AWS/Azure/GCP/OCI)](/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/transit-externaldevice-usecase) * [Aviatrix Transit Gateway to External Devices](/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/transit-externaldevice-usecase) * [Transit Network Design Patterns](/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/transit-externaldevice-usecase) > [!NOTE] > > * ActiveMesh 2.0 is required. To migrate to ActiveMesh 2.0, see > [Migrating to ActiveMesh 2.0](https://legacy.docs.aviatrix.com/documentation/latest/network/activemesh-migration.html). > * This solution is not available to Azure and GCP as they do not support GRE. > * Reachability between Transit VPC CIDR and edge router is your responsibility, > which is typically done by Colocation data center providers. > * Workflow on building underlay connectivity for private network with AWS Direct > Connect here is just an example. Please adjust the topology depending on your > requirements. ## The key ideas for this solution are: * The edge (WAN) router runs a BGP session to AWS VGW via AWS Direct Connect where the edge router advertises its GRE IPs. AWS VGW advertises the AWS Transit VPC CIDR. * Leverage edge router BGP ECMP feature. * Configure multiple GRE tunnels for greater aggregate throughput. > [!NOTE] > > Reachability between Transit VPC CIDR and edge router is the responsibility of > customer. ## Prerequisites * [Upgrade](/docs/enterprise/10.1/guides/maintenance/upgrade-controller-image-overview) Aviatrix Controller to the latest version. * In this example, we are going to deploy the below VPCs in AWS: * AWS Aviatrix Transit VPC (i.e. 10.1.0.0/16) by utilizing Aviatrix feature [Create a VPC](/docs/enterprise/10.1/reference/ui/cloud-resources/cloud-networks#create-vpcvnet) with Aviatrix FireNet VPC option enabled * AWS Aviatrix Spoke VPC (i.e. 192.168.1.0/24) by utilizing Aviatrix feature [Create a VPC](/docs/enterprise/10.1/reference/ui/cloud-resources/cloud-networks#create-vpcvnet) as the previous step or manually deploying it in each cloud portal. Moreover, feel free to use your existing cloud network. * Edge router has high throughput supported on hardware interface(s) and GRE tunnel(s) ## Building Underlay Connectivity with AWS Direct Connect Building AWS Direct Connect is your responsibility. For more information about AWS Direct Connect, please see [Connect Your Data Center to AWS](https://aws.amazon.com/getting-started/projects/connect-data-center-to-aws/). Adjust the topology depending on your requirements. ### Building AWS Direct Connect See [Equinix ECX Fabric AWS Direct Connect](/docs/enterprise/10.1/guides/connectivity/vpn/transit-externaldevice-aws-gre-tunneling) if you select an Equinix solution. This is just an example. Make sure to select 10 Gbps capacity. ### Associating AWS VGW to AWS Transit VPC 1. Log in to the AWS VPC Portal and select **Virtual Private Gateways** under the Virtual Private Network (VPN) sidebar. 2. Select the Virtual Private Gateway that you have the private virtual interface to AWS Direct Connect 3. Click **Actions**. 4. Select **Attach to VPC**. 5. Select the AWS Transit VPC and click **Attach to VPC**. ![aws\_vgw\_attach](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/aws-vgw-attach.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=054fc5d8f43e7504ac727e9f6197d5d1) ## Deploying the Aviatrix Multicloud Transit Solution Refer to [Single Region Transit Network Workflow Instructions](/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/transit-externaldevice-usecase) for the steps below. Please adjust the topology depending on your requirements. ### Step 2.1. Deploy Aviatrix Multicloud Transit Gateway and HA in AWS * Follow this step to [Deploy the Aviatrix Transit Gateway](/docs/enterprise/10.1/reference/ui/cloud-fabric/gateways#view-transit-gateways) to launch Aviatrix Transit gateway and enable HA with insane mode enabled in the AWS Transit VPC. * In this example, sizes c5n.2xlarge and c5n.4xlarge are selected to benchmark [performance](#performance-benchmarks). ### Enabling Route Propagation on the Subnet Route Table where Aviatrix Transit Gateway Locates on AWS Portal 1. Log in to the AWS VPC portal and locate the subnet route table where the Aviatrix Transit Gateway is located. 2. Select **Route Propagation** tab. 3. Click **Edit route propagation**. 4. Locate the AWS VGW that is associated with this Transit VPC and mark the **Propagate** checkbox. 5. Click **Save**. 6. Check that the Propagate status is **Yes**. ![aws\_route\_propagation\_status\_yes](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/aws-propagation-status-yes.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=7695fc2006e55370b537df53bbf4f39b) If the Propagation status is not Yes, you can enable it by clicking **Edit route propagation**. ### Deploying Spoke Gateway and HA [Deploy Spoke Gateways](https://legacy.docs.aviatrix.com/documentation/latest/network/spoke-gateway-attachments.html?expand=true) to launch Aviatrix Spoke gateway and enable HA with insane mode enabled in AWS Spoke VPC. In this example, sizes c5n.2xlarge and c5n.4xlarge are selected to benchmark [performance](#performance-benchmarks). ### Attaching Spoke Gateways to Transit Network [Attach Spoke Gateways to Transit Network](https://legacy.docs.aviatrix.com/documentation/latest/network/spoke-gateway-attachments.html?expand=true). ### Building Connectivity between Edge Router and Aviatrix Transit Gateway Cisco ASR is used as an edge router in this example. ### Checking Whether the edge router has Learned AWS Transit VPC CIDR via the BGP Session Between Edge Router and AWS Direct Connect 1. Log in to the edge router (i.e. Cisco ASR). 2. Check whether edge router has learned AWS Transit VPC CIDR via the BGP session between edge router and AWS Direct Connect by issuing the related "show ip bgp" command. Simple Cisco IOS example: ```text theme={null} #show ip bgp ``` ### Preparing IP for GRE source IP on Edge Router In this example, we use ASR loopback interface with an unique IP address as a GRE source IP. 1. Create a loopback interface and assign an IP to itself as a GRE source IP. Simple Cisco IOS example: ```text theme={null} #configure t (config)#interface Loopback77 (config-if)#ip address 192.168.77.1 255.255.255.255 ``` ### Advertising that GRE source IP on Edge Router to the BGP Session Between Edge Router and AWS Direct Connect The purpose of this step is to let AWS VGW learn the GRE source IP on edge router via BGP session between the edge router and AWS Direct Connect, so that the Aviatrix Transit Gateway can reach the GRE source IP on edge router to form a GRE tunnel over AWS Direct Connect. To demonstrate this concept, we utilize IOS "ip prefix-list" function and apply it on BGP neighbor with direction out function to distribute GRE source IP. 1. Create a prefix list that defines GRE source IP on edge router for BGP advertisement. Simple Cisco IOS example: ```text theme={null} #configure t (config)#ip prefix-list Router-to-VGW description Advertised GRE source CIDRs 192.168.77.X/32 to build GRE tunnels (config)#ip prefix-list Router-to-VGW seq 10 permit 192.168.77.1/32 ``` 2. Apply this prefix list to outgoing BGP advertisements Simple Cisco IOS example: ```text theme={null} #configure t (config)#router bgp 65000 (config-router)#address-family ipv4 (config-router-af)#neighbor 169.254.253.17 prefix-list Router-to-VGW out ``` The IP 169.254.253.17 in this example is the AWS Direct Connect BGP Peer IP. ### Checking Route Propagation Info on AWS Portal 1. Log in to the AWS VPC portal and locate the subnet route table where Aviatrix Transit Gateway is located. 2. Select the VPC > Route tables tab. 3. Check whether there is a route entry "GRE source IP on edge router pointing to AWS VGW." ## Confirming that Edge Router and Aviatrix Transit Gateway can Reach each other's IP for GRE Tunnel In Aviatrix CoPilot, go to **Networking** > **Connectivity** > **External Connections (S2C)** tab. 1. Click **+ Connection**. 2. Enter the following values: | Parameter | Description | | - | - | | Name | Enter a unique name to identify the connection to the external device. | | Connect Public Cloud To | Select the **External Device** radio button. Click on the dropdown menu and select **BGP over GRE**. | | Local Gateway | Select the Transit Gateway. | | Local ASN | Enter the BGP AS number the Transit Gateway will use to exchange routes with the external device. | | Remote ASN | Enter the BGP AS number that the edge router will use to exchange routes with the Transit Gateway. | | Over Private Network | Enable this option since AWS Direct Connect is the underlay network. | | Learned CIDR Approval | Enable this setting to set up an approval process for gateway learned CIDRs for this BGP external connection. This approval process improves security for your network. If an unapproved CIDR address attempts to access the connection, CoPilot sends an approval email to the CoPilot admin so that the admin can approve or block access. | | ActiveMesh Connection | | | Remote Gateway | Click here to add a remote or on-prem gateway instance. | | Remote Gateway Instance IP | Enter the IP address of the remote or on-prem device. | | Local Tunnel IP | Leave it blank in this example. | | Remote Tunnel IP | Leave it blank in this example. | 3. Click **Connect** to generate GRE tunnel and its BGP session. ### Downloading the GRE Configuration Sample In CoPilot, go to the Networking\* > Connectivity > External Connections (S2C) tab. 1. Find the connection you created and click the vertical ellipsis ![25](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/threedot.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=36963cd7f65aa469c6d5fe2021fa71a7). 2. Click **Download Configuration**. 3. Enter the following values: | Parameter | Description | | - | - | | Vendor | Select Cisco. | | Platform | Select ISR, ASR, or CSR. | | Software | Select IOS(XE) for this example. | 4. Click **Download**. ### Configuring GRE tunnel on Edge Router 1. Open the downloaded GRE configuration file. 2. Populate these values as follows based on your setup throughout the Tunnel Interface Configuration. * **\**: the primary GRE tunnel interface number connecting Aviatrix Transit Primary Gateway (i.e. 11) * **\**: the secondary GRE tunnel interface number connecting Aviatrix Transit HA Gateway (i.e. 12) * **\**: the IP which is assigned on the Loopback interface as an GRE source IP (i.e. 192.168.77.1) * **\**: the IP which is assigned on the Loopback interface as an GRE source IP (i.e. 192.168.77.1) 3. Copy and paste the updated Tunnel Interface Configuration into the edge router. Simple Cisco IOS example: ```text theme={null} interface Tunnel 11 ip address 169.254.61.205 255.255.255.252 ip mtu 1436 ip tcp adjust-mss 1387 tunnel source 192.168.77.1 tunnel destination 10.1.0.185 ip virtual-reassembly no keepalive exit interface Tunnel 12 ip address 169.254.173.77 255.255.255.252 ip mtu 1436 ip tcp adjust-mss 1387 tunnel source 192.168.77.1 tunnel destination 10.1.1.27 ip virtual-reassembly no keepalive exit ``` ### Configuring BGP over GRE tunnel on Edge Router 1. Open the downloaded GRE configuration file and copy and paste the BGP Routing Configuration into edge router. Simple Cisco IOS example: ```text theme={null} router bgp 65000 bgp log-neighbor-changes neighbor 169.254.61.206 remote-as 65212 neighbor 169.254.61.206 timers 10 30 30 neighbor 169.254.173.78 remote-as 65212 neighbor 169.254.173.78 timers 10 30 30 ! address-family ipv4 redistribute connected neighbor 169.254.61.206 activate neighbor 169.254.61.206 soft-reconfiguration inbound neighbor 169.254.173.78 activate neighbor 169.254.173.78 soft-reconfiguration inbound maximum-paths 4 exit-address-family ``` 2. Create a prefix list that defines CIDR where server locates in on-prem/co-location for BGP advertisement. Simple Cisco IOS example: ```text theme={null} #configure t (config)#ip prefix-list Router-To-Transit-GRE description Advertised on-prem CIDRs 10.220.5.0/24 (config)#ip prefix-list Router-To-Transit-GRE seq 10 permit 10.220.5.0/24 ``` 3. Apply the prefix list to outgoing BGP advertisements. Simple Cisco IOS example: ```text theme={null} #configure t (config)#router bgp 65000 (config-router)#address-family ipv4 (config-router-af)#neighbor 169.254.61.206 prefix-list Router-To-Transit-GRE out (config-router-af)#neighbor 169.254.173.78 prefix-list Router-To-Transit-GRE out ``` ### Verifying GRE Tunnel Status on Aviatrix Controller 1. In CoPilot, go to Networking > Connectivity > External Connections (S2C) tab. 2. Find the connection that you created. The green dot next to the Connection Name indicates the connection is Up. ![aviatrix\_gre\_status\_1](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/aviatrix-gre-status-one.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=7b105c01e2c8e018ab7c16348c56c07c) 3. Go to Cloud Fabric > Gateways > Transit Gateways. 4. Select the Transit Primary Gateway that was created in the previous step. 5. Click **Details/Diag**. 6. Click the Connections sub-tab. 7. Find the connection that you created with Connection Name in the previous step and check the Tunnel Status. ### Verifying BGP Session Status 1. Go to Diagnostics > Cloud Routes > BGP Info. 2. Find the connection that you created with Connection Name in the previous step and check the status. The Status should be Established. If some external connections for the selected Transit Gateway are Not Established, the overall BGP Status for the Transit Gateway is Partially Established. ## Configuring ECMP Load Balancing for High Performance ### Building Multiple GRE tunnels between Edge Router and Aviatrix Transit Gateway 1. Building multiple GRE tunnels by repeating [Build connectivity between edge router and Aviatrix Transit Gateway](#building-connectivity-between-edge-router-and-aviatrix-transit-gateway). 2. Build multiple BGP over GRE tunnels by repeating [Build GRE tunnel and BGP over GRE](#confirming-that-edge-router-and-aviatrix-transit-gateway-can-reach-each-others-ip-for-gre-tunnel). In this example, we build up to 4 pairs of GRE connections (total up to 8 tunnels) to benchmark [performance](#performance-benchmarks). ### Enabling BGP ECMP feature on Aviatrix Transit Gateway 1. In CoPilot, go to Cloud Fabric > Gateways > Transit Gateways and select the Transit Gateway that was created in the previous step. 2. On the Settings sub-tab, under Border Gateway Protocol, toggle the BGP ECMP setting to **On**. ### Verifying BGP ECMP feature on Aviatrix Controller 1. With the same Transit Gateway selected, click the VPC/VNet Route Tables tab. 2. Select the Transit Primary Gateway that was created in the previous step. 3. Click the Gateway Routes sub-tab. 4. Search for the on-prem CIDR in the Destination column. 5. Check whether there are multiple GRE tunnels with same Metric and Weight under the same route entry. ![aviatrix\_gre\_bgp\_verify\_ecmp\_function](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/aviatrix-gre-verify-ecmp.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=c66f36fc70ccadd5f1c006d0ac8a1ef0) ### Enabling the BGP ECMP feature on Edge Router 1. Configure "maximum-paths" with higher number of equal-cost routes in BGP settings so that BGP will install in the routing table. In this example, we configure "maximum-paths 8" to achieve high performance over multiple GRE tunnels. Simple Cisco IOS example: ```text theme={null} #configure t (config)#router bgp 65000 (config-router)#address-family ipv4 (config-router-af)#maximum-paths 8 ``` 2. Modify ECMP Load Balancing algorithm depending on traffic type. Simple Cisco IOS example: ```text theme={null} #configure t (config)#ip cef load-sharing algorithm include-ports source destination ``` ### Verifying the BGP ECMP feature on Edge Router Check whether BGP install equal-cost routes in the routing table by issuing the related command "show ip bgp." ![asr\_gre\_bgp\_verify\_ecmp\_function](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/asr-gre-verify-ecmp.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=fdeabfb74aac7d86509da2ae2e60929b) ## Ready to Go At this point, run connectivity and performance test to ensure everything is working correctly. ## Performance Benchmarks ### End-to-End traffic via Aviatrix to Cisco ASR #### Multiple flows result by using iperf3 tool with TCP 128 connections | Aviatrix Gateway size | 3 pairs of GRE connections (total 6 tunnels) | 4 pairs of GRE connections (total 8 tunnels) | | - | - | - | | C5n.2xlarge | 8.0 - 8.3 (Gbps) | 8.3 - 9.1 (Gbps) | | C5n.4xlarge | 9.0 - 9.3 (Gbps) | 9.2 - 9.3 (Gbps) | #### Single flow result by using iperf3 tool with TCP 1 connection: 1.6 - 2.4 (Gbps) for both sizes C5n.2xlarge and C5n.4xlarge # Transit Gateway Integration with AWS VGW Workflow Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/vpn/transit-externaldevice-aws-vgw Aviatrix automates the process of discovering and connecting Aviatrix Transit Gateway to AWS VGW. Aviatrix automates the process of discovering and connecting Aviatrix Transit Gateway to AWS VGW. Before connecting an Aviatrix Transit Gateway to AWS VGW, a VGW must have already been created on the AWS console. In the AWS console, note the VGW ID that is created for the VGW. When you setup the connection by providing the VGW ID, a Customer Gateway and a Site2Cloud connection between the VGW and the Aviatrix Transit Gateway will be automatically created. The Site2Cloud IPsec tunnel establishes a BGP session to exchange routes between on-premises and the cloud. After successful creation of the connection, you can view connection in the AWS console by navigating to Customer Gateways and Site-to-Site VPN Connections. > [!TIP] > > You are responsible for building the connection between VGW and on-premises. > The connection is either over the Internet, over Direct Connect, or both. Aviatrix supports two types of connections: Detached VGW and Attached VGW. > [!NOTE] > > The VGW should not be attached to the Transit VPC or VNet. > > The VGW can be attached to a VPC or VNet in a different AWS account in a > different region. ![Transit Gateway to VGW connection](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/connect-vgw.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=6f7615e14b619494a6d37a387079f5ba) ## Connecting Transit Gateway to AWS VGW This document describes the workflow to connect an Aviatrix Transit Gateway to an AWS Virtual Private Gateway (VGW). The procedure below assumes the AWS VGW is already deployed in the Transit VPC. **CoPilot UI** To connect the Transit Gateway to AWS VGW (VPN Gateway): 1. In Aviatrix CoPilot, go to **Networking > Connectivity > External Connections (S2C)** tab. 2. From the **+ External Connection** dropdown menu, select **AWS Virtual Gateway**. 3. Enter the following values: | Field | Description | | - | - | | Name | A name to identify the connection to the VGW. | | Local Gateway | The Transit Gateway to connect to the VGW. | | Local ASN | The BGP AS Number the Transit Gateway will use to exchange routes with the VGW. | | VGW Account Name | The name of the AWS access account with which the VGW was created. | | VGW Region | The AWS region where the VGW is located. | | VGW ID | The unique identifier for the VGW. | | Manual CIDR Approval | This is Off and disabled by default unless the Local Gateway you select has **[Manual Learned CIDR Approval](https://legacy.docs.aviatrix.com/documentation/latest/network/bgp-connection-settings.html#gateway-learned-cidr-approval)** turned On for Connection Level. Then it is On by default (not editable). | 4. Click **Save**. The new AWS VGW connection appears in the table. It takes a few minutes for the VPN connection to come up and routes from VGW to be propagated. When the IPsec tunnel with a VGW is up, the Controller admin should receive an email notification. If you log in to the AWS Console and select *service VPC* in the region where the VGW is, you should see that the Customer Gateway and VPN Connections have been created. Do not delete or modify them from AWS Console. These resources are deleted if you disconnect the VGW. **Terraform** Create the connection with the [aviatrix\_vgw\_conn](https://registry.terraform.io/providers/AviatrixSystems/aviatrix/latest/docs/resources/aviatrix_vgw_conn) resource. This resource requires an existing Aviatrix Transit Gateway and an existing AWS VGW. **1. Create the AWS VGW connection** Add the following `aviatrix_vgw_conn` resource block to your Terraform configuration file: ```hcl theme={null} resource "aviatrix_vgw_conn" "aws_vgw_conn" { conn_name = "aws-vgw-conn-01" gw_name = "transit-gw-01" vpc_id = "vpc-abcdef" bgp_vgw_id = "vgw-0123456789abcdef0" bgp_vgw_account = "aws_account_1" bgp_vgw_region = "us-east-1" bgp_local_as_num = "65001" } ``` > [!NOTE] > > `gw_name` is the name of the Aviatrix Transit Gateway, and `vpc_id` is > the ID of the Transit VPC that gateway is in. `bgp_vgw_id` is the AWS > VGW ID — the VGW must already exist in AWS and must not be attached to > the Transit VPC/VNet. **2. Verify and Apply the Configuration** Run `terraform plan`, review the planned changes, and run `terraform apply`. ## Related Resources * [aviatrix\_vgw\_conn Resource](https://registry.terraform.io/providers/AviatrixSystems/aviatrix/latest/docs/resources/aviatrix_vgw_conn) ## Checking Route Propagation You can check if routes are properly propagated. 1. Go to **Diagnostics > Cloud Routes > BGP Info**. 2. In the table, next to the name of the Transit Gateway used in the above external connection. 3. Click the vertical ellipsis ![vertical ellipsis](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/threedot.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=36963cd7f65aa469c6d5fe2021fa71a7) and select **Show BGP Learned Routes** to display the total number and list of routes propagated from the VGW. ## Checking Tunnel Status You can check the tunnel connection between the AWS VGW and the Transit Gateway from **Diagnostics > Cloud Routes > BGP Info**. Expand the Transit Gateway in the list and check that the Status column shows Established. If some external connections for the selected Transit Gateway are Not Established, the overall BGP Status for the Transit Gateway is Partially Established. ## Related Topics * [Viewing External Connection Details](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-details.html?expand=true) * [Disconnecting an External Connection](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-disconnect.html) # Transit Gateway Integration with Azure VNG Workflow Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/vpn/transit-externaldevice-azure-vng Currently, Aviatrix Multicloud Transit solution requires encryption for connectivity over Azure ExpressRoute or when connecting directly from an external device to on-premises. Currently, Aviatrix Multicloud Transit solution requires encryption for connectivity over Azure ExpressRoute or when connecting directly from an external device to on-premises. However, there are scenarios where encryption is not required and native network connectivity over ExpressRoute is preferred. In such cases, the Aviatrix Transit solution, including Transit FireNet, can only forward traffic between Spoke VNets or inspect east-west traffic, as shown [by these common Azure Transit Network design patterns](/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/planning-azure) . ## Integration with Azure VNG Multicloud Transit Integration with Azure Virtual Network Gateway (VNG) enables Aviatrix Multicloud Transit solution to work cohesively with native Azure Virtual Network Gateway (VNG). This integration allows the Aviatrix Transit Gateway to inspect not only east-west and egress traffic, but also traffic from on-premises to cloud. Both native Spoke VNets and Aviatrix Spoke Gateway based Spoke VNets are supported. ## Data Packet Forwarding and Traffic Inspection Data packets are forwarded natively to on-premises through Azure Virtual Network Gateway (VNG), using either ExpressRoute or Internet connectivity. Aviatrix Transit Gateways are deployed in the data path between VNG and Spoke VNets, enabling advanced functions such as firewall inspection for traffic moving from on-premises to Spoke and between the Spokes. ## Configurtion Workflow Overview This setup is illustrated by a network design featuring two Spoke VNets: one with an Aviatrix Spoke Gateway (172.60.0.0/16) and one native Spoke VNet (172.50.0.0/16). ![topology\_expressroute](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/topology-expressroute.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=b32ab7b03378405e90123a2566127673) The key ideas for this solution are: * The edge (WAN) router establishes a BGP session with Azure VNG via ExpressRoute or VPN. The edge router advertises on-premises routes to Azure VNG, while Azure VNG advertises Spoke VNet CIDRs. * The Aviatrix Controller periodically retrieves route entries from the Transit VNet VNG route table—these routes are advertised from on-premises. The Controller then propagates these routes to both Spoke VNets and the Aviatrix Transit Gateway. * Azure native VNet Peering is configured between each Spoke VNet and the Transit VNet VNG. The *Allow Remote Gateway* attribute is set on the Spoke VNets to automatically advertise routes from the Spoke VNets to VNG to on-premises. * Traffic arriving from on-premises to the VNG is routed to the Azure Load Balancer, which then forwards traffic to both Aviatrix Transit Gateways (for active-mesh deployment). The same Load Balancer also distributes traffic to firewalls for inspection. * Traffic originating from Spoke VNet is routed directly to Aviatrix Transit Gateway, which then forwards the traffic to the Azure Load Balancer. ## Prerequisite [Upgrade](/docs/enterprise/10.1/guides/maintenance/controller-upgrade-planning) Aviatrix Controller to the latest version. > [!TIP] > > We highly recommend to > [create an Azure Transit VNET](/docs/enterprise/10.1/reference/ui/cloud-resources/cloud-networks#create-vpcvnet) > with the *Transit + FireNet* option enabled, and create a VNG in this Transit > VNet. ## Connect VNG on On-Premises > [!NOTE] > > If you have already created the VNG in Transit VNet, skip this section. Setting up Azure ExpressRoute is your responsibility. For details, see: * [Azure ExpressRoute landing page](https://azure.microsoft.com/en-us/products/expressroute/) * [ExpressRoute documentation](https://docs.microsoft.com/en-us/azure/expressroute/) * [Equinix ECX Fabric Microsoft Azure ExpressRoute](https://docs.equinix.com/fabric/connecting-service-provider/microsoft/azure-ne/) if you select the Equinix solution. Follow these steps to configure the workflow. Adjust the network topology as needed for your specific requirements. 1. Create an ExpressRoute circuit. See [Tutorial: Create and modify an ExpressRoute circuit](https://docs.microsoft.com/en-us/azure/expressroute/expressroute-howto-circuit-portal-resource-manager). 2. Create Azure private network for your ExpressRoute circuit. Refer to the private peering section in [Create and modify peering for an ExpressRoute circuit](https://docs.microsoft.com/en-us/azure/expressroute/expressroute-howto-routing-portal-resource-manager). 3. [Create a VNG in Transit VNet](/docs/enterprise/10.1/reference/ui/cloud-resources/cloud-networks#create-vpcvnet) with *Transit + FireNet* enabled. > [!NOTE] > > This process may take up to 45 minutes to complete. For more information, see [Configure a virtual network gateway for ExpressRoute using the Azure portal](https://docs.microsoft.com/en-us/azure/expressroute/expressroute-howto-add-gateway-portal-resource-manager). 4. Connect a virtual network to an ExpressRoute circuit. See [Connect a virtual network to an ExpressRoute circuit using the portal](https://docs.microsoft.com/en-us/azure/expressroute/expressroute-howto-linkvnet-portal-resource-manager). 5. Check ExpressRoute Circuits by accessing the List Routes Table on the Azure portal. ### Check ExpressRoute Circuits Routes Table in Azure 1. Log in to the Azure portal and search for "ExpressRoute circuits" using the search bar. 2. Select the ExpressRoute circuit that you created. 3. Select the Azure private peering row. 4. Select *Get route table* to verify routes learned from on-premises. ## Connect Aviatrix Transit Gateway with VNG Refer to [Global Transit Network Workflow](/docs/enterprise/10.1/guides/connectivity/transit/transitvpc-workflow) instructions for the following steps. Adjust the network topology as needed for your specific requirements. ### Deploy an Aviatrix Multicloud Transit Gateway and HA in Azure 1. [Deploy the Aviatrix Transit Gateway in the Azure Transit VNet](/docs/enterprise/10.1/guides/connectivity/transit/transitvpc-workflow) with High Availability and High Performance Encryption (HPE) enabled. HPE is not required but an optional feature to increase throughput. A minimum instance size of Standard\_D5\_v2 is required for [High Performance Encryption](/docs/enterprise/10.1/concepts-architectures/components/security/hpe-about) to achieve higher throughput. For detail performance information, refer to [High Performance Encryption Performance Benchmarks](/docs/enterprise/10.1/reference/gateways/hpe-perf-benchmark) . 2. [Add Transit FireNet to the Transit Gateway](/docs/enterprise/10.1/guides/security/firenet/transit-firenet-workflow-azure) . ### Connect Transit Gateway to Azure VNG # Connect Transit Gateway to Azure VNG > [!NOTE] > > This procedure assumes the Azure VNG is already deployed in the Transit VNet. To connect the Transit Gateway to Azure VNG (VPN Gateway): 1. In Aviatrix CoPilot, go to **Networking** > **Connectivity** > External Connections (S2C) tab. 2. From the **+ External Connection** dropdown menu, select **Azure Virtual Network Gateway**. 3. Enter the following values: | **Field** | **Description** | | - | - | | Name | A name to identify the connection to VNG (VPN Gateway). | | Aviatrix Gateway | The Transit Gateway to connect to the VNG (VPN Gateway). \[!NOTE] If Azure VNG has not been deployed in the Transit VNet, the connection process cannot complete.\\ | | VNG Name | The name of the Azure VNG (VPN Gateway). | 4. Click **Save**. The new Azure VNG connection appears in the table. **External Connection to Azure Virtual Network Gateway** | **Field** | **Description** | | - | - | | Name | A name to identify the connection to VNG (VPN Gateway). | | Aviatrix Gateway | The Transit Gateway to connect to the VNG (VPN Gateway). \[!NOTE] If Azure VNG has not been deployed in the Transit VNet, the connection process cannot complete.\\ | | VNG Name | The name of the Azure VNG (VPN Gateway). | ### Check Effective Routes Info on Azure Portal 1. Log in to the Azure portal and search for "Network interfaces" using the search bar. 2. Select the interface associated with the Aviatrix Transit Gateway. 3. Go to the *Effective routes* page by selecting *Effective routes* under the Support + troubleshooting section. 4. Review the route entry for On-premises, where the *Next Hop Type* should be *Virtual network gateway*. ![azure\_effective\_routes\_routing\_entry](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/azure-effective-routing-entry.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=3a98c8655108ded0d7a5b9efab16465c) ## Attach Spoke VNet to Aviatrix Transit Gateway 1. Deploy an Aviatrix Spoke Gateway in a Spoke VNet. You may create an Azure VNET for Aviatrix Spoke Gateway by using the Aviatrix feature [Create a VNG in Transit VNet](/docs/enterprise/10.1/reference/ui/cloud-resources/cloud-networks#create-vpcvnet) or manually deploy it via the cloud portal. Alternatively, you can use existing virtual network. 2. [Deploy a Spoke Gateway in the Azure Spoke VNet](/docs/enterprise/10.1/guides/connectivity/transit/transitvpc-workflow) , ensuring High Availability is enabled. High Performance Encryption (HPE) may also be enabled if required, though it is optional. > [!NOTE] > > For optimal throughput with HPE mode, select an instance size of at least > Standard\_D5\_v2. For detail performance information, see > [High Performance Encryption Performance Benchmarks](/docs/enterprise/10.1/reference/gateways/hpe-perf-benchmark) > . 3. Attach the Spoke Gateways to the Transit Network. 1. [Attach Spoke Gateways to Transit Network (Azure)](/docs/enterprise/10.1/guides/connectivity/transit/transitvpc-workflow) 2. To attach the Azure Native VNet Spoke to Aviatrix Transit Gateway, follow the steps in [Attach Native Azure VNET to Transit Network](/docs/enterprise/10.1/guides/connectivity/transit/transitvpc-workflow) . ## Ready to Go You can now send traffic between cloud to on-premises via Azure ExpressRoute. For FireNet deployment, follow the [Transit FireNet workflow](/docs/enterprise/10.1/guides/security/firenet/transit-firenet-workflow-azure) . ## Limitations Only native Spoke VNet peering routes are advertised from VNG to on-premises; non-native Spoke and Transit-to-Transit peerings are not included. # BGP-Enabled Spoke Gateway to External Devices Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/vpn/transit-externaldevice-bgpspoke You can connect an external (or 3rd party) device ( Router or Firewall) to Aviatrix Spoke gateways that are enabled with BGP (and NAT). You can connect an external (or 3rd party) device ( Router or Firewall) to Aviatrix Spoke gateways that are enabled with BGP (and NAT). BGP is run on top of a Site-to-Cloud (S2C) connection that terminates on ActiveMesh Spoke gateways. This document describes connecting an external device to an Aviatrix Spoke gateway that is BGP-enabled. Using BGP-enabled Spoke gateways is currently supported for AWS Commercial and Azure Commercial cloud service providers, including Government regions. ## What is a use case for connecting a BGP-enabled Spoke gateway to an external router? Previously, Software-as-a-Service (SaaS) providers with certain architectures had to deploy Transit gateways for each of their tenants to enable the tenant to connect to their in-cloud network. By using BGP-enabled Spoke gateways in their network architecture, SaaS providers can solve several requirements: * **Requirement**: Connect a large number of tenants (1000+). **Solution**: Distribute the tenants across Spoke gateways, for horizontal scaling and blast radius minimization. * **Requirement**: Provide both dedicated tenant services, and shared services. **Solution**: Host dedicated services in tenant-specific Spoke VPCs. Host shared services in common Spoke VPCs. * **Requirement**: Onboard the tenants with BGP: dynamic control plane that fits their operational model. **Solution**: Terminate BGP on the tenant Spoke gateways. * **Requirement**: Handle overlapping IPs across tenants, and between tenants and shared services. **Solution**: Use NAT on the tenant Spoke gateways. * **Requirement**: Maintain isolation across tenants. **Solution**: Use segmentation domains on the tenant Spoke gateways. * **Requirement**: Provide the highest throughput to tenant services. **Solution**: Horizontal scaling. Tenant services are directly hosted in the Spoke VPC where BGP terminates. They are directly accessed by tenants, without the transit layer to be a bottleneck. ![spokegw\_external\_saas\_sol](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/spokegw-external-saas-sol.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=cb8730604eafd7528f39770c19bc5b5c) ## How does using a BGP-enabled Spoke gateway to an external device work? The Aviatrix Spoke gateway runs a BGP session to an external router to dynamically exchange routes. It also establishes an IPsec tunnel to the router for packet forwarding. BGP is run on top of a S2C connection that terminates on active-mesh Spoke gateways. All Spoke gateways must be active-mesh (no standalone gateway). Each Spoke gateway must have a unique Autonomous System (AS) number. The following features are supported: * Fully integrated with ActiveMesh 2.0 control plane. * Route-based only. * Active/Active HA is supported for ActiveMesh and for on-prem with ECMP across multiple BGP connections. Active/Standby S2C is also supported. * Co-existence of BGP S2C with static S2C connections under the same Spoke gateway is supported. * FireNet is supported. The inspection policy is the entire Spoke gateway including all the BGP routes (not individual S2C BGP sessions). The following features are not supported on a BGP-enabled Spoke gateway: * Static Route-Based (Mapped) external connection) * UserVPN * Egress transit * Customize Spoke VPC Routing Table * Private VPC Default Route * Skip Public VPC Route Table * Select Route Tables upon Spoke Gateway Attachment to Transit The following features configured on a Transit Gateway have no effect on a Spoke VPC equipped with a BGP-enabled Spoke gateway (they still work for any other Spoke VPC): * Customize Attached Spoke VPC Routing Table * Exclude Learned CIDRs to Spoke VPC ![spokegw\_external\_ex\_arch](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/spokegw-external-ex-arch.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=2778e9e85a08c42162d0d48806c99460) ### Interactions with Segmentation Domains When using BGP-enabled Spoke gateways, SaaS providers can use segmentation domains per Spoke gateway to enforce isolation across tenants. When segmentation domains are set per BGP-enabled Spoke gateway, the Site-to-Cloud (S2C) BGP connection respects the domain of the Spoke gateway for traffic enforcement and route advertisement. All S2C connections on a given Spoke gateway belong to the Spoke gateway domain (currently, you cannot have different S2C connections on a given Spoke gateway be assigned to different domains). BGP routes of a tenant are always advertised to all other tenants connected with S2C BGP under the same Spoke gateway. This cannot be controlled with segmentation policies. It is controlled using the [Connection Manual BGP Advertised Network List](https://legacy.docs.aviatrix.com/documentation/latest/network/bgp-connection-settings.html?expand=true). BGP routes of a tenant are propagated into ActiveMesh based on the connection policies of the Spoke gateway. ActiveMesh routes are advertised over BGP based on the connection policies of the Spoke gateway. ### Interactions with NAT * Customized NAT under Gateway config is supported (mapped NAT under S2C config is not currently supported). * S2C BGP connections are available as option in the NAT connection. * ActiveMesh connections are available in the NAT connection but ONLY for non-HPE spoke gateways. * Many:1 and 1:1 NAT are possible. * Active - Active HA for both gateways and S2C connections (with flow affinity) is supported. This means that return traffic goes to the same Spoke Gateway that performed NAT (not valid for Mapped NAT). ### Route Propagation Spoke VPC CIDR + BGP prefixes received on the Spoke gateway are propagated into ActiveMesh (Subnets outside of RFC 1918 are programmed on the VPC RTBs). All CIDRs known to ActiveMesh (Spoke VPCs for all regions and clouds + BGP prefixes + custom advertisements, etc.) are advertised over BGP on the Spoke Gateway S2C BGP connections. ![bgp\_spoke\_route\_propagation](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/bgp-spoke-route-propagation.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=d7909d6be8b3e481d2b001c082c187d7) ### Connected Transit The propagation of BGP routes learned on a Spoke Gateway to other Spoke Gateways under the same Transit complies with Connected Transit. If Connected Transit = Disabled, those routes are not propagated to other Spoke Gateways under the same Transit. In this example, 192.168.200.0/25 learned via BGP on Spoke-1-GW is not propagated to Spoke-2-GW: ![bgp\_spoke\_connected\_transit](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/bgp-spoke-connected-transit.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=38bcee7aeeafb2951c6bfe7090b0842a) ## How do I configure a BGP-enabled Spoke gateway and connect it to an external router? This section describes how to: * Create a Spoke Gateway that is BGP enabled. * Create the S2C BGP tunnel (build a Site2Cloud IPsec BGP attachment for the newly created spoke). * Configure your router with the connection details. * Configure additional settings. ### Create a BGP-Enabled Spoke Gateway > [!NOTE] > > BGP must be enabled at the creation of the Spoke Gateway. A Spoke Gateway > enabled with BGP has a few restrictions compared to a non-BGP Spoke. For > information about these restrictions, see [How does using a BGP enabled spoke > gateway to an external device > work?](#how-does-using-a-bgp-enabled-spoke-gateway-to-an-external-device-work) 1. Navigate to Cloud Fabric > Gateways > Spoke Gateways and [complete these steps](https://legacy.docs.aviatrix.com/documentation/latest/network/spoke-gateway-about.html?expand=true). 2. Ensure that the **BGP** toggle is set to On. You can enable HA for the Spoke gateway if desired. For best practice, the HA gateway should be launched on a different public subnet in a different Availability Zone. > [!NOTE] > > If the Spoke gateway is connected to VGW, you cannot disable the Spoke HA > gateway. 3. After saving the Spoke gateway, navigate to Networking > Connectivity > External Connections (S2C). This is where you connect the Spoke gateway to the external device (device in an on-prem network). In this case, you will build a site-to-cloud (S2C) BGP over IPsec connection as instructed in the next section. ### Create the S2C BGP Tunnel You use the External Device option on the Spoke Gateway to build a BGP tunnel directly to the on-prem device for exchanging routes with a remote site. To create the S2C BGP tunnel: 1. On the Connectivity > External Connections (S2C) tab, click **+External Connection**. 2. In the Add External Connection dialog, enter a name for the connection and select **Connect Public Cloud to External Device**. 3. From the **External Device** list, select one of the BGP options as described in the below table. 4. Specify the rest of the parameters (defined below) and click **Save**. Remember that some of the parameters described below are not applicable to BGP. 5. Fill out the parameters required for your BGP selection and click **OK**. For ActiveMesh design notes, see [ActiveMesh Design Notes](/docs/enterprise/10.1/concepts-architectures/components/connectivity/activemesh-design-notes) . | **Setting** | **Value** | | - | - | | Connect Public Cloud to | External Device | | External Device | BGP over IPsec: select to run BGP and build an IPsec connection to a remote site. BGP over GRE: select to run BGP and build a GRE connection to a remote site. BGP over LAN: select to run BGP and data plane via a LAN interface in the same VPC/VNet. | | Local Gateway | The Spoke gateway you created (must have BGP enabled). | | Local ASN | The BGP AS number the Spoke gateway will use to exchange routes with the external device. | | Remote ASN | When BGP is selected, the remote ASN for backup should be the same as the primary remote ASN. The Remote ASN is the BGP AS number the external device will use to exchange routes with the Transit Gateway. | | BGP ActiveMesh | Enable full mesh BGP connections to the external devices. | | Over Private Network | Select this option if your underlying infrastructure is private network, such as AWS Direct Connect and Azure Express Route. See [How does using a BGP-enabled Spoke gateway to an external device work?](#how-does-using-a-bgp-enabled-spoke-gateway-to-an-external-device-work) for more information. When this option is selected, BGP and IPSEC run over private IP addresses. | | IKEv2 | Select the option to connect to the remote site using IKEv2 protocol. | | Algorithms | Optional parameters for configuring Phase 1 and Phase 2 Authentication, DH Groups, and Encryption. | | Learned CIDR Approval | If BGP is enabled for the selected Spoke gateway, this field is set to On by default. Only Gateway Mode is supported for BGP Spoke Gateways; Connection Mode is not available. See [Gateway Learned CIDR Approval](https://legacy.docs.aviatrix.com/documentation/latest/network/bgp-connection-settings.html?expand=true#gateway-learned-cidr-approval) for more information. | | Pre-shared Key (if Authentication Method is available) | Optional parameter. Leave it blank to auto-generate the pre-shared key. | | Remote Device IP | IP address of the remote device. | | Local Tunnel IP | Optional parameter. This field is for the tunnel inside IP address of the Spoke gateway. Leave it blank. | | Remote Tunnel IP | Optional parameter. This field is for the tunnel inside IP address of the External device. Leave it blank. | | Remote LAN IP (if BGP over LAN is selected) | The private IP of the LAN interface of the third-party cloud primary instance. | | Local LAN IP (if BGP over LAN is selected) | Aviatrix detects the Local LAN IP automatically. | | VPC Name/Site ID | The Spoke VPC ID where Transit GW was launched. | | Connection Name | A unique name to identify the connection to the external device. | | Remote Gateway | Select HA if there are two external devices. Only available for BGP over IPsec, BGP over GRE, Static Route-Based (ActiveMesh). | ### Configure the External Device To configure the external device: 1. On the External Connections (S2C) tab, click the vertical ellipsis ![vertical ellipsis](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/threedot.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=36963cd7f65aa469c6d5fe2021fa71a7) and then select **Download Configuration**. 2. In the Download Configuration dialog, select the following: * Vendor: select the device you are using (any device that is capable of running IPsec and BGP). * Platform: select the applicable platform for the chosen device. * Software: automatically selected based on the Vendor/Platform you select. 3. Click **Download**. 4. Open the downloaded configuration template. 5. Use the information from the template to populate the necessary areas of your external device (for example, your FortiGate firewall or your Cisco router). ### Verify Connection Status Navigate to Diagnostics > Cloud Routes > External Connections. The Status and Tunnel Status columns indicate if the tunnel is Up or Down. ### Verify the BGP Routes Navigate to Diagnostics > Cloud Routes > BGP Info to verify the BGP routes. The Status should be Established. If some external connections for the selected Transit Gateway are Not Established, the overall BGP Status for the Transit Gateway is Partially Established. ### Customize Spoke Advertised VPC/VNet CIDRs You can customize Spoke Advertised VPC/VNet CIDRs for the Spoke gateway included in this configuration. The CIDRs are propagated into ActiveMesh and into BGP as belonging to the Spoke Gateway shown in the example. 1. Navigate to Cloud Fabric > Gateways > Spoke Gateways. 2. Select the Spoke gateway from this configuration. 3. On the Settings tab, on the Customize Spoke Advertised VPC/VNet CIDRs card, enter the CIDRs to advertise. These will display under Advertised Routes on the Diagnostics > Cloud Routes > BGP Info tab. 4. Click **Save**. ![spokegw\_external\_custom\_adv\_cidrs](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/spokegw-external-adv-cidrs.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=8fc42651092a8593f25991427ffa2717) ActiveMesh propagation: those CIDRs are combined with the BGP prefixes received on the S2C BGP connection(s) of the Spoke Gateway. BGP advertisement: those CIDRs are combined with all other ActiveMesh CIDRs from the Aviatrix Transit. ### Set Up BGP Route Control Also on the Settings tab under Border Gateway Protocol (BGP), you can configure the following [BGP route control settings](https://legacy.docs.aviatrix.com/documentation/latest/network/bgp-connection-settings.html?expand=true): * Local AS Number * BGP ECMP * Manual BGP Advertised CIDR List ### Delete the External Connection To disconnect the external device from the BGP-enabled Spoke gateway, on the Networking > Connectivity > External Connections (S2C) tab, click the Delete icon ![delete icon](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/delete-icon.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=59603279dfa3e866cdcd178c41de3347) next to the connection you want to delete. # Transit Gateway to Cisco ASA Over the Internet Workflow Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/vpn/transit-externaldevice-ciscoasa 2. Create a Transit gateway that will connect to the Cisco ASA. 3. To connect the Transit VPC gateway to the Cisco ASA, navigate to Networking > Connectivity > External Connections (S2C) and follow these instructions. 4. Download the configuration. 5. 1. In CoPilot, navigate to Cloud Fabric > Gateways > Transit Gateways. 2. [Create a Transit gateway](https://legacy.docs.aviatrix.com/documentation/latest/network/transit-gateway-about.html?expand=true) that will connect to the Cisco ASA. 3. To connect the Transit VPC gateway to the Cisco ASA, navigate to Networking > Connectivity > External Connections (S2C) and [follow these instructions](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-create-bgp-over-ipsec.html). 4. [Download the configuration](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-download-configuration.html). 5. Ensure that your Cisco ASA is configured based on the information in the downloaded file. The following is a sample configuration based on the external connection configuration above. ![Cisco ASA sample configuration](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/ciscoasa5.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=8dba4d19181f180dfadb8b1d9ea112d2) > [!NOTE] > > The tunnel IP addresses are configured accordingly with the configuration > file downloaded above. 6. After configuring the Cisco ASA, in CoPilot navigate to Diagnostics > Cloud Routes > External Connections to confirm that the tunnel status has changed from Down to Up. 7. Navigate to Diagnostics > Cloud Routes > BGP Info tab to verify the BGP routes. The Status should be Established. If some external connections for the selected Transit Gateway are Not Established, the overall BGP Status for the Transit Gateway is Partially Established. # Transit Gateway to Cisco Router over the Internet Workflow Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/vpn/transit-externaldevice-ciscorouter 2. Create a Transit gateway that will connect to the Cisco Router. 3. To connect the Transit VPC gateway to the Cisco Router, go to Networking > Connectivity > External Connections (S2C) and follow these instructions. 4. Download the configuration. 5. 1. In CoPilot navigate to Cloud Fabric > Gateways > Transit Gateways. 2. [Create a Transit gateway](/docs/enterprise/10.1/reference/ui/cloud-fabric/gateways#view-transit-gateways) that will connect to the Cisco Router. 3. To connect the Transit VPC gateway to the Cisco Router, go to Networking > Connectivity > External Connections (S2C) and [follow these instructions](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-create-bgp-over-ipsec.html). 4. [Download the configuration](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-download-configuration.html). 5. Ensure that your Cisco Router is configured based on the information in the downloaded file. The following is a sample configuration based on the Site2Cloud configuration above. ![Cisco Router Configuration Sample](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/cisco5.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=f27d1918cd3cd4c6422aab33cfce9797) > [!NOTE] > > The tunnel IP addresses are configured accordingly with the configuration file > downloaded above. 6. After configuring the Cisco Router, in CoPilot navigate to Diagnostics > Cloud Routes > External Connections to confirm that the tunnel status has changed from Down to Up. 7. Navigate to Diagnostics > Cloud Routes > BGP Info to check the BGP routes. The Status should be Established. If some external connections for the selected Transit Gateway are Not Established, the overall BGP Status for the Transit Gateway is Partially Established. * [Transit Gateway to External Devices Workflow](/docs/enterprise/10.1/guides/connectivity/vpn/transit-externaldevice-workflow) * [Transit Gateway to Cisco ASA over the Internet Workflow](/docs/enterprise/10.1/guides/connectivity/vpn/transit-externaldevice-ciscoasa) # Transit Gateway to FortiGate Over the Internet Workflow Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/vpn/transit-externaldevice-fortigate This is not a common scenario. You would only set up this type of connection if you want to connect an Aviatrix Transit gateway to a firewall that is outside your Cloud service provider (for example, in a branch office or warehouse). > [!NOTE] > > This is not a common scenario. You would only set up this type of connection > if you want to connect an Aviatrix Transit gateway to a firewall that is > outside your Cloud service provider (for example, in a branch office or > warehouse). 1. In CoPilot, navigate to Cloud Fabric > Gateways > Transit Gateways. 2. [Create a Transit gateway](/docs/enterprise/10.1/reference/ui/cloud-fabric/gateways#view-transit-gateways) that will connect to your FortiGate firewall. 3. To connect the transit VPC gateway to FortiGate, navigate to Networking > Connectivity > External Connections (S2C). 4. Click *+External Connection*. 5. In the Add External Connection dialog, configure the following: | Field | Value | | - | - | | Connect Public Cloud to | External Device > BGP over IPsec | | Local Gateway | Select the Transit gateway you created in step 2 above | | Local ASN | The BGP AS number the Transit gateway will use to exchange routes with the external device. | | Remote ASN | Enter the BGP AS number the external device will use to exchange routes with the Transit Gateway. | | Remote Device IP | This is the FortiGate WAN IP. | > [!NOTE] > > Ensure that the Local/Remote ASN and the Remote Device IP are correct before > saving. 6. Click *Save*. 7. [Download the configuration](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-download-configuration.html). The following is a sample configuration based on the Site2Cloud configuration above. ![Sample configuration based on Site2Cloud](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/fortigate4.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=fbbd6390ee1f6e1e3e49ed156878538d) ## Configuring the Fortinet FortiGate Firewall 1. Login into FortiGate and configure it as follows: 1. Navigate to VPN > IPsec Tunnels. 2. Click *+Create New*, and select *IPsec Tunnel*. 3. In the VPN Creation Wizard, select the Custom template type. 4. Populate the fields according to your preferences. 5. Click *Next*. ### VPN Setup | Field | Expected Value | | - | - | | Name | Any name | | Template Type | Custom | ![VPN Setup wizard](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/fortigate5.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=312ea4ebb33b9f492d25065d7e7d518a) ### New VPN Tunnel Tab 2. Complete the Network fields on the New VPN Tunnel tab as follows: **Network section of New VPN Tunnel tab** | Field | Expected Value | | - | - | | IP Version | IPv4 | | Remote Gateway | Static IP Address | | IP Address | Public IP address of Aviatrix Gateway | | Interface | Select the external port/interface | | Local Gateway | Disabled | | Mode Config | Unchecked | | NAT Traversal | Recommended: Enable | | Keepalive Frequency | Any value | | Dead Peer Detection | On Demand | ![Network section of New VPN Tunnel tab](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/fortigate6.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=dd2bbde1afe586bd54e2c9c1684f4816) **Authentication section of New VPN Tunnel tab** | Field | Expected Value | | - | - | | Method | Pre-Shared Key | | Pre-shared Key | In the Pre-shared Key field, enter the value from the Pre-Shared Key row in the downloaded configuration file. | | IKE Version | 1 | | IKE Mode | Main (ID protection) | ![Authentication section of New VPN Tunnel tab](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/fortigate7.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=9a1ce921566eb77838fae8b8145c840e) **Phase 1 Proposal section of New VPN Tunnel tab** | Field | Expected Value | | - | - | | Encryption | In the Encryption field, enter the value from the Encryption Algorithm row in the downloaded configuration file. | | Authentication | In the Authentication field, enter the value from the Authentication Algorithm row in the downloaded configuration file. | | Diffie-Hellman Group | Select the appropriate value as per the Perfect Forward Secrecy row in the downloaded configuration file. | | Key Lifetime (seconds) | 28800 | | Local ID | | ![Phase 1 Proposal section of New VPN Tunnel tab](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/fortigate9.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=bad6289d94faf847c5c707cf75fb1edb) **XAUTH section of New VPN Tunnel tab** | Field | Expected Value | | - | - | | Type | Disabled | ![XAUTH section of New VPN Tunnel tab](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/fortigate10.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=94c4bc87c423ab65dae0b706f55029ed) **Phase 2 Selectors > New Phase 2 section of New VPN Tunnel tab** | Field | Expected Value | | - | - | | Name | Any string value | | Comments | Any string value | | Local Address | 0.0.0.0/0 | | Remote Address | 0.0.0.0/0 | ![Phase 2 Selectors section of New VPN Tunnel tab](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/fortigate11.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=4f395d42760311cfc0ddb19d6f103f02) **Advanced section of New VPN Tunnel tab** Click *+Advanced* to complete the fields listed below. > [!TIP] > > Obtain the following values from the downloaded configuration file. | Field | Expected Value | | - | - | | Encryption | In the Encryption field, enter the value from the Encryption Algorithm row in the downloaded configuration file. | | Authentication | In the Authentication field, enter the value from the Authentication Algorithm row in the downloaded configuration file. | | Diffie-Hellman Group | Select the appropriate value as per the Perfect Forward Secrecy row in the downloaded configuration file. | | Key Lifetime (seconds) | 28800 | ![Advanced section of New VPN Tunnel tab](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/fortigate13.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=b85f4abe06fd896650696f8a2d57b61d) 3. Click *OK*. 4. Navigate to Network > Interfaces. 5. Click on the Tunnel created above (e.g. aviatrix-gatew) and assign the IP address from the downloaded configuration file. ![Tunnel interface IP assignment](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/fortigate14.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=0661ab1c8b47b2cc1a9ac6b19e18a609) 6. Click *OK*. ### Configure IPv4 Policy 1. Navigate to Policy & Objects > IPv4 DoS Policy. 2. Create two new IPv4 policies: 1. Outbound traffic ![Outbound traffic policy](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/fortigate15.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=381c9892a8d1bf14862b7bd586325aa8) 2. Inbound traffic ![Inbound traffic policy](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/fortigate16.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=3c7ca959bc6aedb8e7c60fce424b3cc7) > [!NOTE] > > The reference to *port2* in the screenshots should be replaced with your own > interface name that represents the internal facing interface. > [!NOTE] > > Be sure to select *ACCEPT* for Action and select *ALL* for Service. ### IPSec Monitor 1. In the Fortigate UI, navigate to Dashboard > Network and click the IPsec widget. 2. Select the Aviatrix tunnel, and click *Bring Up*. 3. You can then check the tunnel status in CoPilot under Diagnostics > Cloud Routes. ### BGP 1. In the FortiGate UI, navigate to Network > BGP. 2. Configure the Local BGP Options as below: * RouterID: Tunnel IP address taken from the configuration file downloaded at step3 * Neighbors: Remote tunnel IP address and ASN * Networks: All the networks needs to be advertised via BGP (here 10.0.3.0 is the local network of FortiGate) ![BGP configuration](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/fortigatebgp.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=4a63ea45f252f806919235652c57b786) 3. In CoPilot, go to Diagnostics > Cloud Routes > BGP Info to verify the BGP Routes. The Status should be Established. If some external connections for the selected Transit Gateway are Not Established, the overall BGP Status for the Transit Gateway is Partially Established. # Transit Gateway to JuniperSRX over the Internet Workflow Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/vpn/transit-externaldevice-junipersrx This document describes how to build a Transit connection between an Aviatrix Transit gateway and a Juniper SRX. This document describes how to build a Transit connection between an Aviatrix Transit gateway and a Juniper SRX. > [!NOTE] > > Aviatrix does not officially support Juniper SRX, but you can configure Juniper > SRX to work with the Aviatrix software. Aviatrix does not actively test our > software with Juniper SRX and you should rigorously test your configuration in a > development environment before deploying in a production system. 1. In CoPilot, navigate to Cloud Fabric > Gateways > Transit Gateways. 2. [Create a Transit gateway](/docs/enterprise/10.1/reference/ui/cloud-fabric/gateways#view-transit-gateways) in AWS that will connect to your Juniper SRX firewall. 3. To connect the Transit VPC gateway to Juniper SRX, navigate to Networking > Connectivity > External Connections (S2C) and [follow these instructions](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-create-bgp-over-ipsec.html). 4. [Download the configuration](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-download-configuration.html). 5. Ensure that your JuniperSRX is configured based on the information in the downloaded file. The following is a sample configuration based on the Site2Cloud configuration above. ![Juniper SRX sample configuration](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/juniper4.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=4d3d25100d9df997519202fec8235401) > [!NOTE] > > The tunnel IP addresses are configured accordingly with information from the > downloaded configuration file. 6. After configuring the router, in CoPilot navigate to Diagnostics > Cloud Routes > External Connections to confirm that the tunnel status has changed from Down to Up. 7. Navigate to Diagnostics > Cloud Routes > BGP Info to check the BGP routes for the Transit gateway you created. The Status should be Established. If some external connections for the selected Transit Gateway are Not Established, the overall BGP Status for the Transit Gateway is Partially Established. # Transit Gateway to Palo Alto VM-Series Workflow Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/vpn/transit-externaldevice-paloalto This is not a common scenario. You would only set up this type of connection if you want to connect an Aviatrix Transit gateway to a firewall that is outside your Cloud service provider (for example, in a branch office or warehouse). > [!NOTE] > > This is not a common scenario. You would only set up this type of connection > if you want to connect an Aviatrix Transit gateway to a firewall that is > outside your Cloud service provider (for example, in a branch office or > warehouse). This document describes how to build a Transit connection between an Aviatrix Transit Gateway and Palo Alto Networks Firewall. To simulate an on-prem Firewall, we use a VM-Series in an AWS VPC. Network setup is as following: * VPC1 (with Aviatrix Transit Gateway) * VPC1 CIDR: 10.5.0.0/16 * VPC1 Public Subnet CIDR: 10.5.3.0/24 * VPC1 Private Subnet CIDR: 10.5.2.0/24 * VPC2 (with Palo Alto Networks VM-series) * VPC2 CIDR: 10.0.0.0/16 * VPC2 Public Subnet CIDR: 10.0.0.0/24 * VPC2 Private Subnet CIDR: 10.0.1.0/24 * Sample subnet advertised with the help of BGP - 192.168.0.24/32 (loopback interface on PaloAlto) Configuration WorkFlow: 1. In CoPilot, navigate to Cloud Fabric > Gateways > Transit Gateways. 2. [Create a Transit gateway](https://legacy.docs.aviatrix.com/documentation/latest/network/transit-gateway-about.html?expand=true) in AWS that will connect to your Palo Alto firewall. 3. To connect the transit VPC GW to Palo Alto, navigate to Networking > Connectivity > External Connections. 4. Click *+External Connection*. 5. In the Add External Connection dialog, configure the following: | Field | Value | | - | - | | Connect Public Cloud to | External Device > BGP over IPsec | | Local Gateway | Select the Transit gateway you created in step 2 above | | Local ASN | Enter the BGP AS Number the Transit Gateway will use to exchange routes with the external device. | | Remote ASN | Enter the BGP AS number configured on the LAN | | Remote Device IP | This is the Palo Alto WAN IP | 6. Click *Save*. 7. [Download the configuration](/docs/enterprise/10.1/guides/connectivity/gateways/external-connection-download-configuration) . 8. Use the information in the downloaded file to configure the router. The following is a sample configuration based on the Site2Cloud configuration above. ![image4](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/paloalto4.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=ad6568b9e307003c6d55ef7722a22089) ## Configuring the Palo Alto VM-Series Firewall Log into Palo Alto Networks VM Series and configure it as follows: 1. Navigate to Network > Interfaces > Tunnel. ![](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/paloalto-create-tunnel.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=a7b1a3bf03843f9a159705cea9a5621e) 2. Click *Add* to create a new tunnel interface. In the Tunnel Interface dialog, on the Config tab, assign the following parameters. ![paloalto5](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/paloalto5.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=0305e8945eeab9dc9bc96cfd9aa4f953) | **Field** | **Value** | | - | - | | Interface Name | Type an ID in the yellow field next to Interface Name (for example, 45; this tunnel will then be named 'tunnel.45') | | Virtual Router | Select the existing *default* virtual router | | Security Zone | Select the layer 3 internal zone from which traffic originates | > [!NOTE] > > If the tunnel interface is in a zone different from the one where the > traffic will originate, you must create a policy to allow the traffic to > flow from the source zone to the zone containing the tunnel interface. Click the IPv4 tab and add the tunnel IP address from the configuration downloaded above. ![paloalto6](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/paloalto6.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=36a0f201d0bc2fc1ddba07424f2de1e8) 3. Click *OK*. 4. Navigate to Network > Network Profiles > IKE Crypto. 5. Click *Add*. 6. In the IKE Crypto Profile dialog, define the IKE Crypto profile (IKEv1 Phase-1) parameters as shown. ![paloalto7](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/paloalto7.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=c0c3280b397b24f6c2cc2eb253d2a20c) 7. Click *OK*. 8. Navigate to Network > Network Profiles > IKE Gateways and click *Add*. 9. In the IKE Gateway dialog, configure the IKE Phase-1 Gateway as shown below. These parameters should match the Site2Cloud configuration [downloaded here](#configuring-the-palo-alto-vm-series-firewall). ![image8](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/paloalto8.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=508eace9e0beb04521c64cc09b5c3a06) | **Field** | **Value** | | - | - | | Name | avx\_ike\_gw | | Address Type | IPv4 | | Interface | Palo Alto Networks WAN port (ethernet 1/1) | | Peer IP Address Type | IP | | Peer Address | Aviatrix Gateway public IP If using remote private IP, the Peer Address should be the remote private IP while Peer Identification should be the remote public IP. | | Authentication | Pre-Shared Key | | Pre-shared Key / Confirm Pre-shared Key | Key from [downloaded configuration](/docs/enterprise/10.1/guides/connectivity/gateways/external-connection-download-configuration) | | Peer Identification | IP Address & Aviatrix Gateway public IP | 10. On the Advanced Options tab, select the *IKE Crypto Profile* created in step 6. 11. Click *OK*. 12. Navigate to Network > Network Profiles > IPSec Crypto. 13. Click *Add* to create a new profile. 14. In the IPSec Crypto Profile dialog, define the IPSec crypto profile (IKEv1 Phase-2). These parameters should match the relevant values in the [downloaded configuration](/docs/enterprise/10.1/guides/connectivity/gateways/external-connection-download-configuration). ![paloalto10](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/paloalto10.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=d38ccb329b94cd4782fe77c885e13e4b) 15. Click *OK*. 16. Navigate to Network > IPSec Tunnels and click *Add* to create a new IPSec Tunnel. 17. Configure the General tab as follows: ![image11](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/paloalto11.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=53c5903e54edfed8220f9fed919fd8bb) | **Field** | **Value** | | - | - | | Tunnel Interface | Tunnel interface created in step 3 | | IKE Gateway | IKE gateway created in step 9 | | IPSec Crypto Profile | IPSec crypto profile created in step 14 | > [!NOTE] > > There is no need to configure proxy-id. Commit the configuration. The > IPSec tunnel should now be green. ![image23](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/paloalto14.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=502bbfcd36e9fb3e4ac31f4de68d3f09) 18. Steps to configure BGP: a. Navigate to Network > Virtual Routers. b. Click the 'default' link. c. In the Virtual Router - default dialog, click the BGP tab, and then the Peer Group tab. d. Click *Add*. e. Name the peer group (e.g bgppeering) and then click *Add* as shown to add BGP peering. ![paloaltobgp1](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/paloaltobgp1.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=08a192ba0ad10e9278d948c7137a2868) f. In the Virtual Router-BGP-Peer Group-Peer dialog, enter the information as shown in the below screenshot. Click *OK*. ![image14](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/paloalto13.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=27384010e899d35cf87f71a9aa4ab77d) ![image15](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/paloaltobgp3.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=8f15f477674934eb50a066be4c3fe53e) g. After everything is created, click *OK* to commit the configuration. The Router ID is taken from the downloaded config file (it should be the IP address of the tunnel created). ![image16](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/paloaltobgp4.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=7db2e1ab990f1692fd8f0c938af2e099) h. Click *OK*. i. Navigate to Network > default > Redistribution Profile. j. On the IPv4 tab, click *Add*. k. In the *Name* field, enter 'redis'. l. Enter a priority. m. Select the *connect* Source Type to create a redistribution profile. ![image12](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/paloaltobgp11.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=088d4115749891760e78523cb17a3ef7) n. Click *OK*. o. On the Virtual Router dialog, click BGP > Redist Rules. p. Click *Add* to add the redistribution rule you just created and click *OK*. ![image18](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/paloaltobgp12.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=037b62bc6cd45d32411b9a0f88cd67ca) q. On the same Virtual Router dialog, navigate to BGP > Export and click *Add*. r. In the Virtual Router - BGP - Export Rule dialog, add a name in the Rules field, and Enable the Export rule. s. Click *Add* to add the Peer Group from which the routes will be imported. Select Match and define the options used to filter routing information. ![image19](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/paloaltobgp7.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=4e94faafde3d95cb511082bb11ff591d) t. Click *OK*. u. After the BGP route has been advertised it looks like the following image. Navigate to Network → More runtime stats → BGP → RIB out. ![image20](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/paloaltobgp8.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=941a9ff6cc7a8757e2a1a14e9dddd404) 19. In the AWS portal, configure the VPC Route Table associated with the private subnet of VPC2. Add a route with VPC1 private subnet as the destination and Palo Alto Networks VM LAN port as the gateway. 20. In CoPilot, go to Diagnostics > Diagnostic Tools > BGP Diagnostics. 21. Select the gateway name from the dropdown list. 22. In the Command field, enter the 'show Ip bgp' command. 23. Click *Run* to verify the BGP Routes. # Transit Gateway to External Devices Workflow Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/vpn/transit-externaldevice-workflow This document describes the workflow to connect an Aviatrix Transit Gateway to an external (or third party) router or firewall. This document describes the workflow to connect an Aviatrix Transit Gateway to an external (or third party) router or firewall. ## Deploy a Transit Gateway If you have not launched an Aviatrix Transit Gateway, then follow the instructions for [deploying an Aviatrix Transit Gateway](https://legacy.docs.aviatrix.com/documentation/latest/network/transit-gateway-about.html?expand=true) in the Multicloud Transit Network Workflow. ## Set up the External Device Connection To set up an external connection from a Transit Gateway to an external device, select one of the following: * [BGP over IPsec](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-create-bgp-over-ipsec.html): connect to an external device using IPsec tunneling protocol and Border Gateway Protocol (BGP) routing. * [BGP over GRE](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-create-bgp-over-gre.html): connect to an external device using GRE tunneling protocol and BGP routing. * [BGP over LAN](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-create-bgp-over-lan.html): connect to an external device using BGP routing without any tunneling protocol. * [Static Route-Based (ActiveMesh)](https://legacy.docs.aviatrix.com/documentation/latest/network/external-connection-create-static-route-based-activemesh.html): connect to an external device using IPsec tunneling protocol and Static Route configuration with full mesh peering from the primary and highly availalble (HA) gateway instances. > [!NOTE] > > Starting with **Controller 9.0**, **Static Routing over IPSec** ActiveMesh > connections to an external device can enable **Proxy-ID** when routing > protocol is **Static** and tunnel protocol is **IPSec**. See > [Proxy-ID for Routed IPSec Connections](/docs/enterprise/10.1/guides/connectivity/vpn/proxy-id-routed-ipsec-connections) > . ## Download an External Connection Configuration If you are connecting an Aviatrix gateway and an on-premises router or firewall, Aviatrix can generate a configuration file that you can apply to your remote router or firewall. The configuration file contains the Aviatrix gateway tunnel details, such as the Public IPv4 and IPv6 addresses, VPC/VNet CIDR, pre-shared key, and encryption algorithm. You can download the configuration file and then import the details to your remote router or firewall to configure the other end of the VPN tunnel. After creating an external connection, to download an external connection configuration: 1. In Aviatrix CoPilot, go to **Networking > Connectivity > External Connections (S2C)** tab. 2. On the External Connections (S2C) tab, locate the connection you created and click the vertical ellipsis ![vertical ellipsis icon](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/connectivity/vpn/vertical-ellipsis-icon.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=dfa275cca2e54747ed314060d27d437d) icon on the right side of the row. 3. Select the following values: **Vendor**: Select your remote site device. * Select **Generic** for anything that is not an Aviatrix gateway. * Select **Aviatrix**, if you are connecting two Aviatrix gateways. **Platform and Software**: * If you selected a Generic vendor, the Platform field is populated as Generic, and the Software field is populated with Vendor Independent. * If you selected the Aviatrix vendor, the Platform field is populated with UCC, and the Software version is 1.0. * If you selected a specific hardware vendor (such as Cisco), select from the available platforms belonging to that vendor are displayed in the Platform field (ISR, ASR, and CSR are for Cisco routers), and the Software field is populated with the related software version. 4. Click **Download**. ## Configure the External Device Use the information provided in the configuration file to configure the other end of the tunnel on the remote router or firewall. See: * [Transit Gateway to Cisco Router](/docs/enterprise/10.1/guides/connectivity/vpn/transit-externaldevice-ciscorouter) * [Transit Gateway to Cisco ASA](/docs/enterprise/10.1/guides/connectivity/vpn/transit-externaldevice-ciscoasa) * [Transit Gateway to PaloAlto](/docs/enterprise/10.1/guides/connectivity/vpn/transit-externaldevice-paloalto) * [Transit Gateway to FortiGate](/docs/enterprise/10.1/guides/connectivity/vpn/transit-externaldevice-fortigate) * [Transit Gateway to JuniperSRX](/docs/enterprise/10.1/guides/connectivity/vpn/transit-externaldevice-junipersrx) ## Disconnect the External Connection To disconnect an external connection: 1. In Aviatrix CoPilot, go to **Networking > Connectivity > External Connections (S2C)** tab. 2. In the table, locate the external connection and click the Delete icon ![delete icon](https://mintcdn.com/aviatrix-14b37c43/1zOMu4WXCFpEtXhJ/images/guides/connectivity/vpn/delete-icon.png?fit=max\&auto=format\&n=1zOMu4WXCFpEtXhJ\&q=85\&s=59603279dfa3e866cdcd178c41de3347) on the right side of the row. 3. To confirm that you want to delete the external connection, click **Delete**. The external connection is deleted. # Troubleshooting IPsec VPN Connection with IKEv2 Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/connectivity/vpn/troubleshooting-ipsec-vpn-ikev2 This article describes how to troubleshoot IPsec VPN connection with IKEv2 on Aviatrix gateway. This article describes how to troubleshoot IPsec VPN connection with IKEv2 on Aviatrix gateway. ## Check External Connection (S2C) Connection Status In CoPilot, go to Networking > Connectivity > External Connections (S2C). Check if there is a green or red dot next to the name of the external connection. You can also check external connection status from Diagnostics > Cloud Routes > External Connections (look at the Status and Tunnel Status columns). If the Tunnel Status is down, you can perform the following procedure. ## Perform the Analysis Diagnostics Action 1. Go to Diagnostics > Diagnostic Tools > Connectivity Diagnostics. 2. Select the Gateway Instance and the related Connection. 3. Select **Analysis** in the Tools list and click **Run**. The screen will display analysis results. ## Troubleshoot the keyword in the Diagnostics Action "Show logs" 1. Go to Diagnostics > Diagnostic Tools > Connectivity Diagnostics. 2. Select the Gateway Instance and the related Connection. 3. Select **Logs** in the Tools list. 4. (optional) Enable or disable verbose logging. 5. Click **Run**. The screen displays the related logs. You can copy the results to the clipboard. ![IKEv2\_show\_log](https://mintcdn.com/aviatrix-14b37c43/Gre8pdluiL7JmlpL/images/guides/connectivity/vpn/ikev2-show-log.png?fit=max\&auto=format\&n=Gre8pdluiL7JmlpL\&q=85\&s=d2fb4407ee5b69a3a1c5174c5b6a6e35) ## Examples of IKEvs Negotiation Failure Here are some examples of negotiation failure related troubleshooting hints: | Keyword | Probable Causes | Suggestions | | - | - | - | | Error: Failed to deliver message to gateway | Aviatrix Controller cannot reach gateway | Troubleshoot connectivity between the Aviatrix Controller and the gateway. | | Establishing IKE\_SA failed, peer not responding | Peer IP address is mismatched, or peer IP address is not reachable. UDP port 500/4500 is not accessible | Troubleshoot connectivity between the Aviatrix Gateway and the peer VPN router. | | NO\_PROPOSAL\_CHOSEN | Peer IP address is mismatched, or peer IP address is not reachable. IKE version is mismatched (one VPN gateway uses IKEv1 and another uses IKEv2). IKEv2 algorithm is mismatched. IPsec algorithm is mismatched | Troubleshoot connectivity between Aviatrix gateway and peer VPN router. Verify that both VPN settings use the same IKEv2 version. Verify that all IKEv2/IPsec algorithm parameters (i.e., Authentication/DH Groups/Encryption) match on both VPN configuration | | AUTHENTICATION\_FAILED | IKE version is mismatched (one VPN gateway uses IKEv1 and another uses IKEv2). Pre-shared key is mismatched. Identifier configuration is mismatched | Verify that both VPN settings use the same IKEv2 version. Verify that pre-shared key match on both VPN configuration. Verify that Identifiers match; by default, Aviatrix utilizes the gateway's public IP as the Local Identifier. | | no shared key found | IKE version is mismatched (one VPN gateway uses IKEv1 and another uses IKEv2). Identifier configuration is mismatched | Verify that both VPN settings use the same IKEv2 version. Verify that identifiers match; by default, Aviatrix utilizes the gateway's public IP as the Local Identifier. | | failed to establish CHILD\_SA, keeping IKE\_SA | IPsec algorithm is mismatched | Verify that all IPsec algorithm parameters (i.e., Authentication/DH Groups/Encryption) match on both VPN configurations. | # Aviatrix Licensing Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/controlplane/aviatrix-licensing Aviatrix offers licenses (unique customer IDs) to customers via Cloud Service Providers. A valid license is required to access and use Aviatrix Cloud Network Controller and Aviatrix Cloud Network CoPilot. Aviatrix offers licenses (unique customer IDs) to customers via Cloud Service Providers. A valid license is required to access and use Aviatrix Cloud Network Controller and Aviatrix Cloud Network CoPilot. > [!NOTE] > > You must subscribe to the Aviatrix license listing to obtain a valid customer > ID to activate any other Aviatrix offers. ## Required Marketplace Offers To deploy the Aviatrix cloud networking platform on AWS, Microsoft Azure, or GCP, *three subscriptions* are required from the CSP Marketplace. For AWS and Azure, you can subscribe to these offerings through the [Launch Aviatrix guided experience](https://launch.aviatrix.com). > [!NOTE] > > Some optional AWS resources, such as S3 buckets, ELB with WAF, and others, are > subject to additional AWS service charges. See the AWS website for pricing for > [S3](https://aws.amazon.com/s3/pricing/?nc2=type_a), > [ELB](https://aws.amazon.com/elasticloadbalancing/pricing/), and > [WAF](https://aws.amazon.com/waf/pricing/?nc2=type_a). These optional > resources are not required for deployment of the Aviatrix platform. For GCP, you can subscribe from the CSP's marketplace. The required subscriptions per CSP are shown in the following table. | **CSP** | **Offer** | **Description** | | - | - | - | | AWS | **Aviatrix Enterprise Contract** | The Aviatrix Enterprise Contract listing provides the license (customer-id) required to active the Aviatrix Cloud Network Controller and Aviatrix Cloud Network CoPilot. A free trial is available with this AWS offer. When the free trial expires, the customer must upgrade to one of the paid options in the listing. | | Azure | **Aviatrix Enterprise Contract** | The Aviatrix Enterprise Contract provides the license (customer-id) needed to access the Aviatrix Cloud Network Controller and Aviatrix Cloud Network CoPilot. This service also calculates Aviatrix bills based on usage. | | GCP | **Aviatrix Secure Networking Metered 2208-Universal 24x7 Support** | This Aviatrix license provides the license (customer-id) needed to access the Aviatrix Cloud Network Controller and Aviatrix CoPilot - BYOL. This service also calculates Aviatrix bills based on usage. | | AWS, Azure, GCP | **Aviatrix Cloud Network Controller** | This Bring Your Own License (BYOL) offer integrates with the Aviatrix license. | | AWS, Azure | **Aviatrix Cloud Network CoPilot** | This Bring Your Own License (BYOL) offer integrates with the Aviatrix license. | | GCP | **Aviatrix CoPilot - BYOL** | This Bring Your Own License (BYOL) offer integrates with the Aviatrix license. | > [!NOTE] > > If you subscribe to the free trial license, you receive notification emails 14, > 7, and 1 day before the free trial expires and billing begins. ## Subscribing to an Aviatrix Systems License ### Upgrading from a Trial License to an Aviatrix Systems License If you subscribe to the Aviatrix Enterprise Contract offer on AWS and you opt to *Try for free*, you have 30 days to use Aviatrix services. After this trial period ends, you must upgrade to a paid listing to continue using the product. ### Available Licenses The following table provides URLs where you can obtain an Aviatrix license for supported cloud service providers (CSPs). | **Cloud Service Providers** | **URL Where you can Subscribe to a Non-Trial License** | | - | - | | **AWS** | [Aviatrix Enterprise Contract](https://aws.amazon.com/marketplace/pp/prodview-5ikm5hj3ibosk?sr=0-5\&ref_=beagle\&applicationId=AWSMPContessa) | | **Azure** | [Aviatrix Enterprise Contract](https://azuremarketplace.microsoft.com/en-us/marketplace/apps?search=aviatrix\&page=1) | | **GCP** | [Aviatrix Secure Networking Metered 2208-Universal 24x7 Support](https://console.cloud.google.com/marketplace/product/aviatrix-public/controller-aviatrix-billing-unit-subscription) | ## Entering the Customer ID in Aviatrix CoPilot 1. After obtaining a valid license (customer-id) from one of the CSP marketplace listings, go to Aviatrix CoPilot and click *Activate License* on the right side of the banner. 2. Enter the license (Customer ID) you received by email when you subscribed through the CSP marketplace. 3. Click *Apply*. Your license information is saved. ## Changing Aviatrix Licenses If you are a current user and you change licenses, you must reset the Customer ID in CoPilot. 1. After subscribing to the new license, log on to CoPilot and go to **Settings > Configuration > License**. 2. Click **Reset** next to **Customer ID**. 3. Enter the new customer ID (license ID). 4. Select the **I understand the implications** checkbox. 5. Click **Reset**. 6. If you need to turn on features that are disabled by default, see [CoPilot Optional Features](/docs/enterprise/10.1/reference/ui/settings/configuration#configure-feature-settings) . # Configuring an AWS Load Balancer with SSL in Front of Aviatrix Controller Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/controlplane/aws-ssl-lb The Aviatrix Controller supports adding an SSL certificate. However, sometimes you may prefer to put an AWS Load Balancer (ALB) in front of the Controller. The Aviatrix Controller supports adding an SSL certificate. However, sometimes you may prefer to put an AWS Load Balancer (ALB) in front of the Controller. ![aws ssl lb](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/controlplane/aws-ssl-lb.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=24fb652cdc9f4ff6e8d26e7e1477344f) Follow these steps to place the Aviatrix Controller behind an AWS ALB: 1. Log into your AWS console. 2. Go to EC2 > Load Balancers in the region where your Aviatrix Controller is running. 3. Click **Create load balancer**. 4. Click **Create** under Application Load Balancer. ![500](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/controlplane/aws-lb-1.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=ff9b42712ca24c0737fef018d6d06010) 5. On the Create Application Load Balancer page, under Basic configuration, enter a name for the load balancer. 6. Also under Basic Configuration, select **Internet-Facing** for the Scheme. ![500](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/controlplane/aws-lb-2.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=f2f5b7abe00c6d93c12c100e54375858) 7. Under Network mapping, select at least two Availability Zones and one subnet per zone. 8. Under Security groups, select the appropriate security group from the Security groups drop-down. This security group should allow traffic on port 443 from your desired source network(s). A default security group may already be selected. ![500](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/controlplane/aws-lb-3.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=bfb57fd2953521433c041afea3789c36) 9. Under Listeners and routing, select **HTTPS** from the Protocol list. 10. Select a target group from the Default action field. 11. (optional) If needed, click **Create target group** under the **Default action** field. This opens the EC2 > Target groups > Create target group Console page in a new web browser tab. ![500](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/controlplane/aws-lb-3a.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=7ab6e113d0588afd7b7eb06cf2ebe585) On the Specify group details page: 1. Select the **Instances** target type. 2. Enter a target group name. 3. Select the HTTPS Protocol and port 443. 4. Select the VPC where the Controller resides (normally named aviatrix-mgt-vpc). 5. Under Health checks, select the HTTPS Protocol and the default Path of '/'. 6. Click **Next** to register the target group. 12. On the Create Application Load Balancer page, under Secure listener settings, select **From ACM** from the Default SSL/TLS Certificate drop-down and select your certificate. If necessary you can [request a new certificate](https://docs.aws.amazon.com/acm/latest/userguide/gs-acm-request-public.html). ![500](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/controlplane/aws-lb-5.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=7fd8b0029fd4f246f96d8392d4a6b742) 13. Review and create the load balancer. 14. Collect the DNS name from the load balancer. ![500](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/controlplane/aws-lb-6.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=d95a33eab3dc60c3a80afe783009b0b7) 15. Create a DNS CNAME record that points your desired name to the load balancer's DNS name. > [!NOTE] > > The DNS CNAME record must match the name used in the SSL certificate or > you will receive a warning. 16. Ensure that your Controller security groups have an inbound allow policy for port 443 for the VPC CIDR, so that the load balancer can talk to the Controller. > [!NOTE] > > If you have enabled HA for your Aviatrix Controller, you can point your auto > scaling group to the target group of your load balancer in the event of a > failover. The Max value should always be 1. Having more than one active > Controller for any given set of services is not supported if deployed behind a > load balancer. # Subscribing Gateway and Firewall Offers to Private Azure Marketplace for Aviatrix Deployments Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/controlplane/azure-powershell-deploy Depending on your company's security policies, you may need to add and subscribe to Aviatrix gateway and firewall offers in your private Azure Marketplace using PowerShell. Depending on your company's security policies, you may need to add and subscribe to Aviatrix gateway and firewall offers in your private Azure Marketplace using PowerShell. This document explains how to use PowerShell commands to add Aviatrix gateways, firewalls, and partner firewall offers to your private Azure Marketplace. Since our gateway images are not publicly available, you cannot subscribe to these offers directly in the Azure portal. Instead, follow the instructions below to complete the subscription process. > [!NOTE] > > The examples in this document demonstrate Azure PowerShell commands for subscribing in the private Marketplace. Because these commands may change, always verify usage in the Azure documentation: > > * [Manage a private Azure Marketplace using PowerShell](https://learn.microsoft.com/en-us/marketplace/manage-private-azure-marketplace-powershell) > * [Get storage account configuration information](https://learn.microsoft.com/en-us/azure/storage/common/storage-account-get-info) ## Subscribing an Aviatrix Gateway Offer to Azure Private Marketplace ### Prerequisites * Verify that the Private Azure Marketplace is enabled for your tenant. If it is not enabled, enable it in the Azure portal. See [Azure documentation](https://learn.microsoft.com/en-us/marketplace/create-manage-private-azure-marketplace-new) for detailed steps. By default, only Microsoft offers are available in the private Azure Marketplace. * Assign the *Marketplace Admin* role to the user account that will manage the private Azure Marketplace. See [Assign the Marketplace Admin role](https://learn.microsoft.com/en-us/marketplace/create-manage-private-azure-marketplace-new#assign-the-marketplace-admin-role). * Create the Private Azure Marketplace if it does not already exist. This is a one-time operation per tenant. When you create it, a single collection called **"Default Collection"** is automatically created. It includes all existing subscriptions in the tenant. See [Create a Private Azure Marketplace](https://learn.microsoft.com/en-us/marketplace/create-manage-private-azure-marketplace-new#create-a-private-azure-marketplace). * Ensure that the Azure collection you plan to use is enabled. By default, a single collection called **"Default Collection"** is created. It includes all existing subscriptions in the tenant, but contains only Microsoft offers by default. You must explicitly add and approve other offers. Make sure the collection is enabled in the Azure *Manage Marketplace* page. See [Enable or disable a collection](https://learn.microsoft.com/en-us/marketplace/manage-private-azure-marketplace-powershell#enable-disable-a-collection). * Identify the Azure subscription that will be used for billing resources. Run the following command to confirm that your current Azure subscription is in the `Enabled` state: ```bash theme={null} az account show --query "{Name:name, SubscriptionId:id, State:state}" -o table ``` ### Subscribing to the Aviatrix Gateway Offer 1. Log in to your Azure account using PowerShell. Use an account that has the Marketplace Admin role in your tenant. ```powershell theme={null} Connect-AzAccount ``` 2. Run the following PowerShell script to subscribe to the Aviatrix Gateway offer in your private Azure Marketplace. Where: * Replace `CollectionName` with your actual collection name. If you do not have a custom collection, use `DefaultCollection` (no space) in PowerShell. * Replace `OfferId` and `SpecificPlanIdLimitation` with the appropriate values for your Controller's software version. See the table below for related values. ```powershell theme={null} # Provide Azure Private Marketplace Collection Name. By default, a collection is scoped to all subscriptions $CollectionName="Default Collection" if ($CollectionName -eq "Default Collection") {$CollectionName="DefaultCollection"} # Install the Az.Marketplace module Install-Module -Name Az.Marketplace -Confirm:$false -Force # Get the Private Store ID $PrivateStore = Get-AzMarketplacePrivateStore if ($PrivateStore.Availability -ne "enabled") { throw "Azure Private Marketplace is not enabled. Follow https://learn.microsoft.com/en-us/marketplace/create-manage-private-azure-marketplace-new#enabledisable-private-azure-marketplace to enable it" } $PrivateStoreId = $PrivateStore.PrivateStoreId # Get the Collection ID $Collection = Get-AzMarketplacePrivateStoreCollection -PrivateStoreId $PrivateStoreId | Where-Object {$_.CollectionName -eq $CollectionName} if (!($Collection.Enabled)) { throw "Azure Private Marketplace Collection: $CollectionName is not enabled. Follow https://learn.microsoft.com/en-us/marketplace/create-manage-private-azure-marketplace-new#enabledisable-a-collection to enable it" } $CollectionId = $Collection.CollectionId # Add the offer to the private Azure Marketplace $Params = @{ privateStoreId = $PrivateStoreId collectionId = $CollectionId offerId = "aviatrix-systems.aviatrix-gateway" SpecificPlanIdLimitation = @("aviatrix-gateway-g3") } # Set the offer to the private Marketplace Set-AzMarketplacePrivateStoreCollectionOffer @Params ``` Use the following table for guidance: | Release Version | Offer ID | Specific Plan ID Limitation | | - | - | - | | >= 6.7 | aviatrix-systems.aviatrix-companion-gateway-v10 | aviatrix-companion-gateway-v10u | | >= 6.8 | aviatrix-systems.aviatrix-companion-gateway-v13 | aviatrix-companion-gateway-v13u | | >= 6.9 | aviatrix-systems.aviatrix-companion-gateway-v15 | aviatrix-companion-gateway-v15u-6-9 | | >= 7.0 | aviatrix-systems.aviatrix-companion-gateway-v16 | aviatrix-companion-gateway-v16 | | >= 7.1.3958 | aviatrix-systems.aviatrix-gateway | aviatrix-gateway-g3 | The Aviatrix Gateway image has now been added to your private Azure Marketplace. ### Example Code Below is a complete example PowerShell snippet demonstrating the workflow. Replace the variables with values specific to your account. ```powershell theme={null} # Replace and (or sku) with appropriate values $OfferId = "aviatrix-systems.aviatrix-gateway" $SpecificPlanIdLimitation = "aviatrix-gateway-g3" # Example for 8.0 G3 Gateway $CollectionName="Default Collection" if ($CollectionName -eq "Default Collection") {$CollectionName="DefaultCollection"} Install-Module -Name Az.Marketplace -Confirm:$false -Force $PrivateStore = Get-AzMarketplacePrivateStore if ($PrivateStore.Availability -ne "enabled") { throw "Azure Private Marketplace is not enabled. Follow https://learn.microsoft.com/en-us/marketplace/create-manage-private-azure-marketplace-new#enabledisable-private-azure-marketplace to enable it" } $PrivateStoreId = $PrivateStore.PrivateStoreId $Collection = Get-AzMarketplacePrivateStoreCollection -PrivateStoreId $PrivateStoreId | Where-Object {$_.CollectionName -eq $CollectionName} if (!($Collection.Enabled)) { throw "Azure Private Marketplace Collection: $CollectionName is not enabled. Follow https://learn.microsoft.com/en-us/marketplace/create-manage-private-azure-marketplace-new#enabledisable-a-collection to enable it" } $CollectionId = $Collection.CollectionId $Params = @{ privateStoreId = $PrivateStoreId collectionId = $CollectionId offerId = $OfferId SpecificPlanIdLimitation = @($SpecificPlanIdLimitation) } Set-AzMarketplacePrivateStoreCollectionOffer @Params ``` ### Accepting the Marketplace Terms for the Subscription After you have added the Aviatrix Gateway offer to your private Azure Marketplace, ensure that any subscription where you will deploy the gateway has accepted the Marketplace terms for the plan. If your Access Account (service principal or app registration) does not have the permission (`Microsoft.MarketplaceOrdering/offerTypes/publishers/offers/plans/agreements/*`), you must accept the terms for the offer at the subscription level. You can do this using PowerShell or Azure CLI: **PowerShell example** ```powershell theme={null} $PublisherId = "aviatrix-systems" $OfferId = "aviatrix-gateway" $PlanId = "aviatrix-gateway-g3" Set-AzMarketplaceTerms -Publisher $PublisherId -Product $OfferId -Name $PlanId -Accept ``` **Azure CLI example** ```bash theme={null} az vm image terms accept --publisher aviatrix-systems --offer aviatrix-gateway --plan aviatrix-gateway-g3 ``` ## Subscribing an Aviatrix Firewall Offer to Your Private Azure Marketplace Repeat the steps above to add a partner firewall offer to your private Azure Marketplace. Use the table below to find the correct *Publisher*, *Offer ID*, and *SKU* values. | *Name* | *Publisher* | *Offer ID (plan product)* | *SKU (plan name)* | | - | - | - | - | | PAN | paloaltonetworks | vmseries1, vmseries-flex | bundle1, bundle2, byol | | Fortinet | fortinet | fortinet\_fortigate-vm\_v5 | fortinet\_fg-vm fortinet\_fg-vm\_payg fortinet\_fg-vm\_payg\_20190624 | | Check Point | checkpoint | check-point-cg-r81, check-point-cg-r8110 | sg-ngtp, sg-ngtx, sg-byol, mgmt-byol | After completing these steps, you can deploy Azure firewalls from your private Azure Marketplace using the Aviatrix Controller. ## Next Steps * In the Azure portal, onboard the service principal created under the subscription you used to add the gateway offer. See the relevant [Microsoft documentation](https://learn.microsoft.com/), if needed. * You can now deploy Aviatrix Gateways for Azure from the Aviatrix Controller. See [Building Your Network](/docs/enterprise/10.1/reference/release-notices/release-notes/copilot/index) in the Aviatrix documentation. # Controller High Availability in AWS Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/controlplane/controller-ha-aws Aviatrix Controller HA in AWS leverages an auto-scaling group and a Lambda function to perform monitoring the health of the current Controller, launching a new Controller and restoring the configuration when the active Controller instance becomes unreachable. ## Overview Aviatrix Controller HA in AWS leverages an auto-scaling group and a Lambda function to perform monitoring the health of the current Controller, launching a new Controller and restoring the configuration when the active Controller instance becomes unreachable. When a new Controller is launched, the existing Controller is terminated, its EIP is associated to the newly launched Controller, and the private IP is created in the new Controller subnet. Existing configuration is restored, resulting in a seamless experience when failover happens. ## Prerequisites * Existing AVX Controller. If you have not yet launched an AVX Controller, please see [AWS Getting Started Guide](/docs/enterprise/10.1/guides/controlplane/getting-started-guide-aws). * Enable Controller [backups](/docs/enterprise/10.1/guides/maintenance/controller-backup-restore#copilot). * AMI *aviatrix\_cloud\_services\_gateway\_043018\_YYYY-xxxxxx* or later. If you are on an older AMI, see [Migrating Your Aviatrix Controller](/docs/enterprise/10.1/guides/maintenance/controller-migration) to migrate to the latest Controller AMI first (Controller > Settings > Maintenance > Backup & Restore). Aviatrix recommends contacting [Aviatrix Support](https://support.aviatrix.com/) before migrating your Controller. * The Controller's VPC should have one or more public subnets, preferably in different AZs for HA across multiple AZ. * To use Controller HA with an ELB, see [Load Balancing](https://legacy.docs.aviatrix.com/documentation/latest/platform-administration/accounts-and-users/iam-policy-requirements-aws.html#udp-load-balance). * Controller has enabled backup function. ### Controller HA Details Aviatrix Controller HA operates by relying on an AWS Auto Scaling Group. This ASG has a desired capacity of 1 (and minimum capacity = 0 and maximum capacity = 1\). If the Controller EC2 instance is stopped or terminated, it will be automatically re-deployed by the ASG. An AWS Lambda script is notified via SNS when new instances are launched by the Auto Scaling Group. This script handles configuration using a recent Controller backup file. The Aviatrix Controller manages these backups once [enabled](/docs/enterprise/10.1/guides/maintenance/controller-backup-restore). Restoring the Aviatrix Controller from a newly built instance requires access to the S3 bucket to retrieve the latest backup file. In order to do this, the newly built EC2 Controller instance must be granted permission to read files in the bucket. The simplest method of doing this is via an [IAM user with programmatic access to the S3 bucket](/docs/enterprise/10.1/reference/general/access-key). The lambda script also requires access to the S3 bucket. It is recommended that the backup bucket is used in the same account that was used to launch the controller. ## Enabling AWS Controller High Availability 1. Log in to the AWS console and switch to the region where your existing AVX Controller is installed. 2. Launch a CloudFormation template stack: * For Controller version 7.2 or later, use this [CloudFormation template](https://console.aws.amazon.com/cloudformation/home#/stacks/new?stackName=AviatrixHA\&templateURL=https://aviatrix-cloudformation-templates.s3.us-west-2.amazonaws.com/aviatrix-aws-existing-controller-ha-v4.json). * For Controller earlier than version 7.2, use this [CloudFormation template](https://console.aws.amazon.com/cloudformation/home#/stacks/new?stackName=AviatrixHA\&templateURL=https://aviatrix-cloudformation-templates.s3-us-west-2.amazonaws.com/aviatrix-aws-existing-controller-ha.json). > [!WARNING] > > CloudFormation template v3 and earlier versions are now deprecated due to security improvements in CloudFormation template v4. > > Template v4 implements enhanced security controls and access restrictions that > better align with AWS security best practices. > > Users running template v3 or earlier should upgrade to template v4 as soon as > possible. See > [Updating from CloudFormation template version 3 to version 4](#how-do-i-update-from-cloudformation-template-version-3-to-version-4) > for more details. > [!NOTE] > > Template v4 creates its own `execute-api` VPC endpoint. If the VPC already has > a centralized `execute-api` endpoint with `PrivateDnsEnabled: true`, it > overrides DNS resolution and redirects traffic away from the template > endpoint, causing a `403 AccessDeniedException`. In this case, you have two > options to resolve the issue: either remove or opt out of the centralized > endpoint from the Controller VPC (recommended), or use template v3 with a > Lambda Function URL as a fallback. 3. Click *Next* on the Create stack page to accept the CloudFormation stack defaults: * Prerequisite - Prepare template: Template is ready * Specify template: Amazon S3 URL 4. On the Specify Stack details page, populate the fields as follows: | Field | Expected Value | | - | - | | Stack name | Any valid stack name. | | **Network Configuration** | | | Enter VPC of existing controller instance | Select the VPC in this region where the AVX Controller is installed. | | Enter one or more subnets in different Availability zones within that VPC | Select a PUBLIC subnet of the controller VPC. Optionally one additional subnet for redundancy. | | **Aviatrix Controller Backup Configuration** | | | Enter Name tag of the existing Aviatrix Controller instance | Enter the **Name** tag for the existing Controller EC2 instance. | | Enter the controller IAM EC2 role name | Enter the controller IAM APP role name if it is different than the default? | | Enter Name tag of the existing Aviatrix Controller instance | Enter the **Name** tag for the existing Controller EC2 instance. | | Enter the controller IAM EC2 role name | Enter the controller IAM APP role name if it is different than the default? | | Enter S3 Bucket which will be used to store backup files. | Name of S3 bucket that stores the backup files from the AVX Controller. | | Enter an email to receive notifications for autoscaling group events | Enter an email address that will be notified whenever a new Controller is provisioned. | > [!NOTE] > > The S3 bucket you use or create for Controller HA and Backups does not need to > have public access enabled and should be configured to restrict general public > access. 5. Click *Next*. 6. Populate any additional CloudFormation Options. 7. Click *Next*. 8. Check "I acknowledge that AWS CloudFormation might create IAM resources with custom names." 9. Click *Create*. 10. Refresh the Stacks page and wait for the status of this stack to change to *CREATE\_COMPLETE* > [!NOTE] > > If the stack fails (and ends with status of *ROLLBACK\_COMPLETE*) check the log > messages in the *Events* section. If you see an error that says "Failed to > create resource. AMI is not latest. Cannot enable Controller HA. Please > backup/restore to the latest AMI before enabling controller HA. ", then follow > the steps outlined > [here](/docs/enterprise/10.1/guides/maintenance/controller-migration) > . > [!NOTE] > > This CloudFormation template creates the following: > > **Core HA Components** > > * An Aviatrix Auto Scaling Group (size 1): Maintains exactly one Controller > instance at all times. If the instance fails, a replacement automatically > launches to ensure continuous availability > * A new security group: Creates fresh firewall rules to control network access > to the Controller instance > * An SNS topic with the same name as the existing Controller instance > * An email subscription to the SNS topic (optional) > * A Lambda function for setting up HA and restoring configuration automatically > * An Aviatrix Role for Lambda with corresponding role policy and required > permissions > > **Private API Endpoint** > > * VPC Endpoint > * Private API Gateway: The Controller can call the Lambda function's > /controller\_version endpoint privately (without going over public internet). > This endpoint is restricted to access from the VPC Endpoint only > [!NOTE] > > Please note that if you change the Controller name or change the backup > destination bucket on S3, your Controller HA will not work as expected. You > would have to delete the Controller HA CloudFormation Stack and redeploy it. > [!TIP] > > Additional instructions and code are available > [here](https://github.com/AviatrixSystems/Controller-HA-for-AWS/). > [!NOTE] > > During spinning up the HA after the current active controller stops or being > terminated by accident, you won't see a new Controller for a few minutes on > AWS console. This is expected. ## Disabling AWS Controller High Availability You can disable Controller HA by deleting the Controller HA CloudFormation stack. * Please take a backup from the Controller first: Go to *Controller > Settings > Maintenance > Backup & Restore > Backup Now*. Verify that the S3 bucket now contains these backup files. * Check the ASG capacity first. It should be minimum capacity=0, maximum capacity=1, desired capacity=1. If these are changed, deleting the Controller HA Cloudformation stack could have an impact on your current Controller. * Log in to AWS Console, go to CloudFormation Service, identify the CloudFormation stack you used to enable Controller HA and delete the stack. > [!WARNING] > > *Please be careful* to delete the CloudFormation stack associated with the > controller HA; do not delete your controller launch CloudFormation stack. ## AWS High Availability FAQ * How can I know which version of the HA script I am running? versions.py file found in the AWS Lambda function with the name \-ha would show the information. You can also see the version in the cloudwatch logs. Only versions from 1.5 and above are visible. * How can I get notifications for H/A events? Enter an email address to receive notifications for autoscaling group events while launching the CFT. You would receive an email to subscribe to SNS. Click on the link from the email to accept SNS event notifications * My H/A event failed. What can I do? You can manually restore the saved backup to a newly launched Controller. Please ensure controller H/A is disabled and re-enabled by deleting and re-creating the CFT stack to ensure that lambda is pointing to the right backup * How do I ensure that lambda is pointing to the right backup? In the AWS Lambda, verify if the INST\_ID environment variable is updated correctly to the current controller instance ID and the PRIV\_IP environment variable is updated to the current controller private IP. * Where do I find logs related to Controller H/A ? All logs related to H/A can be found in AWS Cloudwatch under the log group \-ha * How do I make lambda talk to the Controller privately within the VPC? Launch CFT with Private access set to True. Attach lambda to the VPC from the AWS console. Ensure that the VPC that you have attached the lambda to has internet access via NAT gateway or VPC endpoints. You can also ensure that lambda has internet access by attaching an EIP(Elastic IP) to the lambda ENI(Network Interface). Please ensure that everything is reverted before you destroy the stack. Otherwise the lambda will not have internet access to respond to the CFT (CFT may get stuck on destroy). * Can two Controllers in two different regions be linked such that they can detect if one or the other is down? Is this possible? Our Controller HA script leverages EC2 auto-scaling. EC2 auto-scaling does not support cross regions but it does support cross AZs. The script will automatically bring up a new Controller in case the existing Controller enters an unhealthy state. * Could a Controller in a different region be used to restore a saved configuration in case of disaster recovery? Will the change in the Controller's IP cause any issues? A Controller can be manually launched from a different region and the backed up configuration can be restored on it. The Controller's new EIP should not cause any issues unless SAML VPN authentication is being used. (All peering tunnels will still work). In that case, SAML VPN client will need to reach the Controller IP address. If the FQDN hostname is used for the Controller for SAML, then it should work after changing the Route 53 to resolve to the correct EIP in the different region. * How do I manage the Controller HA stack if the controller instance's disk is encrypted? If EBS Encryption using Customer managed key is enabled, the Autoscaling Group created may not have permissions to launch the instance. You will need to allow the service-linked role created for the Autoscaling group to have permissions to use this key for the cryptographic operation. To do so, go to AWS KMS→Customer managed keys→select the key and add the "AWSServiceRoleForAutoScaling" role to the list of Key Users. * What do I need to do after I change the Controller name? Please delete the CFT stack and then create a new CFT stack using the new Controller name. * How do I update from CloudFormation template version 3 to version 4? If you are currently running a version 7.2 or later Controller AMI with the v3 CloudFormation stack, you can update to the v4 template to benefit from improved security controls and access restrictions. To update your stack: * Direct stack update (recommended): Click *Update Stack* in the CloudFormation console and apply the v4 template. This is faster than deleting and recreating the stack. * Full replacement: If there are changes to the controller configuration, delete the current stack and create a new one with the v4 template. # Planning Your Controller Deployment Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/controlplane/controller-planning-deployment The information in this topic applies to deployments on the marketplaces for Google Cloud and Oracle Cloud Infrastructure (OCI). The information in this topic applies to deployments on the marketplaces for *Google Cloud and Oracle Cloud Infrastructure (OCI)*. > [!TIP] > > For *AWS* (Amazon Web Services) deployments, see [AWS Getting Started Guide](/docs/enterprise/10.1/guides/controlplane/getting-started-guide-aws). > > For Microsoft Azure deployments, see [Azure > Getting Started Guide](/docs/enterprise/10.1/guides/controlplane/getting-started-guide-azure). There are two ways to deploy an Aviatrix Controller with Google Cloud or OCI: 1. Through one of the CSP (Cloud Service Provider) marketplaces. 2. Through [Aviatrix Terraform modules](https://registry.terraform.io/namespaces/AviatrixSystems). > [!NOTE] > > Terraform is the recommended deployment method. See [Terraform & > Infrastructure as > Code](https://aviatrix.com/learn-center/cloud-networking/terraform-and-infrastructure-as-code/) > for more information. Ensure that you have met all of the [Prerequisites for Deploying Through a CSP Marketplace](#prerequisites-for-deploying-through-a-csp-marketplace) before starting the Aviatrix Controller deployment. ## Prerequisites for Deploying Through a CSP Marketplace These prerequisites apply to *Google Cloud and OCI*. For *AWS Marketplace* deployments see [Preparing for Your AWS Deployment](/docs/enterprise/10.1/guides/controlplane/planning-aws-deployment). For Microsoft Azure deployments, see [Preparing for Your Azure Deployment](https://legacy.docs.aviatrix.com/documentation/latest/getting-started/planning-azure-deployment.html). Each CSP has its own set of prerequisites that must be implemented prior to deploying the Aviatrix Controller and CoPilot. ### Google Cloud Prerequisites #### Get a Customer ID from Aviatrix The Aviatrix Controller for Google Cloud is available on the Google Cloud Marketplace for BYOL license. Send an email to [info@aviatrix.com](mailto:info@aviatrix.com) or open a support ticket at [Aviatrix Support Portal](https://support.aviatrix.com) with your organization name to request a customer ID. We offer a 30-day free trial license. ### Creating a Google Cloud Account Aviatrix Cloud Connect is a software product that is launched in your own Google Cloud account. The Controller and the Gateways created from the Controller are all in your own network perimeter and completely under your control. Create a Google Cloud account ([https://cloud.google.com/](https://cloud.google.com/)). If you already have an account, skip this step and go to the next step. Note that the Controller supports multiple accounts and each account is associated with a different Google Cloud project. But you must have at least one account. #### Creating a Google Cloud Project Log in to your Google Cloud account and go to the project page: [https://console.cloud.google.com/project](https://console.cloud.google.com/project) Create a project. Go on to the next step if you have already created one. Note that the project ID will be used in referencing this project by Aviatrix Controller. For example, in a project called Aviatrix-UCC, the project ID is aviatrix-ucc-1214. #### (Optional) Creating Networks This step creates a network in the project created in the previous step. When a new project is created, a default network is created. You may skip this step if you do not need to customize the network address range by creating a new network, or go on to the next step if you have done so. Note that the Aviatrix Controller handles a Google Cloud network like a VPC in AWS. Whenever a network configuration is mentioned for Google, the term VPC is used. (The VNet is used for Azure.) At the Google Cloud console, select the project that you copied the Aviatrix Controller image to. Click the 3 bars. At the dropdown menu, select *VPC Network*. Click *\[+] Create Network*. Use the automatic subnet creation mode; otherwise ensure there is a subnet allocated for each region where a Gateway will be deployed. > [!NOTE] > > If you plan to have multiple projects, we suggest you plan your subnets so > that the network addresses do not overlap. Select *Custom* to create subnets. #### Saving the Management CIDR Range Find and save the public IP address **CIDR range** for the device you will initially use to run the Aviatrix Controller and CoPilot. > [!NOTE] > > To find a device's IP address and determine this CIDR range, search for "what > is my IP" on the browser's search engine. ### OCI Prerequisites Find and save the public IP address **CIDR range** for the device you will initially use to run the Aviatrix Controller and CoPilot. > [!NOTE] > > To find a device's IP address and determine this CIDR range, search for "what > is my IP" on the browser's search engine. ## Controller Requirements This section discusses the system and platform requirements of Aviatrix Controller. For larger environments, the CoPilot-related resource consumption is increased and the Controller must be resized accordingly. The Controller size is defined according to the footprint and the number of gateways it needs to support. Controller size is adjustable based on future requirements and enhancements. For example, if a t3.2xlarge controller instance is deployed in AWS this supports less than 30 Aviatrix gateways (t3.small) to start with, and sizing can be increased as needed when expanding the footprint. See [CoPilot platform requirements](/docs/enterprise/10.1/reference/controlplane/copilot-platform-requirements) . ### Minimum Sizing for Controller For a simple deployment, Controller (with CoPilot) requires a *minimum* of: | Requirement | Minimum Value | | - | - | | Memory & CPU | 4 CPU (or more) | | | 16 GB of RAM (or more) | | | Examples: | | | Amazon EC2 instance type: t3.xlarge | | | Google Cloud: n2-standard-4 | | | Oracle Cloud Infrastructure: VM.Standard3.Flex or VM.Standard.E4.Flex | | | Microsoft Azure virtual machine: Standard\_D4\_v5 | | Storage | 100 GB | ## Next Steps After completing the prerequisites for the specific Google Cloud or OCI CSP marketplace, see the relevant Getting Started Guide to deploy your Controller in one of the CSPs: * [Deploy Aviatrix Controller in GCP CLI](/docs/enterprise/10.1/guides/controlplane/manual-launch-controller-csp) * [OCI Getting Started Guide](https://legacy.docs.aviatrix.com/documentation/latest/getting-started/getting-started-guide-oracle.html) # CoPilot Settings You Can Enable in Controller Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/controlplane/copilot-controller-settings By default, CoPilot is already associated with your Controller after CoPilot deployment. If you want to reset this association, such as to point Controller to a different CoPilot instance or to recover from a lost association, use the following steps. ## Associating Controller with CoPilot By default, CoPilot is already associated with your Controller after CoPilot deployment. If you want to reset this association, such as to point Controller to a different CoPilot instance or to recover from a lost association, use the following steps. 1. In Aviatrix Controller, go to the **Environment** tab. This is the default landing page after you log in. 2. If a CoPilot instance is already associated, click **Reset Association** to clear the existing association before associating a different instance. 3. Click **Deploy CoPilot**, then select **Associate CoPilot** from the dropdown. 4. Enter the **Public IP/FQDN** and **IP Address** of your running CoPilot instance, then click **Save**. > [!NOTE] > > The **Environment** tab does not support entering a separate CoPilot FQDN for > browser-based redirects (such as the redirect Controller performs after a SAML > login) apart from the address used for gateway log delivery. To configure that > field, use Legacy Controller UI, as described in the next section. ## Setting a Separate CoPilot FQDN for Browser Redirects > [!NOTE] > > This field is only available from Legacy Controller UI. Click **Legacy > Controller UI** in the Controller header to access it. 1. From Legacy Controller UI, go to **Settings > CoPilot > CoPilot Association** and check whether your Controller is associated with CoPilot. 2. For **CoPilot FQDN (Optional)**, enter a hostname if your browser needs to reach CoPilot at a different address than the one gateways use to send logs. Use this field when "Public IP" points to an endpoint that is not suitable for browser access, such as a load balancer configured for the mTLS connection that gateways use to deliver logs. When "CoPilot FQDN" is set, Controller and CoPilot use it for browser-based redirects, including the redirect Controller performs after a SAML login. If this field is blank, "Public IP" continues to be used for these redirects, matching the previous behavior. The value must be a valid IP address or DNS hostname. Leave this field blank if a single address already works for both browser access and log delivery. No action is required for an existing CoPilot Association. 3. Click **Save**. ## Enabling Controller to Manage CoPilot Security Groups The CoPilot Security Group Management feature in Controller is available starting from Controller release 6.8. The feature is available for AWS and Azure CSPs and is enabled by default. To enable or disable Security Group Management, see [CoPilot Security Group Management](/docs/enterprise/10.1/reference/controlplane/copilot-security-group-management) for details. # Deploying Your CoPilot Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/controlplane/copilot-deployment This section discusses how to deploy Aviatrix CoPilot. ## About Deploying CoPilot This section discusses how to deploy Aviatrix CoPilot. Aviatrix CoPilot is available as an all-in-one virtual appliance that is hosted in your own IaaS cloud environment. It can be launched as an EC2 instance in AWS, a virtual machine in Azure, or a VM instance in GCP and OCI. For information about the latest Aviatrix CoPilot base image releases, see [Aviatrix Controller and Gateway Image Release Notes](/docs/enterprise/10.1/reference/release-notices/release-notes/controller-images/index). Typically, Aviatrix Controller and Aviatrix CoPilot are run in the same VPC/VNet. However, Aviatrix Controller and CoPilot are not required to be collocated. It is possible to run them in separate VPCs/VNets or separate cloud providers (in multicloud environments). Before launching CoPilot: 1. Ensure that you have reviewed all the topics under the [Planning Your CoPilot Deployment](https://legacy.docs.aviatrix.com/documentation/latest/getting-started/copilot-vm-instance-sizing.html#vmsizing/copilot-planning-deployment). * Take note of the minimum instance (VM) system requirements for CoPilot and guidelines for instance sizing. See [Instance (Virtual Machine) Sizing for CoPilot](https://legacy.docs.aviatrix.com/documentation/latest/getting-started/post-deploy-copilot.html). * Subscribe to the Aviatrix offers in the CSP marketplace: "Aviatrix Cloud Network CoPilot" on AWS and "Aviatrix CoPilot" on Azure, GCP, and OCI. * Perform other prerequisite tasks. See [Planning Your CoPilot Deployment](/docs/enterprise/10.1/guides/controlplane/copilot-planning-deployment) . 2. Choose a deployment method. See [CoPilot Deployment Methods](/docs/enterprise/10.1/guides/controlplane/deployment-methods) . 3. After launching CoPilot, perform all the [Post-Deployment Tasks for CoPilot](/docs/enterprise/10.1/guides/controlplane) . \* You must perform all the initial setups of CoPilot. See [Initial Setup of CoPilot](/docs/enterprise/10.1/guides/controlplane/initial-setup) . \* You must configure integration points for CoPilot to connect and communicate with other components in the Aviatrix platform. See [CoPilot Settings You Can Enable in Controller](/docs/enterprise/10.1/guides/controlplane/copilot-controller-settings) . \* [Verify the CoPilot Deployment and the Connectivity with Controller](/docs/enterprise/10.1/guides/controlplane/copilot-deployment-verification) If your CoPilot deployment is for the purpose of migrating CoPilot data from an existing (source) CoPilot to a newly launched (destination) CoPilot, see [Migrating CoPilot Data](/docs/enterprise/10.1/guides/controlplane/copilot-migrate). ## CoPilot and Controller Interoperability CoPilot is deployed as an all-in-one virtual appliance and is available on multiple clouds including AWS, Azure, GCP, and OCI MarketPlaces. CoPilot works in tandem with Aviatrix Controller; in order to use CoPilot, you must have an operational Aviatrix Controller. CoPilot works with all Controller versions that are not End of Life (EOL). If you need information about [Aviatrix End of Engineering and End of Life (Support) policies](https://support.aviatrix.com/Aviatrix-EOL-Policy) or how to plan and update to the latest software, please contact Aviatrix Support. # Verify the CoPilot Deployment and the Connectivity with Controller Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/controlplane/copilot-deployment-verification You are now successfully logged into CoPilot. To verify Copilot has connected successfully to your Aviatrix Controller, perform the following steps: You are now successfully logged into CoPilot. To verify Copilot has connected successfully to your Aviatrix Controller, perform the following steps: 1. From the CoPilot dashboard, confirm that you can see your resource inventory across all clouds in your multicloud network that is managed by Aviatrix Controller. > [!NOTE] > > The expected page load time in the CoPilot UI is less than 10 seconds. If > you see significantly slower behavior, contact Support for assistance. 2. Confirm that the inventory tiles show the number and status of each of your managed resources and that the global location of your managed VPCs/VNets is represented on the geographic map. 3. After deployment, the CoPilot virtual machine ports 31283 and 5000 will be open for any IP (0.0.0.0/0). It is strongly recommended to remove the 0.0.0.0 entry from the CoPilot security group for these ports and add entries for all of your gateway IP addresses as described in the next steps. > [!NOTE] > > In Controller 6.8, the CoPilot Security Group Management feature (From > Controller User Interface, *SETTINGS > CoPilot > CoPilot Security Group > Management*) is automatically enabled so this configuration is expected to > be completed. This is only a verification step. 4. (Verify NetFlow Agent is enabled in Controller) To use the FlowIQ feature in CoPilot, ensure that the Controller is configured to forward NetFlow logs to CoPilot. > [!NOTE] > > In Controller 6.8, the CoPilot Netflow Agent feature is automatically > enabled so this configuration is expected to be completed. This is only a > verification step. a. Log in to Aviatrix Controller UI. b. Go to *SETTINGS > Logging > NetFlow Agent*. c. Use the static IP address of CoPilot as the Netflow server IP and UDP port 31283 (default, port is configurable). d. Use version 9. e. Click *EDIT OPTIONS* to enable the edit the Netflow Agent settings. Tick the Advanced check-box. In Gateways, verify all of your Aviatrix gateways are in the Include List. f. Click *Enable*. > [!NOTE] > > If you launch new gateways from your Controller later, you must transfer the newly launched gateways to the Include List here. In addition, in your native cloud console, you must open your CoPilot security group for UDP 31283 from each newly launched gateway. 5. (Verify Remote Syslog is enabled in Controller) Remote syslog index 9 is used for the *Settings > Logging > Remote Syslog* page. Ensure the controller is configured to specify CoPilot as the loghost server. > [!NOTE] > > In Controller 6.8, the CoPilot Security Group Management feature (in > Controller > SETTINGS> CoPilot > CoPilot Security Group Management) is > automatically enabled so this configuration is expected to be completed. > This is only a verification step. a. Log in to Aviatrix Controller. b. Go to *SETTINGS > Logging > Remote Syslog*. c. Choose Profile Index 9. d. Click *EDIT OPTIONS* to edit the Remote Syslog settings. e. In Enable Remote Syslog, enter the profile name you want to use, the static IP address of CoPilot as the server, and UDP port 5000 (default). f. If you want to use custom certificates, choose your custom Server CA certificate, Client certificate, and Client Private Key. g. Tick the Advanced checkbox. In Gateways, verify all of your Aviatrix gateways are in the Include List. h. Click *Enable*. > [!NOTE] > > If you launch new gateways from your Controller later, you must transfer the newly launched gateways to the Include List here. In addition, in your native cloud console, you must open your CoPilot security group for UDP 5000 from each newly launched gateway. # Migrating CoPilot Data Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/controlplane/copilot-migrate This section discusses migrating CoPilot data from old CoPilot to new CoPilot. It includes: This section discusses migrating CoPilot data from old CoPilot to new CoPilot. It includes: * [About Migrating CoPilot Data](#about-migrating-copilot-data) * [Supported CoPilot Data Migration Paths](#supported-copilot-data-migration-paths) * [CoPilot Data Migration Preparations](#copilot-data-migration-preparations) * [Perform Prerequisite Tasks for Migrating CoPilot Data](#perform-prerequisite-tasks-for-migrating-copilot-data) * [CoPilot Data Migration Procedure](#copilot-data-migration-procedure) * [Verify CoPilot Data Migration](#verify-copilot-data-migration) * [Post CoPilot Migration Tasks](#post-copilot-migration-tasks) * [Troubleshoot CoPilot Data Migration](#troubleshoot-copilot-data-migration) * [Cancel CoPilot Data Migration](#cancel-copilot-data-migration) ## About Migrating CoPilot Data Starting with CoPilot release 2.0.3, you can migrate data from one (source) CoPilot instance to another (destination) CoPilot instance. Data migration is supported across regions, availability zones, and VPCs/VNets within the same CSP. Migration is not supported across CSPs. The migrated data includes the indexes shown in Settings > Index Management. The indexes are migrated from the data disk (volume) of the source instance to the data disk (volume) of the destination instance. Configuration data for CoPilot functions are also migrated, including but not limited to data for notifications, alerts, network behavior analytics configurations, threat IP configurations, and Geoblocking configurations. You would migrate CoPilot data when upgrading to a new CoPilot *image release* version. After launching a new CoPilot instance based on the new image release version, you would migrate data from the old instance to the new instance. You would also migrate CoPilot data any time you want to deploy a new instance of CoPilot and retain your data from the old instance. You can launch the new instance through the same CSP marketplace portal as your existing instance, by using the controller UI (starting from Controller 6.7.1185 for AWS Cloud only), or by using Terraform scripts. For instructions on migrating CoPilot data from a source to a destination instance, see [Migrate data from one CoPilot instance to another](#migrate-data-from-one-copilot-instance-to-another). ## Supported CoPilot Data Migration Paths CoPilot supports [Migrating data from one CoPilot instance to another](#migrate-data-from-one-copilot-instance-to-another) ### Data Migration Flow for CoPilot Instance(s) #### Migrate Data from One CoPilot Instance to Another Data will be migrated from the old CoPilot data disk to the new CoPilot data disk. ## CoPilot Data Migration Preparations These instructions for migrating CoPilot data apply to the migration paths that are listed in [Supported Migration Types](#supported-copilot-data-migration-paths). The following terms are used in these instructions: * *old copilot* — Refers to your current (source) CoPilot instance that you want to migrate data from. * *new copilot* — Refers to your newly deployed (destination) CoPilot instance that you want to migrate data to. ### Notes about Data Migration Process Please consider the following points about the data migration process in the current release: * Migration of CoPilot data is not supported across CSPs. Data migration is supported across regions, availability zones, and VPCs/VNets of the same CSP. * Aviatrix has tested data migration for infrastructures with up to a total of 500 GB of data. If you have a much larger infrastructure, please contact Aviatrix Support for more information about how to migrate your data. * If data migration fails and you want to retry the migration, please contact Aviatrix Support for assistance. * If the data migration utility fails to migrate all indexes, you cannot revert the migration but the data remains intact on the old copilot. * During the migration process, you cannot make configuration changes to the *old copilot*. * Upon starting the data migration, the netflow and syslog data sent by your Aviatrix gateways is automatically switched to be sent to your *new copilot*. If the migration fails, and you decide to terminate your *new copilot*, the netflow and syslog data that was directed to the *new copilot* during migration will be lost. * You can pick or customize the restore time period. However, we will restore the available data if the chosen time period does not have data. * If data migration fails and you decide to cancel the data migration, you can terminate your *new copilot* and continue to use your *old copilot* (by following the instructions indicated in the procedure). CoPilot data may take a few hours for small environments and a few days for large environments. Environments with 500 GB of data may take 3 to 4 days to complete data migration. The migration process will not cause down-time in your data plane but your CoPilot will not be available to receive new data for about 10 minutes after migration begins and 5 minutes after migration succeeds or fails. It is best practice to plan the migration during a maintenance window. You can perform prerequisite tasks outside your maintenance window to save valuable time during the maintenance window. Please note that while you can use your *old copilot* during the migration process, any changes you make during the migration will NOT be reflected on the *new copilot*. ## Perform Prerequisite Tasks for Migrating CoPilot Data Before you begin the data migration process, perform the following tasks. Prerequisite tasks can be performed outside your maintenance window. When logging in to CoPilot, use a controller user account that has full admin permissions. To confirm that the user account has full admin permissions, log in to your Controller, go to Accounts > Account Users, and verify the "Permissions Groups" column is set to *admin* for the account in question. ### Obtain the Required Information Obtain the following information: * The IP address of your Aviatrix Controller. * The IP address of your *old copilot*. The *old copilot* IP address can be the private IP, public IP, or Elastic IP address (EIP) used for reachability of the instance. * The amount of *storage used* on your *old copilot* for the data volume containing CoPilot data. To look up the storage used, navigate to CoPilot > Settings > Resources and refer to the Used column of the volume. If your CoPilot is based on CoPilot image release version 1.5.1, you will see a *cpltLV* volume listed on the Resources page. In that case, refer to the Used column of the cpltLV volume to take note of the storage used. ### Launch Your New Copilot Make sure your new CoPilot meets the following: * The size of the disk/volume you specify for the Instance is the same size or larger than the *storage used* in your *old copilot*. If you deploy in AWS using the controller UI deploy process, you specify the size in the "Data Disk" column for the instance. * Take note of your *new copilot* IP address. The new copilot IP address can be the private IP, public IP, or Elastic IP address (EIP) used for reachability of the instance. It is highly recommended that a persistent IP is used such as an EIP or statically assigned private IP. * After launch, your *new copilot* will take about an hour to automatically update to the latest software release version. Your *new copilot* and *old copilot* must be the same software version before starting data migration. You will verify this later when following the data migration procedure. ### Open Required Ports on Each CoPilot Instance At the applicable CSP portal, on the **new copilot** VM: > [!NOTE] > > After initial deployment, your *new copilot* ports 31283 and 5000 will be open > for any IP (0.0.0.0/0) . It is strongly recommended to remove the 0.0.0.0 > entry from the inbound access rules for these ports and add entries for all > your gateway IP addresses. * Open port 443 to receive TCP traffic from the *old copilot* (*old copilot* IP address). * Open port 31283 to receive UDP traffic from each of your Aviatrix gateways. * Open port 5000 to receive UDP traffic from each of your Aviatrix gateways. *For private mode*, you open *TCP* port 5000. * Open port 31284 to receive TCP traffic from the Aviatrix Controller and each of your Aviatrix gateways (OpenTelemetry metrics and logs). > [!NOTE] > > If you launched your *new copilot* from the Controller UI starting from > Controller release 6.8, the above security group configurations will be > automatically applied. At the applicable CSP portal, on the **old copilot** VM: * Open port 9200 to receive TCP traffic from the *new copilot* (new copilot IP address). * Open port 443 to receive TCP traffic from the *new copilot* (new copilot IP address). ### Verify the aviatrix-app-policy Permission has been Added to Your Access Account 1. Verify that your Controller instance ([Access Account](/docs/enterprise/10.1/guides/platform-administration/aviatrix-account)) has the IAM role `aviatrix-role-ec2` attached to it. Make sure the `aviatrix-app-policy` policy has been added to the `aviatrix-role-ec2` role. See details about IAM roles in [AWS IAM Overview](/docs/enterprise/10.1/reference/general/iam-role). 2. If the `aviatrix-app-policy` policy has not been added yet, perform the following steps: 1. Go to your Controller UI. 2. From *Access Accounts*, choose your account name. 3. Click the *EDIT* button. 4. On the editing page, click the *LAUNCH CLOUDFORMATION SCRIPT* button to create the `aviatrix-app-policy`. > [!NOTE] > > It is important to note that the "aviatrix-app-policy" policy is a critical > component in order to ensure that you will be able to migrate your CoPilot > data. See more detail about IAM policies in > > [IAM Roles and Policies](/docs/enterprise/10.1/reference/general/iam-role) > > . ## CoPilot Data Migration Procedure > [!NOTE] > > The data migration steps listed below apply to the migration paths that are > listed in [Supported CoPilot Data Migration > Paths](#supported-data-migration-types). To migrate CoPilot data from your *old copilot* to your *new copilot*: 1. Verify you have performed all prerequisite tasks, including launching your new CoPilot and opening required ports as described in [Perform Prerequisite Tasks for Migrating CoPilot Data](#perform-prerequisite-tasks-for-migrating-copilot-data) 2. Open the *new copilot* application in your web browser at `https://`, where `` is the IP address of your *new copilot* instance. For AWS clouds, this would be your Elastic IP (EIP) address. 3. Go through the initial setup process to enter information about your *new copilot* when prompted (see "Initial Setup of CoPilot" for information about each prompt). When prompted to enter the IP address of your controller, make sure you enter the IP address of the same controller your *old copilot* is pointing to. 4. Verify the software version of your *new copilot* and *old copilot* are the same (they should both be software version 2.0.3 or later). 5. Verify the Performance feature version on your *new copilot* and *old copilot* are the same (they should both use Performance v2 or v1). It is likely your *old copilot* is already using Performance v2 (the *Switch to v2* option in the Performance page was activated). In this case, on your *new copilot*, select the Performance sidebar option, and when prompted, click *Switch to v2*. If your *old copilot* is using Performance v1 and your *new copilot* is using v2, click *Switch to v2* on your *old copilot*. 6. Go to the *new copilot* UI. From *Settings*, click *Migration*. 7. Enter the IP address of your *old copilot* for the *Old CoPilot IP* field. 8. Pick a time period for the data that you want to migrate from your *old copilot* to your *new copilot*. * You can choose *All Available Data* to restore all available data. Alternatively, you can choose a specific period from the drop-down list of the *Time Period* field. * You can also provide a custom time period for the migration data. However, if the custom time period does not exist, we use the available data under the available time period for data migration. 9. Check the estimated size and estimated time of the data that you want to restore. 10. Click *Migrate*. Migrating CoPilot data may take a few hours for small environments and a few days for large environments. Environments with 500 GB of data may take 3 to 4 days to complete data migration. If you use your *old copilot* during the migration process, changes you make during migration will NOT be reflected in the *new copilot*. Upon starting the data migration, the netflow and syslog data sent by your Aviatrix gateways is automatically switched to be sent to your *new copilot*. 11. *If data migration succeeds (Migration Complete):* If all data indices migrate successfully, you will get a Migration Complete message. See these sections to [Verify the Data Migration](#verify-the-data-migration) and [Post CoPilot Migration Tasks](#post-copilot-migration-tasks). 12. *If data migration fails* (*Error: Failed to migrate. Please check the log below.*), see [Troubleshoot CoPilot Data Migration](#troubleshoot-copilot-data-migration) ## Verify the Data Migration To verify the data are migrated on the *new copilot*: * Go to *Settings > Index Management*. Verify all indexes are visible. * Navigate to the CoPilot functional screens and verify you can see your configuration data for notifications, alerts, anomalies, threat IPs, Geoblocking, and other configurations. ## Post CoPilot Migration Tasks After your *new copilot* is running with your migrated data intact, you can perform the following tasks: * On your *new copilot*, if you use your own SSL certificate (rather than the CoPilot self signed certificate), update the DNS servers associated with your certificate in Settings > Configuration. * Remove your *old copilot* and associated cloud resources: * EIP * Access/security groups * Disks/Volumes * VM/Instance * On your Aviatrix Controller, verify that the CoPilot association (Controller > Settings > CoPilot) is pointing to your *new copilot* IP. If you used the controller UI to launch your *new copilot*, this was automatically done for you. Additionally, verify that the Remote Syslog and Netflow Agent (Controller > Settings > Logging) are pointing to your *new copilot* IP. ## Troubleshoot CoPilot Data Migration If one or more data indices do not migrate, you will get an *Error: Failed to migrate.* message followed by a list of failed indices in the error message. Try the following methods to troubleshoot. * Retry the migration at least once by doing the following: 1. Click the *Clear Migration* button. This clears the indexes in the status page of the user interface. It does not delete migrated indices. 2. In the *Migrate Data from Old CoPilot* dialog, re-enter the IP address of your *old copilot* and tick the checkbox for acknowledging prerequisites. 3. Click *Migrate*. The migration process will continue to migrate data from where it left off. * If the migration process continues to fail, you have the following options: * If you decide you want to continue to retry the data migration, please contact Aviatrix Support. * If you decide the indices that failed to migrate are not important (for example, they are very old records you no longer need), you can choose to use the *new copilot*. In this case, go to [Verify Data Migration](#verify-the-data-migration) to verify the important data you want is indeed in your *new copilot* before deleting your *old copilot*. ## Cancel CoPilot Data Migration If you decide to cancel the data migration and continue to use your *old copilot*, do the following: 1. At the applicable CSP portal for the *new copilot*, stop the instance/virtual machine. 2. Turn on the Task Server on your *old copilot* (CoPilot > Settings > Services > Task Server). 3. Configure your controller to send netflow data to your *old copilot* (Controller > Settings > Logging > Netflow Agent. See "Enable Netflow for CoPilot FlowIQ Data" for details.) 4. Configure your controller to send syslog data to your *old copilot* (Controller > Settings > Logging > Remote Syslog. See "Enable Syslog for CoPilot Security Audit Data" for details.) 5. At the applicable CSP portal for the *old copilot* VM, you can remove the access rules that were added to open TCP ports 9200 and 443 from the *new copilot* source IP. 6. Remove your new copilot and its associated cloud resources. # Planning Your CoPilot Deployment Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/controlplane/copilot-planning-deployment This section discusses prerequisite information and tasks for deploying Aviatrix CoPilot only with Google Cloud (GCP) and OCI (Oracle Cloud Infrastructure). This section discusses prerequisite information and tasks for deploying Aviatrix CoPilot only with *Google Cloud (GCP) and OCI (Oracle Cloud Infrastructure)*. For *Microsoft Azure* deployment information, see [Azure Getting Started Guide](/docs/enterprise/10.1/guides/controlplane/getting-started-guide-azure). For *AWS deployment* information, see [AWS Getting Started Guide](/docs/enterprise/10.1/guides/controlplane/getting-started-guide-aws) . > [!TIP] > > Ensure you have already > [deployed Aviatrix Controller](/docs/enterprise/10.1/guides/controlplane/controller-planning-deployment) > , either from a CSP or via Terraform, before deploying CoPilot. Aviatrix > CoPilot works in tandem with Aviatrix Controller. ## Subscribe to the Aviatrix CoPilot Offer in the Marketplace For *Microsoft Azure* and *AWS* deployments, use the Aviatrix launch experience at [launch.aviatrix.com](https://launch.aviatrix.com). For more information, see the [Azure Getting Started Guide](/docs/enterprise/10.1/guides/controlplane/getting-started-guide-azure) or [AWS Getting Started Guide](/docs/enterprise/10.1/guides/controlplane/getting-started-guide-aws) . For a CoPilot deployment on *Google Cloud* or *OCI*, the first step is to log in to the CSP marketplace and subscribe to the CoPilot AMI. Search for the "Aviatrix CoPilot" offer. Consult with your Aviatrix Sales Representative for the subscription you require if you are not sure. To subscribe to a CoPilot offer in the cloud marketplace for Google Cloud or OCI use these steps: 1. Log in to the marketplace of your chosen cloud provider using your provider user account credentials. 2. Locate the "Aviatrix CoPilot" subscription offer and click *Continue to Subscribe*. Use the latest base image release version of the CoPilot AMI that is listed on the marketplace. For information about the latest Aviatrix CoPilot base image releases, see [Aviatrix CoPilot Image Release Notes](/docs/enterprise/10.1/reference/release-notices/release-notes/copilot-images/index) . 3. When prompted, review the subscription pricing information and accept the terms and conditions. You might be prompted to confirm your subscription before moving on to configuration. * If you want to deploy CoPilot via the Controller UI or via Terraform scripts, you can stop here and refer to the instructions for each deploy method. # Deleting a CoPilot Instance Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/controlplane/delete-copilot This section discusses deleting a deployed CoPilot instance. This section discusses deleting a deployed CoPilot instance. ## Deleting an Existing CoPilot Instance You may need to delete (terminate) an existing CoPilot instance. For example, after you [migrate CoPilot data](/docs/enterprise/10.1/guides/controlplane/copilot-migrate) to a new CoPilot instance, it is recommended that you delete the old instance. If you deployed your CoPilot by using Terraform scripts or from the CSP marketplace, you must manually delete the CoPilot instance from the cloud provider environment. When deleting the CoPilot from the cloud provider environment, delete associated cloud resources: * EIP (if applicable) * Access/security groups * Disks/Volumes * VM Instance If you deployed your CoPilot from the Controller user interface, you can delete the *last-deployed* CoPilot instance via the Controller user interface as described in the next section. > [!WARNING] > > Do not use that option if you want to delete an old (source) CoPilot instance > after deploying a new (destination) CoPilot instance as that option will delete > your last-deployed new instance. You must manually remove your old (source) > CoPilot instance after your new instance is functional. ## Deleting a CoPilot Instance Deployed from Controller If you deployed your CoPilot from the Controller user interface, you can delete the *last-deployed* CoPilot by using the **Delete Deployed CoPilot Instance** option in **Aviatrix Controller > Settings > CoPilot**. This method of deleting CoPilot is commonly used immediately after deployment when the instance was accidentally deployed into the wrong VPC. > [!WARNING] > > Do not use this option if you want to delete an old (source) CoPilot instance > after deploying a new (destination) CoPilot instance from the Controller as this > option will delete your last-deployed new instance. You must manually remove > your old (source) CoPilot instance after your new instance is functional. # Deploy CoPilot from Your Controller UI (AWS Only) Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/controlplane/deploy-copilot-controller You can deploy CoPilot from the Aviatrix Controller user interface with AWS. However, the recommended deployment method is by using the Launch Aviatrix setup assistant. See the AWS Getting Started Guide for more information. You can deploy CoPilot from the Aviatrix Controller user interface with AWS. However, **the recommended deployment method** is by using the [Launch Aviatrix](https://launch.aviatrix.com) setup assistant. See the [AWS Getting Started Guide](/docs/enterprise/10.1/guides/controlplane/getting-started-guide-aws) for more information. Before you [Launch a CoPilot Instance from Controller UI (AWS Only)](#launch-a-copilot-instance-from-controller-ui-aws-only), make sure you have reviewed all the planning topics. ## Launch a CoPilot Instance from Controller UI (AWS Only) > [!NOTE] > > You can only deploy a CoPilot Instance from the Controller UI with AWS. This > option is not available for other CSPs. However, **the recommended deployment > method** is by using the [Launch Aviatrix](https://launch.aviatrix.com) setup > assistant. See the [AWS Getting Started Guide](/docs/enterprise/10.1/guides/controlplane/getting-started-guide-aws) > for more information. This feature is available starting from Aviatrix Controller release 6.7.1185. If you deployed Aviatrix Controller in AWS and you want to deploy Aviatrix CoPilot in AWS in the same region/AZ as the Controller, you can deploy CoPilot directly from the Controller UI by following these instructions. The instance will be launched in the same AWS region/availability zone as your Controller. If you want to deploy CoPilot in a different AWS region/availability zone or in a different cloud, follow the instructions in the Aviatrix [Getting Started Guide](/docs/enterprise/10.1/concepts-architectures/overview/index) for your cloud provider. Your Controller instance must have the `aviatrix-role-ec2` IAM role attached for the deployment to succeed. ### Prerequisite Tasks for a Single CoPilot Instance Launch Obtain the following information before you begin. You will need it for the initial setup of CoPilot after instance launch: * The login credentials of your Aviatrix Controller user account. * The login credentials for the user account to be used as the CoPilot service account. See [CoPilot User Account Administration](/docs/enterprise/10.1/reference/controlplane/copilot-user-account-administration). * The CoPilot customer ID obtained from your Aviatrix Sales representative. * Verify that your AWS account has been created in your Controller (you create your AWS account by navigating to Controller UI, from **ACCOUNTS > Access Accounts > +ADD NEW**). * Verify that the IAM role `aviatrix-role-ec2` is attached to the Controller instance Access Account. The CoPilot deployment will fail if this role is not attached to your Controller. Make sure The `aviatrix-app-policy` policy is also added to the `aviatrix-role-ec2` role if you need to migrate your CoPilot data in the future. If you are using private mode, note that CoPilot must have Internet access. ### Deploy CoPilot from the Controller UI To deploy CoPilot from the Controller UI (AWS Only): 1. If you haven't already done so, subscribe to a CoPilot offer: a. Log in to your account at the Amazon Web Services (AWS) Marketplace. If your company has multiple AWS accounts, be sure to log in to the same AWS account that you registered (onboarded) in your Controller. b. Locate the Aviatrix CoPilot software offer you want to subscribe to and click **Subscribe**. c. When prompted, review the subscription pricing information and accept the terms and conditions. > [!NOTE] > > You only need to subscribe, review the subscription pricing information, and > accept the terms and conditions in the marketplace before proceeding to the next > step. You would not move on to the configuration steps in the marketplace. 2. Log into Aviatrix Controller UI, and perform one of the following to begin the deployment process: * From the onboarding page, click the Aviatrix CoPilot tile. * From the action bar, click the app icon and select **Deploy CoPilot**. 3. On the **Deploy CoPilot** dialog, perform the following: a. Select **Simple Deployment**. b. (Optional) If you have backed up CoPilot configuration in an old CoPilot instance and want to restore your CoPilot configuration for this new CoPilot deployment, select **Restore from CoPilot Backup**. Then you will not be able to choose **Customize Deployment**. See details in [Copilot Backup Configuration](https://legacy.docs.aviatrix.com/documentation/latest/platform-administration/copilot/copilot-backup-configuration.html). c. (Optional) If desired, click **Customize Deployment** to enable the editing of the configuration. Otherwise, click **Deploy** to use the default configurations for the CoPilot instance. By default, the deployment process will deploy the CoPilot instance in the same region and same VPC as your Controller. d. (Optional) If desired, in **Region**, change the region in which to deploy the instance. Deploying the instance in the same region as the Controller is recommended to reduce latency. If you want to deploy the instance in a different region than where the Controller is deployed, choose the desired region from the drop-down box. e. (Optional) If desired, in **VPC**, change the VPC in which to deploy the instance. By default, the deployment process will deploy the instance in the same VPC as your Controller. If you want to deploy the instance in a different VPC than where the Controller is deployed, select the VPC from the VPC drop-down list. f. (Optional) In **Subnet**, specify the subnet (availability zone) in which to deploy the instance. > [!NOTE] > > Each CoPilot instance must be launched in a *subnet* that has outbound Internet > access. If you are using private mode, you also must select a *subnet* with > outbound Internet access in this step. g. (Optional) In **VM Size**, specify the VM size you want to provision for your instance. The default is a recommended minimum VM size for a single instance. See [Minimum Instance (VM) System Requirements for CoPilot](/docs/enterprise/10.1/reference/controlplane/copilot-platform-requirements) for information about instance sizing. h. (Optional) In **Data Volume (GB)**, you can accept the default disk volume size or specify the size you want. See [CoPilot Disk (Volume) Management](/docs/enterprise/10.1/guides/controlplane/disk-management). i. Click **NEXT**. j. Enter an CoPilot service account username and password. Then click **VERIFY**. k. Click **Deploy**. 4\. Wait until the deployment progress indicator reads **Complete** and then click **Close**. > [!NOTE] > > During the deployment process, the status dialog shows the details of your > CoPilot instance including the instance size, VPC name, region, and associated > AWS account number. You can copy the information from the status dialog to take > note of it. 5. **Wait for approximately 15 minutes.** It takes about fifteen minutes for the CoPilot application to initialize and upgrade. ## Log into CoPilot 1. From the Controller action bar, click the app icon and select **CoPilot**. 2. When prompted, enter the login and password of your Aviatrix Controller user account. 3. When prompted for a **CoPilot Service Account**, enter the login credentials of the user account you created previously on the Controller to be the service account. CoPilot opens to the Dashboard page. # Deploy CoPilot from the Marketplace Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/controlplane/deploy-copilot-marketplace CoPilot is the management interface for Aviatrix Platform. It is required if you want to implement features such as Distributed Cloud Firewall (DCF) or Network Insights API. CoPilot is the management interface for Aviatrix Platform. It is required if you want to implement features such as Distributed Cloud Firewall (DCF) or Network Insights API. To subscribe to Aviatrix CoPilot on AWS, Azure, Google Cloud, or OCI marketplace, search for and select the "Aviatrix CoPilot" AMI. For **AWS** and **Azure**, the quickest and simplest deployment method is to use the [Launch Aviatrix](https://launch.aviatrix.com) experience. If you start the CoPilot subscription from the AWS or Azure marketplace, you will be redirected to the Launch Aviatrix experience after you select the option to deploy CoPilot. See [AWS Getting Started Guide](/docs/enterprise/10.1/guides/controlplane/getting-started-guide-aws) or [Azure Getting Started Guide](/docs/enterprise/10.1/guides/controlplane/getting-started-guide-azure) for more information. To deploy a CoPilot instance on **Google Cloud or OCI** from the cloud provider marketplace, see [Deploy Aviatrix Controller in GCP CLI](/docs/enterprise/10.1/guides/controlplane/manual-launch-controller-csp) or [Oracle Cloud Infrastructure (OCI) Getting Started Guide](https://legacy.docs.aviatrix.com/documentation/latest/getting-started/getting-started-guide-oracle.html). > [!NOTE] > > CoPilot works in tandem with Aviatrix Controller to provide visibility into your > cloud resources managed by the Controller. It is assumed that you have a > Controller customer ID for your existing Controller. # Deploy CoPilot Using Terraform Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/controlplane/deploy-copilot-terraform If you are knowledgeable in deploying infrastructure-as-code using Terraform, you can deploy the Aviatrix CoPilot using a Terraform script rather than via the CSP marketplace. If you are knowledgeable in deploying infrastructure-as-code using Terraform, you can deploy the Aviatrix CoPilot using a Terraform script rather than via the CSP marketplace. For **AWS**, Terraform deployment is available through the [Launch Aviatrix](https://launch.aviatrix.com) setup assistant. ## Launch CoPilot Instance Using Terraform This section provides a summary of steps for launching a CoPilot single instance (simple deployment) using Terraform. > [!TIP] > > This section only provides a summary of steps. Please refer to the > documentation for the Aviatrix Terraform Module for CoPilot on GitHub here: > [https://github.com/AviatrixSystems/terraform-modules-copilot](https://github.com/AviatrixSystems/terraform-modules-copilot) for the most > current and detailed instructions. A CoPilot launched using Terraform is typically based on the latest CoPilot base image version. You can verify with your Aviatrix representative the CoPilot base image version being launched and record it for your records. ### Summary of steps for a CoPilot instance launch via Terraform 1. If you haven't already done so, subscribe to a CoPilot offer in the CSP marketplace. See [Subscribe to the Aviatrix CoPilot Offer in the Marketplace](/docs/enterprise/10.1/guides/controlplane/copilot-planning-deployment#subscribe-copilot-offer). You only need to subscribe, review the subscription pricing information, and accept the terms and conditions in the marketplace before proceeding to the next step. You do not move on to the configuration steps in the marketplace. Note that to use the latest CoPilot features, you must also subscribe to the [Aviatrix license](/docs/enterprise/10.1/guides/controlplane/aviatrix-licensing) . 2. Verify your CSP account credentials and make sure you know which CSP region in which to launch CoPilot. 3. Go to the Aviatrix repository GitHub - AviatrixSystems/terraform-modules-copilot ([https://github.com/AviatrixSystems/terraform-modules-copilot](https://github.com/AviatrixSystems/terraform-modules-copilot)) to access the sample code in the *README* file for the applicable CSP. The *README* file contains the usage for the CoPilot instance launch. 4. In the folder where you put your Terraform scripts based on the sample code, create a *.tf* file to provide your CSP account credentials (for example, provider block below). You can name the *.tf* file whatever you want. > [!TIP] > > Ensure your CSP account credentials are always in a secure location. 5. The following is an example of the format to enter your CSP account credentials. You can refer to your CSP Terraform Registry to verify the latest information that is required. Example of the format to enter your CSP account credentials : AZURE ```asciidoc theme={null} provider "azurerm" { features {} subscription_id = "" client_id = "" client_secret = "" tenant_id = "" } ``` GCP ``` provider "google" { project = "" region = "" zone = "" } ``` OCI ``` provider "oci" { tenancy_ocid = "" user_ocid = "" fingerprint = "" private_key_path = "" region = "" } ``` AWS ``` provider "aws" { region = "" access_key = "" secret_key = "" } ``` Refer to the Terraform Registry for the latest information. 6. Ensure that you set the variable for the region in which to launch the instance in the provider block. 7. Specify at least 1 data disk (volume) for Terraform to attach to your CoPilot instance for data storage. For information about CoPilot storage, see [CoPilot Disk (Volume) Management](/docs/enterprise/10.1/guides/controlplane/disk-management) . The following is sample code for building a single CoPilot instance. The variable default\_data\_volume\_name is required to create the data volume : ``` module "copilot_build_aws" { source = "github.com/AviatrixSystems/terraform-modules-copilot.git//copilot_build_aws" allowed_cidrs = { "tcp_cidrs" = { protocol = "tcp" port = "443" cidrs = ["1.2.3.4/32"] } "udp_cidrs_1" = { protocol = "udp" port = "5000" cidrs = ["0.0.0.0/0"] } "udp_cidrs_2" = { protocol = "udp" port = "31283" cidrs = ["0.0.0.0/0"] } } keypair = "copilot_kp" default_data_volume_name = "/dev/sdf" } ``` 8. Run terraform (terraform apply). The CoPilot instance is launched in the CSP as defined by the script's source field. If you do not specify an availability zone (using the variable `availability_zone`), an AZ that supports the instance type will be used. The AZ that is used will be listed in the output `ec2-info`. The instance launched is typically based on the latest CoPilot base image version. You can verify with your Aviatrix representative the CoPilot base image version being launched and record it for your records. 9. Wait to receive a success image message. 10. Launch the CoPilot application in your web browser: https: //copilot\_static\_ip\_address/ where copilot\_static\_ip\_address is the static IP address of your newly deployed CoPilot software instance/virtual machine. 11. Perform the initial setup of CoPilot. See [Initial Setup of CoPilot](/docs/enterprise/10.1/guides/controlplane/initial-setup). 12. Verify connectivity with your controller. See [Verify the CoPilot Deployment and the Connectivity with Controller](/docs/enterprise/10.1/guides/controlplane/copilot-deployment-verification) # CoPilot Deployment Methods Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/controlplane/deployment-methods For AWS and Microsoft Azure deployments, use the Aviatrix launch experience at launch.aviatrix.com. For more information, see the AWS Getting Started Guide and Azure Getting Started Guide. For **AWS and Microsoft Azure deployments**, use the Aviatrix launch experience at [launch.aviatrix.com](https://launch.aviatrix.com). For more information, see the [AWS Getting Started Guide](/docs/enterprise/10.1/guides/controlplane/getting-started-guide-aws) and [Azure Getting Started Guide](/docs/enterprise/10.1/guides/controlplane/getting-started-guide-azure). You can deploy Aviatrix CoPilot directly from Google Cloud (GCP) or Oracle Cloud Infrastructure (OCI) marketplaces, by using Terraform scripts. ## General Recommendation for Choosing Which Deployment Method General recommendations for which deployment method to use are as follows: * For AWS and Azure, the recommended method is to subscribe to Controller and CoPilot by using the [Launch Aviatrix](https://launch.aviatrix.com) experience. You can also deploy by using Terraform scripts or the Aviatrix Controller user interface (AWS only). * For Google Cloud or OCI, use Terraform scripts. ## Deploy from the CSP Marketplace Deploying from the CSP marketplace takes only a few clicks to provision and launch the instance. You must subscribe to a CoPilot offer in the marketplace as a first step for all deployment methods. See [Deploy CoPilot from the Marketplace](/docs/enterprise/10.1/guides/controlplane/deploy-copilot-marketplace) . ## Deploy by using Terraform Scripts If you are knowledgeable in deploying infrastructure-as-code using Terraform, it is convenient to launch CoPilot by using Terraform scripts. You can launch a single-instance deployment using Terraform scripts. A summary of steps for launching a single instance using Terraform is provided at [Deploy CoPilot Using Terraform](/docs/enterprise/10.1/guides/controlplane/deploy-copilot-terraform) . Please refer to the Aviatrix Terraform Module for CoPilot for the most current instructions and samples on GitHub here: [https://github.com/AviatrixSystems/terraform-modules-copilot](https://github.com/AviatrixSystems/terraform-modules-copilot). ## Deploy from Aviatrix Controller User Interface If your Controller is in AWS, you can launch a CoPilot simple (single) deployment from the Aviatrix Controller user interface. This deployment method has its advantages because the associated auto-deploy process configures security group rules that are required for CoPilot and Controller and gateway interconnectivity. See [Deploy CoPilot from Your Controller UI (AWS Only)](/docs/enterprise/10.1/guides/controlplane/deploy-copilot-controller) . # CoPilot Disk (Volume) Management Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/controlplane/disk-management Allocate data disks (volumes) to your Aviatrix CoPilot deployment to be used for expandable storage. Allocate data disks (volumes) to your Aviatrix CoPilot deployment to be used for expandable storage. When you initially provision CoPilot (from your cloud service provider), you must add a disk (volume) to be used for CoPilot storage. You are required to add at least one disk (volume) that is an additional disk (on the side) from the 25GB root disk that already comes with CoPilot. You create the data disk (volume) in your CSP account and attach it to your CoPilot instance. During instance provisioning, there is no minimum requirement for the disk/volume you add. You can choose the disk type (volume type) you want. You will be able to add more storage after deployment. When you newly launch the CoPilot instance, the initial setup process automatically detects the disk/volume you attached during instance provisioning. An add-disk process prompts you to confirm the disk/volume to use and then formats and attaches your disk(s). A logical disk/volume is created from all physical disks (volumes) you added during provisioning. For example, if you launch Copilot and you added two physical disks that are 8G each, CoPilot will form a 16G volume. The storage you need for CoPilot can increase based on several factors including the number of Aviatrix gateways launched and the type and volume of traffic in your network. Gateway node exporter metrics (`gw-metrics`) in particular grow with gateway count and, at larger fleet sizes, can use as much disk as NetFlow or DCF logs. See [Monitor Daily Size of Gateway Metrics Indices](#monitor-daily-size-of-gateway-metrics-indices) for reference numbers and tuning guidance. When you need more storage, you can add additional disks (volumes) by using the *CoPilot > Settings > Resources* page (Attach Additional Disk). For instructions, see [Add a Disk (Volume) for CoPilot Storage after Deployment](#add-a-disk-volume-for-copilot-storage-after-deployment). After you allocate new disks (volumes), you can only increase storage (you cannot decrease storage). For data disks you already allocated to your CoPilot deployment, you can increase their size. For instructions, see [Expand a Disk (Volume) for CoPilot Storage](#expand-a-disk-volume-for-copilot-storage). > [!NOTE] > > Disk (volume) management for expandable storage became available with the > release of Aviatrix CoPilot base image version 1.5.1. To use the expandable > storage feature and leverage additional disks/volumes, you must launch a new > CoPilot instance from the marketplace based on a CoPilot base image version > 1.5.1 or later (Appliance Version 2). ## Add a Disk (Volume) for CoPilot Storage after Deployment Add a data disk (volume) to your simple Aviatrix CoPilot deployment to be used for expandable storage. This procedure assumes you have a running CoPilot and want to add more storage (add a data disk/volume) above and beyond the storage you added when you first provisioned the CoPilot instance (from your CSP). To add a data disk (volume) for CoPilot expandable storage after deployment: 1. Log in to your CSP account and create the disk (volume) you want to add. Choose the disk type (volume type) that meets your business needs given the size of your environment and performance requirements. 2. Attach the disk (volume) to your CoPilot instance. Do *not* reboot the instance (a disk can be dynamically added to an instance on any CSP without rebooting). 3. After you receive confirmation from the CSP that the disk is attached, log in to CoPilot. 4. Go to *CoPilot > Settings > Resources > Disk Utilization* and click *Attach Additional Disk*. If you do not see the disk (volume) you created in the list, press the refresh icon. 5. Select the disk/volume you created and click START. 6. When the process is complete, click FINISH. ## Expand a Disk (Volume) for CoPilot Storage For data disks you already allocated to your CoPilot deployment, increase their size. To expand a disk (volume) that is allocated to your CoPilot deployment: 1. Log in to your CSP account and locate the disk (volume) attached to your CoPilot instance that you want to expand for CoPilot storage. 2. Increase the size of the disk (volume). 3. After you receive confirmation from the CSP that the disk is resized, log in to CoPilot. 4. Go to CoPilot > Settings > Resources. 5. In the Disk Usage table, expand the tree-pane icon (shown) for the physical volume associated with your resized data disk. ![tree-pane-icon-disk-usage-table](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/controlplane/tree-pane-icon-disk-usage.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=8eb5968aa2596a7ad0218601df02f332) 6. Click the enabled RESIZE button. CoPilot resizes the physical volume to match the size of your expanded disk. ## Monitor Data Disk (Volume) Utilization In the *CoPilot > Settings > Resources > Disk Utilization* page, CoPilot shows you the following: * Disk Usage section Shows a chart of *database disk used* broken down by the top five indices. This shows which indices are currently taking up the most space and the overall amount. * Estimated Disk Runway section Shows, in general, when the database would become full and how fast it is growing each day. Assuming data retention policies are working properly, it is expected the database will not run out of space since data is deleted regularly per the policies. * Data Retention Policy Enables you to set data retention policies for the data in CoPilot's logical disk (volume). ## Set Data Retention Policies for the Logical Disk (Volume) You can set data retention policies for the CoPilot data stored in the logical disk (volume) in *CoPilot > Settings > Resources > Disk Utilization*. You can use the *Critical Capacity Management* setting to configure overall retention for all indices. If you want the index retention to be longer for individual indices, you can scroll down the list of indices and configure their respective retention policies. ## Monitor Daily Size of NetFlow Indices You can check the daily size of NetFlow indices in your CoPilot storage in the *CoPilot > Settings > Resources > Index Management* page. If you click the filter icon and type `netflow` in the value column to create the filter `Index Name contains netflow`, CoPilot shows the netflow indices and their sizes. ## Monitor Daily Size of Gateway Metrics Indices Gateway node exporter metrics (`gw-metrics`) scale with the number of Aviatrix gateways CoPilot is monitoring. At larger fleet sizes, daily `gw-metrics` ingest can approach or exceed NetFlow and DCF logs under moderate traffic, which causes the disk retention policy to trigger sooner than expected. For sizing purposes, plan for approximately **50 MB per gateway per day** of `gw-metrics` storage. This translates to: | Gateway count | Approximate daily `gw-metrics` ingest | | - | - | | 100 gateways | \~5 GB/day | | 550 gateways | \~30 GB/day | | 1,000 gateways | \~50 GB/day | | 1,400 gateways | \~80 GB/day | Actual usage can vary with the number of external interfaces per gateway and other metric cardinality factors. You can check the daily size of `gw-metrics` indices in *CoPilot > Settings > Resources > Index Management*. Click the filter icon and type `gw-metrics` in the value column to create the filter `Index Name contains gw-metrics`. If `gw-metrics` is consuming more disk than expected for your retention window, either add storage (see [Add a Disk (Volume) for CoPilot Storage after Deployment](#add-a-disk-volume-for-copilot-storage-after-deployment)) or shorten the retention setting for the `gw-metrics` index in *Settings > Resources > Disk Utilization*. # Initial Setup of CoPilot Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/controlplane/initial-setup Perform initial setup of CoPilot after you have launched the instance/virtual machine in the cloud provider environment. ## Prerequisites for Initial CoPilot Setup Perform initial setup of CoPilot after you have launched the instance/virtual machine in the cloud provider environment. For initial setup of CoPilot, have the following information available: * The static public IP address of your recently deployed CoPilot software instance/virtual machine (obtained from the cloud provider portal). * The static IP address for your Aviatrix Controller. * The login credentials of your Aviatrix Controller user account. * The login credentials for the user account to be used as the CoPilot service account. > [!NOTE] > > This procedure assumes your Aviatrix Controller is up and running and the > controller instance's inbound rules have port 443 open to the public static IP > address of the CoPilot instance (so that CoPilot can reach your Controller). > If you launched your CoPilot instance from the Aviatrix Controller user > interface, the Controller instance's inbound rules will have port 443 open to > the public static IP address of the CoPilot instance. This procedure also > assumes your CoPilot software instance/virtual machine is up and running. ## Initial Setup of CoPilot To perform an initial setup of CoPilot: 1. (For marketplace deployment only), go to your Controller instance in your cloud service provider. Add your CoPilot IP address into your Controller security group with TCP protocol and port 443 on the cloud service provider so that CoPilot server can communicate with Controller API. 2. Launch CoPilot in your web browser: `https:///` where \ is the static public IP address of your newly deployed CoPilot instance/virtual machine. > [!NOTE] > > If you are deploying a Load Balancer in front of CoPilot, ensure that the > Load Balancer timeout value is at least ten minutes (as described in > [Configuring the Timeout Value for Load Balancer](/docs/enterprise/10.1/guides/controlplane/copilot-planning-deployment) > ). Alternatively, you can launch CoPilot in your web browser directly from Aviatrix Controller as described in the next step. 3. (Optional) To launch CoPilot in your web browser directly from Aviatrix Controller: a. In Aviatrix Controller, under Settings, select CoPilot. b. For the CoPilot Association, set the status to **Enabled**. c. In the **IP Address/Hostname field**, enter the static IP address for your running CoPilot instance/virtual machine. This can be the private or public IP address of your CoPilot instance/virtual machine. The IP address specified here is used for connectivity between the controller and CoPilot for intra-platform communication (such as API message exchanges). If CoPilot is located in the same VPC/VNet as your controller, specifying a private IP can increase operational bandwidth and potentially save on cost. If you enter the private IP address here, then go to the next step to specify the public IP address. d. If you specified the private IP address of your CoPilot instance in "IP Address/Hostname", enter the public IP address of your CoPilot instance in the **Public IP (Optional)** field. This enables you to open CoPilot in your web browser directly from your Controller. e. In the action bar of your Controller homepage, click on the application icon, and then select **CoPilot**. 4. When prompted, enter the username and password of a valid Aviatrix Controller user account and click Login. Enter the static IP address of your controller and click Login. 5. (Customer ID Required) If you are prompted for a **Customer ID**, then you likely deployed Controller without setting the ID. In this case, enter the Customer ID you received when you subscribed to the marketplace license. If you did not receive a Customer ID, please contact [Aviatrix Support](https://support.aviatrix.com). 6. When prompted for a **CoPilot Service Account**, enter the login credentials for a valid user account in Aviatrix Controller to be used as the CoPilot service account. See [CoPilot User Account Administration](/docs/enterprise/10.1/reference/controlplane/copilot-user-account-administration) . ![copilot\_login\_service\_account](https://mintcdn.com/aviatrix-14b37c43/0QYX2iFaLoliwUis/images/guides/controlplane/copilot-login-service-account.png?fit=max\&auto=format\&n=0QYX2iFaLoliwUis\&q=85\&s=9b1eb8a03af8d58e15f97369eb3a1156) 7. If a Data Disk Setup dialog does not appear, skip to step 8 to verify connectivity with your Controller. > [!NOTE] > > Do not toggle or move your mouse during the Data Disk Setup. 8. In Data Disk Setup, select the disk/volume you created for CoPilot storage and click START. When the process is complete, click FINISH. Note that when you launch CoPilot at first your version number will be based on the version of the image release. Within an hour, the CoPilot version will be updated to the latest software release. 9. [Verify the CoPilot Deployment and the Connectivity with Controller](/docs/enterprise/10.1/guides/controlplane/copilot-deployment-verification) . > [!NOTE] > > The expected page load time in the CoPilot UI is less than 10 seconds. If you > see significantly slower behavior, contact Support for assistance. # CA Certificate Rotation of Internal Service Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/controlplane/internal-service-ca-rotation Aviatrix gateways receive X.509 certificate identities from an internal Certificate Authority (CA) running on the Controller. This identity is used for securing the control-plane connections between gateway and controller, and certificate-based authentication for Site-to-Cloud (S2C)... ## Overview Aviatrix gateways receive X.509 certificate identities from an internal Certificate Authority (CA) running on the Controller. This identity is used for securing the control-plane connections between gateway and controller, and certificate-based authentication for Site-to-Cloud (S2C) connections between S2C and on-prem gateways. Rotate this CA when it is approaching expiration or when re-keying is required to maintain a secure gateway identity. This guide includes workflows of rotating the CA certificate for environments with and without certificate-based authentication for Site-to-Cloud (S2C) connections. ## Rotation Options * **Simple Rotation**: This is a standalone CA rotation. This option is for environments without S2C cert-based authentication. * **Staged Rotation**: This is a CA rotation with S2C Cert Authentication. This option is for environments relying on CA-signed S2C certificates. ## Perform a Simple Internal Service CA Rotation ### Prerequisites * Access to **Settings > Configuration > Certificate Store** * No S2C tunnels use certificate authentication * Ensure all gateways are up and running before each step ### Step 1: Prepare a New CA in Certificate Store 1. Go to **Settings > Configuration > Certificate Store**. 2. Click **+ Certificate**. 3. Choose **Generate Certificate** or **Upload Certificate**. We recommend generating a new certificate. For **Generate Certificate** (Recommended): * Enter a name for the new CA. * Select a validity period. For **Upload Certificate**: * Enter a name for the new CA. * Upload the CA Certificate file in PEM format. * Upload the CA Private Key file in PEM format. 4. Click **Add**. ### Step 2: Prepare the New CA for Internal Service 1. Go to **Networking > Connectivity > Settings**. 2. Click **Rotate Certificate**. 3. Select the new CA certificate and click **Prepare**. ### Step 3: Activate the New CA and Remove the Old CA > [!NOTE] > > This step will take some time. Do not close the browser window, navigate to > other pages, or refresh the page while this process is running. 1. From **Networking > Connectivity > Settings**, locate the newly **Prepared** CA entry in the listed Certificate table. 2. Click **Activate**. After activation, the new CA becomes active and the previously active CA is deactivated and will be removed automatically. ## Perform a Staged Internal Service CA Rotation with S2C Certificate Authentication Choose this workflow to rotate the Internal Service CA when using certificate-based authentication for Site-to-Cloud (S2C) connections. ### Prerequisites * The admin should have direct access or an indirect channel to update the trust bundle of on-prem gateways. ### Step 1: Prepare a New CA in Certificate Store 1. Go to **Settings > Configuration > Certificate Store**. 2. Click **+ Certificate**. 3. Choose **Generate Certificate** or **Upload Certificate**. We recommend generating a new certificate. For **Generate Certificate** (Recommended): * Enter a name for the new CA. * Select a validity period. For **Upload Certificate**: * Enter a name for the new CA. * Upload the CA Certificate file in PEM format. * Upload the CA Private Key file in PEM format. 4. Click **Add**. ### Step 2: Prepare the New CA for Internal Service 1. Go to **Networking > Connectivity > Settings**. 2. Click **Rotate Certificate**. 3. Select the new CA certificate and click **Prepare**. ### Step 3: Install the New Trust Bundle 1. Click **Download Trust Bundle** to download the bundle containing both current and new CA certificates. 2. Install the trust bundle on all on-prem gateways configured for certificate-based authentication connections. ### Step 4: Activate the New CA 1. Locate the **Prepared** CA. 2. Click **Activate**. > [!NOTE] > > Once you activate the new CA, it will be used and the old CA will be removed automatically. For any on-prem gateway that did not install the new trust bundle (from Step 3), new certificate-based authentication S2C tunnels cannot be established with Aviatrix S2C gateways after the activation. ### Step 5 (Optional): Install the Pruned Trust Bundle If the previously active CA is considered compromised, you must complete this step to keep S2C connections secure. Otherwise, you may ignore this step. 1. Click **Download Trust Bundle** to download the pruned bundle containing only the newly activated CA certificate. 2. Install this trust bundle on all on-prem gateways. 3. Ensure the previously active CA certificate is removed from each on-prem gateway's trust bundle. > [!TIP] > > This step is critical for maintaining security. Failure to remove the old CA certificate may allow unauthorized S2C connections. ## Related Topics * [Rotate the User VPN CA Certificate](/docs/enterprise/10.1/guides/uservpn/user-vpn-ca-rotation) # Migrate CoPilot to Appliance V3 (AWS) Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/controlplane/migrate-to-appliance-v3 The CoPilot migration to Appliance v3 on AWS is a necessary and essential step to enhance the security, availability, and performance of your CoPilot deployment. As the old version of CoPilot images will soon be deprecated, it is highly recommended to migrate your CoPilot to Appliance V3. The CoPilot migration to Appliance v3 on AWS is a necessary and essential step to enhance the security, availability, and performance of your CoPilot deployment. As the old version of CoPilot images will soon be deprecated, it is highly recommended to migrate your CoPilot to Appliance V3. This migration involves updating the CoPilot image, enabling a seamless transition CoPilot Appliance (v1 or v2) to the latest version (v3), or from CoPilot Appliance v3 to the latest version of Appliance v3. Additionally, it provides support for migrating existing CoPilot deployments running on Appliance v3 to the latest version of Appliance v3. > [!NOTE] > > We only support simple-deployment to simple-deployment migration and > migrations within the same region. Migration to a different region is not > supported. ## Overview The migration of CoPilot to Appliance v3 (AWS) covers all aspects of the CoPilot system, including the CoPilot configuration and CoPilot data. This migration enhances security hardening, expands availability in new regions and improves database performance. As a result, if you are migrating CoPilot to Appliance v3, there is no need to perform a separate data migration from *Settings > Maintenance > Upgrade > CoPilot*. ### Improvements of Appliance V3 * Enhanced Security Hardening: The migration to Appliance v3 includes additional security hardening measures. It is a mandatory security update that helps protect your data and infrastructure from potential vulnerabilities. * Expanded Availability: With the migration, CoPilot will be released in new regions, expanding its availability to a wider user base. Furthermore, the Appliance v3 will become the new default option in all regions, ensuring that you have access to the latest features and improvements. * Improved Database Performance: The migration introduces significant database improvements, resulting in faster queries and cached results. This means you can expect improved performance and efficiency when analyzing and visualizing data using CoPilot. ### Overview of the Migration Phases The CoPilot migration process includes the following phases: #### Phase 1: Backup In this phase, the existing CoPilot, including the CoPilot configuration and CoPilot data, is backed up to ensure no data loss during the migration. #### Phase 2: Deploy New CoPilot The program deploys a new CoPilot instance using the new Appliance v3 image. This updated version provides enhanced security, availability, and performance. #### Phase 3: EIP Switch The Elastic IP (EIP) associated with the CoPilot is switched from the old instance to the new instance. This ensures that the new CoPilot is accessible using the existing IP address. #### Phase 4: Restore Configuration and Data The program restores the CoPilot configuration and data to the new CoPilot instance. This step transfers all the settings and data from the old CoPilot to the new environment. #### Phase 5: Restore CoPilot Data The program initiates the restoration of CoPilot data to the new CoPilot instance. This ensures that all the data is available in the new CoPilot environment. After completing Phase 4, you will be directed to the new CoPilot login page. You can use the same credentials that you used for the old CoPilot to log in. ## Before CoPilot Migration > [!NOTE] > > Before proceeding with the migration of CoPilot to Appliance v3 (AWS), you must upgrade your Controller to the latest patch version of the release you are currently using. This prerequisite task ensures a smooth and successful migration process. > > If you are unable to migrate to a new CoPilot image on 6.8.1148, 6.9.128, > 7.0.1307 or 7.1.1710, please upgrade to the latest minor release or greater. Before proceeding with the CoPilot migration, make sure to complete the following preparation tasks: 1. **Upgrade your Controller**. You must upgrade your Controller to the latest patch version of the release you are currently using. 2. **[Back Up CoPilot Data](https://legacy.docs.aviatrix.com/documentation/latest/platform-administration/copilot/copilot-backup-restore.html?expand=true)**. It is highly recommended to perform a pre-migration backup of your CoPilot data that you want to migrate. This backup can help shorten the migration process and reduces the amount of data that might be lost during the migration. 3. Adjust Maximum Session Duration. Change the maximum session duration of the *AWS App Role* to 12 hours on *AWS > IAM > Roles*. See [Specify the Maximum Session Duration of an IAM Role on AWS](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use.html#id_roles_use_view-role-max-session). 4. Check Current CoPilot Region. During the migration, you will need to select a region for your new CoPilot. Ensure that you choose the same region as your current CoPilot. ### Check CoPilot Migration Notes Read through the migration notices before you perform the CoPilot migration: * Configuration Backup: If you had CoPilot configuration backup enabled on the old CoPilot, it will be disabled after the migration. You'll need to manually re-enable it on the new CoPilot if you want to continue backing up your configuration. * Simple to Simple Migration: We only support simple-deployment to simple-deployment migration and migrations within the same region. Make sure to perform the migration accordingly. * Data Backup: To speed up the migration process, it is highly recommended to perform a data backup of the date range that you want to migrate beforehand. This allows us to deduplicate the data and make the migration faster. In addition, it reduces the amount of data that might be lost during the migration. * Security Groups (SG): The new CoPilot's security groups are managed automatically through CoPilot SG Management feature. If you had custom SG settings and was not using SG management on the old CoPilot, you'll need to manually copy your old CoPilot's SG to the new CoPilot. This is a known issue. * FlowIQ and Performance Page Availability: After deploying the new CoPilot, the FlowIQ and Performance page might not be immediately accessible as netflow/perfmon data restoration is in progress. This is a temporary issue. * EIP Switch and Login: If you are on the old CoPilot UI when the EIP is switched, you might see a "Session timed out" message on the login page. In that case, do not log in. The new CoPilot will initialize within a minute, and you'll see a loading page. This is a known issue. * Potential Data Loss: During the migration, there is a possibility of a small amount of incoming data (such as performance monitoring and netflow data) being lost while the data restore process is ongoing. ### General Data Backup/Restore Notice It is recommended to set a more frequent backup frequency (daily or weekly). This provides greater granularity when selecting which backup to restore. Remember that increasing the backup frequency will not result in additional data stored in S3, as deduplication is applied. ## Migrate CoPilot to Appliance V3 (AWS) Currently, we only support simple-deployed to simple-deployment CoPilot migration. ### Procedures of Migrating CoPilot to Appliance V3 (AWS) Perform the following steps to migrate your CoPilot to Appliance v3. 1. Log in to your CoPilot. 2. From *Settings > Maintenance*, click *Upgrade*. 3. On the *Migrate to Appliance v3 (AWS Only)* tile, click *Migrate*. You can migrate from Appliance v1 to v3, v2 to v3 or from Appliance v3 to v3. 4. In the *Migrate to Appliance v3* window, click *Simple Deployment*. 5. Click *Next* to use the default deployment settings if your Controller and CoPilot are in the same region. Otherwise, you need to turn on the *Customize Deployment* Option to change the region to your current CoPilot region. To customize the deployment, toggle the *Customize Deployment* option to *On*, then perform the following steps: a. Choose *AWS* as the cloud provider. b. Choose your access Account. c. Choose a region as the deployment region. This region must be the same region as your current CoPilot. d. Choose a VPC/VNet. e. Choose a Subnet. f. Choose the VM size for CoPilot. See [CoPilot Requirements](/docs/enterprise/10.1/reference/controlplane/copilot-platform-requirements) for details. The VM size must be equal to or larger than the VM size of your current CoPilot., which is displayed as the default value for the *CoPilot VM Size*. g. Specify a disk size of your CoPilot deployment. The disk size must be equal to or larger than the disk size of your current CoPilot. of your Current CoPilot, which is displayed as the default value for the *Data Disk Size*. 6. Specify the time period, the starting date, and end date for the data restore from drp-down box and the date picker. You can see the estimated data size and estimated migration time. If you have previously backed up data, it can save time during this migration phase. 7. Click *Start Migration* to start the migration process. The migration takes some time to complete. You can choose to continue the migration in the background by clicking *Continue in Background* and close this window. During the CoPilot migration, avoid performing any tasks on CoPilot to prevent the loss of configurations or settings in the new CoPilot after migration. ### View CoPilot Migration Progress Once you have initiated the CoPilot migration, you can close the migration window and continue with other tasks. The migration process will continue running in the background. However, you may want to periodically check the progress of the migration. To view the CoPilot migration progress, follow these steps: 1. From *Settings > Maintenance > Upgrade*, go to the *Migrate to Appliance v3 (AWS Only)* tile under the CoPilot section. 2. Click *View Progress* to view the migration progress page. ## After Deploying the New CoPilot Click the *View Progress* to view the migration progress page. Keep an eye on the progress until you see the message "Finish deploying CoPilot, waiting for it to upgrade to the latest version." This indicates that the new CoPilot has been successfully deployed. ### Add 0.0.0.0/0 to Security Group The new CoPilot has the security group enabled by default. If you had custom security group configurations on the old CoPilot or have more than 500 gateways on the old CoPilot, you need to add the 0.0.0.0/0 rule to the security group of the new CoPilot instance as soon as the new CoPilot has been deployed: 1. Log in to the Controller UI. 2. From *SETTINGS > CoPilot > CoPilot Security Group Management*, toggle the settings to *Disabled*. Then click *SAVE*. 3. Log into your AWS console. Go to *EC2 > Security group*. 4. Edit the inbound rules of the security group associated with the new CoPilot instance. 5. Add a rule to allow incoming traffic from 0.0.0.0/0 for UDP port 31283 (NetFlow) and UDP port 5000 (Syslog). ### Log into the New CoPilot Once you receive a message indicating that your CoPilot webpage will be refreshed and the new CoPilot UI will be loaded, your browser window will automatically refresh. You will then be directed to the login page of the new CoPilot UI. Use your old CoPilot credentials to log in. > [!NOTE] > > Please ignore any "Failed to retrieve CoPilot data backup status" messages you > may receive during this process. These messages are due to the EIP switch > taking place. ### Check the Ongoing CoPilot Migration Task To check the status of the ongoing CoPilot migration task, follow these steps: 1. Click on the Task icon located on the top-right menu of your new CoPilot UI. 2. From the dropdown menu, click *Tasks > Show All Tasks*. 3. Look for the task labeled "CoPilot Migration: Restore CoPilot Data". 4. Check the status of this task to determine if it is completed or still in progress. You can also go to *Settings > Maintenance > Upgrade > Migrate to Appliance v3 (AWS Only) > View Progress* to check a more detailed progress of log for the data restore process. Once the "CoPilot Migration: Restore CoPilot Data" task is marked as completed, it indicates that the CoPilot migration to Appliance v3 has been successfully completed. > [!WARNING] > > After migration, Aviatrix automatically stops the task server and update > process on the old CoPilot. No manual action is needed. If you restart the old > CoPilot after the migration, the task server and update process will resume. > To avoid potential issues, do not restart the old CoPilot VM after migration. ## Troubleshoot the CoPilot Migration If you encounter a "Session timeout. Please log in again" error message on the new CoPilot login page, follow these steps: 1. Wait for 1-2 minutes before attempting to log in again. 2. After the waiting period, log into the new CoPilot using your credentials. For more details about troubleshooting the CoPilot Migration, see [Troubleshoot the CoPilot Migration](/docs/enterprise/10.1/guides/controlplane/migrate-to-appliance-v3-troubleshoot) for more details. # Troubleshoot the CoPilot Migration Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/controlplane/migrate-to-appliance-v3-troubleshoot When performing the CoPilot migration to Appliance V3, you may encounter certain issues or challenges. Here are some troubleshooting steps to help address potential challenges during the migration process: When performing the CoPilot migration to Appliance V3, you may encounter certain issues or challenges. Here are some troubleshooting steps to help address potential challenges during the migration process: ## Errors while Still on the Old CoPilot UI When you encounter errors while still on the old CoPilot UI, you can: 1. Check the error message under *Settings > Maintenance > Upgrade > CoPilot > Migrate to Appliance v3 (AWS) > View Progress*. 2. Address the issue based on the error message and rerun the migration. 3. If the new CoPilot has already been deployed according to the progress log, delete the new CoPilot before rerunning the migration. 4. If the same error persists, reach out to Aviatrix Support for further assistance. ## Errors after Deploying the New CoPilot When you receive error messages resemble any of the following: * "Controller API fails" error * "Migration Failed During CoPilot Initialization" error * "Restore CoPilot Data failed" error It is recommended to contact Aviatrix support for help. ### "Session timeout" Error If you receive a "Session timeout. Please log in again" error message on the login page of the CoPilot after migration: 1. Wait for 1-2 minutes. 2. After the waiting period, attempt to log in to the new CoPilot again. This delay allows the new CoPilot to complete its initialization process, and you should be able to log in the new CoPilot. # Onboard Your AWS Access Account in Controller Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/controlplane/onboard-account-controller-aws You can onboard an AWS account to your Aviatrix Controller from the Controller UI. You can onboard an AWS account to your Aviatrix Controller from the Controller UI. > [!NOTE] > > You need to deploy a separate Controller to use > [AWS China](/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/aviatrix-china-overview) > . 1. In your Controller, navigate to *Onboarding* in the left sidebar, and then click on the *AWS* icon. ![Click AWS Icon](https://mintcdn.com/aviatrix-14b37c43/0QYX2iFaLoliwUis/images/guides/controlplane/click_aws_icon.png?fit=max\&auto=format\&n=0QYX2iFaLoliwUis\&q=85\&s=55ecaf22e95190bcf065c33135f8d9b4) 2. Enter your AWS account's Customer ID. This Customer ID was emailed to you when you subscribed to the license. If you do not have a Customer ID, please contact Aviatrix Support. 3. Skip the Enter Certificate Domain field, which is only relevant for AWS China accounts. 4. Under Create Primary Access Account - AWS, enter: * Account Name: A name for this account. Note that this name is only used within your Controller and does not need to match the name or ID from your AWS account. * AWS Account Number: Enter your 12-digit Account ID. To find this Account ID, open your AWS account and click on the dropdown menu in the top right corner. Select Account. Your Account ID is listed at the top of the page under Account Settings. * IAM role-based: Mark the **Use IAM Roles** checkbox. > [!NOTE] > > If you leave this checkbox unmarked, use the ARN values in the optional > fields in this section to set up user roles. ARN values are only required > if you are onboarding an account that is separate from the one from which > you deployed the Controller. 5. Click *Create* to finish onboarding your AWS account. 6. Verify your email address at *Settings > Controller*. 7. Enter the verification code sent to your email address. You can now use advanced settings to configure your [IAM roles](/docs/enterprise/10.1/reference/general/iam-role), launch gateways, and build a single- or multicloud network architecture. > [!TIP] > > To launch Aviatrix CoPilot, please see the > [CoPilot Deployment Guide](/docs/enterprise/10.1/guides/controlplane/copilot-deployment) > . # Planning AWS Deployment Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/controlplane/planning-aws-deployment You can subscribe to the required Aviatrix Cloud Networking Platform license and subscriptions by using the Launch Aviatrix experience, accessible at launch.aviatrix.com. You can subscribe to the required Aviatrix Cloud Networking Platform license and subscriptions by using the Launch Aviatrix experience, accessible at [launch.aviatrix.com](https://launch.aviatrix.com). See the [Getting Started Guide AWS](/docs/enterprise/10.1/guides/controlplane/getting-started-guide-aws) for details about using the Aviatrix launch experience to deploy the Aviatrix platform. Before starting the Aviatrix deployment process, you should do the following: * Ensure the AWS region you are deploying to has at least 1 VPC, 1 internet gateway, and 2 elastic IPs (EIPs) available prior to deployment. Deployment will fail if you do not have enough VPCs, gateways, and static public IP addresses (EIPs in AWS). If you need to increase your limits, contact AWS. > [!NOTE] > > If you use a shared VPC for Controller and CoPilot, different accounts are > not allowed to see each other's instances. This is by design by AWS. In this > case, put in a feature request with AWS to modify Resource Access Manager > (RAM) permissions. All accounts in each shared subnet need the ability to > see all the instances in the subnet. Otherwise, features like CoPilot AppIQ, > SmartGroups, and others will not work properly in your account. > [!WARNING] > > Make sure you choose the correct region before launching the Controller > instance. After launching a Controller instance, you can only change that > instance's region by stopping that Controller and deploying a new one. * If you already signed up for an Aviatrix license, have the license key available. The key was emailed to you when you signed up. It is not retained anywhere else. If you do not have the key, contact your account manager. * Decide if you want to use an existing VPC for your deployment or create a new one. If you want to use an existing VPC, identify the VPC ID and a public subnet in that VPC. That information will be needed during deployment. If you want to use a new VPC, the VPC is automatically created for you as part of deployment. * Determine which deployment method you will select from Launch Aviatrix. After registering for a license and subscribing to the Aviatrix platform, you complete the launch process by using one of the following. * The automated CloudFormation Quick Create Stack. *This is the recommended method.* It is fast and simple. No prerequisites required. * Terraform manually configured modules. You should be familiar with Terraform modules to use this method. If you choose to deploy with Terraform, you must meet the following prerequisites: * Install [Boto3](https://boto3.amazonaws.com/v1/documentation/api/latest/index.html). * Configure [authentication for AWS](https://docs.aws.amazon.com/cli/v1/userguide/cli-authentication-user.html). * Install a minimum version of [Terraform 0.13](https://www.terraform.io/) and [Python3](https://www.python.org/). > [!TIP] > > As a general cloud security best practice, do not use the root user > credentials of your AWS account to launch the Aviatrix Platform or any other > AWS resources in your AWS account. # Planning Your Controller Deployment Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/controlplane/planning-deployment The information in this topic applies to deployments on the marketplaces for Google Cloud and Oracle Cloud Infrastructure (OCI). The information in this topic applies to deployments on the marketplaces for *Google Cloud and Oracle Cloud Infrastructure (OCI)*. > [!TIP] > > For *AWS* (Amazon Web Services) deployments, see [Getting Started Guide AWS](/docs/enterprise/10.1/guides/controlplane/getting-started-guide-aws). > > For Microsoft Azure deployments, see [Getting > Started Guide Azure](/docs/enterprise/10.1/guides/controlplane/getting-started-guide-azure). There are two ways to deploy an Aviatrix Controller with Google Cloud or OCI: 1. Through one of the CSP (Cloud Service Provider) marketplaces. 2. Through [Aviatrix Terraform modules](https://registry.terraform.io/namespaces/terraform-aviatrix-modules). > [!NOTE] > > Terraform is the recommended deployment method. See [Terraform & > Infrastructure as > Code](https://aviatrix.com/learn-center/cloud-networking/terraform-and-infrastructure-as-code/) > for more information. Ensure that you have met all of the [Prerequisites for Deploying](#prerequisites-for-deploying-through-a-csp-marketplace) before starting the Aviatrix Controller deployment. ## Prerequisites for Deploying Through a CSP Marketplace These prerequisites apply to *Google Cloud and OCI*. For *AWS Marketplace* deployments see [Preparing for Your AWS Deployment](/docs/enterprise/10.1/guides/controlplane/planning-aws-deployment). For Microsoft Azure deployments, see [Getting Started Guide Azure](/docs/enterprise/10.1/guides/controlplane/getting-started-guide-azure). Each CSP has its own set of prerequisites that must be implemented prior to deploying the Aviatrix Controller and CoPilot. ### Google Cloud Prerequisites #### Get a Customer ID from Aviatrix The Aviatrix Controller for Google Cloud is available on the Google Cloud Marketplace for BYOL license. Send an email to [info@aviatrix.com](mailto:info@aviatrix.com) or open a support ticket at [Aviatrix Support Portal](https://support.aviatrix.com) with your organization name to request a customer ID. We offer a 30-day free trial license. ### Creating a Google Cloud Account Aviatrix Cloud Connect is a software product that is launched in your own Google Cloud account. The Controller and the Gateways created from the Controller are all in your own network perimeter and completely under your control. Create a Google Cloud account ([https://cloud.google.com/](https://cloud.google.com/)). If you already have an account, skip this step and go to the next step. Note that the Controller supports multiple accounts and each account is associated with a different Google Cloud project. But you must have at least one account. #### Creating a Google Cloud Project Log in to your Google Cloud account and go to the project page: [https://console.cloud.google.com/project](https://console.cloud.google.com/project) Create a project. Go on to the next step if you have already created one. Note that the project ID will be used in referencing this project by Aviatrix Controller. For example, in a project called Aviatrix-UCC, the project ID is aviatrix-ucc-1214. #### (Optional) Creating Networks This step creates a network in the project created in the previous step. When a new project is created, a default network is created. You may skip this step if you do not need to customize the network address range by creating a new network, or go on to the next step if you have done so. Note that the Aviatrix Controller handles a Google Cloud network like a VPC in AWS. Whenever a network configuration is mentioned for Google, the term VPC is used. (The VNet is used for Azure.) At the Google Cloud console, select the project that you copied the Aviatrix Controller image to. Click the 3 bars. At the dropdown menu, select *VPC Network*. Click *\[+] Create Network*. Use the automatic subnet creation mode; otherwise ensure there is a subnet allocated for each region where a Gateway will be deployed. > [!NOTE] > > If you plan to have multiple projects, we suggest you plan your subnets so > that the network addresses do not overlap. Select *Custom* to create subnets. #### Saving the Management CIDR Range Find and save the public IP address **CIDR range** for the device you will initially use to run the Aviatrix Controller and CoPilot. > [!NOTE] > > To find a device's IP address and determine this CIDR range, search for "what > is my IP" on the browser's search engine. ### OCI Prerequisites Find and save the public IP address **CIDR range** for the device you will initially use to run the Aviatrix Controller and CoPilot. > [!NOTE] > > To find a device's IP address and determine this CIDR range, search for "what > is my IP" on the browser's search engine. This section discusses the system and platform requirements of Aviatrix Controller. > [!TIP] > > For larger environments, the CoPilot-related resource consumption is increased and the Controller must be resized accordingly. > > The Controller size is defined according to the footprint and the number of gateways it needs to support. Controller size is adjustable based on future requirements and enhancements. For example, if a t3.2xlarge controller instance is deployed in AWS this supports less than 30 Aviatrix gateways (t3.small) to start with, and sizing can be increased as needed when expanding the footprint. > > See [CoPilot platform requirements](https://docs.aviatrix.com/documentation/latest/getting-started/copilot-platform-requirements.html). > [!NOTE] > > The requirements listed below are intended as a general guide only. Once you have deployed Controller, it is important to monitor the indexing of data traffic in your specific environment and adjust configuration settings, such as index retention periods, to optimize instance sizing for your business needs. It is also recommended to upgrade the size of virtual machines if Aviatrix platform global health alerts are triggered frequently. For information about platform global health alerts, see [Alert Global Network Health](https://docs.aviatrix.com/documentation/latest/monitoring/alert-global-network-health.html). ## Minimum Sizing for Controller For a simple deployment, Controller (with CoPilot) requires a *minimum* of: | **Requirement** | **Minimum Value** | | - | - | | Memory & CPU | 4 CPU (or more) 16 GB of RAM (or more) Examples: \* Amazon EC2 instance type: t3.xlarge \* Google Cloud: n2-standard-4 \* Oracle Cloud Infrastructure: VM.Standard3.Flex or VM.Standard.E4.Flex \* Microsoft Azure virtual machine: Standard\_D4\_v5 | | Storage | 100 GB | ## Next Steps After completing the prerequisites for the specific Google Cloud or OCI CSP marketplace, see the relevant Getting Started Guide to deploy your Controller in one of the CSPs: * [Google Cloud Manual Launch Controller CSP](/docs/enterprise/10.1/guides/controlplane/manual-launch-controller-csp) * [OCI Getting Started Guide](https://legacy.docs.aviatrix.com/documentation/latest/getting-started/getting-started-guide-oracle.html) # Configure CoPilot for the Aviatrix Platform Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/controlplane/platform-integration-points As a component in the Aviatrix Platform, CoPilot must communicate with other components in the platform to receive the data it requires. This section details the configuration of CoPilot for the Aviatrix platform. As a component in the Aviatrix Platform, CoPilot must communicate with other components in the platform to receive the data it requires. This section details the configuration of CoPilot for the Aviatrix platform. The integration points are typically configured for you as part of the CoPilot deployment process. If you encounter any problems with your CoPilot deployment, you can check to ensure these integration points are configured. ## Integration with Controller CoPilot must be able to reach Controller. See [Associate Controller with CoPilot](#associate-controller-with-copilot). ### Associate CoPilot with Controller In Aviatrix Controller, go to **Settings > CoPilot** and enable the **CoPilot Association** option so that your CoPilot will be associated with your Controller. Once the CoPilot association is enabled, you can click the dotted-square button in the top right to use single sign-on for CoPilot. ### [Associate Controller with CoPilot](/docs/enterprise/10.1/guides/controlplane/copilot-controller-settings#associate-controller-copilot) 1. In the CoPilot UI, go to **Settings > Configuration > General > Associated Aviatrix Controller** and click **Reset Association** to associate your CoPilot with your Controller. 2. On the **Reset Controller Association** page, check **I understand the implications**, and then click **Reset**. 3. When logged out, enter username, password, and Controller IP. If the Controller and CoPilot are in the same subnet, VPC, and region of the same cloud provider, or if the Controller and CoPilot can use a private IP for inbound network access, you can enter the private IP for the Controller. Otherwise, enter the Controller's EIP. 4. Click **Log In**. ### Configure Controller's Access for CoPilot * Assign a static public IP address to CoPilot. For example, in EC2 console, you go to the Elastic IP section and assign an EIP to the CoPilot instance. * On Controller security groups, ensure 443 is open to the public IP of the CoPilot instance. * Configure a dedicated user account on Aviatrix Controller for CoPilot if desired. > [!NOTE] > > If you are using RBAC, as of 1.1.5 CoPilot requires read-only access to ping > and traceroute functions for diagnostic capabilities. ### Setting Session Timeout for the Controller On the Settings > Configuration > General tab, you use the Controller Session Timeout setting to determine how long sessions connected to the Controller can remain inactive before the current sessions time out (in minutes). ## Integration with Gateways CoPilot receives NetFlow data from gateways. Gateways must be able to reach CoPilot. In CoPilot, from **Settings > Configuration > General**, you can enable the CoPilot Security Group Management option so that your Controller can manage your CoPilot's inbound security group rules and allow gateways to access your CoPilot virtual machine. If you choose not to enable the CoPilot Security Group Management option, you must add rules to your CoPilot's inbound security group for specific ports for Aviatrix gateway IPs. For more information about the CoPilot Security Group Management option, see [CoPilot Security Group Management](/docs/enterprise/10.1/reference/controlplane/copilot-security-group-management) . ## Integration with NetFlow CoPilot receives NetFlow data from gateways. ### Enable NetFlow for CoPilot Features To use some features in CoPilot, such as FlowIQ and CostIQ features, ensure that the controller is configured to forward NetFlow logs to CoPilot: 1. Log in to Aviatrix CoPilot UI. 2. Go to **Settings > Configuration > Logging Services > NetFlow Agent**, click **Enable**. 3. The static IP address of CoPilot will be pre-filled as the NetFlow server IP. Use UDP port 31283 (default, port is configurable). For TLS encryption of the NetFlow traffic, select TCP as the protocol. See [TLS Encryption for NetFlow](/docs/enterprise/10.1/reference/controlplane/copilot-security-group-management) for more information. > [!NOTE] > > Changing the protocol may result in network disruption. 4. Use version 9. 5. (Optional) To use the NetFlow L7 mode Preview feature, select L7 mode. See [NetFlow L7 Mode](#netflow-l7-mode). 6. Select the Advanced checkbox. In Gateways, verify all of your Aviatrix gateways are in the Include List.You can choose any gateways to be excluded > [!NOTE] > > If you launch new gateways from your controller later, you must transfer the > newly launched gateways to the Include List also. > > In addition, in your native cloud console, you must open your CoPilot > security group for UDP 31283 from each newly launched gateway. If you enabled > the CoPilot Security Group Management option in CoPilot (**CoPilot > > Settings > Configuration > General > Security > CoPilot Security Group > Management**) this will happen automatically. You should start seeing NetFlow in CoPilot after a few minutes. ### NetFlow L7 Mode NetFlow L7 Mode enables the NetFlow agent on spoke gateways to forward L7 data that are in the flows to your designated NetFlow service point. L7 Mode is a [Preview Feature](/docs/enterprise/10.1/reference/feature-modes/index) . When L7 mode is enabled, Internet traffic that traverses spoke gateways is analyzed for flows that generate L7 data. When these flows are detected, the L7 fields are forwarded to the designated NetFlow service point. Enabling L7 mode may impact traffic throughput for the spoke gateways. After enabling L7 mode, monitor your gateway CPU and throughput telemetry statistics in CoPilot and scale your gateways up if needed. If you use Aviatrix CoPilot as your NetFlow service point, you can view L7 data by going to the CoPilot > Monitor > FlowIQ page, clicking on the Application view, and then opening the Records page. Note that some flows do not generate L7 data. ## Integration with Syslog CoPilot receives syslog data. ### Enable Syslog for CoPilot Audit Data To use audit data in the **CoPilot > Administration > Audit** feature in CoPilot, configure syslog to be sent to CoPilot: 1. Log in to Aviatrix Controller. 2. Go to **Settings > Logging > Remote Syslog**. 3. Choose Profile Index 9. Do not choose another index number. Index 9 is reserved for CoPilot. 4. In Enable Remote Syslog, enter the profile name you want to use, the static IP address of CoPilot as the server, and UDP port 5000 (default). 5. Select TCP for TLS encryption of syslog traffic. > [!NOTE] > > If TCP was enabled before upgrading to Controller version 8.2, manually > enable TLS encryption. > > If TCP and TLS are enabled through API before upgrading to Controller 8.2, > the TLS encryption stays after the upgrade. > > Also, verify the custom certificate after upgrading to Controller 8.2. 6. Tick the Advanced check box. In Gateways, verify all of your Aviatrix gateways are in the Include List. > [!NOTE] > > If you launch new gateways from your controller later, you must transfer the > newly launched gateways to the Include List also. > > In addition, in your native cloud console, you must open your CoPilot > security group for UDP 5000 from each newly launched gateway. If you enabled > the CoPilot Security Group Management option in Controller (Controller > > Settings > CoPilot > CoPilot Security Group Management) this will happen > automatically., 7. Click **Enable**. ## Resetting Controller IP in CoPilot In CoPilot **Settings > Configuration** page, click **Reset Association** to reset the IP address of the Controller with which CoPilot is associated. ## Resetting Service Account in CoPilot On the CoPilot Settings > Configuration page, click **Reset** to reset the account to be used as the CoPilot service account. For instructions to reset the Service Account see, [Initial Setup of CoPilot](/docs/enterprise/10.1/guides/controlplane/initial-setup#prereq-initial-cp-setup) and [CoPilot User Account Administration](/docs/enterprise/10.1/reference/controlplane/copilot-user-account-administration) . > [!WARNING] > > Removing the Service Account will limit CoPilot's ability to perform critical > tasks, including fetching topology data, collecting performance metrics, and > applying configuration changes. ## Setting the Controller FQDN in CoPilot On the CoPilot Settings > Configuration > General page, you use the **Controller Public IP/FQDN** configuration option to specify the public IP address or the FQDN of your Controller. * If your organization's team members log in to Aviatrix Controller via SAML, and you want them to be able to log in to CoPilot via SAML authentication also, this value must match the value you specified for the **Single sign on URL** SAML setting of your IdP application. * If you specified the Controller's IP address in the SSO URL, specify the Controller IP address here. * If you specified the Controller's FQDN in the SSO URL, specify the Controller FQDN here. For more information, see CoPilot Login via SAML in *Aviatrix CoPilot Deployment Guide*. # Post-Deployment Tasks for the Controller Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/controlplane/post-deploy-controller After deploying your Controller through the CSP (Cloud Service Provider) marketplaces or Terraform, use this document to set up your Controller. After deploying your Controller through the CSP (Cloud Service Provider) marketplaces or Terraform, use this document to set up your Controller. ## Saving Your Customer ID If you have a BYOL license or use a community image, you need to have a customer ID provided by Aviatrix to be able to use the product. This Customer ID was emailed to you from [admin@aviatrix.com](mailto:admin@aviatrix.com). Please open a support ticket at Aviatrix Support Portal if you do not have a customer ID. ## Onboarding Steps for Controllers Deployed from CSP Marketplaces See the following documents for onboarding your AWS, Azure, and GCP accounts deployed from the Marketplace: * [AWS IAM Roles](/docs/enterprise/10.1/reference/general/iam-role) * [Azure Account Credentials](/docs/enterprise/10.1/guides/platform-administration/account-azure) * [GCP Account Onboarding](/docs/enterprise/10.1/guides/platform-administration/gcp-account-onboarding) # Re-Encrypt Existing Controller and CoPilot Volumes Using a Custom Key Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/controlplane/reencrypt-custom-key You can re-encrypt your existing Controller and CoPilot volumes using a custom key. You can re-encrypt your existing Controller and CoPilot volumes using a custom key. ## Prerequisites To proceed with the re-encryption process, check the prerequisites below: 1. Make sure the data on the Controller and CoPilot volumes are fully backed up. 2. Make sure your custom key meet the encryption standards and is stored securely. ## Re-encrypt Existing Controller and CoPilot Volumes 1. Log into CoPilot, from *Settings > Resources > Index Management*, click the download button to download the index lists. 2. Identify your old volume ID. Go to your cloud service provider console. Select your Aviatrix CoPilot deployment and under the *Storage* section, locate and note down the volume ID you intend to re-encrypt with your custom key. 3. Create a snapshot. Go to *Elastic Block Store > Snapshots > Create snapshot*. On the *Create snapshot* page, select the old volume ID from the volume drop-down list to generate a snapshot. 4. Create a volume from snapshot. 1. Once the snapshot is created, select it and then click *Action > Create volume from snapshot*. 2. On the subsequent volume settings page,specify your custom KMS key. Then click *Create volume*. 5. Detach the old volume. Click the old volume, then click *Action > Detach volume*, click *Detach* to confirm. 6. Attach the new volume that you encrypted with your custom KMS key. 1. Go to the *Volumes* section, click the newly created volume, from *Actions > Attach volume*. 2. On the *Attach volume* page, choose the intended instance from the drop-down list. 3. Click *Attach volume* to confirm the volume attachment. 7. Verify the new volume with custom KMS key. 1. Go to *EC2 > Instances* page, you can check the *Storage* tab, where the new volumes should be listed. 2. You can also go to CoPilot, from *Settings > Resources > Index Management*, make sure the listed indices match those previously downloaded. ## Related Pages * [CoPilot Disk (Volume) Management](/docs/enterprise/10.1/guides/controlplane/disk-management) * [CoPilot Tuning and Performance](/docs/enterprise/10.1/reference/controlplane/tuning-performance) # CoPilot Simple Deployment Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/controlplane/simple-deployment You can launch a simple deployment on AWS or Azure by using the Launch Aviatrix experience. You can launch a simple deployment on AWS or Azure by using the [Launch Aviatrix experience](https://launch.aviatrix.com). You can launch a simple deployment on GCP and OCI by using Terraform scripts. On AWS, you can also deploy CoPilot from the Controller user interface. See [Deploying Your CoPilot](/docs/enterprise/10.1/guides/controlplane/copilot-deployment) for deployment details. > [!TIP] > > For a simple deployment, the single instance (VM) must meet the minimum > requirements. For instance and system requirements for a simple deployment, > see > [CoPilot Requirements](/docs/enterprise/10.1/reference/controlplane/copilot-platform-requirements) > . # Logging in to CoPilot Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/controlplane/ui-login This section covers how to log in to a running CoPilot. If this is your initial login to CoPilot, see Initial Setup of CoPilot. This section covers how to log in to a running CoPilot. If this is the first time you log in to CoPilot, see [Initial Setup of CoPilot](/docs/enterprise/10.1/guides/controlplane/initial-setup) . For information about supported web browsers, see [Supported Browsers for CoPilot](/docs/enterprise/10.1/reference/controlplane/copilot-platform-requirements) . For information about setting up SAML login for CoPilot, see [Set Up SAML Login for CoPilot](/docs/enterprise/10.1/guides/platform-administration/copilot-saml-authentication). To log in to CoPilot, use the following steps: **1. Launch CoPilot in your web browser** Go to `https:///`, where `` is the static public IP address of your deployed CoPilot software instance/virtual machine. Alternatively, launch CoPilot directly from Aviatrix Controller as described in the next step. **2. (Optional) Launch CoPilot from Aviatrix Controller** 1. In Aviatrix Controller, go to the **Environment** tab. This is the default landing page after you log in. 2. Under **CoPilot**, click **Open CoPilot**. 3. Alternatively, click **Open CoPilot** in the Controller header. This button is available from any page in the Controller. To associate a different CoPilot instance with this Controller, see [Associating Controller with CoPilot](/docs/enterprise/10.1/guides/controlplane/copilot-controller-settings#associate-controller-copilot). **3. Enter your credentials** Enter your Controller account credentials or SAML credentials. **4. (Optional) Configure UI usage analytics sharing** Aviatrix recommends that you keep the **Share UI Usage Analytics with Aviatrix** option selected, then click **Acknowledge and Save**. You can change this selection at any time by going to \*\*Settings > Configuration > General > Sharing Metrics with Aviatrix\*\*. See > [CoPilot Share Metrics](https://legacy.docs.aviatrix.com/documentation/latest/platform-administration/copilot/copilot-share-metrics.html) > for more information. # Deploy Gateways Without Public IPs Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/gateways/deploy-gateways-without-public-ips This guide explains how to deploy an Aviatrix gateway without a public IP address using the Assign Private IP Only setting, covering prerequisites, the CoPilot procedure, and connectivity verification. Deploy an Aviatrix gateway in a private subnet without assigning it a public IP address, using the **Assign Private IP Only** setting. This guide covers prerequisites, the CoPilot procedure for enabling Assign Private IP Only on a new Spoke or Transit Gateway, and how to verify that the gateway keeps connectivity to the Aviatrix Controller (and CoPilot, if deployed) without direct internet access. This guide covers greenfield deployment of a new gateway with Assign Private IP Only enabled. It does not cover gateway high availability — see [High Availability Gateways](/docs/enterprise/10.1/guides/gateways/gateway-ha-about) to configure HA instances after deployment. > [!NOTE] > > Assign Private IP Only is a different capability from the **Private Mode** > global Controller setting described in the > > [Aviatrix glossary](/docs/enterprise/10.1/reference/general/aviatrix-glossary#private-mode) > . Private Mode routes gateway traffic through web proxies and native cloud > constructs as a Controller-wide setting. Assign Private IP Only is a per-gateway > setting that relies on your subnet's route table to reach the Controller. As of > Controller 10.1.0, Private Mode is no longer accessible through the product UI. ## Prerequisites Before you deploy a gateway with Assign Private IP Only, confirm the following: * Your Aviatrix Controller is running version 10.1.0 or later, and CoPilot is running version 4.36 or later. * Assign Private IP Only supports gateway creation and replacement for **AWS** and **Azure** Transit and Spoke Gateways only. It is not available for GCP, OCI, or Alibaba Cloud gateways in this release. * If your Controller currently has the legacy Private Mode global setting enabled, disable it before upgrading. A Controller with the legacy setting enabled cannot upgrade to a version that includes this feature. * The subnet you deploy into does not need direct internet access, but its route table must have one of the following: * A default route (`0.0.0.0/0`) to a NAT gateway or other centralized egress path (for example, a transit gateway, VPN gateway, or peering connection) that can reach the Controller's public IP. * An explicit `/32` route to the Controller's IP address, and to CoPilot's IP address if CoPilot is deployed. * If gateways deployed this way need to tunnel to each other across VPC/VNets, private connectivity must already exist between them (native VPC/VNet peering, a transit gateway, or equivalent) — gateways with Assign Private IP Only enabled do not establish tunnels over the public internet. * The public IP address of the NAT gateway or firewall that provides the gateway's egress path must be added to **NAT-ed Gateway IP Address CIDRs** on the Controller (`aviatrix_controller_security_group_management_config`'s `gateway_egress_cidrs` in Terraform), so the Controller and CoPilot accept management traffic from it. > [!NOTE] > > * This setting applies to the entire gateway, including any HA instance you add > later, and cannot be changed after you create the gateway. To move a gateway > between Assign Private IP Only and a public-IP deployment, delete and recreate > it. > * Assign Private IP Only does not support air-gapped environments — the gateway > still requires an outbound path to the internet, even though the path does not > need to be direct. > * Single-IP SNAT is not supported on a gateway with Assign Private IP Only > enabled. Use Customized SNAT instead, including for Kubernetes egress use > cases. > * A third-party IPsec connection cannot terminate on a gateway with Assign > Private IP Only enabled. ## Enable Assign Private IP Only on a New Gateway **1. Open the gateway creation dialog** In Aviatrix CoPilot, go to **Cloud Fabric** > **Gateways** > **Spoke Gateways** (or **Transit Gateways**), then click **+ Spoke Gateway** (or **+ Transit Gateway**). **2. Configure the gateway and select a subnet** Provide the **Cloud**, **Account**, **Region**, and **VPC/VNet** for the gateway. In **Attach to Subnet**, select a subnet whose route table meets the [Prerequisites](#prerequisites) — a subnet routed through a NAT gateway or other centralized egress path, or a subnet with an explicit route to the Controller's (and CoPilot's) IP address. **3. Expand Advanced Settings** Expand **Advanced Settings**. **4. Turn on Assign Private IP Only** Turn on **Assign Private IP Only**. This deploys the gateway without a public IP address; the gateway still requires internet access to communicate with the Controller through another existing path, such as a NAT gateway on its subnet. > [!NOTE] > > If the gateway also has High Performance Encryption enabled, CoPilot displays an > additional **Gateway Internet Egress Next Hop** field. Specify where the gateway > should route outbound traffic — a NAT gateway ID, an IP address (Azure only), or > any other valid next hop. **5. Complete the remaining gateway settings** Configure any remaining required fields, then click **Save**. ## Verify the Deployment * In **Cloud Fabric** > **Gateways** > **Spoke Gateways** (or **Transit Gateways**), confirm the new gateway's **Status** shows **Up**. * Click the gateway name to open its Details tab. Confirm **GW Instance Public IP** shows **None** and **GW Instance Private IP** shows an address. * If the gateway is attached to a Transit Gateway or has active tunnels, confirm the attachment or connection **Status** shows **Active** (or **Up**). * Confirm the gateway shows as connected to the Controller (and to CoPilot, if deployed) despite having no public IP. > [!NOTE] > > If the gateway does not come up or does not connect to the Controller, recheck > the subnet's route table against [Prerequisites](#prerequisites) — a missing NAT > gateway route, a missing `/32` route to the Controller, or an unreachable > CoPilot path are the most common causes. ## Deploy a Kubernetes Firewall Gateway Use this pattern to deploy an Aviatrix gateway for Distributed Cloud Firewall (DCF) enforcement and source address translation for an existing Kubernetes cluster. The gateway sits inline in the cluster VPC or VNet, between the worker-node subnet and its existing internet egress, so all pod traffic passes through it for inspection. It does not become the cluster's primary routing fabric. > [!NOTE] > > This section assumes your cluster is already onboarded to DCF and worker-node > SNAT is disabled, as described in > > [Kubernetes Onboarding Prerequisites](/docs/enterprise/10.1/reference/dcf/kubernetes-prerequisites) > . DCF identifies workloads by pod source IP; if worker nodes still perform SNAT, > DCF sees only the node IP and per-pod policy does not work. **How this pattern works:** * The Aviatrix gateway runs with **Assign Private IP Only** enabled in the same VPC or VNet as the Kubernetes cluster, so it adds no new public IP surface. * You redirect the worker-node subnet's outbound traffic to the gateway, so DCF can see and enforce policy on individual pod IPs. * Because the gateway, not the worker nodes, now performs source address translation, you configure **Customized SNAT** on the gateway for pod egress instead of relying on node-level SNAT. * Translated traffic continues to your existing centralized egress (a transit attachment to centralized NAT, or the cluster VPC's own NAT gateway). **1. Deploy the gateway into the cluster VPC** Deploy a Spoke Gateway into a private subnet in the same VPC or VNet as the Kubernetes cluster, following [Enable Assign Private IP Only on a New Gateway](#enable-assign-private-ip-only-on-a-new-gateway). In a mature landing zone, the gateway's outbound path is often the transit gateway attachment the cluster already uses to route to centralized egress. **2. Validate management connectivity** Before you change any cluster routing, confirm the gateway shows connected to the Controller (and CoPilot, if deployed) as described in [Verify the Deployment](#verify-the-deployment). **3. Redirect worker-node subnet traffic to the gateway** Point the worker-node subnet's outbound route at the gateway so its traffic is inspected: * **AWS**: Edit the route table associated with the worker-node subnet so the relevant route (for example, `0.0.0.0/0`) targets the Aviatrix gateway's elastic network interface instead of the NAT gateway or transit attachment it currently uses. If the cluster already routes through an AWS Transit Gateway, you can instead enable **Transparent Inspection via AWS TGW** on the Spoke Gateway to inspect the traffic without editing the VPC route table, and use **Customize Inspection by Route Tables** to limit inspection to the worker-node subnet's route table. * **Azure**: Add or edit a user-defined route (UDR) on the worker-node subnet's route table so the relevant route targets the gateway's private IP address with next hop type **Virtual appliance**. > [!WARNING] > > Changing the worker-node subnet's route table affects live traffic. Make this > change during a maintenance window, and confirm the gateway shows **Up** first. **4. Configure Customized SNAT for pod egress** On the gateway, go to **Cloud Fabric** > **Gateways**, select the gateway, open its **Settings** tab, and expand **Network Address Translation (NAT)**. Turn on **Source NAT**, click **Customized SNAT**, then click **+ Rule** and enter: | Field | Value | | - | - | | Source CIDR | The cluster's pod CIDR | | Destination CIDR | Leave blank to match all internet-bound traffic | | SNAT IPs | A secondary private IP on the gateway, or another address you nominate | Click **Save**, then turn on **Apply Route Entry** to commit the rule. **5. Validate pod-to-internet connectivity** From a pod, curl an external address and confirm the response reaches the pod. Then confirm the translation is working as expected: * Check DCF flow logs in CoPilot — connections initiated by the pod should show the pod's original IP as the source before the gateway, and the **SNAT IPs** address after the gateway. * Confirm the traffic continues through your environment's centralized egress (a transit attachment to centralized NAT, or the cluster VPC's existing NAT gateway). > [!NOTE] > > Disable SNAT on Kubernetes worker nodes as described in > > [Disable SNAT for Kubernetes Workloads](/docs/enterprise/10.1/reference/dcf/kubernetes-snat) > . That setting is separate from Customized SNAT on the Aviatrix gateway, which > handles pod-egress translation at the gateway. Size the shared egress path for > both management traffic and NAT'd pod workload traffic. ## Deploy in Terraform Assign Private IP Only is configurable through the Aviatrix Terraform provider using the `private_network` argument on the gateway group resource and the `private_subnet_egress_target` argument on the gateway instance resource: ```hcl theme={null} resource "aviatrix_spoke_group" "this" { name = "private-spoke" cloud_type = 1 # AWS account_name = var.account vpc_id = var.vpc_id private_network = true } resource "aviatrix_spoke_instance" "this" { name = "private-spoke-gw" group_name = aviatrix_spoke_group.this.name private_subnet_egress_target = "nat-xxxxxxxx" # or transit attachment ID # allocate_new_eip and eip do not apply to private-network instances } resource "aviatrix_controller_security_group_management_config" "this" { gateway_egress_cidrs = ["203.0.113.50/32"] # public address of your NAT or firewall egress } ``` `gateway_egress_cidrs` is the Terraform equivalent of **NAT-ed Gateway IP Address CIDRs** on the Controller. For Transit Gateways, use the equivalent `aviatrix_transit_group` and `aviatrix_transit_instance` resources. See the [Aviatrix Terraform provider](/docs/enterprise/10.1/reference/terraform/provider) reference for the full set of arguments. > [!NOTE] > > Do not enable Single IP SNAT (`enable_nat`) on a private-network group. ## Related Topics * [Aviatrix Terraform provider](/docs/enterprise/10.1/reference/terraform/provider) — configure Assign Private IP Only through the Aviatrix Terraform provider. * [High Availability Gateways](/docs/enterprise/10.1/guides/gateways/gateway-ha-about) — configure HA instances for a gateway after deployment. * [Private Mode (legacy glossary entry)](/docs/enterprise/10.1/reference/general/aviatrix-glossary#private-mode) — the separate, older global Controller setting. As of Controller 10.1.0, Private Mode is no longer accessible through the product UI. * [Kubernetes Onboarding Prerequisites](/docs/enterprise/10.1/reference/dcf/kubernetes-prerequisites) — onboard a cluster to DCF and disable worker-node SNAT before deploying a Kubernetes firewall gateway. * [Disable SNAT for Kubernetes Workloads](/docs/enterprise/10.1/reference/dcf/kubernetes-snat) — the worker-node SNAT setting required for per-pod DCF policy. # High Availability Gateways Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/gateways/gateway-ha-about This document provides an overview of the high availability configuration options for Aviatrix Transit Gateway and Spoke Gateway. This document provides an overview of the high availability configuration options for Aviatrix Transit Gateway and Spoke Gateway. ## About Highly Available Gateways The Aviatrix High Availability feature enabled deployment of multiple gateway instances within a VPC or VNet to support high availability and scalability, aiming to minimize network downtime and improve overall network stability and performance. In this setup, each gateway consists of multiple gateway instances operating in Active-Active mode. Network traffic flow is load balanced across all available gateway instances within that VPC or VNet. The Aviatrix Controller automatically updates CSP route tables to distribute traffic among all active gateway instances. ![Gateway HA](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/gateways/gateway-ha.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=2df36ab547808bdf86eb2b0f1d35b4e5) When highly available (HA) gateway instances are deployed, the Aviatrix Controller monitors the status of each instance. If it detects a gateway instance is down, it automatically redirects traffic to the remaining active gateway instances in that VPC or VNet. ## Creating Highly Available Gateway Instances Aviatrix Platform supports up to 15 highly available instances for Transit and Spoke Gateways. At least two instances are required for highly availability. The gateway instances are configured in Active-Active mode by default. A Transit Gateway is limited to two gateway instances in the following scenariios: * When deployed with the FireNet function * When peered to AWS TGW * When external connections are configured A Spoke Gateway is also limited to two gateway instances under these conditions: * When BGP external connections are present * When Site2Cloud external connections are configured, or customized SNAT and DNAT are enabled. A Gateway requires at least one instance for deployment. You can add more instances during or after gateway creation. ![Gateway Spoke Create](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/gateways/gateway-spoke-create.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=edb8035de8bd6eaafa7bde9d52ce5c3e) Aviatrix CoPilot assigns a name to each gateway instance automatically. The initial gateway instance takes the same name as the Gateway, while additional instances are named in the format `-`. ![Gateway Spoke Instances](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/gateways/gateway-spoke-instances.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=5c24c634221b632db85fb0e9dbb0541c) All gateway instances are configured in Active-Active mode and will handle traffic forwarding in the event of tunnel failure between Spoke VPC or VNet and Transit VPC or VNet. For best practice, each gateway instance in a VPC or VNet should be created on a different public subnet in a different availability zone, if available. Gateway instances share the same properties as the Gateway, such as instance size, high performance encryption, and attachments. When a property such as instance size is modified for the Gateway, this change applies to all gateway instances within that VPC or VNet. Aviatrix CoPilot does not support resizing individual gateway instances. However, individual gateway instances can be resized using Terraform resources aviatrix\_transit\_gateway, aviatrix\_spoke\_gateway, or aviatrix\_gateway for regular gateways. For further details, see [Aviatrix Provider](https://registry.terraform.io/providers/AviatrixSystems/aviatrix/latest). The High Performance Encryption setting cannot be modified once the Gateway has been created To create a HA gateway instance, see: * [Creating a Highly Available Gateway Instance](https://legacy.docs.aviatrix.com/documentation/latest/network/spoke-gateway-ha-instance-create.html) * [Creating a Highly Available Transit Gateway Instance](https://legacy.docs.aviatrix.com/documentation/latest/network/gateway-transit-ha-instance-create.html) ## Creating Highly Available Spoke Gateway Instance Using Terraform When deploying the Aviatrix Platform with Terraform, a highly available Spoke Gateway instance can be created using the Terraform resource **aviatrix\_spoke\_ha\_gateway**. ![Gateway HA Terraform](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/gateways/gateway-ha-terraform.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=28336023d4d095fd422327311bb8a69b) Use the **aviatrix\_spoke\_gateway** Terraform resource to create the Spoke Gateway, ensuring **manage\_ha\_gateway** is set to **false** so you can deploy highly available gateway instances with the **aviatrix\_spoke\_ha\_gateway** resource. Avoid any input options starting with "ha" for the Spoke Gateway. For further details, see [Aviatrix Provider](https://registry.terraform.io/providers/AviatrixSystems/aviatrix/latest). ## Removing Highly Available Gateway Instances You can remove one or more highly available gateway instances from a VPC or VNet. When gateway instances are removed, the Aviatrix Controller automatically updates the CSP route tables to forward network traffic to the remaining highly available gateway instances in that VPC or VNet. The initial gateway instance can only be deleted by removing the Gateway itself. Prior to deleting the Gateway, all other gateway instances in the VPC or VNet must be deleted and all gateway peerings must be detached. ### Deleting a Spoke Gateway Instance When a Spoke gateway instance is deleted that is attached to another gateway (spoke-to-transit or spoke-to-spoke), the Aviatrix Controller automatically removes the gateway attachments without impacting other gateway instances in that VPC or VNet. However, for BGP-enabled Spoke Gateway with external connections, you must first remove the connections before deleting the instance. ### Deleting a Transit Gateway Instance If a Transit gateway instance is attached to another gateway (transit-to-spoke or , transit-to-transit), or has external connections, you must first remove these attachments and connections before deleting the instance. To delete a HA gateway instance, see [Deleting a Highly Available Gateway Instance](https://legacy.docs.aviatrix.com/documentation/latest/network/gateway-ha-instance-delete.html). ## Related Topics * [Working with Aviatrix Gateways](https://legacy.docs.aviatrix.com/documentation/latest/network/aviatrix-gateways-overview.html) # Back Up and Restore Your Controller Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/maintenance/controller-backup-restore The Aviatrix Controller is not in the data path, as packet processing and encryption is done by the Aviatrix gateways. ## Overview The Aviatrix Controller is not in the data path, as packet processing and encryption is done by the Aviatrix gateways. When the Controller is down or out of service, your network will continue to be operational and encrypted tunnels and OpenVPN® users stay connected and are not affected. Since most of the data logs are forwarded from the gateways directly, the loss of log information from the Controller is minimal. The only impact is that you cannot build new tunnels or add new OpenVPN® users. This loosely coupled relationship between the Controller and gateways reduces the impact of the availability of the Controller and simplifies your infrastructure. Since the Controller stores configuration data, it should be periodically backed up to the appropriate AWS/Azure/Google account. If a replacement Controller is launched, you can restore the configuration data from your backup. > [!NOTE] > > Before stopping your existing Controller: > > * [Delete your Controller HA configuration](/docs/enterprise/10.1/guides/controlplane/controller-ha-aws#disabling-ha) > to avoid complications. > * Go to *CoPilot UI > Settings > Configuration > General > Associated Aviatrix > Controller*, click *Reset Association* to [reset](/docs/enterprise/10.1/guides/controlplane/copilot-migrate) the IP > address of your newly launched Controller in CoPilot. If you fail to do so, > you may be locked out of your CoPilot after restoring your backup. **CoPilot** Creating regular backups of your Aviatrix Controller is essential to ensure the security and integrity of your network configurations. This section discusses how to back up Controller Configuration and restore the backups on CoPilot UI. > [!NOTE] > > Starting from CoPilot version 3.14, you have the option to back up and restore > your Aviatrix Controller directly from the CoPilot UI. If you are using CoPilot > version 3.13 or earlier, you can perform the Aviatrix Controller backup and > restore directly from the Controller UI. See the [Controller UI](#controller-ui) > tab for more details. ## Check Your Controller Version Compatibility Starting from Controller v7.0, you can back up and restore Controller from CoPilot UI. Refer to the controller and CoPilot version compatibility matrix below for more details. ## Configure External Storage > [!NOTE] > > You must configure external storage before you back up and restore Controller > Configuration. 1. Log in to your CoPilot UI. 2. From *Settings > Maintenance > Back Up & Restore > Manage Backups > External Storage*, click *Configure*. 3. Select the appropriate *Cloud Type* for the *Location*. > **For AWS** > > * Select the *Account* name. \* Populate the *Bucket Name*. > **For GCP** > > * Select the *Account* name. \* Populate the *Bucket Name*. > **For OCI** > > * Select the *Account* name. \* Select a region. \* Populate the *Bucket > Name*. > **For Azure Arm** > > * Select the *Account* name. \* Select a *Region*, *Storage Name*, and > *Container Name*. > **For Azure Government** > > * Select the *Account* name. \* Select a *Region*, *Storage Name*, and > *Container Name*. > **For ARM China** > > * Select the *Account* name. \* Select a *Region*, *Storage Name*, and > *Container Name*. > **For AWS China** > > * Select the *Account* name. \* Select a *Region*, *Storage Name*, and > *Container Name*. > **For AWS GovCloud** > > * Select the *Account* name. \* Select a *Region*, *Storage Name*, and > *Container Name*. 4. Click *Save* to create an association with an external storage. ## Back Up Your Controller Configuration on CoPilot UI To back up your Aviatrix Controller Configuration on CoPilot UI, perform the following steps: 1. Log in to your CoPilot UI. 2. From *Settings > Maintenance > Back Up & Restore > Back Up Controller*, click *Back Up Now* to initiate the backup process. 3. (Optional) Click the caret (^) next to the *Backup Now* button and select *Edit Configuration* to modify the automatic Backup schedule. See [Modify Automatic Backup Schedule for Controller](#modify-automatic-backup-schedule-for-controller) for more details CoPilot will create a backup file containing all your Controller configurations, including network settings, gateways, and security policies. It is recommended to store the backup file in a secure location for future use. ### Modify Automatic Backup Schedule for Controller The Aviatrix CoPilot UI allows you to customize the automatic backup schedule for your Aviatrix Controller. Once you have modified the automatic backup schedule, the CoPilot UI will update the backup settings accordingly. This allows you to customize the backup frequency and timing to align with your specific requirements. To modify the automatic backup schedule for Controller, perform the following steps: 1. Log in to your CoPilot UI. 2. From *Settings > Maintenance > Back Up & Restore > Back Up Controller*, click the caret (^) next to *Backup Now* and select *Edit Configuration* to modify the backup schedule. 3. In the *Modify Automatic Backup Schedule for Controller dialog*, choose the desired frequency for the backups. You can select either daily or weekly backups. 4. Specify the time when you want the backups to be performed. Note that the time is in UTC format. 5. Enter the maximum number of backups that to be saved to the external storage. The default value is set to 1. 6. Click *Save* to save your changes. ## Restore Your Controller Configuration on CoPilot UI Restoring your Aviatrix Controller configuration from a backup is a crucial step in recovering your network settings in the event of a failure or when migrating to a new Controller. 1. Go to *Settings > Maintenance > Back Up & Restore > Restore Controller*, click *Restore*. 2. Select the appropriate *Cloud Type* for the *Location*. 3. Enter the access key, secret key, and bucket name that used for the CoPilot backup. 4. Select the file name of the backup file. 5. Click *Save*. > [!NOTE] > > Note that restoring your Controller configuration will overwrite the existing > configuration on your Controller. Therefore, ensure that you have selected the > correct backup and have a backup of any recent changes or configurations made > after the backup. ### Restore Notes * Make sure your Controller backup and Controller restore take place in the same CSP (Cloud Service Provider): AWS, Azure, or GCP and share the same basic configuration. For example, an AWS backup can only restore to another AWS Controller. * Note that in the case of AWS backups, an AWS Controller set up with IAM roles cannot backup and restore to an AWS Controller set up with a secret key, or vice versa. * You can only restore a Controller backup to a brand new Aviatrix Controller. ### Initialize a New Controller If you are starting from a new Controller, follow these steps to get started: 1. Log in to the Controller with the *admin* username and the default password. 2. Follow the initial steps to get the Controller up and running: 1. Shut down the older Controller. 2. Transfer the IP address to the new Controller. 3. Proceed to configure the new Controller by entering an email address. 4. Change your admin password. 5. Enter or skip the proxy configuration. 6. Allow the upgrade to run. ### Restore Controller Configurations on CoPilot UI Follow the steps below to restore your Controller configuration on CoPilot UI: 1. Log in to your CoPilot UI. 2. From *Settings > Maintenance > Back Up & Restore > Restore Controller*, click *Restore*. 3. On the "Restore Controller" page: Select a *Cloud Type* for the *Location*: > **AWS** > > * If you would like to use an existing account, make sure you create one Access Account only with the *exact* Access Account Name that was used in your previous Controller. Toggle the *Use Cloud Account Name* button to on and select the account. Otherwise, enter an *Access Key* and *Secret Key*. > * Enter the *Bucket* and *File Name* of the file to restore. The file name can be obtained from the external storage that is configured to store your backups. It may display as `CloudN__save-cloudx-config.enc`. > **Azure** > > * Enter the *Subscription ID* and *Directory ID*. > * Enter the *Application Client ID* and *Application Client Secret*. > * Enter the *Storage Name*, *Container Name,* and *File Name* of the file to restore. The file name can be obtained from the external storage that is configured to store your backups. It may display as `CloudN__save-cloudx-config.enc`. > **GCP** > > * Enter the *Bucket Name.* > * Upload the *GCP Project Credentials*. > * Enter the *File Name* of the file to restore. The file name can be obtained from the external storage that is configured to store your backups. > **ARM China** > > * Enter the *Subscription ID* and *Directory ID*. > * Enter the *Application Client ID* and *Application Client Secret*. > * Enter the *Storage Name*, *Container Name,* and *File Name* of the file to restore. The file name can be obtained from the external storage that is configured to store your backups. It may display as `CloudN__save-cloudx-config.enc`. > **OCI** > > * If you would like to use an existing account, make sure you create one Access Account only with the *exact* Access Account Name that was used in your previous Controller. Toggle the *Use Cloud Account Name* button to on and select the account. Otherwise, enter a *Tenancy OCID*, *User OCID*, *Compartment OCID*. And upload an *OCI Private Key*. > * Enter the *Bucket*, *Region* and *File Name* of the file to restore. The file name can be obtained from the external storage that is configured to store your backups. It may display as `CloudN__save-cloudx-config.enc`. 4. Click *Save* to initiate the restoration process. 5. The restoration process will begin, and you will see a message indicating that successfully downloaded a backup file. Once the restoration is completed, you will receive a confirmation message indicating the successful restoration of your Controller configuration. Verify that your network settings and configurations have been restored correctly by checking the functionality of your Aviatrix Controller. ## AWS Encrypted Backups AWS S3 allows uploaded backup files to be encrypted in the server side for more secure storage. The encryption is all done in the AWS S3 server side. This server side secure storage is in addition to the already encrypted Aviatrix Controller backups. Follow the steps below to enable AWS Encrypted backups: 1. Create AWS S3 bucket. ![s3 create](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/maintenance/s3-create.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=6c424ef339e9c6955ae22c9f624acdee) 2. After configuring other bucket properties, configure bucket server side encryption by selecting either *Server-side encryption with AWS Key Management Service keys (SSE-KMS)* or *Dual-layer server-side encryption with AWS Key Management Service keys (DSSE-KMS)*. ![s3 select default encryption](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/maintenance/s3-select-default-encryption.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=2677887f70bf3fc18d12b12d67112514) 3. Click *Create a KMS key* to create a new key. A separate tab opens where you can configure and save the key. 4. After the key is saved, go back to the Create bucket tab and enter the key in the *AWS KMS key ARN* field. ![s3 select encryption](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/maintenance/s3-select-encryption.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=cddc24f7a0fa8988589cec6bcc28e983) 5. Click *Create bucket*. ## Azure Private Link Backups Azure Private Link enables you to access Azure Platform-as-a-Service (PaaS) Services (for example, Azure Storage and SQL Database) and Azure hosted customer-owned/partner services over a private endpoint in your virtual network. Traffic between your virtual network and the service travels the Microsoft backbone network. Exposing your service to the public internet is no longer necessary. Using the Azure private link, the Controller backups happen privately from your VNet so that your blob storage account does not need to be exposed to the outside world. 1. Create an Azure Storage Account. 2. Set up the Storage Account for Private Link: * On the Networking tab for the storage account creation, select Private endpoint for the connectivity method. * Add a new private endpoint with the target of the blob storage resource and enable DNS Integration. > [!NOTE] > > If you currently have existing private endpoints deployed, you may need to use > an existing private zone in another subscription. This must be completed through > the dedicated private endpoint creation workflow. For additional assistance with > this setup, reach out to an Aviatrix Solution Engineer. 3. Verify Backup through Controller. Once successful, backing up traffic from the Controller will be performed privately across private link so that associated storage account does not need to be accessible publicly. **Controller UI** This section discusses how to back up Controller Configuration and restore the backups on Controller UI. * [Backing Up Your Controller Configuration on Controller](#backing-up-your-controller-configuration-on-controller) * [Restoring Your Controller Configuration on Controller UI](#restoring-your-controller-configuration-on-controller-ui) > [!NOTE] > > Starting from CoPilot version 3.14, you have the option to back up and restore > your Aviatrix Controller directly from the CoPilot UI. See the > [CoPilot](#copilot) tab for more details. ## Backing Up Your Controller Configuration on Controller Aviatrix stores the Controller backup in an AWS S3 bucket or an Azure Container. Before you begin, determine where you would like to store the backup and create either the S3 bucket or Azure Container. Creating regular backups of your Aviatrix Controller is essential to ensure the security and integrity of your network configurations. In the event of a failure or the need to migrate to a new Controller, having a backup allows you to quickly restore your settings and minimize downtime. > [!NOTE] > > Make sure your Controller backup and Controller restore take place in the same > CSP (Cloud Service Provider): AWS, Azure, or GCP and share the same basic > configuration. For example, an AWS backup can only restore to another AWS > Controller. Note that in the case of AWS backups, an AWS Controller set up with > IAM roles cannot back up and restore to an AWS Controller set up with a secret > key, or vice versa. (AWS) The S3 bucket you use or create for Controller HA and > Backups does not need to have public access enabled and should be configured to > restrict general public access. 1. Log in to the Controller. 2. Go to ***SETTINGS* > *Maintenance*** > select the ***Backup & Restore*** tab. 3. Under the *Backup* section, select the appropriate *Cloud Type* and *Account Name*. > **AWS** > > Populate the *S3 Bucket Name*. > **Azure** > > Populate the Region, Storage Name, and Container Name. > **GCP** > > Populate the Bucket Name, Gcloud Project Credentials, and File Name. > **OCI** > > Populate the Tenancy OCID, User OCID, Compartment OCID, API Private Key > File, Bucket Name, Region, and File Name. 4. (Optional) You can click *Multiple Backup* to enable multiple backups. See [Multiple Backup](#multiple-backup) for more details. > [!NOTE] > > By default, only the latest configuration data is stored. Each time the > configuration is backed up, it overwrites the previous one. If you would like to > keep every copy, check the box *Multiple Backup*. 5. Click ***Enable***. The first time you enable this feature, the configuration will backed up to your specified location. After this, the configuration data is automatically backed up daily at 12 am. ### Multiple Backup Selecting the *Multiple Backup* checkbox enables the Controller to backup a maximum of 3 rotating backups. Each backup filename will contain the date and time of when the backup is made. Additionally, the backup without any date and time in the filename contains a copy of the latest backup. If you want to force an immediate backup (e.g. for a configuration change) you can accomplish this by clicking *Backup Now*. If multiple backups are not enabled, each time the configuration is backed up, the backup up file will be overwritten. Otherwise, the oldest backed up file will be overwritten. > [!NOTE] > > Selecting the *Multiple Backup* option is recommended. If the backup is already > Enabled, go ahead and Disable it, turn on the Multiple Backup option and then > Enable the backup again. > [!NOTE] > > You should enable cross-region replication in AWS when creating your S3 buckets. > This ensures that an S3 bucket remains accessible if there is a regional CSP > failure. The replacement Controller can retrieve and restore its backup file. ## Restoring Your Controller Configuration on Controller UI > [!NOTE] > > Make sure your Controller backup and Controller restore take place in the same > CSP (Cloud Service Provider): AWS, Azure, or GCP and share the same basic > configuration. For example, an AWS backup can only restore to another AWS > Controller. Note that in the case of AWS backups, an AWS Controller set up with > IAM roles cannot backup and restore to an AWS Controller set up with a secret > key, or vice versa. ### Initialize a New Controller If you are starting from a new Controller, follow these steps to get started: 1. Log in to the Controller with the *admin* username and the default password. 2. Follow the initial steps to get the Controller up and running: 1. Shut down the older Controller. 2. Transfer the IP address to the new Controller. 3. Proceed to configure the new Controller by entering an email address. 4. Change your admin password. 5. Enter or skip the proxy configuration. 6. Allow the upgrade to run. ### Restore Controller Configurations on Controller UI Once you are past the initial configuration steps: 1. Log in to the Controller. 2. Go to *Settings > Maintenance* > select the *Backup & Restore* tab. 3. Under the *Restore* section, select the appropriate *Cloud Type* for the *Location*. > **AWS/ARM** > > * If you would like to use an existing account, make sure you create one > Access Account only with the *exact* Access Account Name that was used in > your previous Controller. Mark the *Use Cloud Account Name* checkbox and > select the account. Otherwise, enter an *Access Key* and *Secret Key*. \* > Enter the *Bucket Name* and *File Name* of the file to restore. The file > name can be obtained from the external storage that you configured to store > your backups. It may display as `CloudN_<Controller Private > IP>_save-cloudx-config.enc`. > **Azure** > > * Enter the *Subscription ID* and *Directory ID*. \* Enter the *Application > Client ID* and *Application Client Secret*. \* Enter the *Storage Name*, > *Container Name*, and *File Name* of the file to restore. The file name can > be obtained from the external storage that is configured to store your > backups. It may display as `CloudN_<Controller Private > IP>_save-cloudx-config.enc`. > **GCP** > > * Enter the *Bucket Name*. \* Upload the GCP Project Credentials. \* Enter the > *File Name* of the file to restore. The file name can be obtained from the > external storage that is configured to store your backups. It may display as > `CloudN_<Controller Private IP>_save-cloudx-config.enc`. > **OCI** > > * If you would like to use an existing account, make sure you create one > Access Account only with the *exact* Access Account Name that was used in > your previous Controller. Mark the *Use Cloud Account Name* checkbox and > select the account. Otherwise, enter a *Tenancy OCID*, *User OCID*, > *Compartment OCID*. Upload an *OCI Private Key*. \* Enter the *Bucket Name*, > *Region* and *File Name* of the file to restore. The file name can be > obtained from the external storage that is configured to store your backups. > It may display as `CloudN_<Controller Private > IP>_save-cloudx-config.enc`. 4. Click *Restore*. ## AWS Encrypted Backups AWS S3 allows uploaded backup files to be encrypted in the server side for more secure storage. The encryption is all done in the AWS S3 server side. This server side secure storage is in addition to the already encrypted Aviatrix Controller backups. Follow the steps below to enable AWS Encrypted backups: 1. Create AWS S3 bucket. ![s3 create](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/maintenance/s3-create.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=6c424ef339e9c6955ae22c9f624acdee) 2. After configuring other bucket properties, configure bucket server side encryption by selecting either *Server-side encryption with AWS Key Management Service keys (SSE-KMS)* or *Dual-layer server-side encryption with AWS Key Management Service keys (DSSE-KMS)*. ![s3 select default encryption](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/maintenance/s3-select-default-encryption.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=2677887f70bf3fc18d12b12d67112514) 3. Click *Create a KMS key* to create a new key. A separate tab opens where you can configure and save the key. 4. After the key is saved, go back to the Create bucket tab and enter the key in the *AWS KMS key ARN* field. ![s3 select encryption](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/maintenance/s3-select-encryption.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=cddc24f7a0fa8988589cec6bcc28e983) 5. Click *Create bucket*. ## Azure Private Link Backups Azure Private Link enables you to access Azure Platform-as-a-Service (PaaS) Services (for example, Azure Storage and SQL Database) and Azure hosted customer-owned/partner services over a private endpoint in your virtual network. Traffic between your virtual network and the service travels the Microsoft backbone network. Exposing your service to the public internet is no longer necessary. Using the Azure private link, the Controller backups happen privately from your VNet so that your blob storage account does not need to be exposed to the outside world. 1. Create an Azure Storage Account. 2. Set up the Storage Account for Private Link: 1. On the Networking tab for the storage account creation, select Private endpoint for the connectivity method. 2. Add a new private endpoint with the target of the blob storage resource and enable DNS Integration. > [!NOTE] > > If you currently have existing private endpoints deployed, you may need to use > an existing private zone in another subscription. This must be completed through > the dedicated private endpoint creation workflow. For additional assistance with > this setup, reach out to an Aviatrix Solution Engineer. 3. Verify Backup through Controller. Once successful, backing up traffic from the Controller will be performed privately across private link so that associated storage account does not need to be accessible publicly. # Upgrade OpenVPN Users Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/maintenance/controller-gateway-openvpn-user-upgrade Most upgrades do not impact connected OpenVPN users. In some cases, OpenVPN service needs to be restarted as part of software upgrade. For example, upgrading to a new SSL version for security patch. Most upgrades do not impact connected OpenVPN users. In some cases, OpenVPN service needs to be restarted as part of software upgrade. For example, upgrading to a new SSL version for security patch. In these cases, connected OpenVPN users are disconnected and need to reconnect after the upgrade. If a release requires stopping and restarting the service, the information is included in the release notes. Rollbacks do disrupt services. If there is only one OpenVPN gateway in service, all user connections are lost and users cannot reconnect until the gateway is available. If there are other OpenVPN gateways available, the disconnected users can attempt to log in again and land on the available gateways. # Upgrade Your Controller and Gateways Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/maintenance/controller-gateway-upgrade-controller This page explains how to perform single-version Controller and Gateway upgrade using either CoPilot (recommended) or Controller UI. Choose a tab below. This page explains how to perform single-version Controller and Gateway upgrade using either **CoPilot** (recommended) or **Controller UI**. Choose a tab below. For multiple-version Controller and Gateway upgrade, see [Upgrade your Controller and Gateways to the Latest Aviatrix Supported Images (AWS and Azure Only)](/docs/enterprise/10.1/guides/maintenance/controller-gateway-upgrade-latest-image). **CoPilot** > [!NOTE] > > If your Controller version is 7.0 or earlier, use Controller UI for the > Controller and Gateway upgrade. See the [Controller UI](#controller-ui) tab. This page instructs you on how to perform single-version Controller and Gateway upgrade via the CoPilot UI. ## Before Upgrade Before upgrading your Controller and Gateways, it is highly recommended to check the [Upgrade Overview](/docs/enterprise/10.1/guides/maintenance/upgrade-overview). * You must upgrade the Controller first, ensuring it is at the desired release version. * Once the Controller upgrade is completed successfully, you proceed to upgrade the Gateways. ### Review the Pre-upgrade Tasks Check and perform all the pre-upgrade tasks. See details in [Pre-upgrade Tasks for Controller and Gateways](/docs/enterprise/10.1/guides/maintenance/controller-upgrade-planning). ### Check the Upgrade Checklist Perform all the items listed in the upgrade checklist. See details in [Upgrade Checklist of the Controller and Gateway Upgrade](/docs/enterprise/10.1/guides/maintenance/controller-upgrade-checklist). The following table outlines the steps involved in upgrading the Controller and Gateways when upgrading between different versions. | Upgrade From | Upgrade To | Steps | | - | - | - | | 7.1 | 7.2 | Upgrade Controller to 7.2, then upgrade Gateways to 7.2 | | 7.1 | 7.2.x | Upgrade Controller to 7.2.x, then upgrade Gateways to 7.2.x | ## Upgrade your Controller Perform the following steps to upgrade your Controller to a desired version: 1. Log into your CoPilot. 2. Go to **Administration** > **Maintenance** > **Controller Upgrade**. 3. Select the **Target Version** to which you want to upgrade. By default, it will upgrade to the latest version of the current release. Alternatively, you can specify a specific release version. 4. Click **Run Readiness Check** and resolve any **Failed (Blocker)** issues before you continue. See [Readiness Check for the Aviatrix Controller Upgrade](/docs/enterprise/10.1/guides/maintenance/controller-upgrade-readiness-check). 5. Follow the on-screen prompts to start the Controller upgrade process. ## Upgrade your Gateways Perform the following steps to upgrade your Gateways to a desired version: 1. Log into your CoPilot. 2. Go to **Administration** > **Maintenance** > **Gateway Upgrade**. 3. Create or select an upgrade group and choose the gateways you want to upgrade. However, please note that the system will upgrade the Controller first and then the gateways. 1. Click **Run Readiness Check** to check for potential issues. 2. (Optional) If any issues are reported, address and fix them accordingly. 3. Click **Prepare For Upgrade** to upgrade the selected gateways. 4. Alternatively, go to **Cloud Fabric** > **Gateways** > **Gateway Management** and click **Software Upgrade**. 1. Select the gateways you want to upgrade. 2. Click **Actions** > **Upgrade Software**. 5. Wait for the upgrade process to complete and verify that you receive a successful upgrade message. | Upgrade HA (High Availability) gateways first, then upgrade primary gateways: | | | - | - | | To ensure proper continuity and system availability, it is recommended to upgrade HA gateways before upgrading primary gateways. This sequence minimizes any potential disruptions during the upgrade process. | After upgrading your HA gateways, upgrade the primary gateways. | If you wish to organize gateways into groups and upgrade them in batches, see [Upgrade Gateways in Groups](/docs/enterprise/10.1/guides/maintenance/gateway-upgrade-groups). ## Verify your Upgrade Status After performing an upgrade, it is important to [verify your upgrade status](/docs/enterprise/10.1/reference/ui/administration/maintenance). **Controller UI** > [!NOTE] > > You can also perform Controller Upgrade and gateway upgrade from CoPilot UI if > your Controller version is 7.1 or later. Go to *CoPilot* > *Settings* > > *Maintenance* > *Upgrade*. See the [CoPilot](#copilot) tab for details. This page instructs on how to perform single-version Controller and Gateway upgrade via the Controller UI. ## Before Upgrade Before upgrading your Controller and Gateways, it is highly recommended to check the [Upgrade Overview](/docs/enterprise/10.1/guides/maintenance/upgrade-overview). * It is important to upgrade the Controller first, ensuring it is at the desired release version. * Once the Controller upgrade is completed successfully, you proceed to upgrade the Gateways. ### Review the Pre-upgrade Tasks Check and perform all the pre-upgrade tasks. See details in [Pre-upgrade Tasks for Controller and Gateways](/docs/enterprise/10.1/guides/maintenance/controller-upgrade-planning). ### Check the Upgrade Checklist Perform all the items listed in the upgrade checklist. See details in [Upgrade Checklist of the Controller and Gateway Upgrade](/docs/enterprise/10.1/guides/maintenance/controller-upgrade-checklist). The following table outlines the steps involved in upgrading the Controller and Gateways when upgrading between different versions. | Upgrade From | Upgrade To | Steps | | - | - | - | | 7.0 | 7.1 | Upgrade Controller to 7.1, then upgrade Gateways to 7.1 | | 7.1 | 7.2 | Upgrade Controller to 7.2, then upgrade Gateways to 7.2 | ## Upgrade your Controller Perform the following steps to upgrade your Controller to a desired version: 1. Log in to your Controller UI. 2. Go to *SETTINGS* > *Maintenance*, click *Upgrade* to open the upgrade panel. 3. Under the *Platform Upgrade* section, enter the release number to which you want to upgrade. By default, it will upgrade to the latest version of the current release. Alternatively, you can specify a specific release version. 4. Click *PLATFORM UPGRADE* to initiate the Controller upgrade process. ## Upgrade your Gateways Perform the following steps to upgrade your Gateways to a desired version: 1. Log in to your Controller UI. 2. Go to *SETTINGS* > *Maintenance*, click *Upgrade* to open the upgrade panel. 3. Go to the *Selective Gateway Upgrade* section, choose the gateways you want to upgrade from the dropdown list. However, please note that the system will upgrade the Controller first and then the gateways. 4. Click *Dry Run* to check for potential issues. 5. (Optional) If any issues are reported, address and fix them accordingly. 6. Click *SOFTWARE UPGRADE* to upgrade the selected gateways. To perform an image upgrade, click *IMAGE UPGRADE*. 7. Wait for the upgrade process to complete and verify that you receive a successful upgrade message. > [!NOTE] > > Upgrade HA (High Availability) gateways first, then upgrade primary gateways: > > To ensure proper continuity and system availability, it is recommended to > upgrade HA gateways before upgrading primary gateways. This sequence minimizes > any potential disruptions during the upgrade process. After upgrading your HA > gateways, upgrade the primary gateways. ## Verify your Upgrade After performing an upgrade, it is important to [verify your upgrade status](/docs/enterprise/10.1/reference/ui/administration/maintenance). # Upgrade your Controller and Gateways to the Latest Aviatrix Supported Images (AWS and Azure Only) Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/maintenance/controller-gateway-upgrade-latest-image This auto-upgrade process only applies to AWS and Azure. Contact Aviatrix Support if your Controller and Gateways are deployed on other cloud service providers. > [!WARNING] > > This auto-upgrade process only applies to AWS and Azure. Contact [Aviatrix > Support](https://support.aviatrix.com) if your Controller and Gateways are > deployed on other cloud service providers. For single-version Controller and Gateway upgrade, see [Single-Version Upgrade for Controller and Gateways](/docs/enterprise/10.1/guides/maintenance/controller-gateway-upgrade-steps) . This documentation aims at providing steps on upgrading your Controller and Gateways to the latest Aviatrix images. During the upgrade, the process transfers all settings from the old Controller to the new Controller, which keeps the same public IP (Elastic IP) but gets a new private IP. This change helps to maintain seamless service during the transition. After the upgrade is complete, the old Controller is stopped. Aviatrix recommends terminating the old Controller to avoid unnecessary costs. ## Major Upgrade Paths Overview The following diagram illustrates the major upgrade paths for the Aviatrix Controller and Gateways. ![Major Upgrade Paths for Controller and Gateways](https://mintcdn.com/aviatrix-14b37c43/gul3YKv39kEEZCPb/images/guides/maintenance/upgrade-path-10.1.11.svg?fit=max\&auto=format\&n=gul3YKv39kEEZCPb\&q=85\&s=080c6d79819aabf20f68716825b709bd) ## Major Upgrade Paths for Controller and Gateways | Current Version | End Version | Upgrade Path 8 | | - | - | - | | 10.0.10 | 10.1.11 (g5) (Latest Controller version) | 10.0.10 → 10.1.11 1. Perform a software upgrade from 10.0.10 to 10.1.11 of the Controller and all the gateways. | | 9.0.20 | 10.1.11 (g5) (Latest Controller version) | 9.0.20 → 10.0.10 → 10.1.11 1. Perform an image upgrade from 9.0.20 to 10.0.10 of the Controller and an image upgrade of all the gateways. 10.0.10 introduces the g5 Controller and gateway image generation. 2. Perform a software upgrade from 10.0.10 to 10.1.11 of the Controller and all the gateways. | | 9.0.x 7 | 10.1.11 (g5) (Latest Controller version) | 9.0.x → 9.0.20 → 10.0.10 → 10.1.11 1. Perform a software upgrade from 9.0.x to 9.0.20 of the Controller and all the gateways. 2. Perform an image upgrade from 9.0.20 to 10.0.10 of the Controller and an image upgrade of all the gateways. 3. Perform a software upgrade from 10.0.10 to 10.1.11 of the Controller and all the gateways. | | 8.2.x 6 | 10.1.11 (g5) (Latest Controller version) | 8.2.x → 8.2.30 → 9.0.20 → 10.0.10 → 10.1.11 1. Perform a software upgrade from 8.2.x to 8.2.30 of the Controller and all the gateways. 2. Perform a software upgrade from 8.2.30 to 9.0.20 of the Controller and all the gateways. 3. Perform an image upgrade from 9.0.20 to 10.0.10 of the Controller and an image upgrade of all the gateways. 4. Perform a software upgrade from 10.0.10 to 10.1.11 of the Controller and all the gateways. | | 8.1.x 5 | 10.1.11 (g5) (Latest Controller version) | 8.1.x → 8.1.50 → 8.2.30 → 9.0.20 → 10.0.10 → 10.1.11 1. Perform a software upgrade from 8.1.x to 8.1.50 of the Controller and all the gateways. 2. Perform a software upgrade from 8.1.50 to 8.2.30 of the Controller and all the gateways. 3. Perform a software upgrade from 8.2.30 to 9.0.20 of the Controller and all the gateways. 4. Perform an image upgrade from 9.0.20 to 10.0.10 of the Controller and an image upgrade of all the gateways. 5. Perform a software upgrade from 10.0.10 to 10.1.11 of the Controller and all the gateways. | | 8.0.x 4 | 10.1.11 (g5) (Latest Controller version) | 8.0.x → 8.0.70 → 8.1.50 → 8.2.30 → 9.0.20 → 10.0.10 → 10.1.11 1. Perform a software upgrade from 8.0.x to 8.0.70 of the Controller and all the gateways. 2. Perform a software upgrade from 8.0.70 to 8.1.50 of the Controller and an image upgrade of all the gateways. 3. Perform a software upgrade from 8.1.50 to 8.2.30 of the Controller and all the gateways. 4. Perform a software upgrade from 8.2.30 to 9.0.20 of the Controller and all the gateways. 5. Perform an image upgrade from 9.0.20 to 10.0.10 of the Controller and an image upgrade of all the gateways. 6. Perform a software upgrade from 10.0.10 to 10.1.11 of the Controller and all the gateways. | | 7.2.x 3 | 10.1.11 (g5) (Latest Controller version) | 7.2.x → 7.2.5105 → 8.0.70 → 8.1.50 → 8.2.30 → 9.0.20 → 10.0.10 → 10.1.11 1. Perform a software upgrade from 7.2.x to 7.2.5105 of the Controller and all the gateways. 2. Perform an image upgrade from 7.2.5105 to 8.0.70 of the Controller and a software upgrade of all the gateways. 3. Perform a software upgrade from 8.0.70 to 8.1.50 of the Controller and an image upgrade of all the gateways. 4. Perform a software upgrade from 8.1.50 to 8.2.30 of the Controller and all the gateways. 5. Perform a software upgrade from 8.2.30 to 9.0.20 of the Controller and all the gateways. 6. Perform an image upgrade from 9.0.20 to 10.0.10 of the Controller and an image upgrade of all the gateways. 7. Perform a software upgrade from 10.0.10 to 10.1.11 of the Controller and all the gateways. | | 7.1.4105 | 10.1.11 (g5) (Latest Controller version) | 7.1.4105 → 7.1.4208 → 7.2.5105 → 8.0.70 → 8.1.50 → 8.2.30 → 9.0.20 → 10.0.10 → 10.1.11 1. Perform a software upgrade from 7.1.4105 to 7.1.4208 of the Controller and all the gateways. 2. Perform a software upgrade from 7.1.4208 to 7.2.5105 of the Controller and all the gateways. 3. Perform an image upgrade from 7.2.5105 to 8.0.70 of the Controller and a software upgrade of all the gateways. 4. Perform a software upgrade from 8.0.70 to 8.1.50 of the Controller and an image upgrade of all the gateways. 5. Perform a software upgrade from 8.1.50 to 8.2.30 of the Controller and all the gateways. 6. Perform a software upgrade from 8.2.30 to 9.0.20 of the Controller and all the gateways. 7. Perform an image upgrade from 9.0.20 to 10.0.10 of the Controller and an image upgrade of all the gateways. 8. Perform a software upgrade from 10.0.10 to 10.1.11 of the Controller and all the gateways. | | 7.1.x 2 | 10.1.11 (g5) (Latest Controller version) | 7.1.x → 7.1.4101 → 7.1.4208 → 7.2.5105 → 8.0.70 → 8.1.50 → 8.2.30 → 9.0.20 → 10.0.10 → 10.1.11 1. Perform a software upgrade from 7.1.x to 7.1.4101 of the Controller and all the gateways. 2. Perform an image upgrade from 7.1.4101 to 7.1.4208 of the Controller and all the gateways. 3. Perform a software upgrade from 7.1.4208 to 7.2.5105 of the Controller and all the gateways. 4. Perform an image upgrade from 7.2.5105 to 8.0.70 of the Controller and a software upgrade of all the gateways. 5. Perform a software upgrade from 8.0.70 to 8.1.50 of the Controller and an image upgrade of all the gateways. 6. Perform a software upgrade from 8.1.50 to 8.2.30 of the Controller and all the gateways. 7. Perform a software upgrade from 8.2.30 to 9.0.20 of the Controller and all the gateways. 8. Perform an image upgrade from 9.0.20 to 10.0.10 of the Controller and an image upgrade of all the gateways. 9. Perform a software upgrade from 10.0.10 to 10.1.11 of the Controller and all the gateways. | | 7.0.x 1 | 10.1.11 (g5) (Latest Controller version) | 7.0.x → 7.0.2250 → 7.1.4101 → 7.1.4208 → 7.2.5105 → 8.0.70 → 8.1.50 → 8.2.30 → 9.0.20 → 10.0.10 → 10.1.11 1. Perform a software upgrade from 7.0.x to 7.0.2250 of the Controller and all the gateways. 2. Perform a software upgrade from 7.0.2250 to 7.1.4101 of the Controller and all the gateways. 3. Perform an image upgrade from 7.1.4101 to 7.1.4208 of the Controller and all the gateways. 4. Perform a software upgrade from 7.1.4208 to 7.2.5105 of the Controller and all the gateways. 5. Perform an image upgrade from 7.2.5105 to 8.0.70 of the Controller and a software upgrade of all the gateways. 6. Perform a software upgrade from 8.0.70 to 8.1.50 of the Controller and an image upgrade of all the gateways. 7. Perform a software upgrade from 8.1.50 to 8.2.30 of the Controller and all the gateways. 8. Perform a software upgrade from 8.2.30 to 9.0.20 of the Controller and all the gateways. 9. Perform an image upgrade from 9.0.20 to 10.0.10 of the Controller and an image upgrade of all the gateways. 10. Perform a software upgrade from 10.0.10 to 10.1.11 of the Controller and all the gateways. | > [!WARNING] > > **Azure gateway users starting at 8.1.x or later**: The software upgrade steps > listed above do not deliver the MANA driver fix. Along your path, the gateway > portion of **at least one** step landing on a MANA-compatible release (8.1.50, > 8.2.30, or 9.0.20) must be performed as an **image upgrade** instead of a > software upgrade. For example, on the **8.1.x → 9.0.20** path, replace the > gateway portion of step 1 (to 8.1.50), step 2 (to 8.2.30), **or** step 3 (to > 9.0.20) with an image upgrade. This requirement does not apply to AWS-only > environments, or to upgrade paths starting at 8.0.x or earlier (where an image > upgrade is already a required step). For details, see the > [MANA Field Notice (FN-2026-AZ-001)](/docs/enterprise/10.1/reference/release-notices/field-notices/index#long-term-fix-mana-compatible-images-and-upgrade-groups) > . > [!NOTE] > > 1 7.0.x refers to any 7.0 release earlier than 7.0.2250. > > 2 7.1.x refers to any 7.1 release earlier than 7.1.4101. > > 3 7.2.x refers to any 7.2 release earlier than 7.2.5105. > > 4 8.0.x refers to any 8.0 release earlier than 8.0.70. > > 5 8.1.x refers to any 8.1 release earlier than 8.1.50. > > 6 8.2.x refers to any 8.2 release earlier than 8.2.30. > > 7 9.0.x refers to any 9.0 release earlier than 9.0.20. > > 8 If you are using a private marketplace in Azure, you must subscribe > to the gateway image offer in the Azure marketplace first. See > [Subscribing Gateway Offers to Azure Private Marketplace for Aviatrix > Deployments](/docs/enterprise/10.1/guides/controlplane/azure-powershell-deploy) > . > > 9 If you are using a private marketplace in Azure, you must subscribe to the controller image offer for 7.2.x in Azure marketplace first. See [Subscribing Controller Offers to Azure Private Marketplace for Aviatrix Deployments](/docs/enterprise/10.1/guides/controlplane/azure-powershell-deploy). ## Major Upgrade Steps > [!WARNING] > > Aviatrix recommends performing all platform software lifecycle management > tasks on the CoPilot UI. If you are not currently using CoPilot, deploy it > before proceeding with any procedures in this documentation. The following steps summarize the major upgrade process: 1. Perform a software upgrade of your Controller and gateways to the latest release version within the current release. See [Upgrade of your Controller and gateways within the current release](/docs/enterprise/10.1/guides/maintenance/controller-gateway-upgrade-steps) . 2. Back up your Controller data. See [Back up your Controller data](/docs/enterprise/10.1/guides/maintenance/controller-backup-restore#copilot) . 3. Perform a [Controller Image Upgrade Dry Run](/docs/enterprise/10.1/guides/maintenance/controller-upgrade-latest-image#controller-upgrade-dry-run) to check for potential errors. 4. Resolve any potential errors according to the [provided solutions](#troubleshooting-your-controller-and-gateway-upgrade). 5. Perform an Image Upgrade of your Controller to the latest Aviatrix supported image. See [Image Upgrade of your Controller to the Latest Aviatrix Supported Images](/docs/enterprise/10.1/guides/maintenance/controller-upgrade-latest-image#controller-image-upgrade) . > [!NOTE] > > If your Controller has been upgraded to 7.1.3958 or later, you can perform > a software upgrade to 7.1.4208 (latest 7.1 version). Then perform a > software upgrade to 7.2.5105. If your Controller is already on 7.2.x, skip > this step. Then perform a software upgrade to 7.2.5105 (latest 7.2 > version). Then, perform an image upgrade from 7.2.5105 to 8.0.70. Then you > can perform a software upgrade from 8.0.70 to 8.1.50. Then you can perform > a software upgrade to 8.2.30. Then you can perform another software upgrade > to 9.0.20. Then, perform an image upgrade from 9.0.20 to 10.0.10. Then, > perform a software upgrade from 10.0.10 to 10.1.11 (latest Controller > version). 6. Verify your [Controller Upgrade Status](/docs/enterprise/10.1/guides/maintenance/controller-upgrade-latest-image#verify-controller-image-upgrade) . 7. Perform an Image Upgrade of all your gateways to the latest Aviatrix supported image. See [Image Upgrade of your Gateways to the Latest Aviatrix Supported Images](/docs/enterprise/10.1/guides/maintenance/gateway-upgrade-latest-image#gateway-image-upgrade) . > [!NOTE] > > If your gateways have been upgraded to 7.1.3958 or later, you can perform a > software upgrade to 7.1.4208 for the latest 7.1 software release. If your > Controller is already on 7.2.x, skip this step. You can then perform > another software upgrade to 7.2.5105 for the latest 7.2 software version. > Then you can perform a software upgrade from 7.2.5105 to 8.0.70. Then, you > can perform an image upgrade from 8.0.70 to 8.1.50. Then you can perform a > software upgrade to 8.2.30. Then you can perform another software upgrade > to 9.0.20. Then, perform an image upgrade from 9.0.20 to 10.0.10. Then, > perform a software upgrade from 10.0.10 to 10.1.11 for the latest software > version. 8. Verify your [Gateway Upgrade Status](/docs/enterprise/10.1/guides/maintenance/gateway-upgrade-latest-image#verify-gateway-upgrade) . ## Prepare for Upgrading your Controller and Gateways to the Latest Aviatrix Supported Images > [!NOTE] > > If you plan to deploy a new Controller from the AWS marketplace or Azure > marketplace before performing the upgrade, make sure to select the correct > Controller version. See > [Aviatrix Controller and Gateway Image Release Notes](/docs/enterprise/10.1/reference/release-notices/release-notes/controller-images/index) > for the latest Controller version. Check the following prerequisites before upgrading your Controller. * Make sure your Controller and CoPilot are deployed on AWS or Azure. For deployments on other cloud service providers, contact [Aviatrix Support](https://support.aviatrix.com) to help you perform the upgrade. * Check your Controller version to ensure it meets the version requirements. Perform a software upgrade of your Controller to make sure the Controller version is 7.1.4101 or later. See [Upgrade your Controller via CoPilot UI](/docs/enterprise/10.1/guides/maintenance/controller-gateway-upgrade-controller#upgrade-controller) for more details. * Ensure that your gateways are in 7.1.4101 version or later. Upgrade any gateways that are not in 7.1.4101 to 7.1.4101. See [Upgrade your Gateways via CoPilot UI](/docs/enterprise/10.1/guides/maintenance/controller-gateway-upgrade-controller#upgrade-gateways) for details. * Make sure you have installed the latest CoPilot version (>=4.27). See [Deploy CoPilot](/docs/enterprise/10.1/guides/controlplane/copilot-deployment) . * Make sure that you have subscribed to the [Required Aviatrix License and Marketplace Offers](/docs/enterprise/10.1/guides/controlplane/getting-started-guide-aws#subscription-overview) and have a valid customer ID (license) issued by the Aviatrix Cloud Network License Service. For legacy licenses/listings below, contact [Aviatrix Support](https://support.aviatrix.com) for the upgrade assistance. \* Aviatrix Secure Networking Platform - Enterprise Subscription \* Aviatrix Secure Networking Platform Metered 2208 - Copilot & 24x7 Support \* Aviatrix Secure Networking Platform PAYG - Metered * Make sure no CloudN gateways are attached to your Controller. If there are any CloudN gateways attached to your Controller, you will first need to migrate to Aviatrix Edge. Contact [Aviatrix Support](https://support.aviatrix.com). * Make sure you have not enabled [Private Mode](https://legacy.docs.aviatrix.com/documentation/latest/platform-administration/controller/private-mode.html?expand=true). If you are using private mode, contact [Aviatrix Support](https://support.aviatrix.com) for the other upgrade options. * For Controller that is configured in HA mode, disable HA before the Controller upgrade. You will then need to enable HA mode after the Controller upgrade. See [Controller High Availability in AWS](/docs/enterprise/10.1/guides/controlplane/controller-ha-aws#controller-ha-details) for details. ## Before your Controller and Gateway Upgrade Ensure you complete the following before starting your Controller upgrade. ### Check the General Controller and Gateway Upgrade Guidance * Check the [General Controller and Gateways Upgrade Guidance](/docs/enterprise/10.1/guides/maintenance/controller-upgrade-workflow) . * Before proceeding with the upgrade in the production environment, perform the upgrade in a testing environment. * It is important to upgrade the Controller first, ensuring it is at the desired release version. * Once the Controller is successfully upgraded, proceed to upgrade the Gateways. ### Go through Checklist of the Controller and Gateway Upgrade 1. Go through the checklist of Controller and Gateway upgrade. See details in [Upgrade Checklist of the Controller and Gateway Upgrade](/docs/enterprise/10.1/guides/maintenance/controller-upgrade-checklist) . 2. Make sure you have [performed all the pre-upgrade tasks of the Controller and Gateway Upgrade](/docs/enterprise/10.1/guides/maintenance/controller-upgrade-planning) . ## Upgrade Controller and Gateways to the Latest Aviatrix Supported Images(AWS and Azure Only) To upgrade Controller and gateways for the latest Aviatrix Supported Images: 1. [Upgrade your Controller for the latest Aviatrix Supported Images (AWS and Azure Only)](/docs/enterprise/10.1/guides/maintenance/controller-upgrade-latest-image) 2. [Upgrade your Gateways for the latest Aviatrix Supported Images (AWS and Azure Only)](/docs/enterprise/10.1/guides/maintenance/gateway-upgrade-latest-image) ## Troubleshooting your Controller and Gateway Upgrade > [!NOTE] > > If the upgrade of your Controller fails, the system reverts the Controller to > its previous state. See [Troubleshooting Controller and Gateway Upgrades](/docs/enterprise/10.1/guides/maintenance/controller-gateway-upgrade-troubleshooting) for the detailed troubleshooting solutions. # Single-Version Upgrade for Controller and Gateways Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/maintenance/controller-gateway-upgrade-steps A single-version Controller and Gateway upgrade refers to: A single-version Controller and Gateway upgrade refers to: * Upgrading from the current version to the latest release version within the current release. * Upgrading from the current release (e.g., version N) to a higher release (N+1 release). Before you upgrade your Controller and Gateways, it is highly recommended to check the [Upgrade Overview](/docs/enterprise/10.1/guides/maintenance/upgrade-overview). > [!WARNING] > > * Before proceeding with the upgrade in the production environment, perform the > upgrade in a testing environment. > > * It is important to upgrade the Controller first, ensuring it is at the desired > release version. > > * Once the Controller is successfully upgraded, proceed to upgrade the Gateways. This section instructs you on how to perform a single-version Controller and Gateway upgrade. To perform a single-version Controller and gateway upgrade: * You can perform a Controller upgrade and gateway upgrade from the Controller UI; see [Upgrade your Controller via the Controller UI](/docs/enterprise/10.1/guides/maintenance/controller-gateway-upgrade-controller) for details. * If your Controller version is 7.1 or later, you can also upgrade Controller and gateways via the CoPilot UI. See [Upgrade your Controller and Gateways via CoPilot UI](/docs/enterprise/10.1/guides/maintenance/controller-gateway-upgrade-controller) for details. For multi-version Controller and Gateway upgrade, see [Upgrade your Controller and Gateways to the Latest Aviatrix Supported Images (AWS and Azure Only)](/docs/enterprise/10.1/guides/maintenance/controller-gateway-upgrade-latest-image). For upgrading Controller and Gateways for the latest Aviatrix supported images, see [Upgrade your Controller and Gateways to the Latest Aviatrix Supported Images (AWS and Azure Only)](/docs/enterprise/10.1/guides/maintenance/controller-gateway-upgrade-latest-image). # Upgrade HA Gateways in An ActiveMesh Topology Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/maintenance/controller-hagateway-activemesh-topolog Gateway traffic is briefly affected and there is a drop in throughput when you perform a gateway image upgrade, and when a gateway software upgrade is rolled back. ## Upgrading HA Gateways in an ActiveMesh Topology Gateway traffic is briefly affected and there is a drop in throughput when you perform a gateway image upgrade, and when a gateway software upgrade is rolled back. If Aviatrix ActiveMesh mode is enabled and only one gateway in an ActiveMesh pair is selected for upgrade, the system gracefully drains the traffic away from one of the gateways so it can be replaced. If both gateways in an ActiveMesh pair are selected, the gateways are replaced simultaneously without any additional safeguards. * If the gateway has BGP peers, the BGP process is shut down and the protocol reconverges to elect alternatives routes. * The tunnel interfaces are shut down. The Controller recalculates alternatives routes and distributes them to the gateways within the Aviatrix network. * If the selected gateway is a spoke, the Controller modifies the underlay cloud routing table of the selected gateway that was acting as the next hop for the default route or RFC1918 routes. The HA peer is selected as the next hop. ![Diagram showing gateway reroute during upgrade](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/maintenance/upgrade-gateway-reroute.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=b69728476599be35389773471b5c5697) ## Related Topics * [Upgrade OpenVPN Users](/docs/enterprise/10.1/guides/maintenance/controller-gateway-openvpn-user-upgrade) * [Unattended Upgrades for Controllers and Gateways](/docs/enterprise/10.1/guides/maintenance/unattended-upgrade) # Upgrade Checklist of the Controller and Gateway Upgrade Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/maintenance/controller-upgrade-checklist This topic provides an overview of the upgrade to-do list of the tasks before you begin your upgrade, during your upgrade, and after the upgrade. This topic provides an overview of the upgrade to-do list of the tasks before you begin your upgrade, during your upgrade, and after the upgrade. Before the Controller and Gateway Upgrade, perform the following steps: ## Perform the Pre-upgrade Tasks for Controller and Gateways Perform all the [Pre-upgrade Tasks for Controller and Gateways](/docs/enterprise/10.1/guides/maintenance/controller-upgrade-planning) before your upgrade. ## Understanding the Release Contents To understand the contents and potential impact of upgrading to a specific software release, see [Aviatrix Controller and Gateway Image Release Notes](/docs/enterprise/10.1/reference/release-notices/release-notes/controller-images/index). To understand the contents and potential impact of upgrading to a specific image release, see [Aviatrix Controller and Gateway Image Release Notes](/docs/enterprise/10.1/reference/release-notices/release-notes/controller-images/index). ### Verify DNS Settings The Aviatrix Controller must have a reliable DNS resolution service available. Aviatrix recommends using the default 8.8.8.8 for the DNS IP address. Using the default address is not required, but your network must be able to resolve public names and have uninterrupted access to the DNS name resolver. #### AWS and Azure DNS Settings If the Controller is running on AWS or Azure, you can go to the controller Settings for the DNS and Disable the VPC or VNET DNS Server to force the Controller to use 8.8.8.8. ## Verify Public Internet Access Verify access to the public internet from the Aviatrix Controller. The Controller must be open for inbound traffic on port 443 and outbound traffic on port 22. Aviatrix recommends you enable security groups to restrict access. Go to the Network tab on the Diagnostics page under Troubleshooting and perform the following tasks. * Ping a widely known public hostname or IP address with the Controller Utility. * Ping security.aviatrix.com from the TCP/443 port with the Network Connectivity Utility. * Ping [www.github.com](http://www.github.com) from the TCP/443 port with the Network Connectivity Utility. * Ping [www.github.com](http://www.github.com) from outbound TCP/22 on Controller with the Network Connectivity Utility. ## Verify Account Permissions and Access Go to the Accounts page and perform the following tasks. * Go to the Accounts Audit tab under Accounts and perform an Account Audit. Correct any reported issues. * Verify all accounts can access all connected cloud resources. * Verify the Aviatrix primary access account is available and that the account credentials are valid. * The IAM policies must be configured as recommended by Aviatrix. For more information, see Controller Instance Requirements. * If you are migrating your Aviatrix Platform Controller to a new image, verify the new image has all required accounts and permissions before migrating the Controller. If you are restoring an image from a backup, the required accounts and permissions should all be available. Migration operations fail if there is not at least one Aviatrix backup file available. ## Verify Controller and Gateway Status Go to the Controller Dashboard and check the status of the Aviatrix Platform Controller and gateways. * Verify all gateways are up and the status is green. * Verify all tunnels are up and the status is green. ## (AWS Specific) Upgrade Checklist ### Verify Controller HA Version You should be running the latest version of the Controller HA application before upgrading. If there is a newer version of Controller HA available, you should upgrade by disabling and re-enabling the Controller HA feature. For more information, see [AWS High Availability](/docs/enterprise/10.1/guides/controlplane/controller-ha-aws). ### Verify Controller HA is Disabled If you use Controller HA you will need to disable your HA configuration before upgrading Controller. If you do not disable Controller HA before upgrading, the system deploys a new Controller and restores the most recent backup. ### Settings for t2 and t3 Instances If your Aviatrix Controller is in AWS and running on a t2 or t3 instance type and you are planning a platform image upgrade, you must set the T2/T3 Unlimited attribute to enabled. For more information, see [Unlimited mode concepts](https://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/burstable-performance-instances-unlimited-mode-concepts.html). # Upgrade Your Controller to the Latest Aviatrix Supported Images (AWS and Azure Only) Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/maintenance/controller-upgrade-latest-image This documentation aims at providing steps on upgrading your Controller to the latest Aviatrix images. This documentation aims at providing steps on upgrading your Controller to the latest Aviatrix images. > [!NOTE] > > This upgrade workflow only applies to AWS and Azure. Contact [Aviatrix > Support](https://support.aviatrix.com) if your Controller and Gateways are > deployed on other cloud service provides. For the major upgrade paths, see [Major Upgrade Paths](/docs/enterprise/10.1/reference/ui/administration/maintenance#major-upgrade-paths). ## Before Upgrade Before you upgrade your Controller and Gateways, it is highly recommended to check the [prerequisites](/docs/enterprise/10.1/guides/maintenance/controller-upgrade-planning) to ensure that your Controller meets the upgrade requirements. ### Perform the Pre-upgrade Tasks Check and perform all the pre-upgrade tasks. See details in [Upgrade Overview](/docs/enterprise/10.1/guides/maintenance/upgrade-overview). ### Check the Upgrade Checklist Perform all the items listed in the upgrade checklist. [Go through Checklist of the Controller and Gateway Upgrade](/docs/enterprise/10.1/guides/maintenance/controller-upgrade-checklist) . ## Upgrade Your Controller to the Latest Aviatrix Supported Images There are four phases of upgrading your Controller: 1. Run the [Readiness Check](#run-the-readiness-check-for-the-upgrade). 2. Review the [Readiness Check Result](#review-the-readiness-check-result). 3. [Perform the Upgrade](#perform-an-image-upgrade-of-your-controller-to-the-latest-aviatrix-supported-images). 4. [Verify the Upgrade](#verify-your-controller-upgrade-status). ## Run the Readiness Check for the Upgrade The Readiness Check reviews compatibility, prerequisites, and known issues to better prepare for a successful upgrade. For full details, see [Readiness Check for the Aviatrix Controller Upgrade](/docs/enterprise/10.1/guides/maintenance/controller-upgrade-readiness-check). Confirm your Controller version meets the required 7.1.4101. You can check your Controller version on the Controller Upgrade page at *CoPilot > Administration > Maintenance > Controller Upgrade*. Perform the following steps: 1. From *CoPilot > Administration > Maintenance > Controller Upgrade*, select the *Target Version* to check against. 2. Click *Run Readiness Check*. ## Review the Readiness Check Result 1. On the *CoPilot > Administration > Maintenance > Controller Upgrade* page, review the **Failed (Blocker)**, **Failed (Non-Blocker)**, **Could Not Run**, and **Passed** counts, then review each listed issue. Solve any blocking issues as indicated. See [Troubleshooting Controller and Gateway Upgrades](/docs/enterprise/10.1/guides/maintenance/controller-gateway-upgrade-troubleshooting). 2. Once you have resolved the identified issues, click *Re-Run Readiness Check* to run the check again. ## Perform an Image Upgrade of your Controller to the Latest Aviatrix Supported Images > [!WARNING] > > During the upgrade, you will not be able to change the settings in Aviatrix > Network and Security. You will be temporarily logged out of CoPilot and unable > to access the Controller. For minimal disruption, we recommend performing this > upgrade during your maintenance window. Your Aviatrix data plane will continue > to function normally during the Controller image upgrade process. Perform the following steps to upgrade your Controller and Gateways to the latest Aviatrix supported images. 1. Enter your upgrade time limit. The upgrade of your Controller will take from 30 to 60 minutes based on your deployment size. You can set a time limit for the upgrade to enable appropriate actions if the process exceeds the expected duration. For large networks with more than 500 gateways, set the time to at least 2 hours. Choose one of the following actions when the upgrade time limit expires: * *Revert the Controller to its Previous State*: This option allows reverting the Controller back to its previous state if the upgrade does not complete within the set time limit. This option helps to prevent prolonged system downtime or instability. * *Continue Attempting the Upgrade*: Selecting this option allows the system to continue attempting the upgrade indefinitely, regardless of the time limit or user inaction. Users can also choose to revert the upgrade or proceed to complete the upgrade. This option might be suitable if the upgrade is critical and must be completed, regardless of how long it takes. You can use the estimated timeout session as your maintenance window. When the upgrade reaches the time limit, the system will send an email to notify you. 2. Click *Start Upgrade*. 3. (Optional) If some error occurs during the upgrade, you can choose to either: * Click *Work with Support*. * Click *Work with Support* and then click *Revert to Old Controller* to cancel the upgrade. If the New Controller Process Status is "Failed", the system will halt the upgrade and revert the upgrade to its previous state and bring back the old Controller. However, once the New Controller Process Status shows "Completed", reverting the upgrade to its previous state is no longer allowed through the CoPilot UI. To revert back to the old Controller at this stage, contact [Aviatrix Support](https://support.aviatrix.com). > [!NOTE] > > You will be blocked from closing the upgrade wizard. After a period of > time, the system will log you out automatically and you will have to wait > 10-15 minutes before you can log back in to CoPilot to check the upgrade > process. The upgrade continues uninterrupted, regardless of whether you are > logged out or logged back in. When prompted to login, enter: * Username * Password 4. After logging back into the CoPilot UI, wait till the *Next* button becomes clickable. Click *Next*. 5. Read all the instructions on the *Verify Upgrade* page. Once completed, click *Done* to exit the upgrade wizard. ### For the Controller Configured with a Load Balancer (Optional) If you have any load balancer configured for the old Controller. Please go to *AWS > EC2 > Load Balancing > Target Groups*, select the new Controller instance and then choose *Include as pending below*. When you are finished adding instances, choose *Register pending targets*. See [Register Controller with your Target Group](https://docs.aws.amazon.com/elasticloadbalancing/latest/application/target-group-register-targets.html). ## Verify Your Controller Upgrade Status After performing an upgrade, it is important to verify the upgrade status to ensure that it has been completed successfully. Perform the following steps to verify that your Controller and Gateways have been upgraded successfully. 1. Go to *CoPilot > Administration > Maintenance > Controller Upgrade* to check the Controller upgrade status. 2. Check if the upgrade window displays a message indicating that the Controller upgrade has been completed successfully. 3. Ensure that the displayed Controller version is updated to the latest version 4. Go to *CoPilot > Administration > Maintenance > Back Up & Migrate > Back Up Controller* to check the Controller backup is shown up. 5. Go to *CoPilot > Cloud Fabric > Gateway Management* to check the status of Controller and gateways. * Verify all gateways are up and the status is green. * Verify all tunnels are up and the status is green. 6. Go to *CoPilot > Diagnostics > AppIQ* to check whether there are some connectivity issue. 7. Go to *CoPilot > Diagnostics > FlowIQ* or perform a ping test on Controller to check if the traffic flows are running as expected. If the above conditions are met, this indicates that your Controller upgrade has been completed successfully. ## Cancel Your Controller Upgrade and Revert Your Controller to the Previous State > [!WARNING] > > If you decide to cancel an ongoing Controller and Gateway upgrade, the > cancellation stops the upgrade from proceeding further and will attempt to > revert your Controller to its previous state. Contact [Aviatrix > Support](https://support.aviatrix.com) before you click the *Force Completion* > or the *Revert to Old Controller* button. During the upgrade, you can: ### Get Assistance from Aviatrix Support The *Work with Support* button displays after the upgrade starts. Click the *Work with Support* button to work with [Aviatrix Support](https://support.aviatrix.com). * Get assistance from [Aviatrix Support](https://support.aviatrix.com). Click *Work with Support* to work with Aviatrix Support. * Click *Enable Remote Support*. You will then see the open remote support port and the log bundle name. * Click *Support* to find solutions or create a support ticket from *Support > Ticket Creation* from [Aviatrix Support website](https://support.aviatrix.com). * You can share your log bundle with [Aviatrix Support](https://support.aviatrix.com) for further assistance. ### Cancel Your Upgrade > [!WARNING] > > Contact [Aviatrix Support](https://support.aviatrix.com) before you click the > *Revert to Old Controller* button. You can cancel and revert your Controller to its previous state under the following scenarios: * During the upgrade * When the upgrade time limit is reached. Click *Revert to Old Controller* to cancel the Controller upgrade and revert Controller to its previous state before the upgrade started. This action will: * *Halt the upgrade process*. Please be noted that this process may take some time. If you need to access the old Controller, contact [Aviatrix Support](https://support.aviatrix.com). * *Revert your Controller to the previous state*. During the revert, the system will clean up all the temporary files and configurations that have been created during the upgrade attempt and bring back the Controller to its previous state. If you revert the Controller upgrade after logging out of the old Controller and logging into the new Controller, you will be logged out and returned to the CoPilot again as we restore the old Controller. Contact [Aviatrix Support](https://support.aviatrix.com) to understand more about potential negative impacts before selecting the *Revert to Old Controller* button. ### Force Completion of the Upgrade > [!WARNING] > > Contact [Aviatrix Support](https://support.aviatrix.com) before you click the > *Force Completion* button. When the upgrade time limit expires, you will be prompted with two options: force completion or revert to old Controller. Click *Force Completion* to continue the upgrade regardless of the expiration of the time limit. Contact [Aviatrix Support](https://support.aviatrix.com) to understand more about potential negative impacts before selecting the Force Completion button. For troubleshooting the Controller upgrade, see [Troubleshooting Controller and Gateway Upgrades](/docs/enterprise/10.1/guides/maintenance/controller-gateway-upgrade-troubleshooting). For upgrading gateways to the latest Aviatrix supported images, see [Upgrade Gateways for the Latest Aviatrix Supported Images (AWS and Azure Only)](/docs/enterprise/10.1/guides/maintenance/gateway-upgrade-latest-image). # Pre-Upgrade Tasks for Controller and Gateways Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/maintenance/controller-upgrade-planning Check the following prerequisites before you upgrade your controller and gateways: Check the following prerequisites before you upgrade your controller and gateways: Before upgrading your Controller and gateways, check the following prerequisites: ## Disable Deprecated Controller-Logging Configurations If you have logging configurations enabled in Controller for the following external log servers, the out-of-the box logging services for these external log servers were deprecated in previous Controller releases and are removed in Controller 7.1 versions - that is, any Controller version that begins with 7.1: * Elastic Filebeat * Splunk Enterprise/Cloud * Sumo Logic You cannot upgrade from any Controller 7.0 version to any Controller 7.1 version until you have disabled these deprecated logging configurations. To disable the deprecated logging configurations: * Depending on your environment, you may want to enable your log forwarding under rsyslog and verify the functionality is working before disabling the deprecated logging configurations. For information about using rsyslog as the logging mechanism to forward Aviatrix platform logs to your external log server, see [Aviatrix Controller Logging](/docs/enterprise/10.1/reference/ui/settings/configuration#configure-logging-services) . * Disable the deprecated logging configurations for Elastic Filebeat/Splunk Enterprise or Cloud/Sumo Logic, as applicable, in the Controller > Settings > Logging page. Locate the applicable external log server's respective option and switch its toggle from Enabled to Disabled. ## Inspect the Current Controller Resource Utilization 1. Inspect the current Controller's overall CPU and memory utilization from *CoPilot UI > Monitor > Performance*: * Ensure that the CPU utilization of the Controller is no more than 50%. * Do not proceed with the upgrade unless the Controller's Memory Available metric is at least 40% of Total Memory. These utilization thresholds should be met before initiating the upgrade. 2. Check the Controller storage usage from *CoPilot UI > Monitor > Performance*: * If you are upgrading to version 6.8, add additional disks. Add approximately 2.5MB per tunnel. * Ensure that the Controller has enough free disk space (at least 30% free) for the upgrade. * If the available disk space is insufficient, [resize the Controller disk](/docs/enterprise/10.1/guides/controlplane/disk-management) to an appropriate size before proceeding with the upgrade. Or you can contact [Aviatrix Support](https://support.aviatrix.com) to migrate the Controller with a large disk size before proceeding with the upgrade. ## Perform Controller and CoPilot Backup Perform a full backup of Copilot and the Controller: * Before initiating the upgrade process, it is crucial to perform a full backup of both Copilot and the Controller. * Save the previous backup in case it is needed for restoration purposes. For Controller backup and restore, see [Back Up and Restore Your Controller](/docs/enterprise/10.1/guides/maintenance/controller-backup-restore) for details. For CoPilot Backup and restore, see [CoPilot Backup and Restore](/docs/enterprise/10.1/guides/maintenance/controller-backup-restore#copilot) for details. ## Check for the Non-HA (without High Availability) Gateways Check for the non-HA gateways: * Please note that the upgrade procedure provided below is designed for environments with HA gateways. * Please ensure that all deployed environments include redundancy, with both primary and high-availability (HA) gateways configured. * If your environment does not have HA gateways and extensively uses Site2Cloud, there will be some downtime during the gateway image upgrade. It is recommended to consult [Aviatrix Support](https://support.aviatrix.com) before proceeding with the upgrade. ## (Optional) Set up a Testing Environment Before proceeding with the upgrade in the production environment, it is highly recommended to establish a dedicated testing environment. This environment should closely mirror the production setup, including hardware, software, and configurations. By doing so, you can simulate the production conditions and assess the compatibility and performance of the upgraded software in a controlled manner. Only when the testing phase is successfully completed, and all identified issues have been resolved, should you proceed with the upgrade in the production environment. # Readiness Check for the Aviatrix Controller Upgrade Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/maintenance/controller-upgrade-readiness-check Run the CoPilot Readiness Check before a Controller upgrade to surface compatibility and prerequisite issues before the maintenance window. Upgrading an Aviatrix Controller requires a healthy environment. CoPilot's **Readiness Check** reviews compatibility, prerequisites, and known issues before the upgrade so problems surface ahead of the maintenance window. > [!NOTE] > > The Readiness Check applies to Controller upgrades only. Gateway upgrades have > their own separate readiness checks under **Gateway Upgrade**. ## Prerequisites * An Aviatrix Controller. * A CoPilot account that can initiate upgrades. ## Run a Readiness Check **1. Open Controller Upgrade** In CoPilot, go to **Administration > Maintenance > Controller Upgrade**. **2. Select a target version** Select the **Target Version** to check against. **3. Run the check** Click **Run Readiness Check** (or **Re-Run Readiness Check** if a previous result is already showing). Readiness checks run in the background; check back in a few minutes for results. **4. Review the results** When the check completes, the page shows counts for **Failed (Blocker)**, **Failed (Non-Blocker)**, **Could Not Run**, and **Passed**, followed by the list of issues. Each issue shows its severity, a description, and a **Support** link. If there are no blocking failures, the upgrade can proceed. If any check reports a **Failed (Blocker)** result, resolve the issue (or contact Aviatrix Support) before you upgrade. > [!TIP] > > Click **Settings** next to **Run Readiness Check** to enable **Auto Readiness > Checks** on a recurring schedule (for example, every 24 hours), so results are > ready without running the check manually. ## Limitations * Only one Readiness Check can run at a time. Starting a new check while one is in progress returns an error. * The Readiness Check covers Controller upgrades only. Gateway upgrade readiness checks are separate. ## Related Topics * [Controller Upgrade Planning](/docs/enterprise/10.1/guides/maintenance/controller-upgrade-planning) * [Controller Upgrade Checklist](/docs/enterprise/10.1/guides/maintenance/controller-upgrade-checklist) * [Controller Upgrade Workflow](/docs/enterprise/10.1/guides/maintenance/controller-upgrade-workflow) # Procedures for Upgrading Aviatrix Controller and Gateways Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/maintenance/controller-upgrade-workflow This section outlines the general Controller and gateway upgrade instructions. This section outlines the general Controller and gateway upgrade instructions. ## General Controller and Gateways Upgrade Guidance This section provides general instructions for upgrading the Controller and Gateways. * Upgrade the Controller before upgrading the Gateways: * It is important to upgrade the Controller first, ensuring it is at the desired release version. * Once the Controller is successfully upgraded, proceed to upgrade the Gateways. * Upgrade from the current version to the latest release version within the current release: * Verify that the latest release version is available for your current release. * Consult the documentation specific to your current release version for detailed upgrade instructions. * Follow the provided steps to upgrade both the Controller and Gateways to the latest release version. * Upgrade from the current release (e.g., version N) to a higher release (N+1 release): * Note that neither the Controller nor the Gateways support multi-hop upgrades. * Upgrade sequentially from one adjacent version to another. * Determine the higher release version (N+1) to which you wish to upgrade. * Upgrade HA (High Availability) gateways first, then upgrade primary gateways: * To ensure proper continuity and system availability, it is recommended to upgrade HA gateways before upgrading primary gateways. This sequence minimizes any potential disruptions during the upgrade process. > [!NOTE] > > It is recommended to schedule upgrades during a maintenance window when short > periods of traffic disruption can be tolerated. In HA setups, the disruption > should be minimal. ## The Controller and Gateway Upgrade Steps **Upgrade Steps Outline** | Step No. | Description | | :-: | - | | 1 | Back up Copilot and the Controller | | 2 | Upgrade Controller to the latest release version of current release. | | 3 | Upgrade HA Gateways to the latest release version of current release. | | 4 | Upgrade primary gateways to the latest release version of current release. | | 5 | Upgrade Controller from current release (e.g., version N) to a higher release (N+1 release) | | 6 | Upgrade HA Gateways from current release (e.g., version N) to a higher release (N+1 release) | | 7 | Upgrade primary gateways from current release (e.g., version N) to a higher release (N+1 release) | > [!NOTE] > > There is a performance enhancement in the latest image of gateways. It is > highly recommended to do image upgrades for Transit gateways and optional for > spoke gateways for Controller version 7.0 and Controller 7.1. For detailed instructions about how to perform Controller and Gateway upgrade on CoPilot, see: * [Single-Version Controller and Gateway Upgrade Workflow](/docs/enterprise/10.1/guides/maintenance/controller-gateway-upgrade-steps) * [Upgrade your Controller and Gateways to the Latest Aviatrix Supported Images (AWS and Azure Only)](/docs/enterprise/10.1/guides/maintenance/controller-gateway-upgrade-latest-image) * Pre-upgrade Tasks for Controller and Gateways * Single-Version Upgrade for Controller and Gateways # Gateway Drain and Undrain During Image Upgrade and Maintenance Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/maintenance/gateway-drain-undrain-upgrades In 9.0, the Aviatrix Controller drains HA gateway pairs before image upgrade, delete, and resize operations, and undrains them only after the data plane has converged. In 9.0, the Aviatrix Controller drains HA gateway pairs before performing image upgrade, gateway delete, and gateway resize operations. The gateway is removed from Aviatrix routing and from the CSP route tables before the operation begins. After the operation, the gateway is undrained only when the data plane has converged. The drain and undrain sequence is automatic. There is no manual control over drain state in this release. ## How It Works When you start an in-scope maintenance operation on an HA gateway pair, the Controller runs this sequence on the gateway being changed: **1. Drain** The Controller removes the gateway from the Aviatrix routing fabric and updates CSP route tables so all traffic flows through the HA peer. **2. Run the operation** The image upgrade, delete, or resize operation runs while traffic flows through the HA peer. **3. Convergence verification** The Controller waits for the gateway to confirm that tunnels, BGP sessions, and dependent services are stable. **4. Undrain** Once convergence is confirmed, the Controller returns the gateway to active routing. CSP route tables are updated to include the gateway again. ## In-Scope Operations | Operation | Drain applies | | - | - | | Gateway image upgrade | Yes | | Gateway delete | Yes | | Gateway resize | Yes | Operations outside this list are not in scope for the drain and undrain sequence in 9.0. ## Changes from Prior Releases | Behavior | Before 9.0 | 9.0 | | - | - | - | | CSP route tables updated to redirect traffic before the operation | Inconsistent | Always updated as part of the drain | | Traffic restored to the gateway before convergence | Possible | No; convergence is verified first | | Drain during gateway delete | No | Yes | | Drain during gateway resize | No | Yes | ## Error Handling If the HA peer is unavailable when the drain sequence starts, the operation fails. Bring the HA peer back to a healthy state before retrying. ## Limitations * Drain and undrain apply to HA gateway pairs only. Single gateways without an HA peer are not protected by this sequence. * Manual drain and undrain control is not available in this release. ## Related Topics * [Overview of Controller and Gateway Upgrades](/docs/enterprise/10.1/guides/maintenance/upgrade-overview) * [Gateway Image Upgrade](/docs/enterprise/10.1/guides/maintenance/gateway-image-upgrade) # Upgrading Your Gateway Image Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/maintenance/gateway-image-upgrade A gateway image is a virtual resource or template that contains all the information required to launch, backup, or restore a gateway in your cloud network. Aviatrix periodically releases new gateway images that include updates, enhancements, and security improvements. A gateway image is a virtual resource or template that contains all the information required to launch, backup, or restore a gateway in your cloud network. Aviatrix periodically releases new gateway images that include updates, enhancements, and security improvements. A best practice is to plan to upgrade your gateways at least once a quarter. ## Overview > [!NOTE] > > For major security issues or software issues, Aviatrix sends out a field notice > to notify you to upgrade to the newest image. Before upgrading, review any > applicable > [Field Notices](https://legacy.docs.aviatrix.com/documentation/latest/release-notices/field-notices/field-notices.html) > that may affect your gateway type. You may need to upgrade your gateway image outside of a periodic upgrade in the following situations: * Aviatrix has released a new gateway image as part of a security update or product enhancement. * A gateway requires significant repair. This document shows you how to upgrade an Aviatrix Gateway to a new image. > [!NOTE] > > A gateway image upgrade is also known as a gateway replacement. During the > Gateway image upgrade, the EIP will remain unchanged. ## Before Gateway Image Upgrade ### Prerequisites * Before upgrading any gateway images, [upgrade](/docs/enterprise/10.1/reference/ui/administration/maintenance) your Controller to the latest software version. This software upgrade ensures that you can update to the latest gateway image and reduce downtime for gateway image upgrades. * Check the [current software version](/docs/enterprise/10.1/guides/maintenance/controller-migration) of your Controller. You cannot upgrade your gateways to a newer version than your Controller. * Schedule the gateway image upgrade every quarter or if you receive a field notice about a new image. * Schedule the gateway image upgrade for an off-peak time on your network or during a maintenance window. These upgrades do require some downtime, but they have minimal impact. * Consider enabling HA (High Availability) on the Transit and Spoke Gateways that require an image upgrade, if you have not done so. HA helps minimize downtime. * If you do not have HA configured, a gateway image upgrade requires downtime. * If you have HA configured, when you perform a gateway image upgrade, your Controller routes all traffic to the gateway that is not being replaced. Performance during the upgrade depends on the size of the gateway and the amount of traffic. > [!TIP] > > Before upgrading, consider increasing the size of your gateway, if the traffic > load is high. > [!WARNING] > > Even with HA configured, if you have high traffic during a gateway image > upgrade, the gateway that remains up could receive too much traffic. Schedule > gateway image upgrades during a low-traffic period. ## Identify Image Upgrades by Gateway Type The process and best practices for upgrading a gateway image can differ based on the type of gateway. Review this list to decide how to structure and schedule your gateway image upgrades. | Gateway Type | Image Upgrade Notes | | - | - | | OpenVPN Gateway | When you have an Open VPN Gateway deployed behind a load balancer, you can upgrade images in batches without causing an outage, depending on the number of users you have. For OpenVPN Gateways that are not deployed with a load balancer, you should expect an outage. If any users are already on an OpenVPN gateway, they will be bumped when the gateway goes through an image upgrade. If you have more than one OpenVPN gateway, the end user can connect immediately to another gateway. | | FQDN Gateways with HA | The Controller does not reroute all traffic for these gateways. | | Public Subnet Filtering (PSF) Gateway | This gateway type does not have HA. | | Transit and Spoke Gateways | If your network has many Spoke Gateways, replacing the Transit primary or HA Gateways takes more time. Wait for one group of image upgrades to complete before beginning another. | | Site2Cloud Gateways | A best practice is to upgrade one gateway at a time. | | Edge Gateways | It is not recommended to upgrade your Edge Gateway unless a field notification has been sent to notify you to upgrade to a new image. It is not recommended to change Edge Gateway size. | ## Upgrading your Gateway Image If your Controller version is 7.1 or later, you can also upgrade your gateway image via the CoPilot UI. See details on [Upgrade your Gateway Image via CoPilot UI](/docs/enterprise/10.1/guides/maintenance/controller-gateway-upgrade-copilot) . For Controller version 7.0 and earlier, use Controller UI for the image upgrade. In your Controller, go to *Settings > Maintenance*. 1. In the *Selective Gateway Upgrade* window, select the gateways that require an upgrade. The system automatically selects the platform Controller current software version and the compatible gateway image version for that software version. > [!TIP] > > * Your Controller can replace up to 15 gateways in parallel. Try to group your > image upgrades in groups of no more than 15. > * For greater simplicity and efficiency, combine all your HA gateways and all > primary gateways in separate operations. > * To organize multiple image upgrades, consider spreading out groups of upgrades > in separate windows on your browser. 2. Click *IMAGE UPGRADE*. You can follow the status in the progress window. > [!NOTE] > > Replacing a gateway can take 5-7 minutes. After the gateway is up, it takes more > time for the tunnels to come up. The total length of time required varies > depending on the number of tunnels. For example, depending on the software > version of the Controller, it may take up to one hour to upgrade 4,000 tunnels. ## Upgrade Gateway Images in Batch To upgrade gateway images, you can use batch mode for ActiveMesh gateways and for some non-ActiveMesh gateways. Check if your gateway is ActiveMesh enabled using the instructions in [Check Whether a Gateway is ActiveMesh](https://legacy.docs.aviatrix.com/documentation/latest/platform-administration/controller/check-activemesh-gateway.html). ### For ActiveMesh Gateways 1. In your Controller, go to *Settings > Maintenance*. 2. In the *Selective Gateway Upgrade* window, select multiple gateways that require an upgrade. 3. Click *IMAGE UPGRADE*. 4. Click *OK* to confirm the upgrade. ### For Non-ActiveMesh Gateways Before upgrading a non-ActiveMesh gateway, it is important to check whether it has any peerings. #### Check Whether a Gateway has any Peerings Follow the steps below to check whether a gateway has any peerings: 1. In your Controller, go to *NATIVE PEERING > Encrypted*. 2. Scroll down to the display window and check if the gateway you want to upgrade is listed. 3. If the gateway is listed, it has at least one peering. #### Supported Batch Image Upgrade Scenarios for Non-ActiveMesh Gateway You can upgrade multiple non-ActiveMesh gateways' images in batch mode in these two scenarios: * Upgrade multiple non-ActiveMesh gateways without any peerings in a single batch process. * Upgrade multiple non-ActiveMesh gateways with at most one of the selected gateways having peering in a batch image-upgrade session. > [!NOTE] > > Only one image-upgrade session is allowed for non-ActiveMesh gateways. This > means that all desired gateways must be included in a single upgrade session. > However, as mentioned in the scenarios above, multiple non-ActiveMesh gateways > can be upgraded simultaneously as part of a single upgrade session. ## After Gateway Image Upgrade ### Verify the Gateway Image Upgrade Verify the gateway upgrade by reviewing the gateway information in the *Current Image Version* column in the *Selective Gateway Upgrade* window. For information about migrating your Controller to a new image, see [Migrating Your Aviatrix Controller](/docs/enterprise/10.1/guides/maintenance/controller-migration) . ### (Optional) Migrating Unmanaged Disk to Managed Disk (Azure) If you have a gateway deployed on Azure and you intend to migrate your unmanaged disk to a managed disk, it is recommended to follow the process of [gateway image upgrade](#upgrading-your-gateway-image). This gateway image upgrade will automatically migrate the unmanaged disk to a managed disk after the gateway image upgrade. > [!TIP] > > It is highly recommended to migrate your unmanaged disk to a managed disk as > soon as possible, as Azure will be > [retiring](https://learn.microsoft.com/en-us/azure/virtual-machines/unmanaged-disks-deprecation) > unmanaged disks soon. # Upgrade Gateways in Groups Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/maintenance/gateway-upgrade-groups The gateway group upgrade feature optimizes the gateway upgrade process by allowing you to categorize and manage gateway upgrades more efficiently. It enables the predefined grouping of gateways based on Cloud, Account, Region, Gateway type, Current image version, and Kernel version. The gateway group upgrade feature optimizes the gateway upgrade process by allowing you to categorize and manage gateway upgrades more efficiently. It enables the predefined grouping of gateways based on Cloud, Account, Region, Gateway type, Current image version, and Kernel version. By assigning a priority order of the gateway groups, the gateway group upgrade can be operated in a sequence determined by the priority levels, which can minimize the potential disruption during the gateway upgrade. From *CoPilot > Administration > Upgrade > Upgrade Plan*, you can access the gateway group upgrade feature. ## Subgroup Division Upon creating the gateway groups, each gateway upgrade group is automatically subdivided into two subgroups: * HA Group (High Availability Group): This subgroup includes gateway instances that are part of the High Availability setup. These instances are critical for ensuring uninterrupted network services during the upgrade process. * Primary Group: This subgroup consists of primary gateway instances, which handle the regular traffic and operations within the network. ## Upgrade Gateways in Groups This gateway Upgrade group feature applies to both the software and image upgrade. To upgrade gateways in groups, perform the following steps: 1. Create Gateway Groups. From *Administration > Upgrade > Upgrade Plan>*, click the *+ Upgrade Group* and enter the group name, matching conditions, and *group order number*. Then click *Save*. See [Create a Gateway Group](#create-a-gateway-upgrade-group) for more details. 2. Select the groups that you want to upgrade from the listed groups. 3. Check and adjust the orders of gateway upgrade groups. The upgrade runs in the sequence listed in the **Order** column. You can update the numeric order directly in the table, or reorder groups using the *Move* menu (Above, Below, Top, Bottom, or **By Order**). See [Reorder Gateway Groups](#reorder-gateway-upgrade-groups). 4. Click *Prepare For Upgrade* next to the *+Upgrade Group*. Review the listed gateway groups on the pop-up window. 5. Choose *Software Upgrade* or *Image Upgrade*. For the software upgrade, make sure the software upgrade option is toggled on. By default, both the dry run and software upgrade are on. Then click *Begin* to initialize the upgrade process. 6. (Optional) For multiple upgrade groups, click *Start Next Upgrade* to start the upgrade of the next group after each group upgrade completes. 7. Monitor the upgrade group status. From the *Upgrade Group Status* column, you can check the upgrade group status. See [Check Upgrade Group Status](#check-upgrade-group-status) for more details. ### Create a Gateway Upgrade Group To create a gateway upgrade group, perform the following steps: 1. From *CoPilot > Administration > Upgrade > Upgrade Plan>*, click *+ Upgrade Group* to create a gateway upgrade group. 2. In the Create Upgrade Group dialog, enter a group name. 3. Select matching conditions: Gateway type, cloud, region, account, gateway name, current image version, and kernel version. You can combine the conditions with *AND or OR*. You can enable the Preview toggle to view the gateways that match the selected criteria. > [!NOTE] > > For those unassigned gateways, they will remain in the *Unassigned* group and will be placed at the bottom of the gateway list. ### Reorder Gateway Upgrade Groups By default, the gateway groups are upgraded in an ascending sequence of the **Order** column. To change the upgrade order of a gateway group: 1. Select the checkbox next to the gateway group name. 2. Click the reorder icon (up-and-down arrows). 3. In the pop-up window, select one of the following: * *Above* or *Below* – place the group relative to another group. * *Top* or *Bottom* – move the group to the start or end of the list. * **By Order** – directly enter a new numeric order for the group. 4. Alternatively, edit the **Order** column value directly in the Upgrade Plan table. 5. Click *Save in Draft* to save the changes without applying them. 6. Click *Commit* to confirm and apply the changes. ## View Gateway Upgrade Process To monitor the progress of the gateway upgrade, from *Gateway Upgrade*, click *View Progress*. This will display a detailed view of the current gateway upgrade status, including the completed steps, ongoing tasks, and any pending actions. ### View Gateway Upgrade Groups After creating a Gateway Upgrade Group, you can click the table icon next to the gateway instance in the Upgrade Plan table to view the details of the Upgrade Group. ![Upgrade group detail view showing gateway instances and their status](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/maintenance/upgrade-group-view.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=7e8a80e90fcb608f327701c70c5a8ebc) If the Last Upgrade/Rollback Status shows as anything but Complete (green), you cannot upgrade the Gateway Group. ![Upgrade group view showing Last Upgrade/Rollback Status](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/maintenance/upgrade-group-view2.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=725d52bd5ac805a8a7b44d80eaa61198) ## Check Upgrade Group Status To check the upgrade group status: 1. Review the *Upgrade Group Status* column of each upgrade group. A label of *Completed* indicates that the upgrade of this upgrade group is finished. 2. To check the detailed upgrade information of each gateway within the group, click the group name in the *Software Upgrade* section. 3. In the pop-up window, scroll left to view the *Last Upgrade/Rollback Status*. ## Rollback Gateway Upgrade You cannot roll back the entire Gateway Group Upgrade. However, you can roll back the upgrade of a specific gateway by following the steps below: 1. Go to the *CoPilot > Cloud Fabric > Gateways > Gateway Management* page 2. Click *Gateway Rollback* to roll back a specific gateway upgrade. > [!TIP] > > Be aware that a gateway upgrade rollback will cause temporary service disruption. ## Upgrade all Gateway Groups If you want to upgrade all gateway groups in a listed sequence, perform the following steps: 1. From *CoPilot > Administration > Upgrade > Upgrade Plan>*, click *Prepare For Upgrade* under the *Gateways Requiring Upgrade*. 2. In the *Upgrade Gateways in All Upgrade Groups*, select either *Software Upgrade* or *Image Upgrade*. 3. (Optional) Make sure the *Software Upgrade* is toggled on for the Software Upgrade. By default, the dry run and software upgrade options are both on. 4. Click *Begin*. 5. Click *Start Next Upgrade* when each group upgrade has completed. 6. You can click *View Progress* to check the status of Upgrading Status. ## Cancel All Pending Gateway Upgrades You cannot cancel or stop a Gateway Group Upgrade in progress. However, you can cancel the pending gateway upgrades once the first gateway group upgrade completes. To cancel all the pending gateway upgrades: 1. Confirm the completion of the first gateway group upgrade. 2. Click the dropdown caret adjacent to *Start Next Upgrade* within the *Software Upgrade* section. 3. From the drop-down menu, click *Stop All Upcoming Upgrade* to cancel all pending Gateway Group Upgrades. # Upgrade Gateways for the Latest Aviatrix Supported Images (AWS and Azure Only) Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/maintenance/gateway-upgrade-latest-image Before upgrading your Controller and Gateways, it is highly recommended to check the Upgrade Overview. Before upgrading your Controller and Gateways, it is highly recommended to check the [Upgrade Overview](/docs/enterprise/10.1/guides/maintenance/upgrade-overview). * It is important to upgrade the Controller first, ensuring the Controller has been upgraded to the latest image. See [Upgrade Your Controller to the Latest Aviatrix Supported Images (AWS and Azure Only)](/docs/enterprise/10.1/guides/maintenance/controller-upgrade-latest-image) * Once the Controller upgrade is completed successfully, you can upgrade your Gateways to the latest image. For the major upgrade paths, see [Major Upgrade Paths.](/docs/enterprise/10.1/reference/ui/administration/maintenance#major-upgrade-paths) ## Perform an Image Upgrade of all your Gateways to the Latest Aviatrix Supported Images This section describes how to upgrade your gateways to the latest Aviatrix supported images. To perform a gateway image upgrade, follow these steps: 1. Go to *CoPilot UI > Cloud Fabric > Gateways > Gateway Management* to upgrade your gateways. Alternatively, you can go to *CoPilot UI > Administration > Upgrade* to use the upgrade plan and upgrade groups for bulk upgrade. See [Upgrade Gateways in Groups](/docs/enterprise/10.1/guides/maintenance/gateway-upgrade-groups). 2. From *Image Upgrade*, select the gateways that you want to upgrade. The system will automatically choose the compatible gateway image version based on the current software version of the platform Controller. 3. Click *Image Upgrade* and closely monitor the progress in the status window. The upgrade process may take a while to complete. 4. After the upgrade is finished, verify the gateway upgrade by reviewing the gateway information in the *Current Image Version* column. Ensure that the upgraded gateways display the expected image version. See [Verify Gateway Upgrade Status](#verify-gateway-upgrade-status) for details. > [!NOTE] > > After the gateway upgrade, several Network Metrics Interfaces and Connection > Status Interfaces have been renamed. Therefore, it is necessary to reconfigure > the alerts on any of these renamed Network Metrics Interfaces and Connection > Status Interfaces. See > [Renamed Network Metrics Interfaces and Connection Status Interfaces](/docs/enterprise/10.1/reference/release-notices/release-notes/copilot/index) > . ## Verify Gateway Upgrade Status After you have completed the upgrade, you can: 1. Go to *CoPilot UI > Gateways > Gateway Management > Upgrade Controller* to check the gateway upgrade status. Alternatively, you can go to *CoPilot UI > Administration > Upgrade* to use the upgrade plan and upgrade groups for bulk upgrade. Please note that the Upgrade Plan is a *Preview Feature*. See [Upgrade Gateways in Groups](/docs/enterprise/10.1/guides/maintenance/gateway-upgrade-groups). 2. Look for the *Update Status* field. * If the *Update Status* displays "complete", it indicates that the gateway upgrade has been successfully completed. Alternatively, you can check the current version on the *Controller Upgrade* card. * If the *Update Status* shows any other status, it means that your gateway upgrade has failed. 3. Go to *CoPilot > Cloud Fabric > Gateway Management* to check the status of gateways. 4. Verify all gateways are up and the status is green. 5. Verify all tunnels are up and the status is green. ## Roll Back the Gateway Software To roll back the gateway software to a previous version, see [Troubleshooting Controller and Gateway Upgrades](/docs/enterprise/10.1/guides/maintenance/controller-gateway-upgrade-troubleshooting). ## Troubleshooting your Controller and Gateway Upgrade To troubleshoot your Controller and Gateway upgrade, see [Troubleshooting Controller and Gateway Upgrades](/docs/enterprise/10.1/guides/maintenance/controller-gateway-upgrade-troubleshooting) for the detailed troubleshooting solutions. # Back Up and Restore Your Controller from the Minimum Controller UI Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/maintenance/min-controller-ui-maintenance Configure external storage, schedule automatic backups, and restore your Aviatrix Controller from the Minimum Controller UI Maintenance page. ## Overview The Minimum Controller UI is a lightweight web interface hosted directly on the Aviatrix Controller. It stays available even when CoPilot is unreachable, so you can perform bootstrap and lifecycle operations, including Controller backup and restore, without depending on a healthy CoPilot deployment. Use it for initial (Day-0) setup and for break-glass access when critical functions are not yet available in CoPilot. > [!NOTE] > > This page covers backup and restore from the **Minimum Controller UI**. If your > Controller has a healthy CoPilot deployment, use > > [Back Up and Restore Your Controller from CoPilot](/docs/enterprise/10.1/guides/maintenance/controller-backup-restore-copilot) > instead. ## Prerequisites * Network access to the Controller's IP address or FQDN on the port the Minimum Controller UI listens on. * Admin credentials for the Controller. * A cloud account (AWS, Azure, GCP, or OCI) already onboarded on the Controller for the external storage location you plan to use. ## Access the Maintenance Page 1. Log in to the Minimum Controller UI. 2. Go to **Maintenance**. 3. Expand the **Back Up and Restore** section. If external storage is not configured yet, an alert reads *You need to **Configure External Storage** before you can back up Controller data.* Click **Configure External Storage** in the alert, or use the **External Storage** card, to continue. ## Configure External Storage Configure the external storage location the Controller uses for both backup and restore operations. 1. On the **External Storage** card, click **Enable**. 2. In the **Configure External Storage** dialog, select the **Cloud** for the storage location. 3. Select the **Account** to use. > **For AWS** > > Enter the **External Storage (S3 Bucket)** name. > **For Azure** > > Select the **Region**, then enter the **Storage** account name and > **Container** name. > **For Oracle Cloud Infrastructure** > > Select the **Region**, then enter the **Bucket** name. 4. Click **Save**. The **External Storage** card now shows the configured **Account** and storage location. To change the configuration later, click **Edit Configuration** on the card. > [!WARNING] > > Removing the external storage association clears all backup schedules for this > Controller. In the **Configure External Storage** dialog, click **Remove > Association** to disassociate the storage location. You must configure a new > external storage location before you can back up or restore this Controller > again. ## Back Up Your Controller ### Enable Automated Backups 1. On the **Back Up Controller** card, click **Enable**. 2. In the **Enable Automated Backup Schedule for Controller** dialog, under **Frequency and Retention**, select **Repeat**: **Daily** or **Weekly**. 3. If you selected **Weekly**, select the day under **Every**. 4. Set the backup time under **At**. 5. Enter the number of backups to keep under **Backup Retained**. Older backups beyond this count are deleted automatically from external storage. 6. Click **Save**. The **Back Up Controller** card now shows the **Frequency**, **Backups Retained**, **Last Backup**, and **Next Scheduled Backup** for this Controller. To change the schedule, click the caret next to **Backup Now** and select **Edit Configuration**. To stop automatic backups, click **Cancel Schedule** in the same dialog. ### Back Up Now To trigger an immediate backup outside the automatic schedule, click **Backup Now** on the **Back Up Controller** card. While the backup runs, click **View Progress** on the card to see live backup status. > [!NOTE] > > If a scheduled or manual backup fails, a notification reads *The controller > backup failed.* Click **Task Details** in the notification to investigate, or > **Dismiss** to close it. ## Restore Your Controller > [!WARNING] > > Restoring a Controller overwrites its existing configuration. Confirm you > selected the correct backup file, and that any configuration changes made after > that backup was taken are accounted for elsewhere, before you restore. Restore is intended for a new, unconfigured Controller. Transfer the public IP address from the old Controller to the new Controller before you start the restore process. 1. On the **Restore Controller** card, click **Restore**. 2. In the **Restore Controller** dialog, select the **Location** (cloud) that holds your backup. > **For AWS** > > To connect with a cloud account already onboarded on this Controller, turn > on **Use Existing Account** and select the **Account**. Otherwise, enter > the **Access Key** and **Secret Key** for the account that owns the > backup. > > Enter the **External Storage (S3 Bucket)** name. > **For Microsoft Azure** > > Enter the **Subscription ID**, **Directory ID**, **Application Client > ID**, and **Application Client Secret**. > > Enter the **Storage** account name and **Container** name. > **For Google Cloud** > > Enter the **Bucket** name, then upload the **Gcloud Project Credentials** > file. > **For Oracle Cloud Infrastructure** > > To connect with a cloud account already onboarded on this Controller, turn > on **Use Existing Account** and select the **Account**. Otherwise, enter > the **Tenancy OCID**, **Compartment OCID**, **Application Client ID**, and > **Application Client Secret**, and upload the **API Private Key** file. > > Select the **Region**, then enter the **Bucket** name. 3. Enter the **File Name** of the backup to restore. You can find this file name in the external storage location that held backups for the old Controller. 4. (Optional) Select **Use the same external storage for future backups** to reuse this location as the external storage for this Controller going forward. 5. Click **Save** to start the restore. While the restore runs, click **View Progress** on the **Restore Controller** card to see live restore status. ### Restore Notes * Restore only to a new, unconfigured Controller. Restoring overwrites the target Controller's existing configuration. * Back up and restore must use the same cloud provider. For example, a backup taken on an AWS Controller can only be restored to another AWS Controller. * An AWS Controller configured with IAM roles cannot restore a backup taken from an AWS Controller configured with an access key and secret key, or the reverse. * A Controller allows only one successful restore. Attempting to restore a Controller that was already restored successfully returns an error and the operation does not proceed. ## Related Topics * [Back Up and Restore Your Controller from CoPilot](/docs/enterprise/10.1/guides/maintenance/controller-backup-restore-copilot) * [Back Up and Restore Your Controller from the Controller UI](/docs/enterprise/10.1/guides/maintenance/controller-backup-restore-controllerui) # Unattended Upgrades Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/maintenance/unattended-upgrade Aviatrix unattended upgrades automatically apply operating system security updates to Aviatrix Controllers and Gateways. This capability helps you maintain a strong security posture by ensuring that critical OS‑level vulnerabilities are patched without requiring manual intervention. Aviatrix unattended upgrades automatically apply operating system security updates to Aviatrix Controllers and Gateways. This capability helps you maintain a strong security posture by ensuring that critical OS‑level vulnerabilities are patched without requiring manual intervention. Unattended upgrades are designed to reduce exposure to known vulnerabilities while minimizing operational overhead. ## What Unattended Upgrades Do ### What Is Included Unattended upgrades apply only to the underlying operating system packages. This includes, but is not limited to: * Security updates for the Linux distribution * Core OS components (for example, OpenSSL and shell utilities) These updates are delivered using the standard Linux `unattended-upgrades` mechanism and focus on security‑related patches. ### What Is Not Included Unattended upgrades do **NOT** update Aviatrix software components, including but not limited to: * Controller application services * Gateway dataplane or control‑plane software * Aviatrix features, functionality, or configuration > [!NOTE] > > Aviatrix software upgrades must still be performed manually through the > Controller UI or CoPilot UI, following standard upgrade procedures. See > [Controller Gateway Upgrade Latest Image](/docs/enterprise/10.1/guides/maintenance/controller-gateway-upgrade-latest-image) > . ## Supported Versions Unattended upgrades are available starting with the following versions. Earlier versions do not include this capability. | Component | Minimum Version | Status | | - | - | - | | Controller | 8.0 | Supported | | Gateway | 8.1 | Supported | | CoPilot | 4.36 | Supported | ## How Unattended Upgrade Are Applied After Initial Deployment When a Controller or Gateway is first deployed, unattended upgrades do not take effect immediately. The upgrade process relies on two independent background processes, each running on its own 24‑hour schedule: 1. **Package Index Refresh**: The periodic `apt` package index refresh downloads the latest list of available security updates from Debian repositories. 2. **Security Upgrade Process**: The unattended security upgrade process installs any eligible patches identified during the most recent index refresh. > [!WARNING] > > Because these two timers are not synchronized, they may run out of phase. After a Controller or Gateway is first launched, it can take **up to 48 hours** for all eligible operating system security updates to be fully applied. > > This behavior is expected and does not indicate a failure or misconfiguration. > During this period, the system will progressively update itself as each > scheduled process runs. ## Network Requirements For unattended upgrades to function, the Controller and Gateways must have outbound (egress) connectivity on TCP port 443 to the following Debian official package repositories: * `deb.debian.org` * `security.debian.org` > [!WARNING] > > If outbound access is restricted (for example, by firewall rules or proxy > configuration), unattended upgrades may not be able to retrieve security > updates. Ensure your network policies allow egress traffic to the repositories > listed above. ## Reboot Behavior Unattended upgrades do **NOT** automatically reboot Controllers or Gateways. If an operating system security update includes a new kernel or otherwise requires a reboot, you are responsible for scheduling and performing the reboot in accordance with your maintenance and change‑management practices. > [!TIP] > > We recommend incorporating reboot checks into your regular maintenance windows > to ensure kernel‑level patches take full effect. ## Visibility and Status There is currently no UI indicator or status view showing when Unattended Upgrades are running or what updates have been applied. If you require confirmation of applied updates or need assistance, contact [Aviatrix Support](https://support.aviatrix.com). # Upgrade Your Aviatrix Controller Image Overview Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/maintenance/upgrade-controller-image-overview Aviatrix strongly recommends that you contact Aviatrix Support before upgrading your Controller image. > [!CAUTION] > > Aviatrix strongly recommends that you contact Aviatrix Support before > upgrading your Controller image. Upgrading the Aviatrix Controller image is an essential maintenance task to ensure you have the latest features, bug fixes, and security enhancements. ## When to Upgrade Your Aviatrix Controller Image You may need to upgrade your Aviatrix Controller image in the following situations: * If your Controller uses an old machine image, and you are trying to upgrade to a new software version that requires the latest image. * If you need to transition to a newer machine image for your Controller based on a recommendation from Aviatrix Support. * If you want to switch your Aviatrix license from any other licenses to the latest Aviatrix license. ## Upgrade Your Aviatrix Controller Image on UI Aviatrix provides a convenient upgrade mechanism through the CoPilot UI and manually. ### Upgrade on CoPilot UI (for CoPilot UI User) Starting from CoPilot version 3.14, you have the option to upgrade your Aviatrix Controller image directly from the CoPilot UI. This streamlined process allows you to update your Controller image without switching to the Controller UI. See [Upgrade your Controller Image via CoPilot UI (AWS)](/docs/enterprise/10.1/guides/maintenance/controller-migration) for more details. ## Manually Upgrade Your Aviatrix Controller Image If needed, you can also manually upgrade your Aviatrix Controller Image. Aviatrix strongly recommends that you contact Aviatrix Support before manually upgrading your Aviatrix Controller Image. # Overview of the Aviatrix Controller and Gateways Upgrade Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/maintenance/upgrade-overview Aviatrix encourages you to keep your platform Controller and gateways up to date to ensure you are operating the most secure and highest performing versions available. To facilitate less disruptive upgrades and reduce maintenance windows Aviatrix provides a rolling selective upgrade process. ## Overview Aviatrix encourages you to keep your platform Controller and gateways up to date to ensure you are operating the most secure and highest performing versions available. To facilitate less disruptive upgrades and reduce maintenance windows Aviatrix provides a rolling selective upgrade process. You can choose to upgrade all Aviatrix gateways in all regions simultaneously or select specific gateways and regions to upgrade in logical groups conforming to your network update policies and maintenance windows. > [!TIP] > > Always backup your system configuration before performing an upgrade. For more > information, see > [Back Up and Restore Your Controller on Controller UI](/docs/enterprise/10.1/guides/maintenance/controller-backup-restore#controller-ui) > . ## About Aviatrix Upgrade There are two types of upgrades for the Aviatrix Platform and gateways: * **Software Upgrade (Controller and Gateway)**: Update the Aviatrix software without creating new instances. This software upgrades replace the relevant Aviatrix Controller and gateway packages, configuration files, and binaries without disrupting network traffic or replacing the gateways. The software upgrade causes minimal disruption to network traffic. > [!WARNING] > > If you have DCF with WebGroups configured and applied to rules, traffic > disruption will occur on those gateways. * **Controller Image Upgrade**: Updates the base image of the controller, which is needed for operating system or kernel upgrades and may include an Aviatrix software upgrade. This results in minimal disruption to network traffic. * **Gateway Image Upgrade**: Updates the base image of the gateway, potentially causing brief traffic disruptions. To minimize impact, deploying gateways with high availability (HA) is recommended, and upgrades should be performed on one gateway at a time within the HA set. Gateway image upgrades are usually required after a Controller Image Upgrade to ensure that the Aviatrix Platform and gateways are operating with the same base software version. There are two types of patch updates: * **Security Patches** Security patches are released when security updates to underlying software components become available. Review the patch release notes for more details. * **Software Patches** Software patches are released to address compatibility issues when they arise. You should apply the patches to the Aviatrix system when they become available if you are using any applications or configurations affected by the patch. Review the patch release notes for more details. ## About Release Numbers Aviatrix release numbers follow the Major.Minor.Build format. For example, the release number 7.1.100 indicates: * 7 is the major release number. * 1 is the minor release number. * 100 is the build number. Each release type has different functionality parameters. * **Major** Includes new features and updates that affect the platform infrastructure and user interfaces. * **Minor** Includes modified and new small features and updates that may affect the platform infrastructure and user interfaces. * **Build** Corrected issues and feature enhancements. ## Upgrade Options When you initiate an upgrade, Aviatrix automatically presents the most recently published build for the selected major or minor release version. ### Upgrading Minor Releases of Controller and Gateways When upgrading from one minor version of a major release to another, it is necessary to follow a sequential upgrade process and cannot skip over intermediate minor release versions. Each minor release must be upgraded sequentially. For instance, when upgrading from 6.5.current to 6.8.latest, the intermediate releases 6.6.latest and 6.7.latest must be upgraded first. Valid upgrade paths to a new minor release are determined by the current build (the one currently running) and the latest build available on the Aviatrix server. ## Upgrading Major Releases of Controller and Gateways When upgrading from one major release to another, it is required to perform a sequential upgrade and not skip over intermediate major release versions. Each major release must be upgraded in sequence. You also need to go through all the minor releases within each major release before moving on to the next major release. For example, if you are currently on version 6.6.current and want to upgrade to version 8.0, you need to: 1. Upgrade from 6.6 to the latest minor release in the 6.x series (e.g., 6.6.current to 6.6.latest, then to 6.7.latest, then 6.7.latest to 6.8.latest, and so on) until you reach the latest minor release in the 6.x series. 2. Once you have upgraded through all the minor releases in the 6.x series, you can then move on to the major release 7.0. 3. From 7.0, continue upgrading through all the minor releases in the 7.x series until you reach the latest minor release in the 7.x series. 4. Upgrade from the latest minor release in the 7.x series to version 8.0. See [Upgrade your Controller and Gateways to the Latest Aviatrix Supported Images (AWS and Azure Only)](/docs/enterprise/10.1/guides/maintenance/controller-gateway-upgrade-latest-image) for more detailed examples. Valid upgrade path to a new major release is determined by the current build (the one currently running) and the latest build available on the Aviatrix server. ## Upgrade Parameter Definitions ### Platform Upgrade Window Parameter Definitions | Parameter | Description | | - | - | | Previous Version | Previous version of the Controller. | | Current Version | Current version of the Controller. | | Kernel Version | Version of the Controller's Linux kernel. | | Release Versions | The upgrade path between the currently running version of the Controller and the latest release available on the Aviatrix release server. For example, if you are running Aviatrix Platform 6.4.321 and the latest release available on the release server is 6.6.123, the Release Version field displays: UserConnect-6.6.123 (6.5,6.6). This indicates you must successively upgrade to 6.5 then upgrade to 6.6 to bring the platform up to the latest available version. | | Target Release Version | New version of the Aviatrix Platform to which you are upgrading. If you do not specify a release number, the system automatically selects the latest build of the major and minor release currently running on the platform Controller. The version cannot be a version earlier than the release currently running on the platform Controller. | ### Selective Gateway Upgrade Window Parameter Definitions | Parameter | Description | | - | - | | Current Version | Current software version running on the gateway. | | Previous Version | If the gateway has never been upgraded there is no version number. If the gateway has been upgraded at least once, this is the software version the gateway ran before the last upgrade. | | Target Version | Software version to which the gateway can be upgraded. It is the same version as the current version of the platform Controller. | | Previous Image Version | If the gateway OS has never been upgraded there is no version number. If the gateway OS has been upgraded at least once, this is the image version the gateway ran before the last upgrade. | | Current Image Version | Current version of the gateway underlying OS. | | Target Image Version | Every gateway software version matches a unique recommended OS version that may change over time. This version is determined by a compatibility matrix. This field displays the OS version that will be used in case of an OS upgrade. | | Kernel Version | Version of the gateway OS kernel. | | Rollback Version | Software version to which the gateway can be rolled back. It is the same version as the previous version of the platform Controller. | | Rollback Image Version | OS version that will be used in case of a gateway software rollback. Depending on the system compatibility matrix, this version can be higher, lower, or the same OS version currently running on the gateway. | | Account | Account attached to the gateway. | | Cloud | Cloud provider hosting the gateway. | | Region | Cloud region where the gateway is deployed. | | Gateway Type | Gateway persona: transit, spoke, or standalone. | | Gateway Role | Primary or secondary. | # Troubleshooting Controller and Gateway Upgrades Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/maintenance/controller-gateway-upgrade-troubleshooting This section provides some solutions for resolving some commonly encountered issues during the upgrade process for your Controller and Gateways. This section provides some solutions for resolving some commonly encountered issues during the upgrade process for your Controller and Gateways. > [!NOTE] > > If the Controller upgrade fails, your Controller is reverted to its previous > state. ## Troubleshooting Errors During the Controller and Gateway Upgrade Dry Run Check out the following solutions for the errors you encountered during the Controller and Gateway Upgrade Dry Run. ### "Check Backup" Error *Symptom:* During dry run, you encounter a Check Backup blocker issue. *Solution:* 1. Go to *CoPilot > Settings > Maintenance > Backup & Restore > External Storage*, make sure an external storage is configured. If not, configure a new external storage. 2. Go to *CoPilot > Settings > Maintenance > Backup & Restore > Back Up Controller*, click *Backup Now*. Then follow the backup wizard to back up the Controller to an available S3 bucket. ### "Check All Gateway Versions" Error *Symptom:* During the dry run, you encounter a "Check All Gateway Versions" block issue. *Solution:* Go to *CoPilot > Cloud Fabric > Gateways > Gateway Management*, select the required gateways. Then click *Actions > Upgrade* software to upgrade gateways to match the Controller's version. ### "Check All Gateway State" Error *Symptom:* During the dry run, you encounter a "check all gateway states" issue. *Solution:* Reboot gateways and make sure all gateway states are up and all gateways are all reachable from the Controller. ### "Invalid max\_duration value" Error *Symptom:* During the dry run, you encounter an "invalid max\_duration value" error. *Solution:* Enter a valid time limit between 40 to 1440 minutes. ### "Check Private Mode" Error *Symptom:* During the dry run, you encounter a "check private mode" issue. *Solution:* Private Mode has been removed as of Controller 10.1.0, so this setting cannot be toggled off from CoPilot. Contact [Aviatrix Support](https://support.aviatrix.com) for help resolving this issue. ### "Check Disk Usage" Error *Symptom:* During the dry run, you encounter a "check disk usage" issue. *Solution:* Reduce disk usage on Controller and make sure the disk usage is under at least 95%. See details in [Disk Usage](/docs/enterprise/10.1/guides/troubleshooting/troubleshoot-diagnostic-result#disk-usage). ### "Only members of the group 'admin' may modify Controller setting" Error *Symptom:* During the dry run, you encounter an "Only members of the group 'admin' may modify Controller setting" error. *Solution:* 1. Go to *CoPilot UI > Cloud Resources > Cloud Account*, click the account that you want to perform Controller upgrade. Then edit the permission. 2. Assign this account with admin permission. ### "Check Controller Migration" Error *Symptom:* During the dry run, you encounter a "Check Controller Migration" issue. *Solution:* Wait for the current migration to complete. Ensure that no other processes interfere with the migration, and monitor the system's status dashboard or relevant logs for any updates. ### "Controller Upgrade/Restore in Progress" Error *Symptoms:* During the dry run, you receive a "Controller Upgrade/Restore in Progress" Error. *Solution:* Wait for the current upgrade to complete. Ensure that no other processes interfere with the migration, and monitor the system's status dashboard or relevant logs for any updates. ### "The available memory should be no less than 536870912 bytes " Error *Symptom:* During the dry run, you encounter an "Available memory should be no less than 536870912 bytes" issue. *Solution:* This issue might be temporary, so try again. If it continues, try resizing your Controller or contact [Aviatrix Support](https://support.aviatrix.com) for assistance. ### "No identity-based policy allows the ec2:AllocateAddress Action" Error *Symptom:* During the dry run, you encounter a "No identity-based policy allows the ec2:AllocateAddress Action" Error. Or you receive a "Check AWS Allocate EIP" blocker issue. *Solution:* Modify the AWS account permission on EC2 role to allow EIP allocation. ### "Check AWS Associate EIP" Error *Symptom:* During the dry run, you encounter a "Check AWS Allocate EIP" issue. *Solution:* Modify the AWS account permission on EC2 role to allow EIP allocation. ### "Check AWS Disassociate EIP" Error *Symptom:* During the dry run, you encounter a "Check AWS Disassociate EIP" issue. *Solution:* Modify the AWS account permission on EC2 role to allow EIP disassociation. ### "Check AWS Add SG Ingress Rule" Error *Symptom:* During the dry run, you receive a "Check AWS Add SG Ingress Rule" blocker issue. *Solution:* Modify the AWS account permission to add an ingress rule to the EC2 role. ### "Check AWS Remove SG Ingress Rule" Error *Symptom:* During the dry run, you encounter a "Check AWS Remove SG Ingress Rule" issue. *Solution:* Modify the AWS account permission to remove an ingress rule from the EC2 role. ### "Check AWS Run Instance" Error *Symptom:* During the dry run, you encounter a "Check AWS Run Instance" issue. *Solution:* Modify the AWS account permission in the EC2 role to give this account proper permission to run EC2 instances. ### "Check AWS Stop Instance" Error *Symptom:* During the dry run, you encounter a "Check AWS Stop Instance" issue. *Solution:* Modify the AWS account permission in the EC2 role to give this account proper permission to stop EC2 instances. ### "Check AWS Terminate Instance" Error *Symptom:* During the dry run, you receive a "Check AWS Terminate Instance" issue. *Solution:* Modify the AWS account permission in the EC2 role to give this account proper permission to terminate EC2 instances. ### "Check AWS Disable Stop Protection" Error *Symptom:* During the dry run, you encounter a "Check AWS Disable Stop Protection" blocker issue. *Solution:* Modify the AWS account permission in the EC2 role to enable stop protection on EC2 instances. ### "Check AWS Describe Volume" Error *Symptom:* During the dry run, you encounter a "Check AWS Describe Volume" blocker issue. *Solution:* Modify the AWS account permission to allow this account to describe Controller volume. ### "Check AWS Quota EIP" Error *Symptom:* During the dry run, you encounter a "Check AWS Quota EIP" issue. *Solution:* In an AWS account, you need to increase the quota for Elastic IP (EIP) addresses. ### "Check Controller HA" Error *Symptom:* During the dry run, you encounter a "Check Controller HA" issue. *Solution:* 1. Delete the CloudFormation Stack before the Controller Upgrade. 2. Redeploy the Cloud Formation after the Controller Upgrade completes. See [Controller High Availability in AWS](/docs/enterprise/10.1/guides/controlplane/controller-ha-aws) for details. ### "Check AWS Describe Instance" Error *Symptom:* During the dry run, you encounter a "Check AWS Describe Instance" issue. *Solution:* Modify the account permissions to allow the account to describe Controller instance. ### "Check AWS Controller EIP" Error *Symptom:* During the dry run, you encounter a "Check AWS Controller EIP" issue. *Solution:* 1. Go to *CoPilot UI > Settings > Configuration > General*, check the stored EIP, which is associated with Aviatrix Controller. 2. Modify the EIP in AWS to match the EIP stored in Controller. ### "Check Universal License Type" Error *Symptom:* During the dry run, you encounter a "Check Universal License Type" issue. *Solution:* Subscribe to a universal license before you perform Controller upgrade. ### "Check Registered CloudN Instance" Error *Symptom:* During the dry run, you receive a "Check Registered CloudN Instance" issue. *Solution:* Contact [Aviatrix Support](https://support.aviatrix.com) for the details about upgrading Controller with CloudN. ### "Dry Run of upgrade to software userconnect version latest failed" *Symptom:* During the dry run of the upgrade, you encounter the error message: "Dry Run of upgrade to software UserConnect version latest failed" with the error code "AVXERR-UPGRADE-CTRL-0099". *Solution:* From Controller UI, go to *Troubleshoot > Diagnostics > mTLS > Spire Nodes*. Choose *controller*, then click *RE-ATTEST*. ### "Check Azure Quota Compute Blocking" Error *Symptom:* During the dry run, you encounter a "Check Azure Quota Compute Blocking" issue. *Solution:* In an Azure account, release the unused resources or increase the resource quota limit. ### "Check Azure Export Resource Group" Error *Symptom:* During the dry run, you encounter a "Check Azure Export Resource Group" issue. *Solution:* Update your Azure account permission to allow export of resource group. ### "UPGRADE\_IMAGE threads.tasks execute" Error *Symptom:* During the upgrade dry run, you encounter a "UPGRADE\_IMAGE threads.tasks execute" issue. *Solution:* * Check the IAM policy to ensure that the correct IAM policy is applied. * Verify the resource availability to make sure that resources are available in the specified region for deployment. ### "Check Azure Get VM" Error *Symptom:* During the dry run, you encounter a "Check Azure Get VM" issue. *Solution:* Update your Azure account permission to allow the account to get Controller VM details. ### "Azure Controller Public IP" Error *Symptom:* During the dry run, you encounter an "Azure Controller Public IP" issue. *Solution:* Update your Azure Controller EIP to match it with the stored IPv4 address. Alternatively, you can also contact [Aviatrix Support](https://support.aviatrix.com) to modify the stored IPv4 address in Controller. ## Troubleshooting Errors During Controller and Gateway Upgrade Check out the following solutions for the errors you encountered during the Controller and Gateway Upgrade. ### "Customer ID is not set up, enter one." Error *Symptom:* When you are prompted to log back into CoPilot after upgrade, a "Customer ID is not set up, enter one." error occurs. *Solution:* Wait for some time to allow the upgrade to complete on the backend. Then enter your username and password to login. ### "EIP Switch Azure Failure" Error *Symptom:* During upgrade, you see an "EIP move failed" error. *Solution:* Change your Azure EIP from `basic` to `standard` EIP. ### "Migration Timeout - x gateways did not register to new Controller" Error *Symptom:* During upgrade, you see a "Migration Timeout - x gateways did not register to new Controller" error. *Solution:* Contact [Aviatrix Support](https://support.aviatrix.com) to register all your gateways to your new Controller. # Verify Your Upgrade Status Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/maintenance/controller-gateway-upgrade-verify After performing an upgrade, it is important to verify the upgrade status to ensure that it has been completed successfully. After performing an upgrade, it is important to verify the upgrade status to ensure that it has been completed successfully. ## Verify Controller Upgrade Status 1. Go to your Controller upgrade window from *Controller UI > Settings > Maintenance > Upgrade*. Alternatively, you can go to **CoPilot > Administration > Maintenance > Controller Upgrade**. 2. Check if the upgrade window displays a message indicating that the Controller upgrade has been completed successfully. 3. Ensure that the displayed Controller version is updated to the latest version. If the above conditions are met, it means that your Controller upgrade has been successfully completed. ## Verify Gateway Upgrade Status After you have completed upgrade, you can: 1. Go to *CoPilot UI > Gateways > Gateway Management > Upgrade Controller* to check the gateway upgrade status. Alternatively, you can go to *CoPilot UI > Administration > Upgrade* to use the upgrade plan and upgrade groups for bulk upgrade. Please note that the Upgrade Plan is a [Preview Feature](/docs/enterprise/10.1/reference/release-notices/release-notes/controller/index). See [Upgrade Gateways in Groups](/docs/enterprise/10.1/guides/maintenance/gateway-upgrade-groups). 2. Look for the *Update Status* field. * If the *Update Status* displays "complete", it indicates that the gateway upgrade has been successfully completed. Alternatively, you can check the current version on the *Controller Upgrade* card. * If the *Update Status* shows any other status, it means that your gateway upgrade has failed. 3. Go to *CoPilot > Cloud Fabric > Gateway Management* to check the status of gateways. * Verify all gateways are up and the status is green. * Verify all tunnels are up and the status is green. # Controller Migration Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/maintenance/controller-migration Aviatrix strongly recommends that you contact Aviatrix Support before migrating your Controller. > [!CAUTION] > > Aviatrix strongly recommends that you contact Aviatrix Support before > migrating your Controller. ## Introduction The Aviatrix multicloud networking platform is delivered via two images, a Controller image and a gateway image. Both should be maintained with the latest version for managing security and support for the product. Aviatrix intends to publish 2 new images per year. You may need to migrate your Aviatrix Controller in the following situations: * If your Controller uses an old machine image, and you are trying to upgrade to a new software version that requires the latest image. * If you need to transition to a newer machine image for your Controller based on a recommendation from Aviatrix Support. > [!NOTE] > > * If you launched an existing AWS, Azure, or GCP Controller with a BYOL license and have it updated to software version 6.7.1185 or later, you only need to subscribe to the Aviatrix Secure Networking Platform Metered 2208-Universal 24x7 Support through your Controller. Log in as an admin and go to Settings > Controller > License to subscribe. You do not need to follow the migration steps in this document. > * A machine image is named an "AMI (Amazon Machine Image)" in AWS, a "VM image" (Virtual Machine Image) in Azure, a "machine image" in GCP, and a "custom image" in OCI. The Controller Migration process involves the following steps: * Fulfill the prerequisites, including backing up your old Controller. * Delete your old Controller in the CSP Account from which you originally launched it: AWS, Azure, GCP, or OCI. * Launch a new Controller from the relevant CSP marketplace. * Restore the data from your old Controller to your new Controller. This document explains how to migrate from one type of machine image to another for each CSP, including: * [Prerequisites for all clouds](#prerequisites-for-all-clouds) * [Prerequisites for CoPilot users](#prerequisites-for-copilot-users) * [Controller Migration in AWS](#controller-migration-in-aws) * [Controller Migration in Azure](#controller-migration-in-azure) * [Controller Migration in GCP](#controller-migration-in-gcp) * [Controller Migration in OCI](#controller-migration-in-oci) * [Setting up your new Controller](#setting-up-your-new-controller) * [Controller Migration during Disaster Recovery](#controller-migration-during-disaster-recovery) > [!TIP] > > In a Disaster Recovery (DR) situation in which you cannot access the old > Controller, please see the [Controller Migration During Disaster > Recovery](#controller-migration-during-disaster-recovery) section below. ## Prerequisites for all Clouds * Make sure you have the relevant CSP storage container (an AWS S3 bucket, Azure Blob Storage, Google Cloud Storage, or OCI Object storage service) linked with your Aviatrix Controller. This storage container enables you to back up your data. > [!TIP] > > As a security best practice, enable versioning on the destination storage > container to preserve, retrieve, and restore every version of every object > stored. * (For AWS, Azure, and GCP Controllers) Get your Customer ID either through Controller > Settings > Controller > License or from your Aviatrix Account Manager. If you are migrating a metered image, subscribe to the Aviatrix Secure Networking Platform Metered 2208-Universal 24x7 Support offer to receive your Customer ID by email. > [!NOTE] > > To ensure that you receive the license email, please whitelist these email addresses: > > * **[appalerts@aviatrix.com](mailto:appalerts@aviatrix.com)** > * **[admin@aviatrix.io](mailto:admin@aviatrix.io)** * Run an audit on the Controller's primary access account and all the remaining secondary accounts to make sure that the IAM roles and policies are set up as suggested. In your Controller, go to Accounts > Access Accounts > select an account > click **Audit**. Repeat these steps for all access accounts for the CSP where your Controller instance is located (AWS, Azure, GCP, or OCI). * Enable a [Controller backup](/docs/enterprise/10.1/guides/maintenance/controller-backup-restore) using the access account for the CSP from which you launched the Controller (Azure, GCP, or OCI). > [!NOTE] > > In case of a Disaster Recovery (DR) scenario, considering keeping current > backups in separate regions of each CSP. * Back up your existing Controller. In your Controller, go to Settings > Maintenance > Backup & Restore > Backup. Click **Backup now**. > [!TIP] > > As a security best practice: > > * When the backup completes, verify that it is the correct size. > * Make a local copy of the backup file (object). * Schedule the migration during a maintenance window. * Check the current software version of your [gateways](/docs/enterprise/10.1/guides/maintenance/gateway-image-migration#upgrade-gateway-image). You cannot upgrade your Controller unless all gateways are on the same version as the Controller. * Upgrade to the latest build of your current release. Note that Aviatrix software version upgrade is a version-to-version upgrade. Therefore, you may need to perform multiple upgrades until you reach the latest version on your existing Controller. * [Disable](/docs/enterprise/10.1/guides/controlplane/controller-ha-aws) your Controller's HA configuration if HA is set up. You can re-enable HA on the new Controller once migration is complete. > [!WARNING] > > Deleting your HA configuration is crucial. If you do not delete your HA > configuration before migration, the backup configuration may become corrupted, > and the migration may fail. * If you are using SAML login for either the Controller login (Settings/Controller/SAMLLogin) and/or for openvpn authentication (OpenVPN/Advanced/SAML), please make sure that the endpoints configured on the Controller and the SAML applications in the IdP match exactly. > [!NOTE] > > Do not add any new configurations to your old Controller before migrating, as > those updates may be lost in the migration process. ## Prerequisites for CoPilot Users > [!WARNING] > > **(Aviatrix CoPilot users)** When you migrate your Controller, your Controller IP address will change. Depending on which version of controller you are migrating from, perform the following tasks before you start the migration. This is to avoid being unable to log in to your CoPilot after the migration: > > If you are migrating a Controller earlier than version 6.8.1088 or earlier than > version 6.9.161: > > * In Copilot Home > Settings > Configuration, click **Reset Controller IP**. > This will bring you to the CoPilot login page where you will enter your new > Controller's IP address once it's available after the Controller migration. If > your new Controller was deployed from the Controller UI, the IP address will > be the private IP address of the new Controller. > > If you are migrating a Controller version 6.8.1088 or later or 6.9.161 or later: > > * In the CSP environment of your CoPilot, confirm that your old Controller's IP > address (the Controller you are migrating from) is set in your CoPilot's > security group inbound rule for port 443. ## Controller Migration in AWS For Controllers launched from AWS, there are two methods you can use to migrate your Controller AMI (Amazon Machine Image): 1. **Controller-driven method** (simplest method) - You can easily migrate your existing (BYOL, Metered, or another license) AMI directly from the Controller to a BYOL AMI. The BYOL AMI requires a valid customer ID. 2. **Manual method** (longer method which offers more visibility) - You can migrate your Controller by going through the AWS marketplace and stopping your Controller instance, disassociating your EIP, and so on. This method requires more steps but allows you to see the network and account changes involved in each step. You can also use this method to change licenses if needed. > [!NOTE] > > The Controller-driven method can only migrate between AMIs using the same > license. To change your license, use the manual method. * To migrate to a BYOL license model, please contact your Aviatrix Sales Account Manager or email [sales@aviatrix.com](mailto:sales@aviatrix.com). * To migrate to a Metered license model, subscribe to the Aviatrix Secure Networking Platform Metered 2208-Universal 24x7 Support offer on the AWS Marketplace. ### Controller-Driven Method (AWS) Before starting, make sure you have completed the [prerequisites for all clouds](#prerequisites-for-all-clouds). 1. In your Controller, go to Settings > Maintenance > Migration. 2. Click **Migrate**. 3. The Controller will back up the current configuration, launch a new Controller instance with the latest AMI, and restore the configuration on the new Controller. 4. Once the migration is complete, the new Controller will be accessible at the same IP address as the old Controller. > [!NOTE] > > The Controller-driven migration method retains the same EIP (Elastic IP) > address as the old Controller. ### Manual Method (AWS) Before starting, make sure you have completed the [prerequisites for all clouds](#prerequisites-for-all-clouds). 1. Record the settings of your existing Controller instance, including: * VPC ID * Subnet * Security Group(s) * IAM role * EIP (Elastic IP address) * Private IP address * Instance type 2. Go to your AWS Console > EC2 > Instances. Select the old Controller instance. 3. Disassociate the EIP from your old Controller instance: * Go to EC2 > Elastic IPs. * Select the EIP associated with the old Controller. * Click Actions > Disassociate Elastic IP address. * Confirm the disassociation. 4. Stop or terminate the old Controller instance: * Go to EC2 > Instances. * Select the old Controller instance. * Click Instance State > Stop instance (or Terminate instance). > [!WARNING] > > Do not terminate the old Controller instance until you have verified the new > Controller is working correctly. 5. Subscribe to the new AMI on the AWS Marketplace: * Go to the AWS Marketplace and subscribe to the appropriate Aviatrix offer. * If you are migrating to BYOL, subscribe to the Aviatrix Secure Networking Platform BYOL offer. * If you are migrating to Metered, subscribe to the Aviatrix Secure Networking Platform Metered 2208-Universal 24x7 Support offer. 6. Launch a new Controller instance from the new AMI: * Use the same VPC, subnet, and security group settings as the old Controller. * Use the same instance type (or a larger one if needed). * Assign the same IAM role. 7. Associate the EIP with the new Controller instance: * Go to EC2 > Elastic IPs. * Select the EIP that was previously associated with the old Controller. * Click Actions > Associate Elastic IP address. * Select the new Controller instance and associate the EIP. 8. Proceed to [Setting up your new Controller](#setting-up-your-new-controller). ## Controller Migration in Azure Before starting, make sure you have completed the [prerequisites for all clouds](#prerequisites-for-all-clouds). 1. Record the settings of your existing Controller VM, including: * Resource Group * Virtual Network and Subnet * Network Security Group (NSG) * Public IP address * Private IP address * VM size 2. Disassociate the public IP address from your old Controller: * Go to the Azure Portal > your Controller VM > Networking. * Click on the network interface. * Go to IP configurations and disassociate the public IP. 3. Stop or delete the old Controller VM: * Go to the Azure Portal > Virtual Machines. * Select the old Controller VM. * Click Stop (or Delete). > [!WARNING] > > Do not delete the old Controller VM until you have verified the new Controller > is working correctly. 4. Launch a new Controller from the Azure Marketplace: * Go to the Azure Marketplace and search for the appropriate Aviatrix offer. * Deploy a new Controller VM using the same Resource Group, Virtual Network, Subnet, NSG, and VM size as the old Controller. 5. Associate the public IP address with the new Controller VM: * Go to the Azure Portal > your new Controller VM > Networking. * Click on the network interface. * Go to IP configurations and associate the public IP that was previously used by the old Controller. 6. Proceed to [Setting up your new Controller](#setting-up-your-new-controller). ## Controller Migration in GCP Before starting, make sure you have completed the [prerequisites for all clouds](#prerequisites-for-all-clouds). 1. Record the settings of your existing Controller instance, including: * Project * VPC network and Subnet * Firewall rules * External (static) IP address * Internal IP address * Machine type * Zone 2. Disassociate the external IP from your old Controller instance: * Go to the GCP Console > VPC network > External IP addresses. * Change the assignment of the static external IP from the old Controller instance to None. 3. Stop or delete the old Controller instance: * Go to the GCP Console > Compute Engine > VM instances. * Select the old Controller instance. * Click Stop (or Delete). > [!WARNING] > > Do not delete the old Controller instance until you have verified the new > Controller is working correctly. 4. Launch a new Controller from the GCP Marketplace: * Go to the GCP Marketplace and search for the appropriate Aviatrix offer. * Deploy a new Controller instance using the same Project, VPC network, Subnet, Firewall rules, Machine type, and Zone as the old Controller. 5. Associate the external IP with the new Controller instance: * Go to the GCP Console > VPC network > External IP addresses. * Change the assignment of the static external IP to the new Controller instance. 6. Proceed to [Setting up your new Controller](#setting-up-your-new-controller). ## Controller Migration in OCI Before starting, make sure you have completed the [prerequisites for all clouds](#prerequisites-for-all-clouds). 1. Record the settings of your existing Controller instance, including: * Compartment * VCN (Virtual Cloud Network) and Subnet * Security List / Network Security Group * Public IP address (Reserved) * Private IP address * Shape (instance type) 2. Disassociate the reserved public IP from your old Controller instance: * Go to the OCI Console > Networking > IP Management > Reserved Public IPs. * Select the reserved public IP associated with the old Controller. * Click on it and remove the assignment. 3. Stop or terminate the old Controller instance: * Go to the OCI Console > Compute > Instances. * Select the old Controller instance. * Click Stop (or Terminate). > [!WARNING] > > Do not terminate the old Controller instance until you have verified the new > Controller is working correctly. 4. Launch a new Controller: * Deploy a new Controller instance using the same Compartment, VCN, Subnet, Security List/NSG, and Shape as the old Controller. 5. Associate the reserved public IP with the new Controller instance: * Go to the OCI Console > Compute > Instances > select the new Controller instance. * Under Resources, go to Attached VNICs > select the primary VNIC. * Edit the IP configuration to assign the reserved public IP. 6. Proceed to [Setting up your new Controller](#setting-up-your-new-controller). ## Setting up your New Controller After launching your new Controller from the CSP marketplace, follow these steps to complete the migration: 1. Open a browser and navigate to the new Controller's IP address (`https://`). 2. Follow the initial setup wizard: * Set the admin password. * Set the admin email address. * Optionally set a proxy configuration if required. 3. When prompted, enter your Customer ID. 4. Run the initial setup and wait for the Controller to initialize. > [!NOTE] > > Do not upgrade the software version on the new Controller before restoring the > backup. The restore process requires that the new Controller is on the same > software version as the old Controller at the time of backup. 5. Restore the backup from your old Controller: * Go to Settings > Maintenance > Backup & Restore > Restore. * Select the CSP storage where your backup is stored (S3, Azure Blob, GCS, or OCI Object Storage). * Enter the required details (bucket name, file name, access account, etc.). * Click **Restore**. 6. Wait for the restore to complete. The Controller will reboot automatically after the restore. 7. After the Controller reboots, log in with your previous admin credentials (from the old Controller). 8. Verify that the configuration has been restored correctly: * Check that all access accounts are present under Accounts > Access Accounts. * Run an audit on all access accounts. * Verify that all gateways are up and running. * Check that all tunnels and peerings are in the correct state. 9. If you previously disabled Controller HA, re-enable it on the new Controller: * Go to Settings > Controller > Redundancy and re-enable Controller HA. 10. Update any DNS records that point to the old Controller IP address if the IP address has changed. 11. **(For CoPilot users)** If your Controller IP address changed: * If you are on a Controller version earlier than 6.8.1088 or earlier than 6.9.161: Log in to CoPilot and enter the new Controller IP address when prompted. * If you are on Controller version 6.8.1088 or later or 6.9.161 or later: Update the CoPilot security group to allow inbound 443 from the new Controller's IP address. 12. If you are using SAML authentication, update the SAML application in your IdP with the new Controller IP address or FQDN if it has changed. 13. Once you have verified that the new Controller is fully operational, you can terminate the old Controller instance in your CSP console. ## Controller Migration during Disaster Recovery In a Disaster Recovery (DR) scenario, you may not be able to access your old Controller. In this case, follow these steps: > [!NOTE] > > Restoring a Controller to a different region than the old Controller > (cross-region restore) is officially validated and supported for AWS only. > Cross-region restore is not supported for Azure, GCP, or OCI. If your > Controller runs on Azure, GCP, or OCI, contact Aviatrix Support before > attempting to restore to a different region. 1. Verify that you have a recent backup of your Controller configuration in your CSP storage (S3, Azure Blob, GCS, or OCI Object Storage). 2. Launch a new Controller from the AWS marketplace in the desired region. Use the same CSP account that was used for the old Controller's primary access account. 3. Follow the steps in [Setting up your new Controller](#setting-up-your-new-controller) to initialize the new Controller and restore the backup. > [!NOTE] > > If you are launching the new Controller in a different region, make sure the > backup file is accessible from that region. You may need to copy the backup > file to a storage container in the new region. 4. After restoring the backup, verify that all gateways are reachable from the new Controller. If the gateways were in a different region from the old Controller, they should still be accessible as long as the network connectivity is in place. 5. Update any DNS records, firewall rules, or security group rules to reflect the new Controller's IP address. > [!WARNING] > > Gateway security group rules that reference the old Controller's IP address > are not automatically updated after a cross-region restore. You must manually > update these rules to allow traffic from the new Controller's IP address, or > the gateways may lose connectivity to the Controller. 6. Re-enable Controller HA if it was previously configured. > [!WARNING] > > In a DR scenario, some data or configuration changes made after the last > backup may be lost. Ensure that you maintain frequent and regular backups to > minimize data loss. # Installing Software and Security Patches Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/maintenance/copilot-installing-patches The CoPilot > Settings > Maintenance > Upgrade tab > Install Patches section in Aviatrix CoPilot lists all available security and software patches and indicates whether or not they have been installed. The **CoPilot > Settings > Maintenance > Upgrade** tab > Install Patches section in Aviatrix CoPilot lists all available security and software patches and indicates whether or not they have been installed. When you [upgrade to the latest release](/docs/enterprise/10.1/guides/maintenance/upgrade-overview), install any patches that are not currently installed. ## Applying a Security Patch To apply a Security Patch that was not previously installed: 1. [Back up](/docs/enterprise/10.1/guides/maintenance/controller-backup-restore) your Aviatrix CoPilot. 2. Apply the security patch. Go to your **Aviatrix CoPilot > Settings > Maintenance > Upgrade** tab, and on the Install Patches card, click **Install**. 3. In the Install Patches window, the Patches Not Installed tab opens by default and lists uninstalled patches. Mark the checkbox next to each patch you wish to install. The window displays relevant information or warnings about each Security Patch. 4. Click **Install** to install the Security Patches. 5. The window closes and a confirmation message appears. 6. To see the status of the patch installation, click on the Tasks icon in the top right. 7. [Back up](/docs/enterprise/10.1/guides/maintenance/controller-backup-restore) your Aviatrix CoPilot again to save the new configuration. ## Applying a Software Patch To apply a Software Patch that was not previously installed: 1. [Back up](/docs/enterprise/10.1/guides/maintenance/controller-backup-restore) your Aviatrix CoPilot. 2. Apply the software patch. Go to your **Aviatrix CoPilot > Maintenance > Upgrade tab > Install Patches** section and click Install. 3. In the Install Patches window, select the Software Patches tab. Mark the checkbox next to each patch you wish to install. The window displays relevant information or warnings about each software patch. 4. Click **Install** to install the Security Patches. 5. The window closes and a confirmation message appears. 6. To see the status of the Software Patch installation, click on the Tasks icon in the top right. 7. [Back up](/docs/enterprise/10.1/guides/maintenance/controller-backup-restore) your Aviatrix CoPilot again to save the new configuration. # Upgrading Gateway Image Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/maintenance/gateway-image-migration A gateway image is a virtual resource or template that contains all the information required to launch, backup, or restore a gateway in your cloud network. Aviatrix periodically releases new gateway images that include updates, enhancements, and security improvements. A gateway image is a virtual resource or template that contains all the information required to launch, backup, or restore a gateway in your cloud network. Aviatrix periodically releases new gateway images that include updates, enhancements, and security improvements. A best practice is to plan to upgrade your gateways at least once a quarter. > [!TIP] > > For major security issues or software issues, Aviatrix sends out a field > notice to notify you to upgrade to the newest image. You can review past Field > Notices and > [Aviatrix Controller and Gateway Image Release Notes](/docs/enterprise/10.1/reference/release-notices/release-notes/controller-images/index) > . You may need to upgrade your gateway image outside of a periodic upgrade in the following situations: * Aviatrix has released a new gateway image as part of a security update or product enhancement. * A gateway requires significant repair. This document shows you how to upgrade an Aviatrix Gateway to a new image. > [!NOTE] > > A gateway image upgrade is also known as a gateway replacement. During the > Gateway image upgrade, the EIP will remain unchanged. ## Prerequisites * Check the [current software version](/docs/enterprise/10.1/guides/maintenance/controller-migration) of your Controller. You cannot upgrade your gateways to a newer version than your Controller. * Schedule the gateway image upgrade every quarter or if you receive a field notice about a new image. * Schedule the gateway image upgrade for an off-peak time on your network or during a maintenance window. These upgrades do require some downtime, but they have minimal impact. * Consider enabling HA (High Availability) on the Transit and Spoke Gateways that require an image upgrade, if you have not done so. HA helps minimize downtime. * If you do not have HA configured, a gateway image upgrade requires downtime. * If you have HA configured, when you perform a gateway image upgrade, your Controller routes all traffic to the gateway that is not being replaced. Performance during the upgrade depends on the size of the gateway and the amount of traffic. > [!TIP] > > Before upgrading, consider increasing the size of your gateway, if the > traffic load is high. > [!WARNING] > > Even with HA configured, if you have high traffic during a gateway image > upgrade, the gateway that remains up could receive too much traffic. > Schedule gateway image upgrades during a low-traffic period. * Before upgrading any gateway images, [upgrade](/docs/enterprise/10.1/guides/maintenance/controller-gateway-upgrade-steps) your Controller to the latest software version. This software upgrade ensures that you can update to the latest gateway image and reduce downtime for gateway image upgrades. ## Image Upgrades by Gateway Type The process and best practices for upgrading a gateway image can differ based on the type of gateway. Review this list to decide how to structure and schedule your gateway image upgrades. | Gateway Type | Image Upgrade Notes | | - | - | | OpenVPN Gateway | When you have an Open VPN Gateway deployed behind a load balancer, you can upgrade images in batches without causing an outage, depending on the number of users you have. For OpenVPN Gateways that are not deployed with a load balancer, you should expect an outage. If any users are already on an OpenVPN gateway, they will be bumped when the gateway goes through an image upgrade. If you have more than one OpenVPN gateway, the end user can connect immediately to another gateway. | | FQDN Gateways with HA (gateways on the Specialty Gateways tab that have 'fqdn' in the name) | The Controller does not reroute all traffic for these gateways. | | Public Subnet Filtering (PSF) Gateway | This gateway type does not have HA. | | Transit and Spoke Gateways | If your network has many Spoke Gateways, replacing the Transit primary or HA Gateways takes more time. Wait for one group of image upgrades to complete before beginning another. | | Site2Cloud Gateways | A best practice is to upgrade one gateway at a time. | | Edge Gateways | It is not recommended to upgrade your Edge Gateway unless a field notification has been sent to notify you to upgrade to a new image. It is not recommended to change Edge Gateway size. | ## Upgrade Gateway Image 1. In your Controller, go to *Settings* > *Maintenance*. 2. In the *Selective Gateway Upgrade* window, select the gateways that require an upgrade. The system automatically selects the platform controller current software version and the compatible gateway image version for that software version. > [!TIP] > > * Your Controller can replace up to 15 gateways in parallel. Try to group > your image upgrades in groups of no more than 15. \* For greater simplicity > and efficiency, combine all your HA gateways and all primary gateways in > separate operations. \* To organize multiple image upgrades, consider > spreading out groups of upgrades in separate windows on your browser. 3. Click *IMAGE UPGRADE*. You can follow the status in the progress window. > [!NOTE] > > Replacing a gateway can take 5-7 minutes. After the gateway is up, it takes > more time for the tunnels to come up. The total length of time required varies > depending on the number of tunnels. For example, depending on the software > version of the Controller, it may take up to one hour to upgrade 4,000 > tunnels. ## Upgrade Gateway Images in Batch To upgrade gateway images, you can use batch mode for ActiveMesh gateways and for some non-ActiveMesh gateways. Check if your gateway is ActiveMesh enabled using the instructions in [Check Whether a Gateway is ActiveMesh](#check-whether-a-gateway-is-activemesh). ### For ActiveMesh Gateways 1. In your Controller, go to *Settings* > *Maintenance*. 2. In the *Selective Gateway Upgrade* window, select multiple gateways that require an upgrade. 3. Click *IMAGE UPGRADE*. 4. Click *OK* to confirm the upgrade. ### For Non-ActiveMesh Gateways Before upgrading a non-ActiveMesh gateway, it is important to check whether it has any peerings. #### [Check Whether a Gateway is ActiveMesh](https://legacy.docs.aviatrix.com/documentation/latest/platform-administration/controller/check-activemesh-gateway.html) 1. In your Controller, go to the gateway detail page for the gateway you want to check. 2. Verify whether ActiveMesh is enabled for the gateway. #### Check Whether a Gateway has any Peerings Follow the steps below to check whether a gateway has any peerings: 1. In your Controller, go to *NATIVE PEERING* > *Encrypted*. 2. Scroll down to the display window and check if the gateway you want to upgrade is listed. 3. If the gateway is listed, it has at least one peering. #### Supported Batch Image Upgrade Scenarios for Non-ActiveMesh Gateway You can upgrade multiple non-ActiveMesh gateways' images in batch mode in these two scenarios: * Upgrade multiple non-ActiveMesh gateways without any peerings in a single batch process. * Upgrade multiple non-ActiveMesh gateways with at most one of the selected gateways having peering in a batch image-upgrade session. > [!NOTE] > > Only one image-upgrade session is allowed for non-ActiveMesh gateways. This > means that all desired gateways must be included in a single upgrade session. > However, as mentioned in the scenarios above, multiple non-ActiveMesh gateways > can be upgraded simultaneously as part of a single upgrade session. ## Verify Verify the gateway upgrade by reviewing the gateway information in the *Current Image Version* column in the *Selective Gateway Upgrade* window. For information about migrating your Controller to a new image, see [Migrating Your Aviatrix Controller](/docs/enterprise/10.1/guides/maintenance/controller-migration). ## Migrating Unmanaged Disk to Managed Disk (Azure) If you have a gateway deployed on Azure and you intend to migrate your unmanaged disk to a managed disk, it is recommended to follow the process of [gateway image upgrade](#upgrade-gateway-image). This gateway image upgrade will automatically migrate the unmanaged disk to a managed disk after the gateway image upgrade. > [!TIP] > > It is highly recommended to migrate your unmanaged disk to a managed disk as > soon as possible, as Azure will be > [retiring](https://learn.microsoft.com/en-us/azure/virtual-machines/unmanaged-disks-deprecation) > unmanaged disks soon. # Controller and Gateway Logging Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/monitoring/aviatrix-logging The Aviatrix Controller and all of its managed gateways can be configured to forward their logs to well known log management systems. The Controller and all the managed gateways will forward the logs directly to the logging server. ## Overview of Controller and Gateway Logging The Aviatrix Controller and all of its managed gateways can be configured to forward their logs to well known log management systems. The Controller and all the managed gateways will forward the logs directly to the logging server. As such, the Controller and each managed gateway needs network connectivity to the logging server. Aviatrix supports using Remote Syslog (rsyslog) for forwarding log messages. Remote Syslog as the log forwarder is both efficient and the industry standard. Most log collectors support rsyslog as a log forwarder. Log data collected from Aviatrix Controller and all the managed gateways can be forwarded by Remote Syslog to your log server, such as to: * Datadog * NetFlow * AWS CloudWatch See [Aviatrix Logging Copilot](https://legacy.docs.aviatrix.com/documentation/latest/platform-administration/copilot/aviatrix-logging-copilot.html) for information about configuring the log server agents with CoPilot. In addition to standard information on syslog, Aviatrix logs also provide informational insights into UserVPN connections, VPN user TCP sessions, security rule violation statistics, gateway stats and FQDN filter violations. The chosen log management system can sift through the Aviatrix logs to get meaningful trend charts that can help monitor the network connectivity and UserVPN sessions. See [Aviatrix Log Formats](#aviatrix-log-formats) for a list of useful Aviatrix logs which can be parsed on the log management system to display relevant analytics of data collected from Aviatrix Controller and gateways. The process the Gateways and Controller use for exporting their log files is as follows: > [!NOTE] > > The process the Gateways and Controller use for exporting their log files is > different in Controller software versions earlier than 7.0.1726. In addition, > the log file prefix previously included the log filename and log timestamp. > For details, see Field Notice 42. Aviatrix gateways and Controller stream the log lines being written to the *syslog* and *auth.log* files. When you use the default rsyslog server configuration suggested in [Aviatrix Documentation](#remote-syslog-configuration-on-controller), the logs streamed from the Controller and gateways have multiple files. Each file is named with the application that generated the log. For example, all logs generated by the *avx-gw-state-sync* application are re-directed to a file named *avx-gw-state-sync* " on the log server. The log format is shown below. Your syslog collectors and any related automation for ingesting logs must accept the log format. ``` Format: [Timestamp] GW-[Name of Gateway]-[Public IP of Gateway] [Name of Application generating log][Application Process ID]: [Log message] ``` *Example of log format:* ``` Mar 23 19:17:50 GW-UdpGateway-50.17.41.173 avx-gw-state-sync[11249]: warn#011gateway_launcher/gateway_launcher.go:212#011daemon exited ``` ``` Prefix of log: [Timestamp] GW-[Name of Gateway]-[Public IP of Gateway] ``` ``` Example prefix: Mar 23 19:17:50 GW-gg-aws-usw2-s127-35.162.124.66 ``` ## Configuring Aviatrix to Send Logs to Your Log Server For information about configuring Aviatrix Controller and its managed gateways to send logs to your log server, see the following topics: * [Enabling Aviatrix Controller Logging](#enabling-aviatrix-controller-logging) * [Remote Syslog Configuration on Controller](#remote-syslog-configuration-on-controller) * [Remote Syslog Configuration on the Remote syslog Server](#remote-syslog-configuration-on-the-remote-syslog-server) * [Using Rsyslog with Datadog](#using-rsyslog-with-datadog) ## Enabling Aviatrix Controller Logging To enable logging at the Aviatrix Controller, go to *Controller > Settings > Logging* page. After setting the logging configuration in this page, you must click *Enable* for the configuration to be saved. After logging is enabled, both the Controller and all gateways will forward logs directly to the logging server. > [!NOTE] > > A total of 10 profiles from index 0 to 9 are supported for remote syslog, while index 9 is reserved for CoPilot. > > Newly deployed gateways will be added to a profile if it is the only profile > enabled in the index range of 0 to 8, > > If more than one profile is enabled in the range of 0 to 8, the newly deployed > gateway will not be added to any profile in the range of 0 to 8. You can use the > advanced options in the logging "Edit Options" window to edit the exclude and > include list. > > Newly deployed gateways will always be added to profile 9 which is reserved for > CoPilot to monitor. ## Remote Syslog Configuration on Controller On the Aviatrix Controller, go to *Controller > Settings > Logging > Remote Syslog* tile: 1. In Profile Index, select a profile to edit and then click the Status button to expand the configuration pane. 2. Profile Name: The name of this profile. 3. Server: FQDN or IP address of the remote syslog server 4. Port: Listening port of the remote syslog server. By default, it is 6514. 5. Server CA Certificate: Certificate Authority (CA) certificate 6. Client Certificate: Client certificate signed by the same CA 7. Client Private Key: Private key of the client that pairs with the public certificate 8. Protocol: TCP or UDP (TCP by default) 9. Optional Custom Template: Useful when forwarding to 3rd party servers like Datadog (Details below) 10. Click *ENABLE*. ## Remote Syslog Configuration on the Remote syslog Server On the Remote syslog server: 1. Install rsyslog and rsyslog-gnutls packages 2. Create a new config file in /etc/rsyslog.d with similar content as shown below depending on your rsyslog version to enable the tls connection. Please make sure key paths are readable by the syslog user. 3. Make sure the output directory /var/log is writable by rsyslog user/daemon. 4. Restart rsyslog service and check port is listening and that there are no errors in /var/log/syslog. 5. Confirm the port is allowed in the security group / fireware for incoming traffic. ```bash theme={null} $ModLoad imtcp $InputTCPServerRun 514 $DefaultNetstreamDriver gtls #Certificate location $DefaultNetstreamDriverCAFile /etc/cert/rsyslog-ca.pem $DefaultNetstreamDriverCertFile /etc/cert/rsyslog-crt.pem $DefaultNetstreamDriverKeyFile /etc/cert/rsyslog-key.pem $InputTCPServerStreamDriverAuthMode x509/certvalid $InputTCPServerStreamDriverMode 1 # run driver in TLS-only mode # Re-direct logs to host specific directories $template TmplMsg, "/var/log/aviatrix/%HOSTNAME%/%PROGRAMNAME%" *.info,mail.none,authpriv.*,cron.none ?TmplMsg & ~ ``` (version >=8) : ```bash theme={null} global( DefaultNetstreamDriver="gtls" DefaultNetstreamDriverCAFile="/etc/cert/rsyslog-ca.pem" DefaultNetstreamDriverCertFile="/etc/cert/rsyslog-crt.pem" DefaultNetstreamDriverKeyFile="/etc/cert/rsyslog-key.pem" ) template(name="TmplMsg" type="list") { constant(value="/var/log/aviatrix/") property(name="hostname") constant(value="/") property(name="programname" SecurePath="replace") constant(value="") } ruleset(name="remote"){ *.info;mail.none;authpriv.*;cron.none action(type="omfile" DynaFile="TmplMsg") } module( load="imtcp" StreamDriver.Name="gtls" StreamDriver.Mode="1" StreamDriver.Authmode="anon" ) input(type="imtcp" port="514" ruleset="remote") ``` Then 1. Go to /var/log/aviatrix directory. 2. Find the directory of the desired Controller or gateway: * Controller's directory name is in a format of `Controller-public_IP_of_controller` * Gateway's directory name is in a format of `GW-gateway_name-public_IP_of_gateway` 3. Each controller/gateway directory should have: * auth.log * syslog ## Format for CA Certificates from External Logging Servers The Aviatrix Controller expects certificates in PEM format. Convert any certificates downloaded from your external logging server's documentation into PEM format. Attempting to upload the wrong format may return an Exception Error. ## Using Rsyslog with Datadog Go to Controller/Settings/Logging/Remote Syslog and enable the service. 1. Server: intake.logs.datadoghq.com 2. Port: 10514 3. Protocol: TCP 4. For Optional Custom Template, copy the following string and replace the string DATADOG\_API\_KEY with your own key. ``` DATADOG_API_KEY <%pri%>%protocol-version% %timestamp:::date-rfc3339% %HOSTNAME% %app-name% - - - %msg%\\n ``` ## Aviatrix Log Formats The Aviatrix Controller and all of its managed gateways can be configured to forward their logs to well known log management systems. This section describes Aviatrix log keywords that can be identified by log management systems for further analysis. ### Aviatrix Log Keywords The following types of Aviatrix log keywords can be identified by the Log Management System for further analysis: * AviatrixVPNSession * AviatrixUser * AviatrixLicenseVPNUsers * AviatrixRule * AviatrixGwMicroSegPacket * AviatrixGwNetStats * AviatrixGwSysStats * AviatrixFQDNRule * AviatrixTunnelStatusChange * AviatrixCMD * AviatrixBGPOverlapCIDR * AviatrixBGPRouteLimitThreshold * AviatrixGuardDuty * AviatrixFireNet * AviatrixVPNVersion * AviatrixGatewayStatusChanged #### AviatrixVPNSession: This log is for gateways that have VPN enabled. Logs with this prefix come from the Controller and contain information such as the VPN username, the VPN gateway IP address and name where the user connects to, client virtual IP address, connection duration, total received bytes, total transmitted bytes, and login and logout time. Two logs will be generated for each VPN connection. One is when the connection is established, the other when it is disconnected. Example logs: Connect Log: ``` Aug 17 22:07:39 ip-172-31-46-24 cloudx_cli: AviatrixVPNSession: User=demo, Status=active, Gateway=demo, GatewayIP=52.52.76.149, VPNVirtualIP=192.168.0.6, PublicIP=N/A, Login=2016-08-17 22:07:38, Logout=N/A, Duration=N/A, RXbytes=N/A, TXbytes=N/A ``` Disconnect log: ``` Aug 17 22:26:37 ip-172-31-46-24 cloudx_cli: AviatrixVPNSession: User=demo, Status=disconnected, Gateway=demo, GatewayIP=52.52.76.149, VPNVirtualIP=192.168.0.6, PublicIP=N/A, Login=2016-08-17 22:07:38, Logout=2016-08-17 22:26:37, Duration=0:0:18:59, RXbytes=2.1 MB, TXbytes=9.03 MB ``` #### AviatrixUser: This log is for gateways that have VPN enabled. Logs with this prefix come from each VPN gateway managed by the Controller. The log contains the information for the TCP session, such as inbound and outbound interface, source IP address, destination IP address, TTL value, protocol name, and packet length. The log record is for each packet that passes through the VPN connection from the client to the destination. Two example logs: ``` Aug 17 22:15:47 ip-10-100-0-60 kernel: [14167.983249] ***AviatrixUser***:IN= OUT=eth0 SRC=192.168.0.6 DST=68.67.154.85 LEN=64 TOS=0x00 PREC=0x00 TTL=63 ID=28916 DF PROTO=TCP SPT=50428 DPT=443 WINDOW=65535 RES=0x00 SYN URGP=0 Aug 17 22:15:47 ip-10-100-0-60 kernel: [14167.968275] ***AviatrixUser***:IN= OUT=eth0 SRC=192.168.0.6 DST=10.100.0.2 LEN=66 TOS=0x00 PREC=0x00 TTL=254 ID=13309 PROTO=UDP SPT=64775 DPT=53 LEN=46 ``` #### AviatrixLicenseVPNUsers: This log is for gateways that have VPN enabled. Logs with this prefix come from the Controller and can be used to monitor the license usage of active vpn users connected to all vpn gateways. One example log: ``` Sep 25 23:40:19 ip-10-40-0-133 cloudxd: AviatrixLicsenseVPNUsers: users=2 ``` > [!NOTE] > > There is a typo in some versions (as noted in the above example) that > incorrectly shows this entry as `AviatrixLicsenseVPNUsers` instead of > `AviatrixLicenseVPNUsers`. #### AviatrixRule: You need to configure security policies to see AviatrixRule log. Logs with this prefix come from each gateway managed by the Controller. Any packet that triggers the security policy rule will generate a log record of this type with the first 100 bytes of the packet. It contains the information such as gateway IP address, inbound and outbound interface, MAC address, TTL value, protocol name, source IP address, destination IP address and packet length. An example for a deny rule event is shown below. The log event prefix is "AvxRl gw1 D:", where the gateway name is gw1, "D" represents Drop. ``` 2019-04-10T23:33:47.217018+00:00 ip-10-240-0-44 kernel: [ 4976.320353] AvxRl gw1 D:IN=eth0 OUT=eth0 MAC=02:bd:e5:4f:d0:e2:02:d8:14:81:fc:48:08:00 SRC=10.240.1.60 DST=10.230.1.23 LEN=84 TOS=0x00 PREC=0x00 TTL=63 ID=45312 DF PROTO=ICMP TYPE=8 CODE=0 ID=2833 SEQ=1 ``` Another example for an accept rule event is shown below. The log event prefix is "AvxRl StatefulGW2 A:", where the gateway name is StatefulGW2, "A" represents Accept. ``` 2019-04-10T23:34:47.602166+00:00 ip-10-240-0-44 kernel: [ 5036.705845] AvxRl StatfulGW2 A:IN=eth0 OUT=eth0 MAC=02:bd:e5:4f:d0:e2:02:d8:14:81:fc:48:08:00 SRC=10.240.1.60 DST=10.230.1.23 LEN=84 TOS=0x00 PREC=0x00 TTL=63 ID=48453 DF PROTO=ICMP TYPE=8 CODE=0 ID=2834 SEQ=1 ``` #### AviatrixGwMicroSegPacket: You need to configure [Distributed Firewalling micro-segmentation policies](/docs/enterprise/10.1/guides/security/dcf/dcf-settings) to see AviatrixGwMicrosegPacket logs. Logs with this prefix come from your configured Distributed Firewalling micro-segmentation policies. These logs contain the following information: * timestamp * source IP * destination IP * protocol (for example, ICMP or TCP) * port number * if a policy is enforced * if a policy was allowed or denied * gateway name * policy ID A micro-segmentation log example is shown below: ``` 2022-05-25T15:57:43.088860+00:00 ip-10-4-179-71 /usr/local/bin/avx-gw-state-sync[1168]: 2022/05/25 15:57:43 AviatrixGwMicrosegPacket: POLICY=54ea65c4-313e-4b3d-8db3-1ecc4f0981db SRC_MAC=16:06:11:d7:a1:11 DST_MAC=16:54:ec:50:09:17 IP_SZ=84 SRC_IP=10.4.187.253 DST_IP=10.5.144.38 PROTO=ICMP SRC_PORT=0 DST_PORT=0 DATA=0x ACT=PERMIT ENFORCED=true ``` #### AviatrixGwNetStats: Logs with this prefix come from each gateway managed by the Controller. These logs are sampled every minute and give details about gateway network interface. Two example logs: ``` 2020-06-09T17:29:31.372628+00:00 GW-test-10.23.183.116 perfmon.py: AviatrixGwNetStats: timestamp=2020-06-09T17:29:31.371791 name=test public_ip=10.23.183.116.fifo private_ip=172.31.78.160 interface=eth0 total_rx_rate=10.06Kb total_tx_rate=12.77Kb total_rx_tx_rate=2.85Kb total_rx_cum=207.16MB total_tx_cum=1.2MB total_rx_tx_cum=208.36 2020-06-12T08:30:09.297478+00:00 GW-test-10.23.183.116 perfmon.py: AviatrixGwNetStats: timestamp=2020-06-12T08:30:09.296752 name=test public_ip=10.23.183.116.fifo private_ip=172.31.78.160 interface=eth0 total_rx_rate=8.84Kb total_tx_rate=8.45Kb total_rx_tx_rate=17.29Kb total_rx_cum=4.63MB total_tx_cum=6.8MB total_rx_tx_cum=11.44MB ``` #### AviatrixGwSysStats: Logs with this prefix come from each gateway managed by the Controller. These logs are sampled every minute and give details about gateway memory, cpu and disk load. Two example logs: ``` 2020-06-09T17:29:31.372822+00:00 GW-test-10.23.183.116 perfmon.py: AviatrixGwSysStats: timestamp=2020-06-09T17:29:31.371791 name=test cpu_idle=68 memory_free=414640 memory_available=1222000 memory_total=1871644 disk_total=16197524 disk_free=10982084 2020-06-12T08:22:09.295660+00:00 GW-test-10.23.183.116 perfmon.py: AviatrixGwSysStats: timestamp=2020-06-12T08:22:09.294333 name=test cpu_idle=99 memory_free=919904 memory_available=1264792 memory_total=1871644 disk_total=16197524 disk_free=11409716 ``` #### AviatrixFQDNRule You need to configure [FQDN allow lists](/docs/enterprise/10.1/guides/security/egress/egress-protect-vpc-vnets) in order to see these logs. Logs with this prefix come from each gateway managed by the Controller. Domain name filtering can be configured per gateway via Controller. And every time a gateway tries to access a domain name, it will check if the domain name passes the configured filters. If it does, access will be allowed with the state as MATCHED, otherwise it will be discarded with state as NO\_MATCH. Two example logs: ``` 2019-12-12T04:33:46.892381+00:00 ip-172-32-0-6 avx-nfq: AviatrixFQDNRule2[CRIT]nfq_ssl_handle_client_hello() L#281 Gateway=spoke1-fqdn S_IP=172.32.1.144 D_IP=52.218.234.41 hostname=aviatrix-download.s3-us-west-2.amazonaws.com state=MATCHED Rule=*.amazonaws.com;1 2019-12-12T04:36:53.173210+00:00 ip-172-32-0-6 avx-nfq: AviatrixFQDNRule1[CRIT]nfq_ssl_handle_client_hello() L#281 Gateway=spoke1-fqdn S_IP=172.32.1.144 D_IP=98.137.246.7 hostname=www.yahoo.com state=NO_MATCH drop_reason=NOT_WHITELISTED ``` #### AviatrixTunnelStatusChange Logs with this prefix come from the Controller whenever a tunnel status changes. old\_state means old state of the tunnel, and new\_state is the new changed state of tunnel. Example log: ``` 2019-11-30T15:44:52.718808+00:00 ip-172-32-0-226 cloudxd: AviatrixTunnelStatusChange: src_gw=oregon-transit(AWS us-west-2) dst_gw=100.20.53.124(NA NA) old_state=Down new_state=Up ``` #### AviatrixCMD Logs with this prefix come from the Controller whenever a CLI command is issued. It contains information on the CLI command that was issued, the results of the execution, reason a message if there is a failure and who issued the command. Example log: ``` 2019-11-19T20:13:44.585942+00:00 ip-172-32-0-226 cloudxd: AviatrixCMD: action=USERCONNECT_UPGRADE_TO_VERSION, argv=['--rtn_file', '/run/shm/rtn957594707', 'userconnect_upgrade_to_version', 'upgrade-status', ''], result=Success, reason=, username=admin 2019-11-19T18:01:59.796230+00:00 ip-172-32-0-226 cloudxd: AviatrixCMD: action=TRANSIT_SPOKE_LIST, argv=['--rtn_file', '/run/shm/rtn2091225061', 'transit_spoke_list', '--spoke_only'], result=Success, reason=, username=admin ``` #### AviatrixBGPOverlapCIDR Log messages with this prefix come from the Controller whenever it detects overlapping CIDRs between on-prem learned and Spoke VPC CIDRs. Example log: ``` 2018-09-24T20:28:58.330708+00:00 ip-172-31-23-128 cloudxd: AviatrixBGPOverlapCIDR: Time Detected: 2018-09-24 20:28:58.329881 Spoke/Manual CIDRs ['10.0.0.0/8'] have a conflict with BGP Learned CIDRs [u'10.2.0.0/16', u'30.2.0.0/16'] in VPC vpc-782bb21f on connection vgw-bgp-ha. ``` #### AviatrixBGPRouteLimitThreshold Log messages with this prefix come from the Controller whenever it detects that total BGP routes exceed the 80 routes. (AWS VGW has a total 100 route limit.) Example log: ``` 2018-09-24T20:24:50.600144+00:00 ip-172-31-23-128 cloudxd: AviatrixBGPRouteLimitThreshold: This message is alerting you that the VGW listed below currently has 89 routes, which is approaching the VGW route limits (100). You can reduce the number of routes on VGW both from on-prem side and on Aviatrix Transit gateway by enabling Route Summarization feature. Time Detected: 2018-09-24 20:24:50.599822 Connection Name: vgw-bgp-ha VGW Id: vgw-0942b724a5150bc6a ``` #### AviatrixGuardDuty Log messages with this prefix come from the Controller whenever it receives an alert message from AWS GuardDuty. Example log: ``` 2018-09-23T00:00:50.369963-07:00 ip-172-31-89-197 cloudxd: AviatrixGuardDuty: Account [aws], Region [us-east-1], Instance ID [i-0a675b03fafedd3f2], at 2018-09-23T02:05:35Z, 163.172.7.97 is performing SSH brute force attacks against i-0a675b03fafedd3f2. Please tighten instance security group to avoid UnauthorizedAccess:EC2/SSHBruteForce threat. 2018-09-23T00:00:50.332066-07:00 ip-172-31-89-197 cloudxd: AviatrixGuardDuty: Account [aws], Region [us-east-1], Instance ID [i-0a675b03fafedd3f2], at 2018-09-23T06:35:40Z, Unprotected port on EC2 instance i-0a675b03fafedd3f2 is being probed. Please tighten instance security group to avoid Recon:EC2/PortProbeUnprotectedPort threat. ``` #### AviatrixFireNet Log messages with this prefix come from the Controller whenever a firewall instance state changes. Example log: ``` 2019-11-19T09:38:40.070080-08:00 ip-172-31-93-101 cloudxd: AviatrixFireNet: Firewall i-021f23187b8ac81c9~~tran-fw-1 in FireNet VPC vpc-0f943cd05455358ac~~cal-transit-vpc-1 state has been changed to down. 2019-11-19T09:39:03.066869-08:00 ip-172-31-93-101 cloudxd: AviatrixFireNet: Firewall i-021f23187b8ac81c9~~tran-fw-1 in FireNet VPC vpc-0f943cd05455358ac~~cal-transit-vpc-1 state has been changed to unaccessible. 2019-11-19T09:40:12.878075-08:00 ip-172-31-93-101 cloudxd: AviatrixFireNet: Firewall i-021f23187b8ac81c9~~tran-fw-1 in FireNet VPC vpc-0f943cd05455358ac~~cal-transit-vpc-1 state has been changed to up. ``` #### AviatrixVPNVersion Log messages with this prefix come from the Controller whenever it rejects an Aviatrix VPN client connection. Example log: ``` 2020-02-07T11:38:48.276150-08:00 Controller-52.204.188.212 cloudxd: AviatrixVPNVersion: The VPN connection was rejected as it did not satisfy the minimum version requirements. Current version: AVPNC-2.4.10 Required minimum version: AVPNC-2.5.7 . The rejected VPN user name is tf-aws-52-tcplb-user1 ``` #### AviatrixGatewayStatusChanged These log messages will be seen from the Controller's syslogs when a gateway's status changes Example log: ``` 2020-03-29T00:09:13.201669+00:00 ip-10-88-1-63 cloudxd: AviatrixGatewayStatusChanged: status=down gwname=EMEA-ENG-VPNGateway ``` ## AWS CloudWatch Integration > [!NOTE] > > Only AWS gateways and Controllers are supported. Other cloud types are not > supported. AWS gateways created from an access account with an AWS secret key > and access key are not supported. ### Configuring AWS CloudWatch For Aviatrix Controllers and gateways in different AWS accounts to send/update logs to the collector's AWS account, follow the instructions below to set up IAM role and policies on the collector's AWS account. 1. Go to AWS console, create an IAM role with a name aviatrix-role-cloudwatch. 2. Add Trust-Relationships for Aviatrix Controllers' and all gateways' AWS accounts. If you are already using CloudWatch for logs from all your AWS accounts, you may have already built the trust relationship between accounts. If this is the case, skip this step. 3. Attach AWS IAM Cloudwatch policy to the role aviatrix-role-cloudwatch. a: Create an IAM role aviatrix-role-cloudwatch, make sure the role name is "aviatrix-role-cloudwatch". ![create role 1](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/monitoring/create-cloudwatch-role-01.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=c041e698a4515eb4a25f213506090b4f) ![create role 2](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/monitoring/create-cloudwatch-role-02.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=2cff46acc1f0256c43633fe04e0e71f0) ![create role 3](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/monitoring/create-cloudwatch-role-03.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=3faa7c15a80006c9843f477d2710f5e8) ![Create role 4](https://mintlify.s3.us-west-1.amazonaws.com/aviatrix-14b37c43/images/guides/monitoring/aviatrix-cloudwatch-iam-role-v2.PNG) b: Add Trust-Relationships for controllers and gateways AWS accounts ![trust relationship 1](https://mintlify.s3.us-west-1.amazonaws.com/aviatrix-14b37c43/images/guides/monitoring/adding-trust-relationships-role-v2.PNG) ![trust relationship 2](https://mintlify.s3.us-west-1.amazonaws.com/aviatrix-14b37c43/images/guides/monitoring/trust-relationships-syntax-example-v2.PNG) c: Attach AWS IAM policy for "CloudWatchAgentServerPolicy" to the role ![iam policy](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/monitoring/attach-aws-iam-policy-v2.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=8f70e11e4be5127542bd0e71725a3ad9) d: Retrieve the ARN of the IAM Role ![retrieve arn](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/monitoring/copy-role-arn.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=80b1605070136a73a2e0d83653942975) ### Enable CloudWatch log on the Controller > [!NOTE] > > ARN of IAM role: Specify the ARN of the IAM role in the collector's AWS > account. \* Region: Specify which region you wish to store your logs. ![enable cloudwatch](https://mintlify.s3.us-west-1.amazonaws.com/aviatrix-14b37c43/images/guides/monitoring/enable-aviatrix-cloudwatch-v3.PNG) ### Verifying your AWS CloudWatch integration In AWS CloudWatch: ![verify in aws 1](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/monitoring/aws-cloudwatch-result-01.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=119c01ef2edc91c2ea9fcf0c7a277a1d) ![verify in aws 2](https://mintlify.s3.us-west-1.amazonaws.com/aviatrix-14b37c43/images/guides/monitoring/aws-cloudwatch-result-02.PNG) To view the Aviatrix Controller's and Gateways' CloudWatch Service Status: !["view service status](https://mintcdn.com/aviatrix-14b37c43/B-K895tPeepPiqhY/images/guides/monitoring/troubleshoot-v2.png?fit=max\&auto=format\&n=B-K895tPeepPiqhY\&q=85\&s=c1e3a102891183e8d1bf4520391b2de5) > [!NOTE] > > Logs from CloudWatch can be exported to S3 buckets. Please see the [AWS > Documentation](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/S3Export.html). ## Related Links for Log Collectors Aviatrix supports using Remote Syslog (rsyslog) for forwarding log messages. Most log collectors support rsyslog as a log forwarder. The following are links related to log collectors and rsyslog: * Sumo document about using rsyslog to send logs to Sumo * Sumo document about creating a cloud syslog source on your collections * Splunk document about creating a listener in Splunk * Logstash document about creating a TCP listener in Logstash * [Controller High Availability in AWS](/docs/enterprise/10.1/guides/controlplane/controller-ha-aws) * [Aviatrix Log Formats](#aviatrix-log-formats) # AWS CloudWatch Integration Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/monitoring/logging-integration-cloudwatch If you use CloudWatch, the Aviatrix platform offers a CloudWatch agent for sending syslog from Aviatrix Controller and Aviatrix gateways to your configured AWS CloudWatch instance. If you use CloudWatch, the Aviatrix platform offers a CloudWatch agent for sending syslog from Aviatrix Controller and Aviatrix gateways to your configured AWS CloudWatch instance. > [!NOTE] > > * Only AWS gateways and Controllers are supported. Other cloud types are not supported. > * AWS gateways created from an access account with AWS secret key and access key are not supported. ## Configuring AWS CloudWatch In order for Aviatrix Controllers and gateways in different AWS accounts to send/update logs to the collector's AWS account, follow the instructions below to set up IAM role and policies on the collector's AWS account. 1. Go to AWS console, create an IAM role with a name aviatrix-role-cloudwatch. 2. Add Trust-Relationships for Aviatrix Controllers' and all gateways' AWS accounts. If you are already using CloudWatch for logs from all your AWS accounts, you may have already built the trust relationship between accounts. If this is the case, skip this step. 3. Attach AWS IAM Cloudwatch policy to the role aviatrix-role-cloudwatch. **a: Create an IAM role aviatrix-role-cloudwatch**, make sure the role name is "aviatrix-role-cloudwatch". ![](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/monitoring/create-cloudwatch-role-01.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=c041e698a4515eb4a25f213506090b4f) ![](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/monitoring/create-cloudwatch-role-02.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=2cff46acc1f0256c43633fe04e0e71f0) ![](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/monitoring/create-cloudwatch-role-03.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=3faa7c15a80006c9843f477d2710f5e8) ![image1](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/monitoring/aviatrix-cloudwatch-iam-role-v2.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=6700c32165a820ab78ca7f2d9c332bd2) **b: Add Trust-Relationships for controllers and gateways AWS accounts** ![image2](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/monitoring/adding-trust-relationships-role-v2.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=2896cd9941ddd529025d6fefb49073d1) ![image3](https://mintcdn.com/aviatrix-14b37c43/B-K895tPeepPiqhY/images/guides/monitoring/trust-relationships-syntax-example-v2.png?fit=max\&auto=format\&n=B-K895tPeepPiqhY\&q=85\&s=468ead65336edb2e1eb67569765c76e4) **c: Attach AWS IAM policy for "CloudWatchAgentServerPolicy" to the role** ![](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/monitoring/attach-aws-iam-policy-v2.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=8f70e11e4be5127542bd0e71725a3ad9) **d: Retrieve the ARN of the IAM Role** ![](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/monitoring/copy-role-arn.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=80b1605070136a73a2e0d83653942975) ## Enable CloudWatch log on the Controller If you are using the Aviatrix Controller user interface, you configure the Aviatrix CloudWatch agent in **Controller > Settings > Logging > CloudWatch Agent**. ![image5](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/monitoring/enable-aviatrix-cloudwatch-v3.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=df25d5d427d6e6fad27e4e825eb3bdd4) > [!NOTE] > > * ARN of IAM role: Specify the ARN of the IAM role in the collector's AWS account. > * Region: Specify which region you wish to store your logs. If you are using the Aviatrix CoPilot user interface, you configure the Aviatrix CloudWatch agent in **CoPilot > Settings > Configuration > Logging Services > CloudWatch Agent**. ## Verifying your AWS CloudWatch integration In AWS CloudWatch: ![](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/monitoring/aws-cloudwatch-result-01.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=119c01ef2edc91c2ea9fcf0c7a277a1d) ![image7](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/monitoring/aws-cloudwatch-result-02.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=1e958bcee3744d789f7c8df5ff7146f1) To view the Aviatrix Controller's and Gateways' CloudWatch Service Status: ![](https://mintcdn.com/aviatrix-14b37c43/B-K895tPeepPiqhY/images/guides/monitoring/troubleshoot-v2.png?fit=max\&auto=format\&n=B-K895tPeepPiqhY\&q=85\&s=c1e3a102891183e8d1bf4520391b2de5) > [!NOTE] > > Logs from CloudWatch can be exported to S3 buckets. Please follow [AWS Documentation](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/S3Export.html). # External Monitoring with the Metrics and Status APIs Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/monitoring/metrics-api-enable The Aviatrix Metrics and Status APIs allow you to integrate CoPilot with third-party monitoring platforms such as Datadog, Splunk, Grafana, or Prometheus. The Aviatrix Metrics and Status APIs allow you to integrate CoPilot with third-party monitoring platforms such as Datadog, Splunk, Grafana, or Prometheus. This guide covers which APIs to use, what data is available, how to configure alerting that stays consistent with CoPilot's built-in monitoring, and how to avoid common pitfalls like double-counted traffic or noisy metrics. CoPilot exposes two complementary APIs for external consumption: > **Metrics API** > > **Endpoint:** `/metrics-api/v1/gateways` > > Performance metrics including CPU, memory, throughput, and packet drops. > > **Scrape interval:** Every 5 minutes > **Status API** > > **Endpoint:** `/status-api/v1/` > > Availability status for gateways, tunnels, and BGP peerings. > > **Scrape interval:** Every 1 minute Both APIs support Prometheus text format and JSON output. All data transmissions are encrypted using industry-standard protocols. ## Authentication Both APIs use the same API key, passed as a Bearer token: ``` Authorization: Bearer ``` The API key is generated in CoPilot under **Settings > Configuration > General > Features**. A single key grants access to both endpoints. See [Enabling the API](#enabling-the-api) for the steps. ## Enabling the API To use the Metrics and Status APIs, enable Network Insights API access in CoPilot and generate an authentication key. > [!NOTE] > > The Aviatrix API uses port 443, the same port as the CoPilot UI. Ensure that > port 443 is accessible and not restricted by any security groups. > [!WARNING] > > The API key shown when you generate it will not be accessible again. Copy it > and save it in a secure place. If you lose the key, you must reset it, which > invalidates the previous key. 1. In CoPilot, navigate to **Settings > Configuration > General**. 2. Scroll down to the **Features** section. 3. Next to **Network Insights API**, toggle the switch on. 4. In the **Network Insights API Key** dialog that opens, click **Generate API Key**. 5. Copy the displayed key and save it in a secure place. 6. Click **Close**. The Network Insights API is now enabled, and your scripts can authenticate to the Metrics and Status APIs using the generated key. ## Downloading the Specifications The Metrics API and Status API each ship with an OpenAPI `.yaml` specification that documents endpoints, request and response schemas, and examples. Download the latest versions from CoPilot rather than relying on external copies, as the specifications are updated with each CoPilot release. You can download a specification either from the **Generate API Key** dialog (see step 4 in [Enabling the API](#enabling-the-api)) or from the Features section at any time after the API is enabled: 1. In CoPilot, navigate to **Settings > Configuration > General**. 2. Scroll down to the **Features** section. 3. Under **Network Insights API**, in the **API Specifications** dropdown, select **Metrics API** or **Status API**. 4. Click **Download**. ## Testing API Access Verify access to your CoPilot instance with a `curl` command: ```bash theme={null} curl -k -X 'GET' \ 'https:///metrics-api/v1/gateways?format=json' \ -H 'Authorization: Bearer ' ``` A `200` response with metric data confirms the API is working. A `403` response indicates an incorrect or expired key. ## Metrics API ### Endpoints ``` GET https:///metrics-api/v1/gateways GET https:///metrics-api/v1/gateways?format=json ``` The default format is Prometheus text. Append `?format=json` for structured JSON output. ### How Data Is Collected CoPilot collects performance data from every managed gateway once per minute via the Aviatrix Controller. The Metrics API serves a snapshot of the most recent collection, rounded to the nearest 5-minute boundary with a 15-minute lookback window. Values represent point-in-time samples, not averages or aggregations. > [!TIP] > > Recommended scrape interval: **5 minutes**. Polling more frequently returns > the same cached data. ### Gateway-Level Metrics These metrics are reported once per gateway, with a `gateway` label. | Metric | Description | Unit | | - | - | - | | `cpu_idle` | CPU idle time | Percent (0-100) | | `cpu_used_per` | CPU utilization | Percent (0-100) | | `cpu_us` | CPU time spent in user space | Percent | | `cpu_ks` | CPU time spent in kernel space | Percent | | `cpu_wait` | CPU time waiting on I/O | Percent | | `memory_available` | Memory available to applications | Bytes | | `memory_free` | Completely unused memory | Bytes | | `memory_cached` | Memory used for disk cache | Bytes | | `memory_buf` | Memory used for kernel buffers | Bytes | | `memory_swpd` | Memory written to swap | Bytes | | `memory_used_per` | Memory utilization | Percent (0-100) | > [!NOTE] > > `cpu_used_per` and `memory_used_per` are available in CoPilot 4.32+. On older > versions, derive CPU utilization as `100 - cpu_idle`. For memory percentage on > older versions, you must know the instance type's total memory from your cloud > provider. ### Per-vCPU Metrics On gateways with multiple virtual CPUs, the API reports per-core utilization with `gateway` and `vcpu_name` labels. | Metric | Description | Unit | | - | - | - | | `vcpu_avg_usage` | Average CPU usage for this vCPU | Percent (0-100) | | `vcpu_min_usage` | Minimum CPU usage for this vCPU | Percent (0-100) | | `vcpu_max_usage` | Maximum CPU usage for this vCPU | Percent (0-100) | These metrics are useful for identifying core imbalance, for example one vCPU pegged at 100% while others are idle, which may indicate a single-threaded bottleneck. ### Interface-Level Metrics These metrics are reported per gateway and per network interface, with `gateway` and `interface` labels. | Metric | Description | Unit | | - | - | - | | `rate_received` | Inbound throughput | Bits/sec | | `rate_sent` | Outbound throughput | Bits/sec | | `rate_total` | Combined throughput | Bits/sec | | `rx_drop` | Inbound packet drops (cumulative) | Count | | `tx_drop` | Outbound packet drops (cumulative) | Count | | `rate_rx_drop` | Inbound drop rate | Drops/sec | | `rate_tx_drop` | Outbound drop rate | Drops/sec | | `rate_pkt_drop` | Combined drop rate | Drops/sec | | `bandwidth_ingress_limit_exceeded` | Times ingress bandwidth limit was exceeded (cumulative) | Count | | `pps_limit_exceeded` | Times packets-per-second limit was exceeded (cumulative) | Count | > [!NOTE] > > Throughput metrics (`rate_sent`, `rate_received`, `rate_total`) are reported > in **bits per second**, not bytes. The `pps_limit_exceeded` and > `bandwidth_ingress_limit_exceeded` counters are cumulative counts of packets > throttled by the cloud provider's instance-type network limits (AWS ENA > driver). These counters are only present on AWS instances. ### Example Prometheus Output ``` cpu_idle{gateway="spoke-gw-us-east-1"} 74.3 1734006896789 cpu_used_per{gateway="spoke-gw-us-east-1"} 25.7 1734006896789 memory_available{gateway="spoke-gw-us-east-1"} 3221225472 1734006896789 memory_used_per{gateway="spoke-gw-us-east-1"} 30.2 1734006896789 memory_swpd{gateway="spoke-gw-us-east-1"} 0 1734006896789 rate_received{gateway="spoke-gw-us-east-1", interface="eth0"} 125400.5 1734006896789 rate_pkt_drop{gateway="spoke-gw-us-east-1", interface="eth0"} 0 1734006896789 pps_limit_exceeded{gateway="spoke-gw-us-east-1", interface="eth0"} 0 1734006896789 vcpu_avg_usage{gateway="spoke-gw-us-east-1", vcpu_name="0"} 30.5 1734006896789 vcpu_avg_usage{gateway="spoke-gw-us-east-1", vcpu_name="1"} 20.9 1734006896789 ``` ## Status API ### Endpoints ``` GET https:///status-api/v1/ GET https:///status-api/v1/?format=json ``` ### How Data Is Collected CoPilot polls gateway, tunnel, and BGP status every minute. The Status API serves the latest cached state with no windowing or aggregation. > [!TIP] > > Recommended scrape interval: **1 minute**. The Status API refreshes at this > rate, and availability events are time-sensitive. ### Gateway Status Reported as `status` with a `gateway` label. | Value | Meaning | | - | - | | `1` | Up -- gateway is operating normally | | `0` | Down -- gateway is not responding | | `-1` | Degraded -- keepalive failure, configuration failure, upgrade failure, or transitional state | **Prometheus** ``` status{gateway="transit-gw-1"} 1 1734006896789 status{gateway="spoke-gw-1"} 0 1734006896789 ``` **JSON** ```json theme={null} { "gateways": { "transit-gw-1": { "status": "up", "lastUpdatedTimestamp": "2024-12-12T19:47:01.000Z", "lastModifiedTimestamp": "2024-12-10T08:30:00.000Z" } } } ``` The `lastModifiedTimestamp` indicates when the status last changed, while `lastUpdatedTimestamp` indicates when the status was last polled. ### Tunnel Status Includes both gateway-to-gateway peering tunnels and Site-to-Cloud (S2C) tunnels, reported as `status` with a `tunnel` label. | Value | Meaning | | - | - | | `1` | Up | | `0` | Down | ``` status{tunnel="spoke-gw-1.transit-gw-1"} 1 1734006896789 status{tunnel="s2c-branch-office"} 0 1734006896789 ``` ### BGP Peering Status Reported as `bgp_status` with `gateway` and `bgp_neighbor` labels. | Value | Meaning | | - | - | | `1` | Established | | `0` | Not established | ``` bgp_status{gateway="transit-gw-1",bgp_neighbor="vgw(169.254.254.185)"} 1 1734006896789 bgp_status{gateway="transit-gw-1",bgp_neighbor="peer(10.0.0.1)"} 0 1734006896789 ``` ## Network Interface Guidance Each gateway reports interface-level metrics for every network interface on the instance. Understanding which interfaces to monitor is essential for accurate dashboards and alerts. ### Interfaces to Monitor | Interface | Description | Recommendation | | - | - | - | | `eth0` | Primary management and data interface | **Always monitor** -- carries production traffic | | `eth1`, `eth2`, etc. | Additional data-plane interfaces (if present) | **Monitor** -- carries production traffic | ### Interfaces to Exclude | Interface | Description | Recommendation | | - | - | - | | `lo` | Loopback (127.0.0.1) | **Exclude** -- internal-only, no operational value | | `tun-*` | IPsec/GRE tunnel interfaces | **Exclude in most cases** -- see note below | ### Why Exclude Tunnel Interfaces? Traffic on `tun-*` interfaces is a subset of traffic already counted on the underlying `eth` interface. Including both leads to double-counted bandwidth in dashboards and inflated throughput numbers. For example, a packet traversing an IPsec tunnel from spoke-gw to transit-gw is counted once on `eth0` (encrypted) and once on `tun-abc123` (decrypted). Summing both overstates actual bandwidth consumption. > [!TIP] > > If you need per-tunnel visibility (for example, to identify which specific S2C > tunnel is experiencing packet drops) you may collect `tun-*` metrics > separately. In that case, do not sum them with `eth` interface metrics. ### Recommended Filter In Prometheus-based systems, apply a relabel or query filter: ```promql theme={null} # Throughput on production interfaces only rate_total{interface=~"eth.*"} ``` ```yaml theme={null} # Drop loopback and tunnel metrics at ingestion metric_relabel_configs: - source_labels: [interface] regex: "(lo|tun-.*)" action: drop ``` ## Matching CoPilot's Built-in Alerts CoPilot ships with three default alert definitions. This section maps each to the equivalent external alert you can configure in your monitoring platform. ### Global Control Plane Health CoPilot monitors Controller and CoPilot CPU, memory, and disk health. Any condition true for 15 minutes triggers the alert. | CoPilot Condition | External Equivalent | Notes | | - | - | - | | CPU Usage > 90% | `cpu_used_per > 90` for 15m | CoPilot 4.32+. On older versions: `(100 - cpu_idle) > 90` | | Memory Usage > 90% | `memory_used_per > 90` for 15m | CoPilot 4.32+. On older versions: `memory_available < (total_memory * 0.10)` | | Disk Free \< 5% | Not available via API | Use CoPilot webhook | ### Global Network Health CoPilot monitors all gateways and their network interfaces. Any condition true for 15 minutes triggers the alert. | CoPilot Condition | External Equivalent | API | | - | - | - | | Gateway Status: DOWN, KEEPALIVE\_FAIL, CONFIG\_FAIL, UPGRADE\_FAIL | `status{gateway=~".+"} != 1` for 5m | Status API | | PPS Limit Exceeded > 1% | `increase(pps_limit_exceeded[5m]) > 0` | Metrics API | | Packet Failure > 5% | Not available via API | Use CoPilot webhook | | Memory Usage > 90% | `memory_used_per > 90` for 15m | Metrics API | ### Global Memory Swap Surge CoPilot monitors all gateways for unexpected swap usage on instances with sufficient RAM. All conditions must be true for 15 minutes. | CoPilot Condition | External Equivalent | API | | - | - | - | | Swap > 0 bytes | `memory_swpd > 0` for 15m | Metrics API | | Total Memory > 1 GB | Filter by instance type (known RAM) | External knowledge | ## Recommended Alert Configuration ### Tier 1 -- Critical Alerts Configure these in every deployment. They cover the most impactful failure conditions. | Alert Name | Source | Expression | Duration | Severity | | - | - | - | - | - | | Gateway Down | Status API | `status{gateway=~".+"} != 1` | 5 min | Critical | | Tunnel Down | Status API | `status{tunnel=~".+"} == 0` | 5 min | Critical | | BGP Peer Down | Status API | `bgp_status != 1` | 5 min | Critical | | High CPU | Metrics API | `cpu_used_per > 90` | 15 min | Critical | | Memory Exhaustion | Metrics API | `memory_used_per > 90` | 15 min | Critical | | Swap Active | Metrics API | `memory_swpd > 0` | 15 min | Warning | > [!TIP] > > Adjust the memory threshold based on your gateway instance sizes. A gateway > with 2 GB of RAM should alert at a different absolute threshold than one with > 16 GB. ### Tier 2 -- Operational Alerts These provide early warning for capacity issues and performance degradation. | Alert Name | Source | Expression | Duration | Severity | | - | - | - | - | - | | Cloud PPS Limit Hit | Metrics API | `increase(pps_limit_exceeded{interface=~"eth.*"}[5m]) > 0` | 15 min | Warning | | Cloud BW Limit Hit | Metrics API | `increase(bandwidth_ingress_limit_exceeded{interface=~"eth.*"}[5m]) > 0` | 15 min | Warning | | Packet Drops | Metrics API | `rate_pkt_drop{interface=~"eth.*"} > 0` | 5 min | Warning | | I/O Wait | Metrics API | `cpu_wait > 20` | 15 min | Warning | | High Throughput | Metrics API | `rate_total{interface="eth0"} > <80% of instance limit>` | 15 min | Info | ### Tier 3 -- CoPilot Webhook Alerts The following conditions cannot be monitored via the external APIs. Configure these as CoPilot alert definitions with a webhook notification channel that forwards events to your monitoring platform. | Condition | Why It Requires a Webhook | | - | - | | Disk Free \< 5% | Disk metrics are not exposed in the Metrics API | | Packet Failure Rate > 5% | The `per_pkt_fail` metric is not exposed in the Metrics API | | Underlay Connection Down | DPD status is tracked internally but not exposed in the Status API | To configure a webhook channel in CoPilot, navigate to **Notifications > Alert Configuration > Channels** and create a webhook channel pointing to your monitoring platform's ingest URL. ## Scrape Configuration **Prometheus** ```yaml theme={null} scrape_configs: # Performance metrics - 5-minute resolution - job_name: 'aviatrix-metrics' scrape_interval: 5m scrape_timeout: 30s metrics_path: /metrics-api/v1/gateways scheme: https authorization: type: Bearer credentials: '' static_configs: - targets: [''] # Exclude loopback and tunnel interfaces metric_relabel_configs: - source_labels: [interface] regex: '(lo|tun-.*)' action: drop # Availability status - 1-minute resolution - job_name: 'aviatrix-status' scrape_interval: 1m scrape_timeout: 15s metrics_path: /status-api/v1/ scheme: https authorization: type: Bearer credentials: '' static_configs: - targets: [''] ``` **Datadog** Use the Datadog Agent's OpenMetrics integration to scrape both endpoints: ```yaml theme={null} # /etc/datadog-agent/conf.d/openmetrics.d/conf.yaml instances: - openmetrics_endpoint: https:///metrics-api/v1/gateways namespace: aviatrix.metrics min_collection_interval: 300 # 5 minutes headers: Authorization: Bearer exclude_metrics_by_labels: interface: - lo - tun-.* - openmetrics_endpoint: https:///status-api/v1/ namespace: aviatrix.status min_collection_interval: 60 # 1 minute headers: Authorization: Bearer ``` **Generic HTTP (Splunk, Custom)** For platforms that don't natively support Prometheus scraping, use the JSON format: ```bash theme={null} # Metrics GET https:///metrics-api/v1/gateways?format=json Authorization: Bearer # Status GET https:///status-api/v1/?format=json Authorization: Bearer ``` Parse the JSON response and map fields to your platform's metric and event models. ## Data Freshness and Timing | Aspect | Metrics API | Status API | | - | - | - | | Collection frequency | Every 1 minute (internal) | Every 1 minute | | API cache resolution | 5-minute intervals | Real-time (latest cache) | | Recommended scrape | Every 5 minutes | Every 1 minute | | Lag vs. CoPilot alerts | \~4-5 minutes | \~1 minute | CoPilot's built-in alert engine evaluates metrics every 60 seconds against a real-time internal cache. External monitoring introduces a small lag: * **Status alerts** (gateway/tunnel/BGP down): Expect approximately 1-2 minutes of lag compared to CoPilot's built-in alerting. * **Performance alerts** (CPU, memory, drops): Expect approximately 5-10 minutes of lag due to the Metrics API's 5-minute caching window. This lag is by design. CoPilot's built-in alerts default to 15-minute evaluation windows to avoid alert fatigue from transient issues. If more granular data is required, you can leverage CoPilot as a drill-down tool, use CoPilot webhook-based alerts, or leverage Syslog via the Aviatrix SIEM connector. ## CoPilot Performance Page vs. Metrics API Values shown on CoPilot's **Monitor > Performance** page may not exactly match Metrics API output. This is expected -- the two serve different purposes. | Aspect | CoPilot Performance Page | Metrics API | | - | - | - | | **Purpose** | Processed insights and trend analysis | Raw data for external processing | | **Data points** | Time-series of aggregated values over a selected range | Single latest raw sample per gateway | | **Aggregation** | Configurable: Average (default), Min, or Max | None -- returns the most recent raw data point | | **Time range** | User-selectable (last hour to 60+ days) | Fixed: latest sample from last 15 minutes | | **Time resolution** | Dynamic buckets (1 min for last hour, 30 min for last 24 hours, etc.) | Single snapshot rounded to 5-minute boundary | | **Metrics available** | \~90 metrics including derived fields | \~24 curated metrics (4.32+) | Aggregation is the primary cause of differences. The Performance page displays the average of all 1-minute samples within each time bucket. The Metrics API returns a single raw sample. Dashboards built from the Metrics API will appear noisier than CoPilot's Performance charts. Your external monitoring platform should apply its own aggregation and smoothing functions. > [!NOTE] > > Trends should align. If CoPilot's Performance page shows CPU usage climbing > over time, your external dashboards should show the same trend. The individual > data points may differ, but the direction and magnitude of changes will be > consistent. ## API Coverage Reference | Data | Metrics API | Status API | Alternative | | - | - | - | - | | Gateway up/down | No | **Yes** | -- | | Tunnel up/down | No | **Yes** | -- | | BGP peer state | No | **Yes** | -- | | CPU utilization | **Yes** | No | -- | | Memory utilization | **Yes** | No | -- | | Swap usage | **Yes** | No | -- | | Throughput | **Yes** | No | -- | | Packet drops | **Yes** | No | -- | | Cloud instance limits | **Yes** | No | -- | | Disk utilization | No | No | CoPilot webhook | | Packet failure rate | No | No | CoPilot webhook | | Underlay connection state | No | No | CoPilot webhook | | Memory total | No | No | Cloud provider API or known instance specs | ## Resetting the API Key You can reset the API key from CoPilot. The **Reset API Key** option appears under Network Insights API only when the feature is enabled. > [!WARNING] > > Resetting the key generates a new API key and the current key will no longer > work. You must update any scripts that use the old key. 1. Navigate to **Settings > Configuration > General**. 2. Scroll to the **Features** section. 3. Under **Network Insights API**, click **Reset API Key**. 4. Select the checkbox for **I understand the implications**, and then click **Reset**. 5. Copy the new key and close the confirmation window. ## Related Resources * [Metrics Monitored for Aviatrix Resources](/docs/enterprise/10.1/reference/monitoring/metrics-monitor) \-- complete list of all metrics CoPilot tracks * [Notifications (Alerts) About Network Events](/docs/enterprise/10.1/guides/monitoring/user-alerts-network) \-- configuring CoPilot's built-in alerts * [Aviatrix Integration with Prometheus and Grafana](https://community.aviatrix.com/t/y7hkf2k/aviatrix-integration-with-prometheus-and-grafana) \-- community walkthrough # Monitoring Business Unit Costs with CostIQ Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/monitoring/network-costs This section describes the CostIQ feature of Aviatrix CoPilot. This section describes the CostIQ feature of Aviatrix CoPilot. ## Enabling CostIQ This feature is off by default. You can enable CostIQ in CoPilot. 1. Go to **Configuration > Settings > General**. 2. Under **Features**, set CostIQ to **On** and click **Save**. The CostIQ tab will now be visible under Monitoring in the left navigation. 3. Go to **Monitoring > CostIQ**. 4. Click **Enable CostIQ** in the notification banner at the top of the page and follow the instructions in the configuration panes to create **Cost Centers** and **Shared Services**. ## Working with CostIQ The CostIQ feature gives you visibility into the traffic data of your deployed networks that are managed by the Aviatrix Controller as they relate to your cost centers. This traffic data are reported as percentages of traffic generated by end instances in Spoke VPC/VNets. The cost information displayed in CostIQ is grouped by: * Cost Center - A group of resources categorized by associated VPCs/VNets. These CoPilot Cost Centers contain resources used by your real-life cost centers or business units. * Shared Service - A cloud or network resource shared by multiple teams or cost centers. You define Shared Services by listing the IP addresses or IP CIDR ranges of the shared resource hosts. > [!NOTE] > > * The CostIQ feature requires that your Controller use software version **6.8 or > later**. > * The VPC/VNets defined in each cost center are typically mutually exclusive. > You can include the same VPC/VNet in multiple cost centers, but doing so can > skew cost-reporting results. You access CostIQ in CoPilot by going to **Monitor > CostIQ** or typing **CostIQ** in the navigation search. ### Overview of CostIQ The CostIQ Overview tab shows cost information about Aviatrix-managed resources across your entire multicloud network. The charts and tables included show the traffic of your Cost Centers and Shared Services. Use the Time Period options at the top of the page to adjust the time period for these data visualizations. | Chart or Graph | Description | | - | - | | Top Cost Centers | Top Cost Centers lists the Cost Centers in your network with the highest costs. The percentage on the right shows each Cost Center's expenditure as a percentage of total network costs. | | Top Shared Services | Top Shared Services lists the Shared Services in your network with the highest costs. The percentage on the right shows each Shared Service's expenditure as a percentage of total network costs. | | Cost Center Traffic Trends | This graph shows traffic trends for each Cost Center in your network. Each Cost center is represented as a colored line on the graph. Each data point in each line represents one Cost Center's percentage of traffic out of the total traffic for all Shared Services. Use this graph to review increases or decreases in the traffic of each Cost Center. | | Shared Service Traffic Trends | This graph shows traffic trends for each Shared Service in your network. Each Shared Service is represented as a colored line on the graph. Each data point in each line represents one Shared Service's percentage of traffic out of the total traffic for all Shared Services. Use this graph to review increases or decreases in the traffic of each Shared Service. | ### Cost Centers in CostIQ The Cost Centers tab lists the Cost Centers you have created, the Cloud Tags that contain the Cost Center resources, and the percentage of total traffic for which each Cost Center is responsible. Select a Cost Center to drill down on its information. A panel opens on the right to show more data. Use the Time Period options at the top of the panel to adjust the time period for these data visualizations. The table below describes the type of data displayed for each instance in the Cost Center: | Column | Description | | - | - | | Resource Name | The name of this instance. | | Rel. Traffic | The percentage of this instance's traffic out of the total Cost Center traffic. | | Abs. Traffic | The percentage of this instance's traffic out of the total Cost Center's traffic. | | Traffic | The absolute value of this instance's traffic. | For some VPCs and VNets in this panel, you can select a filter icon next to the VPC or VNet name to the drill down on the Spoke Gateways it includes. The following table describes the type of data displayed: | Column | Description | | - | - | | Gateway | The gateway in this VPC or VNet. | | Rel. Traffic | The percentage of this gateway's traffic out of the total VPC or VNet traffic. | | Abs. Traffic | The percentage of this gateway's traffic out of the total Cost Center's traffic. | | Traffic | The absolute value of this VPC or VNet's traffic. | ### Adding a New Cost Center To add a new Cost Center, go to **Monitor > CostIQ > Cost Centers**. #### Adding New Cost Centers 1. On the Cost Centers tab, click **+ Cost Center**. 2. Name the Cost Center. * You can use the following values for Cost Center names: alphanumeric characters, whitespace, hyphens, and underscores. * The maximum length of a name is 128 characters. * Special characters are not allowed. 3. Select the **Spoke VPC/VNets** to comprise this Cost Center. #### Selecting Spoke VPC/VNets To add resources to a Cost Center by selecting Spoke VPC/VNets, click on the Associate VPCs dropdown menu and select the necessary VPC/VNets. Note that these VPCs or VNets should be the ones used by your real-life cost centers. The VPC/VNets defined in each cost center are typically mutually exclusive. You can include the same VPC/VNet in multiple cost centers, but doing so can skew cost-reporting results. For example, if you define 2 cost centers with VPC/VNets that overlap, the shared services report will show them both as consumers of the shared service. After selecting which resources to include in this Cost Center, click **Save**. The new Cost Center appears in the table. ### Editing Cost Centers To edit a Cost Center, in the panel on the right, click the Edit icon. You can rename the Cost Centers or add and remove the resources included. ### Deleting Cost Centers 1. To delete a Cost Center, in the panel on the right, click the Delete icon. 2. Click **Confirm**. Deleting a Cost Center does not affect the VPCs/VNets it includes or any data; it only removes the Cost Center as a way of organizing and visualizing historical traffic data for your network. ## Shared Services in CostIQ The Shared Services page displays cost data for your Shared Services, or networking resources used by multiple teams or departments in your network. Shared Services are organized by the IP addresses of the resources hosts. For example, a Shared Service could be an AWS Direct Connect connection that multiple teams use. The Shared Services view displays what percent of the connection's total traffic each team is responsible for. You can break down or organize the Shared Services drilldown by: * Cost Center - Select this view to see the traffic to the Shared Services for each Cost Center in your network. * Cloud Account - Select this view to see the traffic organized by each individual Cloud Account in your network (in AWS, Azure, GCP, or OCI). Select a Cost Center to drill down on its information. A panel opens on the right to show data for each VPC and VNet in this Cost Center. Use the Time Period options at the top of the page to adjust the time period of for these data visualizations. Use the **Amount for Time Period** field to enter the amount of your bill for the specified time period. CoPilot creates a data visualization of the cost of this shared service for each resource defined in the cost center. The table below shows data for each Cost Center or Cloud Account, depending on how you filtered the Breakdown of the data, by Cost Center or Cloud Account: | Column | Description | | - | - | | Cost Center or Cloud Account | The Cost Center or Cloud Account in this Shared Service. | | Cloud | The Cost Center or Cloud Account's region. | | Rel. Traffic | The percentage of this Cost Center or Cloud Account's traffic to the Shared Shared Service out of total Shared Service traffic. | | Traffic | The absolute value of this Cost Center or Cloud Account's traffic. | Select a Cost Center or Cloud Account in this panel to drill down further to the VPC/VNets it includes: | Column | Description | | - | - | | VPC/VNet | Each VPC/VNet in this Shared Service. | | Rel. Traffic | The percentage of this VPC or VNet's traffic to the Shared Service out of the total Cost Center or Cloud Account's traffic to the Shared Service. | | Abs. Traffic | The percentage of this VPC or VNet's traffic to the Shared Service out of the Shared Service's traffic. | | Traffic | The absolute value of this VPC/VNet's traffic. | Select a VPC or VNet in this panel to drill down further to the gateways it includes: ![CostIQ Shared Service drilldown showing gateway-level traffic data](https://mintcdn.com/aviatrix-14b37c43/5QuE0MF5ilKE1Qjg/images/monitoring/costiq-shared-service-drilldown-2.png?fit=max\&auto=format\&n=5QuE0MF5ilKE1Qjg\&q=85\&s=2e0c3559e537a689f14982a115fbc7bc) | Column | Description | | - | - | | Gateway | Each gateway in this VPC or VNet. | | Rel. Traffic | The percentage of gateway traffic to the Shared Service out of the total VPC traffic to the Shared Service. | | Abs. Traffic | The percentage of this gateway's traffic to the shared Service out of the total Shared Service traffic. | | Traffic | The absolute value of this gateway's traffic. | ### Adding Shared Services To add a new Shared Service, click **+ Shared Service** near the top left. 1. Name the Shared Service. * You can use the following values for Shared Service names: alphanumeric characters, whitespace, hyphens, and underscores. * The maximum length of a name is 128 characters. * Special characters are not allowed. 2. Add the individual IP addresses or IP CIDR ranges to include in the Shared Service. 3. Click **Save**. ### Editing Shared Services To edit a Shared Service, in the panel on the right, click the Edit icon. You can rename the Shared Services or add and remove the IP CIDR ranges it includes. ### Deleting Shared Services 1. To delete a Shared Service, in the panel on the right, click the Delete icon. 2. Click **Confirm**. Deleting a Shared Service does not affect the IP addresses or CIDRs it includes or any data; it only removes the Shared Service as a way of organizing and visualizing historical cost data for your network. # Configure Telemetry Export Profiles Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/monitoring/otel-export-profiles Create and manage Telemetry Export Profiles to forward Aviatrix telemetry from the Universal OTEL Export pipeline to your external SIEM, observability, or archival systems. ## Overview Telemetry Export Profiles let you send Aviatrix telemetry to your own external SIEM, observability, or archival systems. Instead of building a separate integration for each telemetry type, you configure one or more **Telemetry Export Profiles**, and each profile controls what telemetry it forwards, the connection it uses, and where it is sent. Each profile can forward the following telemetry source types: * Distributed Cloud Firewall (DCF) security logs * Gateway infrastructure metrics Each profile forwards telemetry using one of the following connection types: * **OTLP over gRPC**, optionally with server TLS * **OTLP over HTTP**, optionally with TLS > [!NOTE] > > Syslog is not a supported export protocol for Telemetry Export Profiles in > this release. A Controller supports a maximum of **10 Telemetry Export Profiles**. For the complete field-level breakdown of each connection type's configuration options, see Telemetry Export Profile Reference. ## Prerequisites * Aviatrix Controller running version 10.1 or later. * CoPilot running version 4.35 or later, deployed and connected to the Controller. * Administrator access to CoPilot. * Network connectivity from Aviatrix Gateways to the destination endpoint (firewall rules and security groups permitting outbound traffic on the destination port). * The destination endpoint (SIEM, observability platform, data lake, or compliance archive) ready to receive OTLP/gRPC or OTLP/HTTP traffic, including any required TLS certificates or authentication credentials. * Fewer than 10 Telemetry Export Profiles already configured on the Controller. ## Create a Telemetry Export Profile ![Telemetry Export tab on the Settings > Configuration page, listing configured profiles](https://mintcdn.com/aviatrix-14b37c43/4Mpqbk9Ymohi3r60/images/monitoring/otel-export-profiles-1.png?fit=max\&auto=format\&n=4Mpqbk9Ymohi3r60\&q=85\&s=ecd529a5632481e087f1c0af4594c865) **1. Open Telemetry Export** In Aviatrix CoPilot, go to **Settings > Configuration > Telemetry Export**. **2. Start a new profile** Click **+ Profile**. > [!NOTE] > > A Controller allows a maximum of 10 Telemetry Export Profiles. If you > have already reached this limit, delete or consolidate an existing > profile before creating a new one. **3. Name the profile and select data sources** Enter a descriptive display name. Use a name that identifies the destination system and purpose, for example `splunk-dcf-security`. Select one or more data source cards for the telemetry this profile forwards: * **DCF Logs** * **Gateway Metrics** > [!NOTE] > > Data sources cannot be changed after the profile is created. To > forward a different set of sources, create a new profile. **4. Select the gateway scope** Choose which gateways this profile applies to: * **All Gateways** * **Gateways by Name** — select specific gateways from an autocomplete list (shows VPC/VNet name and region). * **Gateway by Conditions** — match gateways dynamically, for example by region or by a gateway name pattern. **5. Set the forwarding status** Leave **Forwarding Status** set to **On** so the profile forwards telemetry once saved, or turn it **Off** to save the profile without activating it. **6. Select the connection method** On the **Connection** step, select one of the following methods: | Method | Typical use | | - | - | | gRPC with Server TLS | The gateway verifies the collector's identity. Default method for an OTLP/gRPC collector. | | gRPC with Mutual TLS | Both sides verify each other's certificates. Use when your collector requires mTLS. | | HTTP with TLS | The gateway verifies the collector's identity, delivered over OTLP/HTTP. | The fields available in the remaining steps depend on the method you select. See Telemetry Export Profile Reference for the full field list per method. **7. Configure the destination endpoint** Enter the **OTLP Destination URL** for your external system, as a URL or in `host:port` format. Optionally select a **CA Certificate** file (PEM format) to validate the destination. Leave this blank to use the system CA trust store — a custom CA certificate is only required for destinations using a private or internal CA. **8. Configure custom headers (optional)** Optionally add one or more **Custom Headers** (for example, an `Authorization` header carrying a bearer token, or a vendor-specific API key header) to authenticate to the destination. Headers are optional — leave this section empty if your destination does not require them. > [!NOTE] > > Header values added here are not visible again after you save the > profile. Add all required headers before saving — you cannot view an > existing header's value later to confirm or copy it. **9. Save the profile** Click **Save**. ## Verify That Export Is Flowing **1. Check the profile status** In **Settings > Configuration > Telemetry Export**, confirm the profile shows a **Status** of **Enabled** in the profile table. > [!NOTE] > > **Status** reflects the profile's **Forwarding Status** toggle > (Enabled/Disabled), not a live connectivity or health check. Confirm > actual data delivery at the destination system itself, as described in > the next step. **2. Confirm data arrives at the destination** In your external system (SIEM, observability platform, data lake, or compliance archive), confirm that telemetry from the assigned data sources is arriving as expected. **3. Review for export errors** If telemetry does not appear at the destination, use the profile's **Edit** action to review the destination, CA certificate, and header configuration for mistakes. > [!NOTE] > > A dedicated per-profile error or connectivity-status indicator may not > be available. Rely on the destination system's own ingestion logs to > confirm delivery. ## Related Documentation * Telemetry Export Profile Reference * [Controller and Gateway Logging](/docs/enterprise/10.1/guides/monitoring/aviatrix-logging) * [Metrics Monitored for Aviatrix Resources](/docs/enterprise/10.1/reference/monitoring/metrics-monitor) # Viewing Recent Changes Made to Your Network Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/monitoring/topology-replay Shortly after a problem occurs in the network, you can use CoPilot to see if recent changes to network constructs may have caused the problem. Using the CoPilot Topology Replay feature, you can pinpoint when changes occurred and analyze the details of changed constructs. Shortly after a problem occurs in the network, you can use CoPilot to see if recent changes to network constructs may have caused the problem. Using the CoPilot Topology Replay feature, you can pinpoint when changes occurred and analyze the details of changed constructs. See the **CoPilot > Cloud Fabric > Topology > Replay** page. ## View Topology Changes (Topology Replay) This section describes how to use the Topology Replay feature of Aviatrix CoPilot. You use Topology Replay to see what changed in your environment and when it changed. This feature can be used, for example, shortly after a problem occurs in the network to see if recent changes to network constructs may have caused the problem. See also [Topology Replay Properties](#topology-replay-properties). Topology Replay shows when route, credential, and other metrics in your cloud network constructs changed. The changes are shown for all constructs regardless if they are located in one or multiple clouds. You can view the changes (additions, modifications, and deletions) that were made to the properties of subnets, gateways, and gateway connections. A time series visualization graphs change-set data as points (blue dots) with each change set collected at one-minute intervals. Clicking on a change set in the time series panel loads it into the topology map and changes details pane where you can inspect the data. For more information about UI controls in topology replay, see [Topology Replay Properties](#topology-replay-properties). To view topology changes in your environment, use the following steps: 1. In CoPilot, go to **CoPilot > Cloud Fabric > Topology** and then click the **Replay** tab. The topology replay page opens showing a topology map, a changes details pane, and a time series panel. 2. Locate and load the changes you want to inspect: * (**To locate the most recent changes**) When the page first opens or is refreshed, the most recent change set is automatically loaded in the topological map and details pane. In the time series panel, the change set is shown as a dark blue box that indicates the number of changes within it. For example, **120 Changes** (this is the default title of the change set). * (**To locate earlier changes**) By default, in the time series panel, CoPilot shows changes that occurred within one-minute increments. Each Expand icon has a number which represents the number of change sets that exist in that focused time period. Click on an Expand icon to zoom into that time period. > [!NOTE] > > Alternatively, on the track pad, pinch or stretch out two fingers to quickly > zoom out of or zoom into a time period. The bottom of the time series visualization shows changes that were recorded over the last week. There, you can drag the time-window control lines (blue vertical lines) to encompass a day (or multiple days) to quickly zoom into change sets for that time period. Keep zooming into a time period until you see a View icon. The View icon indicates a change set at that point in time. Click on the View icon to load the change set. It may take a few seconds to load. After it loads, the View icon is replaced by a blue box with the title of the change set (the title reflects the number of changes in the change set). Hovering over the blue box displays the end time of the change set. * (**To locate a specific change by title or note**) You can change the titles of change sets. You can also associate notes with them. To search for a specific change set based on a previously applied title or note, click the search icon in the changes details pane, enter any text that is part of the title or note, and then click Search. Change sets matching the search criteria are listed. Click on one from the list to load it. 3. **Inspect changes for a loaded change set.** You inspect changes for a change set in the changes details pane. When inspecting changes: * Network constructs are organized by construct type in a tree view. Expand the tree for each contruct type to drill down to properties of that construct that were added, modified, or deleted. Changes are shown as green for additions, orange for modifications, and red for deletions. * Click on a network construct in the topology map to bring its change details into focus in the details pane. * For each selected construct in the changes details pane, select or deselect the eyeball icon to toggle between showing only changed data for a construct or showing all data for a construct (to put the changes in context). * Click the AUDIT button to open the Audit tab of the Security section where you can see which users made API calls to Aviatrix Controller during the time period of the change set. 4. (**To modify the title of a change set**) To modify the title of a change set that is loaded in the changes details pane, click on the red notepad icon to open note view. The title is displayed over the dates (the default title is the number of changes in the change set). Click the blue pen icon. Select the title and change it, and then click the Save icon. When you search the timeline for change sets, you can now search by your title text. 5. (**To associate a note with a change set**) To associate a note with a change set that is loaded in the changes details pane, click on the red notepad icon to open note view. Click the blue pen icon, type your note, and then click the Save icon. When you search the timeline for change sets, you can now search by your note text. 6. (**To delete a change set**) To delete a change set that is loaded, in the time series panel, click on the red X associated with the dark blue box that represents it. It will be permanently deleted from the topology-replay database. You can also choose to delete all previous change sets recorded up to that point. ### Topology Replay Properties This section describes properties of the Topology Replay feature in Aviatrix CoPilot. #### Topology Replay - Topology Map The topology map shows network constructs that were changed in the environment for changes associated with the currently loaded change set. Properties of the topology map include: * Show full topology Click **Show full topology** to see how the changed constructs relate to all constructs in your Aviatrix managed environment (the full topology is shown at that point in time). * Disable physics Click **Disable physics** if you do not want objects to move on their own and have them stay in the location you place them (when you click and drag them there). * Hide Highlights Click **Hide highlights** to remove the circles from the objects in the map that denote they represent changed constructs. #### Topology Replay - Changes Details Pane The changes details pane shows all details for the currently loaded change set, including: * The name of the change set (reflecting the total number of changes in it). * The start time and end time for which changes were recorded. * The number of changes by type that were made to the properties of constructs: * Additions * Deletions * Modifications * The network constructs that are associated with the changes organized by construct type in a tree view. Detailed information about what metrics changed for each construct is shown. For modifications, the old value is shown striked out near the new value. Properties of the changes details pane include: * **AUDIT** Click the **AUDIT** button to open the Audit tab (of the CoPilot Security page) where you can view the users that made API calls to Aviatrix Controller during the time period of the change set. * **NOTES** Click the red notepad icon to associate a note with the loaded change set or to change the change set's title. Click the blue pen icon, then edit the title or add/edit a note, and then click the Save icon. The title is displayed over the dates (the default title is the number of Changes in the change set). When you search the timeline for change sets, you search by your title or note text. * **Search** Use the search to search for change sets by your custom change-set title or note text (for notes you associated with changes sets). * **Show only changed data/ Show all changes** Use the eyeball icon to toggle between showing only changed data for a construct or showing all data for a construct (to put the changes in context). This option can be set per construct, per construct type, or globally. #### Topology Replay - Time Series Panel The time series visualization graphs change-set data as points (blue dots) with each change-set collected at one-minute intervals shown in the change timeline (top half of the panel). The overview timeline in the time series panel (bottom half of the panel) shows the duration of all replay data (from the first replay date to the current date). The most recently recorded change set is shown as a dark blue box labeled with the number of changes in that change set. Properties of the time series panel include: * Currently loaded change set The time series panel highlights the current change set as a dark blue box labeled with the number of changes the change set contains. Hover over the box to view the end time of the change set. The green box represents the base of the currently loaded change set. * **Expand** controls to zoom into a time period In the change timeline (top half of the panel) each Expand icon has a number which represents the number of change sets that exist in that change-set cluster. On your track pad, pinch or stretch out two fingers to quickly zoom into or zoom out of a time period to view change sets that occurred within a more narrow timeframe. You can also zoom into a time period by clicking on the Expand icons. You can click on and drag the top of the panel backward and forward to view changes that happened minutes earlier/later. * Time-window control lines In the overview timeline (bottom half of the panel), after you zoom into a time period by clicking on the Expand icons or by using your track pad, two time-window control lines (blue vertical lines) display near the current time (the red line). Drag the time-window control lines where needed to focus in on the day or days you want to locate change sets in. ![time-control-image](https://mintcdn.com/aviatrix-14b37c43/B-K895tPeepPiqhY/images/guides/monitoring/topology-replay-time-preview.png?fit=max\&auto=format\&n=B-K895tPeepPiqhY\&q=85\&s=da3b96dbfd0082ccf359bbd063abdc7c) * **View** controls to load a change set The View icon indicates a change set at that point in time. Click on a View control to load a change set; this populates the network constructs associated with the changes in the topology map and displays the details for their changes in the changes details pane. The constructs associated with the changes are circled in the map. # Gaining Visibility into Your Network Topology Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/monitoring/topology-visibility When you build your Aviatrix Multicloud Transit Network by launching Aviatrix Gateways and other constructs, Aviatrix CoPilot automatically draws a topology map that shows your current network environment. When you build your Aviatrix Multicloud Transit Network by launching Aviatrix Gateways and other constructs, Aviatrix CoPilot automatically draws a topology map that shows your current network environment. In Topology, you can search for any objects that are plotted. This allows you to quickly isolate and identify resources that you are looking for in your entire environment and across clouds. You can run diagnostics for individual gateways in your multicloud network directly from the topology map. [Performing diagnostics from Topology](#access-gateway-vm-details-from-topology-map) can dramatically reduce the time spent troubleshooting issues. You can use views to show only the parts of your network environment you want to see. You can filter the views using the various properties of your managed resources. ## Working with Topology This section describes the Topology feature of Aviatrix CoPilot. Topology provides a visual representation of deployed networks (VPCs/VNets that contain VMs), gateways, instances, and gateway connections. From this map, you can also identify FireNet gateways, Edge gateways, external connections (S2C), and more. The New Topology Experience option is *On* by default, and the views on the Overview page described below relate to the New Topology Experience. If you turn this *Off*, you will see the [Classic topology map](https://legacy.docs.aviatrix.com/documentation/latest/monitoring/topology-classic.html) (if the Old Topology Experiences toggle under Settings > Configuration > General is *On*). The Topology feature gives you visibility into your network as follows: * *Network View (Overview page)* You can access Network View from the menu options below the topology map on Topology > Overview. In Network View, CoPilot displays a network topology map that shows the logical and physical layout of how network resources are connected across multiple clouds. Topology provides a visual representation of deployed networks (VPCs/VNets), gateways, instances, and gateway connections. CoPilot automatically draws the map when it connects to Aviatrix Controller. In the right-hand pane, you can alter the content of the topology map by creating and saving views with selected filters, including latency. The Topology Overview displays both managed and unmanaged VPCs and VNets. However, only VPCs and VNets that contain VMs are included, so empty VPCs/VNets are not shown. A VPC/VNet can be marked as *Aviatrix managed* where: * *Aviatrix managed = Yes* — Indicates an Aviatrix gateway exists in the VPC/VNet. * *Aviatrix managed = No* — Indicates no Aviatrix gateways exist in the VPC/VNet. * *Geo View (Overview page)* You can access Geo View from the menu options below the topology map on Topology > Overview. In Geo View, CoPilot displays a geographical map that shows the regions of your managed network resources and their related connections. You can click on the regions to display their VPC/VNets. In Geo View, under Layers Control, you can modify the map to display VPC/VNets and geoblocked countries. > [!NOTE] > > Geoblocking is a legacy feature and only available if you had it enabled prior to Controller version 7.2.4820. If Geoblocking is disabled (from Configuration > License tab) then the Geo Blocked Countries layer option is not available. Users with admin permissions and running on Controller version 7.2.4820 or earlier can enable the feature. * *Transit View (Transit page)* > [!NOTE] > > This view is deprecated. Using the New Topology Experience on the Overview page is recommended instead. Under Topology > Transit, CoPilot shows the topology of your Aviatrix transit network in relation to your deployed Aviatrix Transit Gateways. By clicking on the Aviatrix transit icon, you can see all the transit VPCs/VNets that are managed by Aviatrix Controller. By clicking on a region icon, you can see the spoke VPC/VNets that the Controller currently manages. By clicking on a spoke VPC/VNet, you can see all network constructs inside that spoke. You can use the search field to find specific resources. * *Topology Replay (Replay page)* In Topology Replay, CoPilot shows what changed in your environment and when it changed. CoPilot shows when route, credential, and other metrics in your cloud network constructs have changed over time. A timeline panel shows you all of the changes (as change sets) that were recorded over the last month. You can analyze the additions, modifications, and deletions recorded in each change set. You can delete change sets when you no longer need them. ## Show Managed Resources by Using Filters You can filter your network managed resources in the topology map to show only the resources you want. See [Create a Topology View](https://legacy.docs.aviatrix.com/documentation/latest/monitoring/topology-view-create.html) for details. In Cloud Fabric > Topology > Overview, you can filter the resources that display for Network View. ![image of filters pane](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/monitoring/topology-filters-pane.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=1714bc56fc7d00c18b891fc95284433b) Filters allow you to narrow your focus to only the resources you want to see. You can then save the selected filters as a View. See [Create a Topology View](https://legacy.docs.aviatrix.com/documentation/latest/monitoring/topology-view-create.html) for details. You can filter the topology map by using search and by selecting managed and unmanaged resources, CSP vendor and region, associated properties and tags, and conditions. You can use the Search tool to filter the Topology map by name, ID, type, etc. using letters, numbers, or characters. You can configure the following filter criteria: | Filter | Description | | - | - | | Resources | Managed: shows VPCs/VNets and related resources that are managed by Aviatrix. Managed VPCs and VNets have been onboarded to the Aviatrix Platform and contain Aviatrix resources. Unmanaged: shows unmanaged VPCs/VNets and related resources. Unmanaged VPCs and VNets have not been onboarded and therefore do not contain any Aviatrix resources. Both: shows both of the above | | Cloud Regions/Sites | Select one or more cloud providers: AWS or Azure Select a region or regions that correspond with the selected cloud providers. | | Types | Spoke VPC/VNet, Subnet, Virtual Machine (VM), Gateway VM | | SmartGroups | Select a previously created SmartGroup. You can enter multiple values for SmartGroups. | You can also filter by condition. When you select *+ Condition*, you must select the following: 1. Condition type 2. Qualifier 3. Value ![image of the condition fields](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/monitoring/topology-condition-equals.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=46eae1ef8eb2b3af8b89dd78d9300717) From the Condition Type field, you can choose a Properties or Cloud Tags condition type. These same properties are displayed on the Details pane when you select a resource in the map. For information about the properties you can filter on, see [Topology Resource Property Reference](#topology-resource-property-reference). After you choose a condition type, two more fields display. You then select an operator to associate with the condition type, and enter or select a value in the third field. > [!TIP] > > If the operator requires text input for the value, you must press Enter to generate the map output. You can add up to ten filter conditions. | Condition Type | Operator | | - | - | | Properties | The most common qualifiers are: *contains, does not contain, equals, does not equal, starts with, does not start with, ends with, does not end with* | | Cloud Tags | The most common qualifiers are: *contains, does not contain, equals, does not equal, starts with, does not start with, ends with, does not end with* | Depending on the operator selected, different types of values can be selected. Some of the available options include the following: * Select or enter multiple values * Select true/false * Select a greater than/less than or equals/does not equal value * Select a contains/does not contain or starts with/ends with * Enter a time value (License Expiry, Launch Time) You can [save the selected filters as a reusable View](https://legacy.docs.aviatrix.com/documentation/latest/monitoring/topology-view-create.html). ## Details Pane When you click on a resource in the Topology map, you can view information about that resource in the Details pane on the right side of the page. ![details panel](https://mintcdn.com/aviatrix-14b37c43/Y73dDPMr0NB7yfrH/images/guides/monitoring/topology-connection-hyperlink-detail.png?fit=max\&auto=format\&n=Y73dDPMr0NB7yfrH\&q=85\&s=13722b83758ff775e78e929ed48bd6ff) The Details pane has a Properties tab and a Resources tab. The content of these tabs changes depending on the resource selected. For lists and descriptions of the properties by selected resource, see [Topology Resource Property Reference](#topology-resource-property-reference). The Properties tab includes information about the object you selected. The specific properties vary with the object selected, but can include details such as the name of the object, the CSP region it's in, its public and private IP addresses, associated CIDRs (IPv4 and IPv6), the status, whether or not it's Aviatrix-managed (onboarded), among others. The Resources tab shows the names and types of resources associated with the resource you select in the map. If you select an object in the Resources tab, the map adjusts to focus on that resource and the Properties tab displays for that resource. ## Topology Resource Property Reference The topology map's supported filter fields include identifiers and tags for constructs, health, and status metrics. The filters also include associative properties like CSP Vendor and Region. This section describes the properties of managed resources you can filter on in the map for Transit Gateway VPC/VNets, Spoke Gateway VPC/VNets, AWS TGWs, and user instances (virtual machines). ### Transit VPC/VNet Property Descriptions You can filter your Transit VPC/VNet topology in the topology map using the following fields for Transit VPCs (innermost circle of the [topology map](#topology-map-new-topology-experience)). | **Transit VPC/VNet Property** | **Description** | | - | - | | Account Name | The CSP Account associated with this VPC. | | Cloud | The CSP that this VPC/VNet belongs to (AWS, Azure, etc.). | | Managed | Whether or not this VPC is managed by an Aviatrix Gateway. | | Name | The VPC's name tag. | | NAT Gateways | List of the VPC's NAT Gateways. | | Peer Connections | List of the VPC's peer connections. | | Region | The CSP region where the VPC is located. | | S2C (Site2Cloud) Connections | List of the VPC's Site2Cloud tunnels. | | Type | The Topology construct type (in this case, VPC). | | Virtual Machines | Number of VM's (instances) the VPC contains. | | VPC/VNet CIDR | List of the VPC/VNet's CIDRs (IPv4 and IPv6). | | VPC/VNet ID | The VPC/VNet's unique identifier. | ### Spoke VPC/VNet Property Descriptions You can filter your Spoke VPC topology in the topology map using the following fields for Spoke VPC/VNets (second from inner circle in the [topology map](#topology-map-new-topology-experience)). | **Spoke VPC/VNet Property** | **Description** | | - | - | | Account Name | The CSP Account associated with this VPC. | | Cloud (not shown in list) | The CSP that this VPC/VNet belongs to (AWS, Azure, etc.). | | Managed | Whether or not this VPC is managed by an Aviatrix Gateway. | | Name | The VPC's name tag. | | NAT Gateways | List of the VPC's NAT Gateways. | | Peer Connections | List of the VPC's peer connections. | | Region (in a different dropdown) | The CSP region where the VPC is located. | | S2C Connections | List of the VPC's Site2Cloud tunnels. | | Type | The Topology construct type (in this case, VPC). | | Virtual Machines | Number of VM's (instances) the VPC contains. | | VPC CIDR | List of the VPC's CIDRs (IPv4 and IPv6). | | VPC ID | The VPC's unique identifier. | ### Transit Gateway Property Descriptions You can filter your Transit Gateway topology in the topology map using the following fields for Transit Gateways virtual machines (second from outermost circle in the [topology map](#topology-map-new-topology-experience)). | **Transit Gateway VM Property** | **Description** | | - | - | | Name | The name of the instance. | | Type | The Topology construct type (in this case, VPC). | | Region | The CSP region in which the instance is located. | | Instance ID | ID of the image from which the instance was built. | | Cloud | The CSP that this VPC belongs to (AWS, Azure, etc.). | | Account Name | The CSP Account associated with this VPC. | | VPC ID | The ID of the instance's VPC. | | VPC Name | The name of the instance's VPC. | | Subnet ID | The ID of the instance's subnet. | | Instance Status | Denotes whether the instance is running. | | Managed | Whether this VPC is managed by an Aviatrix Gateway. true = managed false = unmanaged | | Public IP | The public IP of the instance. | | Private IP | The private IP of the instance (IPv4 and IPv6). | | Public DNS Name | The public DNS name of the instance. | | Private DNS Name | The private DNS name of the instance. | | Launch Time | The timestamp when the VM (Gateway in this case) was launched. | | Public IP | The Public IP of the instance (IPv4 and IPv6). | | Instance Size | The size of the instance (e.g. "t2.micro,"). | | Hypervisor | The instance's hypervisor. | | Kernel | The Linux kernel version of the Gateway instance. | | Tags | The resource tags associated with this gateway. | | High Performance Encryption Mode | Whether the gateway has high performance encryption active. | | Source NAT | Denotes whether Source NAT is active on this gateway. | | Associated Gateway | The Aviatrix Gateway with which this VM is associated. | | License Expiry | The timestamp when the gateway's license expires. | | License ID | The unique identifier of the instance's license. | | Stateful Firewall | Denotes whether stateful firewall rules are enabled or disabled on the gateway. | ### Spoke Gateway Property Descriptions You can filter your Spoke Gateway topology in the topology map using the following fields for Spoke Gateways virtual machines (second from outermost circle in the [topology map](#topology-map-new-topology-experience)). | **Spoke Gateway VM Property** | **Description** | | - | - | | Name | The name of the instance. | | Type | The Topology construct type. | | Region | The CSP region in which the instance is located. | | Instance ID | ID of the image from which the instance was built. | | Cloud | The CSP of the VPC. | | Account Name | The CSP Account associated with this VPC. | | VPC ID | The ID of the instance's VPC. | | VPC Name | The name of the instance's VPC. | | Subnet ID | The ID of the instance's subnet. | | Instance Status | Denotes whether the instance is running. | | Managed | Whether this VPC is managed by an Aviatrix Gateway. \* true = managed \* false = unmanaged | | Public IP | The Public IP of the instance. | | Private IP | The private IP of the instance (IPv4 and IPv6). | | Public DNS Name | The Public DNS name of the instance. | | Private DNS Name | The Private DNS name of the instance. | | Launch Time | The timestamp when the VM (Gateway in this case) was launched. | | Public IP | The Public IP of the instance (IPv4 and IPv6). | | Instance Size | The size of the instance (e.g. "t2.micro,"). | | Hypervisor | The instance's hypervisor. | | Kernel | The Linux kernel version of the Gateway instance. | | Tags | The resource tags associated with this gateway. | | High Performance Encryption Mode | Whether the gateway has high performance encryption active. | | Source NAT | Denotes whether Source NAT is active on this gateway. | | Associated Gateway | The Aviatrix Gateway with which this VM is associated. | | License Expiry | The timestamp when the gateway's license expires. | | License ID | The unique identifier of the instance's license. | | Stateful Firewall | Denotes whether stateful firewall rules are enabled or disabled on the gateway. | ### External Connection Property Descriptions You can filter your Site2Cloud external connection topology using the following fields (outermost circle on the [topology map](#topology-map-new-topology-experience)). | External Connection Property | Description | | - | - | | Name | The name of the external connection. | | Type | The Topology construct type (in this case, external connection). | | Cloud | The cloud that the external connnection belongs to (AWS, Azure, etc.). | | Status | Denotes whether the external connection is running. | | Remote CIDR | The CIDR of the external device. | | Associated Gateway | The Aviatrix Gateway with which this external connection is associated. | | Tunnel Type | The configured tunnel, such as Spoke\_BGP, Site2Cloud, etc. | | Auth Type | Authentication type, if applicable. | | IKE Version | Internet Key Exchange (IKE) protocol version, if applicable. | ### AWS TGW Property Descriptions You can filter your AWS TGW topology in the topology map using the following fields for AWS TGWs (innermost circle on the [topology map](#topology-map-new-topology-experience)). | **AWS TGW Property** | **VPC/VNet CIDR Description** | | - | - | | Name | The name of the AWS Transit Gateway. | | Account Name | The Aviatrix account that corresponds to an IAM role or account in AWS. | | Region | One of the AWS regions. | | AWS TGW ASN | TGW ASN number. The default AS number is 64512. | | AWS TGW CIDR | The TGW CIDR ranges. | | Instance ID | ID of the image from which the AWS TGW was built. | ### User Virtual Machine Property Descriptions You can filter your Virtual Machine topology in the topology map using the following fields for user virtual machines that are in Aviatrix-managed VPCs/VNets (second from outermost circle on the [topology map](#topology-map-new-topology-experience)). | **User Virtual Machine Property** | **Description** | | - | - | | Account Name | The CSP Account associated with this VPC. | | Associated Gateway | The Aviatrix Gateway with which this VM is associated. | | Cloud | The CSP that this VPC belongs to (AWS, Azure, etc.). | | Hypervisor | The instance's hypervisor. | | Image ID | ID of the image from which the instance was built. | | Instance ID | ID of the image from which the instance was built. | | Instance Size | The size of the instance (e.g. "t2.micro,"). | | Kernel | The Linux kernel version of the Gateway instance. | | Launch Time | The timestamp when the VM (Gateway in this case) was launched. | | Name | The name of the instance. | | Private IP | The private IP of the instance (IPv4 and IPv6). | | Private DNS Name | The Private DNS name of the instance. | | Public DNS Name | The Public DNS name of the instance. | | Public IP | The Public IP of the instance (IPv4 and IPv6). | | Region | The CSP region in which the instance is located. | | Status | Denotes whether the instance is running. | | Subnet ID | The ID of the instance's subnet. | | Type | The Topology construct type (in this case, Virtual Machine). | | VPC ID | The ID of the instance's VPC. | | VPC Name | The name of the instance's VPC. | ### Subnets Property Descriptions You can filter your Subnet topology in the topology map using the following fields for subnets that are Aviatrix-managed (third from outermost circle on the [topology map](#topology-map-new-topology-experience)). | **Subnet Property** | **Description** | | - | - | | Account Name | The CSP Account associated with this VPC. | | Cloud | The CSP that this VPC belongs to (AWS, Azure, etc.). | | Interface ID | The ID of the gateway interface the subnet is on. | | Name | The name of the subnet. | | Region | The CSP region in which the instance is located. | | Subnet CIDR | The CIDR of the subnet. | | Subnet ID | The ID of the instance's subnet | | Type | The Topology construct type (in this case, Subnet). | | VPC ID | The ID of the instance's VPC. | | VPC Name | The name of the instance's VPC. | ### Edge Site Property Descriptions You can filter your Edge Site topology in the topology map using the following fields for Edge Sites (innermost circle on the [topology map](#topology-map-new-topology-experience)). | **Edge Gateway Property** | **Description** | | - | - | | Name | The name of the Edge site. | | Type | The topology construct type (in this case, Edge Site). | | Account Name | The account associated with this Edge Site. | | Cloud | The CSP that the Edge Site belongs to (self-managed, Equinix, Megaport). | | Site ID | Site name for the Edge Gateway | | Status | Up or Down | | Managed | Aviatrix managed = True or False | | S2C Connection | The name of the Site2Cloud Connection, if one exists | ### Edge Gateway Virtual Machine Property Descriptions You can filter your Edge Gateway topology in the topology map (second from outermost circle on the [topology map](#topology-map-new-topology-experience)), by using the following fields. | **Edge Gateway VM Property** | **Description** | | - | - | | Name | The name of the Edge Gateway. | | Type | The topology construct type (in this case, Virtual Machine). | | Instance ID | ID of the image from which the instance was built. | | Cloud | The CSP that the Edge Gateway belongs to (self-managed, Equinix, Megaport). | | Account Name | The CSP account associated with this Edge Gateway. | | Site ID | Site name for the Edge Gateway | | Subnet ID | ID for the subnet | | Instance Status | Unknown/No | | Managed | Aviatrix managed, True/False | | Public IP | WAN interface IP (IPv4 and IPv6) | | Private IP | Private IP address (IPv4 and IPv6) | | High Performance | Yes/No | | Source NAT | True/False | ### Edge External Connection Property Descriptions You can filter your Edge External Connection topology in the topology map (second from outermost circle on the [topology map](#topology-map-new-topology-experience)), by using the following fields. | **Edge Gateway VM Property** | **Description** | | - | - | | Name | The name of the Edge Gateway. | | Type | The topology construct type (in this case, Virtual Machine). | | Cloud | The CSP that the Edge Gateway belongs to (self-managed, Equinix, Megaport). | | VPC/VNet Name | The name of the associated Edge Site. | | Status | Up/Down | | Remote CIDR | The CIDR of the external device. | | Associated Gateway | The Edge Gateway with which this VM is associated. | | Tunnel Type | The configured tunnel, such as Spoke\_BGP, Site2Cloud, etc. | | Auth Type | Authentication type, if applicable. | | IKE Version | Internet Key Exchange (IKE) protocol version, if applicable. | | Managed | Aviatrix managed, True/False | ## Latencies for Links in Your Network Topology In Cloud Fabric > Toplogy > Overview (New Topology Experience), you can click the links between resources to show the *Connection and Average Latency* information. ## Viewing a Map of Your Network Topology When you build your Aviatrix Multicloud Transit Network by launching Aviatrix Gateways and other constructs, Aviatrix CoPilot automatically draws a map to show your current network topology. To view the topology map, go to *Cloud Fabric > Topology* or type Topology in the navigation search. The Network View is displayed by default. You can filter the topology map on multiple fields to show only the gateways and constructs you want to see in your network. You can save the conditions to create a topology filter. ## Topology Map (New Topology Experience) In CoPilot release 3.0.1, a new topology map was introduced (Topology New Experience). The new map can display larger network topologies. Use the toggle at the top of the map to switch between the **New Topology Experience** and the old map. The illustration below shows the network constructs that are laid out in the five circles of the topology map (Network View). The following resources are placed on the circle working from the outermost circle to the innermost circle: 1. External Connections (S2C) (outermost circle) 2. Instances (Virtual Machine instances for Transit, Spoke, and Edge Gateways, and User VM) 3. Subnets 4. Spoke Gateway and Edge Site VPC/VNets 5. Transit Gateway and Edge Site VPC/VNets (innermost circle) The following illustration shows a topology map populated with nodes that represent the constructs in the network architecture: You can apply any number of filters to search for resources in the map. You can also use the icons below the map to expand or collapse all VPCs and VNets. > [!TIP] > > If you do not see resources you expect to see represented in the map, expand the VPCs/VNets and check the filter settings. If you click any badge in the Topology map that has grouped entities, hyperlinks for each entity display in the details panel. You can click the hyperlinks to display details for an entity. After you are finished you can click the icon in the upper left of the panel to go back to the previous view. ### Topology Legend #### Network View The topology legend in the Network View shows what the colored lines and icons in the Topology indicate. The colored lines in the Topology map (Network View) denote the following: * Green solid line: the attached gateways have *Max Performance* enabled, and the connection is up. * Green dotted line: there is a peering or gateway attachment connection between the two gateways and the connection is up. * Red solid line: the attached gateways have *Max Performance* enabled, and the connection is down. * Red dotted line: there is a peering or gateway attachment connection between the two gateways and the connection is down. If you click this connection you see a *No data found* message. Move your cursor over the connections to see the connection endpoints. Click on the connections to open a side panel with details. The colored badges represent the various cloud providers, resource types, Edge resources, FireNet gateways, and unmanaged assets. User VMs are virtual machines that are not associated with a gateway. They display as Virtual Machine in the Topology. Gateways display as Gateway Virtual Machine in the Topology. Unmanaged assets are VPC/VNets and their related resources that do not contain any Aviatrix gateways. #### Geo View The topology legend in the Geo View shows what the colored lines and icons in the topology indicate. The colored lines in the Geo View map indicate: * Green solid line: all the connections between the regions are up. * Orange solid line: some connections between the regions are up, and others are not. * Red solid line: all connections between the regions are down. ### Topology Map Controls The following shows the zoom-in, zoom-out, node-expand, and node-collapse topology controls. For large topologies that have many constructs, you can zoom in to the map for selection of granular nodes to display their properties. For smaller monitors, you can zoom out of the map to better see the external connections in the outermost circle. By default, all nodes for your constructs are plotted on the map when you open the Topology page. You can show or hide all subnets and instances under VPC/VNets in your map by using the expand-node and collapse-node controls. You can also show, hide, or truncate labels in the Topology map. ### Access Gateway VM Details from Topology Map In Network View, when you select a Gateway Virtual Machine (in the second from outermost circle), options display below Properties in the side panel. * Click **View Gateway** to open **Cloud Fabric > Gateways** with details about the Gateway. * Click **Tools > Gateway Diagnostics** to [run relevant diagnostics](https://legacy.docs.aviatrix.com/documentation/latest/monitoring/diagnostics-gateway.html) on the selected instance. > [!TIP] > > These options display only when you select individual, not grouped, gateway VMs in the Topology map. You either drill down into the Topology map or click the hyperlinks in the side panel before the options display. ## Viewing Connections in the Topology Map Gateways that have a peering connection (Transit-Transit, Transit-Spoke, Spoke-Spoke), or a gateway attachment connection (such as Specialty Gateways) show a green or red solid or dotted connection between them. Move your cursor over the connections to see the connection endpoints. Click on the connections to display the Connection and Average Latency details in the right pane. The table below the *average latency* graph shows the attachments for the connected gateways and the latency between them in milliseconds (ms). Connections for Transit-Spoke/Spoke-Spoke peerings are merged when you zoom out on the Topology (make sure no filter criteria are applied). You can click on the connection while zoomed out to open a panel with details about the connection. ## Setting the Default Topology View If you have created and saved any Topology views, you can make one of these views the default. On the Topology Network View (New Topology Experience), expand the Views drop-down and select **Manage Views**. The Manage Views dialog displays. Click the vertical ellipsis next to the view that you want to use as the default. Click **Set as Default View**. Click **Close**. In the Manage Views dialog, you can also remove a view as the default, or delete the view entirely. # Monitoring Traffic Flows Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/monitoring/traffic-flows FlowIQ provides you with critical visibility capability to the traffic that traverses your network. You access FlowIQ in CoPilot by going to Monitor > FlowIQ or typing FlowIQ in the navigation search. FlowIQ provides you with critical visibility capability to the traffic that traverses your network. You access FlowIQ in CoPilot by going to **Monitor > FlowIQ** or typing **FlowIQ** in the navigation search. FlowIQ displays metadata about traffic that flows across each link in your network. FlowIQ provides visualization of traffic flows, enabling you to inspect any network traffic that is moving across any gateway managed by the Aviatrix Controller in your Aviatrix transit network (multicloud or single cloud network). FlowIQ enables you to identify where data in your network is going to and where it is coming from and you can filter for detailed information about the traffic down to the packet level. > [!NOTE] > > NetFlow Agent must be enabled in CoPilot to use FlowIQ. You can enable or edit the NetFlow Agent from Settings > Configuration > Logging Services > NetFlow Agent. ## Advantages of FlowIQ You can use CoPilot's FlowIQ feature to determine if a gateway is actually compromised, or if a compromised system is attempting a connection. Sometimes when you use an Aviatrix DNS resolution feature (Egress FQDN, Single IP SNAT, User SSL VPN) you might see a message that an Aviatrix gateway is compromised, when it is the endpoint itself that is compromised. Use FlowIQ to view the source of the traffic that generated an alert. ## Interacting with the Flows FlowIQ provides various views for visualizing traffic records. The views respond to filters that are selected. The filters that you set are carried across all of the views. ## FlowIQ Overview Page In the FlowIQ overview page, CoPilot provides an overview of all the traffic that has traversed across your Aviatrix transit network over the last hour, day, week, month, or over a custom timeframe. The traffic information is broken down into various categories displayed in pie charts. By default, the pie charts show details for all traffic. You can filter the information to show only the traffic you are interested in analyzing. When you click on any pie-chart slice, CoPilot automatically creates a filter that narrows down the information displayed across all pie charts. Each time you select another slice, CoPilot adds another rule to your filter group. After you analyze traffic data based on one or more filters (you can create up to ten filters), clear the filter(s) so that CoPilot returns to showing data for all traffic. ## FlowIQ Trends Page In the FlowIQ trends page, CoPilot shows an overview of traffic as it moves over time for traffic based on: * A specified destination port. * A specified source address. * A specified destination address. * Total bandwidth based on direction of traffic, ingress or egress. By default, the graphs show details for the top ten results. You can filter the information for the graphs to show only the traffic you are interested in analyzing. When you click on any pie-chart slice (or listed value), CoPilot automatically creates a filter that narrows down the information displayed across all graphs. Each time you select another slice, CoPilot adds another rule to your filter group. After you analyze traffic data based on one or more filters, clear the filter(s) so that CoPilot returns to showing data for the top ten results. ## FlowIQ Geolocation Page In the Flow IQ geolocation page, CoPilot provides an overview of where traffic is coming from and going to within your cloud fabric over the last hour, day, week, month, or over a custom timeframe. The map shows the approximate location of your Aviatrix managed network constructs across the globe. When you set the time period to Last Day, you can more easily see where most of the traffic is coming from and going to. Security teams can use the geolocation view to easily identify which countries the traffic coming into their network is coming from to help determine if unexpected traffic poses a security vulnerability. The geolocation traffic information is broken down into various categories displayed in pie charts. By default, the pie charts show details for all traffic. You can filter the information for the pie charts to show only the traffic you are interested in analyzing. When you click on any pie-chart slice, CoPilot automatically creates a filter that narrows down the information displayed across all pie charts. Each time you select another slice, CoPilot adds another rule to your filter group. After you analyze traffic data based on one or more filters, clear the filter(s) so that CoPilot returns to showing data for all traffic. ## FlowIQ Records Page In the FlowIQ records page, CoPilot shows detailed information about all the traffic flows seen by your multicloud transit no matter which cloud the traffic is on. The Flow Records table shows you the detailed records of the traffic down to the packet level. You can filter the flow records in the Flow Records table by hovering over any value in the table and selecting the filter icon. You can export the flow records data table to CSV if you want to save them for later viewing or import them into your own analytics platform. ## FlowIQ Flows Page In the Flow IQ flows page, CoPilot shows all the traffic that is currently seen by your multicloud transit no matter which cloud the traffic is on. You can view how much traffic was sent in the last hour, day, week, month, or a custom timeframe. The pie charts show which source addresses and destination addresses receive the most and least bytes of data for the top 10 addresses. Using the Sankey graph, you can easily identify which source and destination hosts have the most traffic being exchanged between them (top talkers) in your network by the thickest colored bars. Hover over each colored bar to see what destination host the source host is sending traffic to based on how much traffic is being sent between them. # Notifications (Alerts) About Network Events Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/monitoring/user-alerts-network The CoPilot Notifications page enables you to configure alerts so that you can be notified about changes in your Aviatrix transit network, see all triggered alerts, pause alerts, and set preferred notification channels for receiving alerts. The CoPilot Notifications page enables you to configure alerts so that you can be notified about changes in your Aviatrix transit network, see all triggered alerts, pause alerts, and set preferred notification channels for receiving alerts. ## Working with Notifications You access Notifications in CoPilot from *Monitor > Notifications* or by typing *Notifications* in the navigation search. > [!NOTE] > > CoPilot alerts now aggregate exclusively on the main node. Performance data graphs continue to display data per node, with no support for aggregated views across nodes. This change does not affect Controller or Gateway alerts. When alert conditions are met for a metric, Copilot records the event in the *Monitor > Notifications > Alerts page*. This page is a tabular view of Aviatrix platform *triggered* alerts, with search and filter capability. How you set a condition threshold to trigger an alert depends on different factors. Work with your network operations team to determine the metric conditions that will trigger alerts in your environment. For example, for system metrics, the instance size can influence what condition threshold makes sense. For metrics associated with cloud provider-maintained infrastructure, the desired condition threshold may vary between cloud service providers. Alerts can be based on events such as changes in telemetry data for managed resources. For example, you can receive a notification when the total memory of a gateway instance falls below a specified threshold. The alerts can also be based on the status of a resource or connection. For example, when a gateway or gateway connection goes from an up to down status. See [Configuring Alerts](https://legacy.docs.aviatrix.com/documentation/latest/monitoring/notifs-alerts-create.html). CoPilot supports webhook alerts. Webhooks allow you to send notifications to third-party automation systems such as Slack. You can send a webhook to any system that can take an HTTPS callback. A single alert can notify multiple systems/people. For information on how to customize the webhooks CoPilot generates, see [Setting Up Webhook Channels for Sending Alert Notifications](https://legacy.docs.aviatrix.com/documentation/latest/monitoring/notifs-webhook.html). You can pause alerts. You may want to pause alerts before performing maintenance tasks on the network that will trigger pre-configured alerts and then unpause them when the maintenance is complete. See [Pausing Alerts](https://legacy.docs.aviatrix.com/documentation/latest/monitoring/notifs-alerts-pause.html). In *Monitor > Notifications*, CoPilot lists all alerts and shows if they are in a triggerd (open) or closed state. You can open an alert from the list to view its lifecycle. CoPilot closes the alert automatically when the alert metric no longer meets the condition to trigger the alert. The alert lifecycle provides a history for every alert that happens in your network environment. See [Viewing Alert Notifications](https://legacy.docs.aviatrix.com/documentation/latest/monitoring/notifs-alerts-display.html). See also [CoPilot Notifications Page Reference](https://legacy.docs.aviatrix.com/documentation/latest/monitoring/notifications-views.html) for descriptions of each tab available under Notifications and the views they provide. ## About Metrics used for Triggering Notifications For Aviatrix Controller and Aviatrix gateways, you can configure alerts to be notified to events that occur in your network such as performance bottlenecks or other problems. You configure alerts and the channels to be notified using the notifications feature in Aviatrix Copilot. For information about the system and network metrics on which notifications can be based, see [Metrics Monitored for Aviatrix Resources](/docs/enterprise/10.1/reference/monitoring/metrics-monitor). ## CoPilot Alert Condition Display Names You can see a list of all alert condition display names shown by CoPilot in [Metrics Monitored for Aviatrix Resources](/docs/enterprise/10.1/reference/monitoring/metrics-monitor). Many of the alerts are associated with detecting changes in resource utilization (telemetry) data for managed resources. Some alerts are associated with enhanced security services such as detecting network anomalies or detecting threat IPs identified by a well known threat-IP source. ## Underlay Connection Status Alert The Underlay Connection Status alert indicates a potential underlay communication issue. Potential causes of this alert include the following: * A CSP link outage. * A misconfigured security group or ACL. * A firewall blocking traffic. * A router dropping packets. This alert applies to connections between hosts running Aviatrix gateways. Monitored hosts will alert on syslog traffic where it is the source or the destination When you set this alert for a host(s), CoPilot monitors the syslog from any connection that includes the host as the source or destination. When syslog data indicates a potential problem from each direction of the connection between that host and another host within 30 seconds of the other, the alert is triggered. On the same connection, if the syslog data later indicates the problem is resolved from either direction, the alert is automatically resolved. # Setting Up Azure Account Credentials Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/platform-administration/account-azure Aviatrix Controller uses Azure APIs extensively to launch Aviatrix gateways and configure features. ## Azure Account Credential Setup ### Overview Aviatrix Controller uses Azure APIs extensively to launch Aviatrix gateways and configure features. In order to use Azure API, you need to first create an Aviatrix [Access Account](/docs/enterprise/10.1/guides/platform-administration/aviatrix-account) on the Aviatrix Controller. This access account corresponds to a valid Azure subscription with API credentials. You need to create an access account for each subscription. This document describes, for a given subscription, how to set up your Azure account credentials and onboard this Azure account to your Aviatrix Controller. Then, your Aviatrix Controller can execute APIs on that subscription. ### API and Permission Setup Setting up Azure permission for Aviatrix involves the following steps. 1. [Registering Aviatrix Controller Application with Microsoft Entra ID](#registering-your-aviatrix-controller-application) 2. [Assigning a role to the Aviatrix Controller Application](#assigning-a-role-to-the-aviatrix-application) 3. [Creating a Secret Identifier (Application Key)](#creating-a-secret-identifier) 4. [Setting API Permissions for the Aviatrix Controller Application](#setting-api-permissions-for-the-aviatrix-controller-application) 5. [Onboarding Your Azure Access Account in the Aviatrix Controller](#onboarding-your-azure-access-account-in-the-aviatrix-controller) As you complete the first four steps, find and save these four values: your **Azure Subscription ID**, **Directory ID**, **Application ID**, and **Application Key value** (from your Client secret) to complete the last step, onboarding your Azure Access Account in the Aviatrix Controller. ## Registering Your Aviatrix Controller Application > [!NOTE] > > Complete the following steps in order. 1. Log into the [Azure portal](https://portal.azure.com) and go to **All services**. 2. Search for **Microsoft Entra ID** and click **Microsoft Entra ID** from the search results. 3. Click **Add** > **App registration**. 4. Enter a unique and memorable name for your Aviatrix Controller application. 5. Select **Supported account types** as **Accounts in this organizational directory only**. 6. Click **Register**. The page displays details of your Aviatrix Controller application registration. 7. Copy the **Application ID** and **Directory ID** into a text file and save the file. You will use the name of your Aviatrix Controller application and these ID values later to onboard your Azure access account in the Aviatrix Controller. ## Assigning a Role to the Aviatrix Application After registering your Aviatrix Controller as an app, assign this app a role to set up the connection between your Azure account and your Aviatrix Controller. 1. Log in to the Azure portal, go to **All services**, and search for "Subscriptions." ![screenshot of Subscriptions page in Azure](https://mintcdn.com/aviatrix-14b37c43/k65bKcXF8onWe87v/images/guides/platform-administration/login-portal-azure.png?fit=max\&auto=format\&n=k65bKcXF8onWe87v\&q=85\&s=e9e49e595b7bef8581ef0844fbc8db5f) 2. Copy the Subscription ID to the text file where you saved the Application ID and Directory ID. 3. Click the **Subscription ID** to open more details. 4. On the Subscriptions page, select **Access control (IAM)** on the left. 5. On the Access control (IAM) page, click **+ Add > Add role assignment**. Alternatively, you can click Add role assignment, under Grant access to this resource. ![screenshot of add role assignment page in azure](https://mintcdn.com/aviatrix-14b37c43/mOZ0lmP7pRPIryx3/images/guides/platform-administration/sub-contrib-azure.png?fit=max\&auto=format\&n=mOZ0lmP7pRPIryx3\&q=85\&s=09f6297818590cefa782f843c71ddba8) 6. On the Role tab, select **Privileged administrator roles**. Select the **Contributor** role for this app. If the Contributor role is too broad, you can later replace it with a custom role with specific permissions. Refer to [Use Azure IAM Custom Role](/docs/enterprise/10.1/guides/platform-administration/custom-role-azure) for instructions. 7. On the Members tab, click **Select members**, and in the Select search field, enter the name of your Aviatrix Controller, that you registered in the [Registering Your Aviatrix Controller Application](#registering-your-aviatrix-controller-application) task. 8. Select your Aviatrix Controller app and click **Select**. 9. On the Add role assignment page, click **Review + assign**. Your Aviatrix Controller app is now assigned a Contributor role for this Azure subscription. As an alternative to steps 3-8 above, you can run the following PowerShell commands from your Azure AZ PowerShell module, or Azure Cloud Shell, to set up the Contributor role: ``` az ad sp create-for-rbac --name "name you want to use here" --role="Contributor" --scopes=/subscriptions/xxxx-xx-xxxx-xxxx (replace Xs with subscription id) az ad sp list --show-mine --output table ``` ## Creating a Secret Identifier After registering your Aviatrix Controller as an app and assigning it the Contributor role, create a Secret identifier. Microsoft Entra ID uses this Secret identifier to authenticate the Aviatrix Controller application. > [!NOTE] > > When you onboard your Azure account in your Aviatrix Controller, you must > enter this Secret Identifier value in the Application Key field. 1. Navigate to **All services > Microsoft Entra ID > App registrations** and click the application name. 2. Under Essentials, click **Add a certificate or secret**. 3. On the **Client secrets** tab, click **+New client secret**. 4. On **Add a client secret**, enter: * **Description** - Aviatrix * **Expires** - Set the time period to meet your corporate requirements. > [!NOTE] > > Make a note of the expiration date and use an alerting system to send a > reminder before the client secret expires. Azure does not send an alert prior > to invalidating the client secret. If the client secret expires, you can have > issues with your Controller. 5. Click **Add**. 6. On **Certificates & secrets > Client secrets**, copy and save the **Value**, and identify the Value as the Application Key to be used in Aviatrix Controller onboarding. > [!NOTE] > > This is the only opportunity to save this Value. You cannot view it again > after you leave the Certificates & secrets window. It is not stored in Azure > or in Aviatrix Controller. You should save this Value ID (Application Key) with the Account ID, Directory ID, and Subscription ID that you previously saved. These four values are necessary to onboard this Azure account in the Aviatrix Controller. ## Setting API Permissions for the Aviatrix Controller Application The API permission provides the Aviatrix Controller application permission to access Azure APIs. 1. Navigate to **All services > Microsoft Entra ID > App registrations**. 2. Click on the Aviatrix Controller application link. 3. From the left sidebar under **Manage**, select **API permissions**. 4. Under Configured Permissions, click **+ Add a permission**. 5. On Request API permissions, scroll down and click **Azure Service Management**. 6. On the Request API permissions for Azure Service Management page, under Permissions, select **user\_impersonation** and then click **Add permissions**. You can now use the four values you saved to onboard your Azure account in your Aviatrix Controller. ## Onboarding Your Azure Access Account in the Aviatrix Controller 1. Open your Aviatrix Controller and in the left sidebar, select **Onboarding**. 2. Select Microsoft Azure from the list of Cloud Service Providers (CSPs). > [!NOTE] > > Make sure to select **Microsoft Azure**, not Azure Government. 3. Under Create Primary Access Account, enter an Account Name, or use the default, for this Azure subscription. This name is only used in the Aviatrix Controller to identify the account and does not need to be a specific value from your Azure account. 4. In the fields provided, enter the four values you previously saved from Azure: | Access Account Setup Input Field | Value | | - | - | | Subscription ID | From the [Assigning a Role to the Aviatrix Application](#assigning-a-role-to-the-aviatrix-application) section | | Directory ID | From the [Registering Your Aviatrix Controller Application](#registering-your-aviatrix-controller-application) section | | Application ID | From the [Registering Your Aviatrix Controller Application](#registering-your-aviatrix-controller-application) section | | Client Secret Value (Application Key) | From the [Creating a Secret Identifier](#creating-a-secret-identifier) section | For the Application Key, use the Client Secret value you saved from your Azure account. 5. Click **Create**. Your Primary Access Account for Azure should be successfully onboarded. To troubleshoot onboarding issues, see the [Aviatrix Support website](https://support.aviatrix.com). ### Additional References If you need additional information about how to access resources, refer to the Azure documentation [Register a Microsoft Entra app and create a service principal](https://learn.microsoft.com/en-us/entra/identity-platform/howto-create-service-principal-portal). ## Azure China notes ### Deploying the Aviatrix Gateway in the Azure China Cloud Prerequisites: You must already have a Microsoft Azure China account and Aviatrix Controller in AWS China to deploy an Aviatrix Gateway in the Azure China Cloud. 1. Create the Aviatrix Controller in your AWS China Cloud. Go to Onboarding and select Azure China. 2. Enter the Aviatrix Customer ID. 3. Enter the Certificate Domain. 4. Create the Primary Access Account. 5. Deploy Aviatrix Gateway from the Gateway page in the Aviatrix Controller or the Multi-Cloud Transit Solution page. For more information, see [Acquiring a China ICP License](/docs/enterprise/10.1/concepts-architectures/architecture/connectivity/aviatrix-china-overview) . # Admin Users and Duo Sign in Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/platform-administration/admin-users-duo This document describes a reference design using the Aviatrix Controller console's user management and Duo authentication capability to manage multiple users with admin privileges. ## Admin Users and Duo Sign in ### Objectives This document describes a reference design using the Aviatrix Controller console's user management and Duo authentication capability to manage multiple users with admin privileges. As the cloud Ops team continues to expand to manage more cloud deployments, each team member often needs their own username and password with admin privilege. In addition to a username and password for login credentials, a 2FA authentication can be added for enhanced security to manage cloud Controller. Duo authentication is one of the supported methods. When enabled, it requires the user to accept a push message on the user's mobile device from Duo service in addition to username and password at the user login time. The following diagram illustrates the user relationship in a typical cloud Ops department. In this example, the Ops team has created three secondary access accounts. An access account is associated with one or more distinct cloud provider's API credentials. Typically, a cloud account corresponds to an IAM account of a distinct AWS and/or Azure account with a credit card. A default user "admin" is created by the system. In the picture below, admin has created three secondary access accounts. Additional users in admin and access account are added by the admin or admin users. ![account-structure](https://mintcdn.com/aviatrix-14b37c43/k65bKcXF8onWe87v/images/guides/platform-administration/acct-structure.png?fit=max\&auto=format\&n=k65bKcXF8onWe87v\&q=85\&s=b58aa018d00fb983799a9088f4b88249) ## Configuration Workflow for Admin Users ### Adding a New Admin User The page can be found at Accounts > Account Users > `+New User`. After the primary access account is created during onboarding, additional admin users can be added from this page. When an admin user is created, changed, or deleted, an email is sent to the admin's email address as a record for bookkeeping purposes. After the user is added, the user can log in to the console with the specified username and password. The user then has full access to the console like the admin. When the user logs in to the console, the admin username will be displayed in the top right corner. ### Deleting an Admin User The same page can be used to delete an admin user when the user leaves the group or the user role changes. After the delete button is clicked, a confirmation email is sent to the admin's email address. Note that an admin cannot be deleted by himself or herself though the user has the full console access. Typically, the admin user is added or deleted by the special username admin. ### Changing an Admin User's Password The admin user's password can be changed at the same page. An email notification is sent to the admin's email address after the change is successfully done. ### Disabling an Admin Login > [!NOTE] > > You need a local user with admin privileges to be created before you can > disable the "admin" account. If you need to disable admin login for security reasons, go to Settings > Controller > Login Customization. Click `Disable` to disable the admin login. ## Configuration Workflow for Duo Authentication ### Getting DUO API Credentials Follow the [instructions](https://duo.com/docs/duoweb) to set up Duo API credentials on the DUO Security website. ### Creating Duo Authentication To enable Duo, go to Settings > Controller > Duo Login. Enter the Duo integration key, secret key, and API hostname of your account in Duo website described earlier. Currently, only Duo push is supported. Once it is created successfully, the Duo push login applies to all users (admin is exempt). Every user (listed in Settings > Manage Accounts > Users) who wants to log in to the system must have a matching username in their Duo account. ### Removing Duo Authentication The Duo authentication setup can be removed completely by clicking the Remove button on the same page. ### Disabling/Enabling Duo Authentication The authentication can be disabled or enabled without deleting the Duo credential configuration. ### API Server Check This button can be used to troubleshoot Duo API server connectivity when the API failure is occurring. # Managing Access Accounts Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/platform-administration/aviatrix-account You can download the full set of logs to audit user access activities: ## Access Account The Aviatrix Controller is a multicloud and multi-accounts platform. The Controller uses your cloud provider API credentials to make API calls, for example, to launch an Aviatrix gateway instance, on behalf of your cloud accounts. One cloud credential is represented as an Aviatrix access account on the Controller. The Controller supports multiple Aviatrix accounts. One Aviatrix account can have multiple service accounts from different clouds, one from each cloud. For example, an Aviatrix account named DevOps can have an access account for AWS, Azure ARM credentials, and GCP credentials. * Starting from release 3.2, an access account for AWS only consists of the 12-digit account ID. * For Azure, the account information consists of [Azure ARM credentials.](/docs/enterprise/10.1/guides/platform-administration/account-azure) * For GCP (Google Cloud), the account information consists of [GCP Credentials](https://docs.aviatrix.com/documentation/latest/getting-started/onboard-acct-gcp.html). * For AWS China, please refer to [Account with Access Key](/docs/enterprise/10.1/reference/general/access-key) . The Aviatrix account structure is shown in the diagram below, where admin is the default user for the primary access account. ![account-structure](https://mintcdn.com/aviatrix-14b37c43/0QYX2iFaLoliwUis/images/platform-administration/acct-structure.png?fit=max\&auto=format\&n=0QYX2iFaLoliwUis\&q=85\&s=69bc3a41931161da03a74a7f29ab4eb3) To add more admin users, refer to [Admin Users and Duo Sign in](/docs/enterprise/10.1/guides/platform-administration/admin-users-duo) . The primary access account is the first account on the Controller. This is the account you used to launch the Controller through the AWS, Azure, GCP, or OCI marketplaces, and the account where that Controller remains. For example, if you launched your Controller through the AWS marketplace, your primary access account is an AWS account. After setting up your primary access account, you can: * Launch Aviatrix Gateways in the VPC/VNets that belong to this account. * Add access accounts from other Cloud Service Providers. For example, if you launched your Aviatrix Controller through the Azure marketplace, your can add access accounts for AWS, GCP, and OCI. An Aviatrix Cloud Account corresponds to one cloud account of one cloud type. You can create multiple Cloud Accounts to support multi cloud and multi account deployment environment. For AWS, a [primary access account](/docs/enterprise/10.1/guides/platform-administration/aviatrix-account#primary-access-account) is created during the onboarding process. Using this account credential, the Controller can launch gateways and build connectivity on VPCs that belong to this AWS account. See [AWS IAM Roles and Policies](/docs/enterprise/10.1/reference/general/iam-role) for the IAM roles and policies that make this possible. **CoPilot UI** ## Setting Up Additional Access Accounts for Different Clouds After you go through the onboarding process and create the primary access account, you can create additional Aviatrix access accounts on the Controller. This allows you to launch gateways and build connectivity across different cloud accounts. For example, if you create a primary access account in Azure, where you launched your Controller, you can add additional access accounts for AWS, GCP, and OCI. To launch an additional access account: 1. Go to your Aviatrix Controller > Accounts > Access Accounts. 2. Click *+ Add New* to create this new access account. 3. Enter a unique account name: for example, BU-Group-3. 4. Mark the radio button for the appropriate Cloud Service Provider. The fields below change based on which Cloud Service Provider you chose. See the following documents for more information on adding access accounts in each cloud: * [AWS access accounts](/docs/enterprise/10.1/reference/general/iam-role) * > [!NOTE] > > If you launched your Controller in Azure, GCP, or OCI, leave the IAM roles > checkbox *unmarked* for any additional AWS access accounts, and provide an > access key and secret instead. See [AWS IAM Roles and Policies](/docs/enterprise/10.1/reference/general/iam-role) for why IAM roles > do not work in this case. * [Azure access accounts](/docs/enterprise/10.1/guides/platform-administration/account-azure#onboarding-your-azure-access-account-in-the-aviatrix-controller) * [GCP access accounts](/docs/enterprise/10.1/guides/platform-administration/gcp-account-onboarding) * [OCI access accounts](https://docs.aviatrix.com/documentation/latest/getting-started/getting-started-guide-oracle.html#onboarding-oci-in-aviatrix-copilot) * [Alibaba access accounts](/docs/enterprise/10.1/guides/platform-administration/aviatrix-account-alibaba) 5. After entering the information required, scroll down and select any [RBAC](/docs/enterprise/10.1/guides/platform-administration/rbac-faq) or permission groups this account should belong to. 6. Click *OK*. * The new access account is created. 7. Now you can create connectivity between two VPC/VNets in different cloud accounts. **Terraform** If you use Terraform to create more AWS access accounts, you need to run the CloudFormation script on each secondary account first, then use Terraform account resource to create the account. The CloudFormation script creates the IAM roles and policies and establishes a trust relationship with the primary account (the account where the Controller is launched). See [AWS IAM Roles and Policies](/docs/enterprise/10.1/reference/general/iam-role#trust-relationships-default) for the default trust relationship and how to harden it. Add an [aviatrix\_account](https://registry.terraform.io/providers/AviatrixSystems/aviatrix/latest/docs/resources/aviatrix_account) resource block for each additional access account: **1. Create the additional access account** ```hcl theme={null} resource "aviatrix_account" "bu_group_3" { account_name = "BU-Group-3" cloud_type = 1 aws_account_number = "234567890123" aws_iam = true aws_role_app = "arn:aws:iam::234567890123:role/aviatrix-role-app" aws_role_ec2 = "arn:aws:iam::234567890123:role/aviatrix-role-ec2" } ``` > [!NOTE] > > If the Controller was launched in Azure, GCP, or OCI, set `aws_iam` to > `false` and provide `aws_access_key`/`aws_secret_key` instead for any > additional AWS access accounts — IAM roles only work for the account > where the Controller was launched. **2. Verify and Apply the Configuration** Run `terraform plan`, review the planned changes, and run `terraform apply`. ## Related Resources * [aviatrix\_account Resource](https://registry.terraform.io/providers/AviatrixSystems/aviatrix/latest/docs/resources/aviatrix_account) ## Account Audit Logs You can download the full set of logs to audit user access activities: 1. Go to **Administration > Audit** page on the CoPilot UI. 2. Select a desired time period. Then choose any filters as needed. Click **Apply**. 3. Click the Download icon ![download](https://mintcdn.com/aviatrix-14b37c43/k65bKcXF8onWe87v/images/guides/platform-administration/download.png?fit=max\&auto=format\&n=k65bKcXF8onWe87v\&q=85\&s=7ee27e3301ac18dcff8d87d4368630ad) to download the API logs. # Setting up Alibaba Cloud Account Credentials Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/platform-administration/aviatrix-account-alibaba Access your Alibaba account info in the Alibaba UI. Click on the User Icon so you can access the Alibaba Cloud Account ID, Cloud Access Key ID, and Cloud Secret Key. You need the Alibaba account information to create the Alibaba Primary Access Account in the Aviatrix Controller. ## Creating an Alibaba Primary Access Account Access your Alibaba account info in the Alibaba UI. Click on the User Icon so you can access the Alibaba Cloud Account ID, Cloud Access Key ID, and Cloud Secret Key. You need the Alibaba account information to create the Alibaba Primary Access Account in the Aviatrix Controller. ![alibaba\_user\_icon](https://mintcdn.com/aviatrix-14b37c43/k65bKcXF8onWe87v/images/guides/platform-administration/alibaba-user-icon.png?fit=max\&auto=format\&n=k65bKcXF8onWe87v\&q=85\&s=e396c07927d3d14fc3f09b35d31167c5) In the Alibaba UI, create an AccessKey pair for authenticating the Aviatrix Controller. Click on the User Icon and navigate to AccessKey Management > Create Access Key. ![alibaba accesskey](https://mintcdn.com/aviatrix-14b37c43/k65bKcXF8onWe87v/images/guides/platform-administration/alibaba-accesskey.png?fit=max\&auto=format\&n=k65bKcXF8onWe87v\&q=85\&s=81da41e1eeb17a89d786b7ec7ab2f159) In the Aviatrix Controller, navigate to Accounts > Access Accounts and select Alibaba Cloud. Add an Account Name and enter the Alibaba Cloud Account ID, Cloud Access Key ID, and Cloud Secret Key. Optional - add any RBAC Groups that should have access to the Primary Access Account. ## Alibaba Cloud Default Limitations The EIP bandwidth limit is 200 Mbit/s. The Aviatrix Spoke to Transit and Transit to Spoke connections maximum bandwidth is 400 Mbit/s. You can purchase different plans to increase throughput and bandwidth. A maximum of 48 routes in each route table is supported by default. If you require more routes in each route table, contact Alibaba Support. The Alibaba API takes 1-2 seconds to add or delete one route in one VPC route table. No route update requests are accepted while a route is being added or deleted. Outgoing traffic to public non-RFC1918 IP address from an instance with a public IP does not look at the VPC route table. Even non-RFC1918 routes are configured on VPC route table. If you want to improve this non-RFC1918 traffic routing behavior on public instance, contact Alibaba Support. ## Deploying the Aviatrix Gateway in your Alibaba Cloud You must satisfy the prerequisites in [Creating an Alibaba Primary Access Account](#creating-an-alibaba-primary-access-account) before Deploying the Aviatrix Gateway in your Alibaba Cloud. 1. Access your Alibaba account info in the Alibaba UI. Click on the User Icon and record your Alibaba Account ID. 2. Communicate your Alibaba Account ID to your Aviatrix Support representative. 3. Your Aviatrix Support representative shares the Aviatrix gateway image with your Alibaba account. 4. Verify your Alibaba account can access the Aviatrix gateway image. Go to Elastic Compute Service > Instances & Images > Images > Shared Image to view the image. ![alibaba\_share\_image](https://mintcdn.com/aviatrix-14b37c43/mOZ0lmP7pRPIryx3/images/guides/platform-administration/share-alibaba.png?fit=max\&auto=format\&n=mOZ0lmP7pRPIryx3\&q=85\&s=6955aa6ecad508c8a30a5438990008b4) 5. Create an Alibaba Primary Access Account in the Aviatrix Controller. 6. Deploy the Aviatrix Gateway in the Alibaba cloud. # Connecting a Cloud Account Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/platform-administration/cloud-account-create To connect accounts for edge platforms, see Setting up Accounts for Edge Platforms. To connect accounts for edge platforms, see [Setting up Accounts for Edge Platforms](/docs/enterprise/10.1/guides/connectivity/edge/edge-platform-account) . **CoPilot UI** To add a cloud account in CoPilot: 1. Go to Cloud Resources > Cloud Accounts. 2. Click **Onboard Cloud Account**. 3. Enter the parameter values: | Parameter | Description | | - | - | | Account Name | Enter the name of the cloud account. | | Cloud Account | Select the Cloud Service Provider (CSP) for this account. | 4. Select the parameters specific to the Cloud Account you are using. [See the tables below](#aws) for the parameters you can select when connecting cloud accounts. 5. Click **Save**. The new cloud account appears in the table. **Terraform** Connect a cloud account with the [aviatrix\_account](https://registry.terraform.io/providers/AviatrixSystems/aviatrix/latest/docs/resources/aviatrix_account) resource. The same resource is used for every CSP; only the arguments change based on `cloud_type` and the credential type (IAM role vs. access key for AWS, or the CSP-specific fields for Azure, GCP, OCI, and Alibaba). See the [full argument reference](https://registry.terraform.io/providers/AviatrixSystems/aviatrix/latest/docs/resources/aviatrix_account) for the fields required for each cloud type. **1. Connect the cloud account** Add an `aviatrix_account` resource block to your Terraform configuration file. For example, to connect an AWS account using IAM roles (the recommended, CloudFormation-based option): ```hcl theme={null} resource "aviatrix_account" "aws_account" { account_name = "aws-account" cloud_type = 1 aws_account_number = "123456789012" aws_iam = true aws_role_app = "arn:aws:iam::123456789012:role/aviatrix-role-app" aws_role_ec2 = "arn:aws:iam::123456789012:role/aviatrix-role-ec2" } ``` > [!NOTE] > > The IAM roles referenced by `aws_role_app` and `aws_role_ec2` must > already exist and trust the Controller, either from running the > CloudFormation template beforehand or from a prior onboarding of > the same AWS account. To use access keys instead, set `aws_iam` to > `false` and provide `aws_access_key` and `aws_secret_key`. **2. Verify and Apply the Configuration** Run `terraform plan`, review the planned changes, and run `terraform apply`. [The tables below](#aws) provide descriptions of the onboarding parameters for each cloud service provider (CSP). They also identify which fields can be modified for each CSP. After onboarding a cloud account, you can do the following on the Cloud Account page: * Modify the settings by clicking the Edit icon in the row of the cloud account. * Audit the account from either the Actions menu or the vertical ellipses menu. * Update the IAM Policy for any AWS cloud account from the Actions menu. ## Kubernetes Clusters If you have any Kubernetes clusters in your AWS or Azure cloud accounts, they are discovered automatically during the AWS/Azure cloud account onboarding process and displayed on the Cloud Assets > Kubernetes Clusters tab. After they are displayed on this tab they can be [onboarded](/docs/enterprise/10.1/guides/security/dcf/kubernetes-onboard) . All necessary [EKS and AKS permissions](/docs/enterprise/10.1/reference/dcf/kubernetes-prerequisites) are added as part of the AWS/Azure cloud account onboarding process. > [!NOTE] > > Any clusters you created outside of AWS/EKS can be > [onboarded manually](/docs/enterprise/10.1/guides/security/dcf/kubernetes-onboard#manual-onboard) > . *** **AWS** ## AWS Cloud Account Parameters When you select AWS as the CSP, you can click the dropdown menu on the AWS icon to select Standard, China, or GovCloud. You can onboard using CloudFormation or using access keys. CloudFormation is the recommended option. Access keys should only be used in nonproduction environments. | Parameter | Description | Can Be Modified | Comments | | - | - | - | - | | Account Name | The name assigned to the account. | No | Assign a descriptive and unique name. | | Cloud Type | The cloud service provider for the account. | No | Select the AWS logo. | | IAM Role-Based | Toggles IAM role-based access **On** or **Off**. | Yes | If the IAM Role-Based toggle is set to Off, you are prompted to provide a key and secret for access. | | Launch the CloudFormation template | The template runs in AWS to establish trust with your access account. (Skip if you have already run the template.) Only displays if IAM Role-Based option is On. | No | The template cannot be rerun to change settings. For AWS Standard and GovCloud, the template opens. For AWS China, the template downloads. | | AWS Account Number | The 12-digit number for the AWS account. | Yes | If the account number is changed, the launch CloudFormation option displays and you are prompted at the bottom of the page to create a secondary account. | | AWS Access Key ID | Part of an access key that provides long-term credentials to sign programmatic requests to the AWS CLI or API. | Yes | Only displays if IAM Role-Based is set to Off. Not recommended for production environments. | | AWS Secret Key | Part of an access key that provides long-term credentials to sign programmatic requests to the AWS CLI or API. | Yes | Only displays if IAM Role-Based is set to Off. Not recommended for production environments. | | AWS App Role ARN (Optional) | Enter the AWS App Role ARN. ARN values are only required if you are creating an access account that is separate from the one from which you deployed the Controller. | Yes | Available if IAM Role-Based is On. | | AWS EC2 Role ARN (Optional) | Enter the EC2 Role ARN. This IAM role gives Aviatrix temporary security credentials to make API requests. | Yes | Available if IAM Role-Based option is On. | | Add to RBAC Groups (Optional) | Select the Role-Based Access Control (RBAC) groups that should be able to access this account. | No | Available if IAM Role-Based option is On. | | Confirm the CloudFormation script has been run to set up this secondary access account. | Select this checkbox to create a secondary AWS account. | --- | This option displays if you change the AWS Account Number. | **Azure** ## Azure Cloud Account Parameters Note that when you select Azure as the cloud for this account, you can click the dropdown menu on the icon to select Global, China, or GovCloud. | Parameter | Description | Can Be Modified | Comments | | - | - | - | - | | Account Name | The name assigned to the account. | No | Assign a descriptive and unique name. | | Cloud Type | The cloud service provider for the account. | No | Select the Azure logo. | | ARM Subscription ID | The unique ARM Subscription ID from your Azure account. | Yes | Identifies a specific subscription within the Azure account. | | Directory ID | The unique Entra ID from your Azure account, assigned to a tenant. | Yes | Allows users to access Microsoft services. | | Application ID | The unique identifier assigned to an application in your Azure account. | Yes | Allows the application to authenticate and access Azure services. Also called Client ID. | | Application Key | The Secret Key Value saved from your Azure account. | Yes | Used to authenticate an application and provide programmatic access to Azure services. | | Add to RBAC Groups (Optional) | Lists the Role-Based Access Control (RBAC) groups to choose from for access to this account. | No | | **GCP** ## GCP Cloud Account Parameters | Parameter | Description | Can Be Modified | Comments | | - | - | - | - | | Account Name | The name assigned to the account. | No | Assign a descriptive and unique name. | | Cloud Type | The cloud service provider for the account. | No | Select the Google Cloud Platform (GCP) logo. | | GCP Project ID | The unique ID that identifies a specific project in your GCP account. | Yes | | | GCP Project Credentials | The service account credentials file downloaded from GCP. | Yes | You must upload a project credentials file. | | Add to RBAC Groups (Optional) | Lists the Role-Based Access Control (RBAC) groups to choose from for access to this account. | No | | **OCI** ## OCI Cloud Account Parameters | Parameter | Description | Can Be Modified | Comments | | - | - | - | - | | Account Name | The name assigned to the account. | No | Assign a descriptive and unique name. | | Cloud Type | The cloud service provider for the account. | No | Select the OCI logo. | | OCI Tenancy ID | The unique ID for your OCI account (tenancy). | Yes | | | OCI User ID | The ID for the user who should have access to this account through CoPilot. | Yes | | | OCI Compartment ID | The unique label for a logical container (compartment) within your OCI tenancy. | Yes | Allows you to control access to compartment resources using policies. | | OCI API Private Key File | The private key file you downloaded from your OCI account. | Yes | The secret part of a key pair used to authenticate programmatic requests. | | Add to RBAC Groups (Optional) | Lists the Role-Based Access Control (RBAC) groups to choose from for access to this account. | No | | **Alibaba** ## Alibaba Cloud Account Parameters | Parameter | Description | Can Be Modified | Comments | | - | - | - | - | | Account Name | The name assigned to the account. | No | Assign a descriptive and unique name. | | Cloud Type | The cloud service provider for the account. | No | Select the Alibaba logo. | | Alibaba Account ID | A unique ID for your Alibaba Cloud account. | Yes | | | Access Key | The access key ID from your Alibaba account. | Yes | Part of the AccessKey pair that provides access to APIs and resources. | | Secret Access Key | The Secret Access Key from your Alibaba account. | Yes | Part of the AccessKey pair that provides access to APIs and resources. | | Add to RBAC Groups (Optional) | Lists the Role-Based Access Control (RBAC) groups to choose from for access to this account. | No | | # DNS Server Configurations for the Aviatrix Controller Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/platform-administration/copilot-dns-controller This page provides guidance on configuring the DNS server for the Aviatrix Controller. The Aviatrix Controller requires DNS resolution to communicate with various services, including: Licensing, CoPilot services, telemetry services, Public APIs, and optional integrations. This page provides guidance on configuring the DNS server for the Aviatrix Controller. The Aviatrix Controller requires DNS resolution to communicate with various services, including: Licensing, CoPilot services, telemetry services, Public APIs, and optional integrations. By default, the Controller uses Google Public DNS servers. This ensures that DNS resolution is reliable and independent of customer-specific network configurations. However, in environments with private DNS requirements or restricted outbound access, using the DNS server provided by the VPC/VNet DHCP option may be more appropriate. ## Configure DNS server for the Controller To configure the DNS server for the Controller: 1. Go to **Settings > Configuration > General** in the Aviatrix CoPilot user interface. 2. Locate the **DNS Server for Controller** tile. ## Configuration Options ### Aviatrix Default DNS Server (Default) The Aviatrix Default DNS server uses Google Public DNS servers. This option is suitable for environments with unrestricted internet access and ensures the Controller has consistent and reliable DNS resolution regardless of customer infrastructure. ### Cloud VPC/VNet DNS Server This option uses DNS servers configured through the VPC/VNet DHCP options. By enabling this option, Aviatrix Controller can leverage the DNS servers provided by the Virtual Private Cloud (VPC) or Virtual Network (VNet), ensuring that DNS queries are resolved within the cloud infrastructure. This setup is particularly beneficial for environments where Controller need to interact with cloud-based resources and services. Recommended for environments that require: * Resolution of private domain names. * DNS routing through internal enterprise infrastructure * Compliance with organizational policies that restrict access to public DNS servers ## Related Topics * [DNS Server Configuration for Aviatrix Services](/docs/enterprise/10.1/guides/platform-administration/dns-server-general) * [Aviatrix Service Time Synchronization Policy](/docs/enterprise/10.1/reference/controlplane/time-sync-policy) # CoPilot SAML Authentication Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/platform-administration/copilot-saml-authentication This guide explains how to use Aviatrix CoPilot to configure your Aviatrix Controller to authenticate to an IdP. When SAML is used for Controller access authentication, the Controller acts as the Identity Service Provider (ISP) that redirects browser traffic from the client to the IdP (e.g.,... ## Overview This guide explains how to use Aviatrix CoPilot to configure your Aviatrix Controller to authenticate to an IdP. When SAML is used for Controller access authentication, the Controller acts as the Identity Service Provider (ISP) that redirects browser traffic from the client to the IdP (e.g., Okta) for authentication. CoPilot redirects to the Controller, which performs the SAML integration with the IdP. All CoPilot user logins are directed to the Controller, which handles all accounts and users. The Aviatrix CoPilot SAML login supports multiple SAML endpoints with varying access and different IdPs. Links to each IdP integration are provided below. > [!NOTE] > > [Setting up SAML authentication for the VPN client](/docs/enterprise/10.1/guides/uservpn/user-vpn-saml-authentication) > is something separate, although the interfaces are similar. ## SAML Configuration Checklist Before configuring SAML integration between Aviatrix and an IdP, make sure the following is completed: * The Aviatrix CoPilot associated with your Controller is up and running * You have a valid IdP account with admin access An IdP refers to an identity provider for SAML. This could be any provider that supports a SAML endpoint such as Okta, OneLogin, or Azure AD. You will require administrator access to create IdP endpoints for SAML. Check [IdP-specific SAML Integration](#idp-specific-saml-app-integration) to see a list of guides for supported IdP's. ## Configuring SAML Authentication Follow these steps to configure Aviatrix to authenticate against an IdP: 1. [Creating a Temporary Aviatrix SAML Endpoint](#creating-a-temporary-aviatrix-saml-endpoint). 2. [Creating a SAML App for Aviatrix CoPilot with the IdP](#creating-a-saml-app-for-aviatrix-copilot-with-the-idp). 3. [Retrieving IdP Metadata](#retrieving-idp-metadata). 4. [Updating the Aviatrix SAML Endpoint](#updating-the-aviatrix-saml-endpoint) with IdP metadata. 5. [Validating the integration setup](#validating-the-integration). ### Creating a Temporary Aviatrix SAML Endpoint This creates the SAML endpoint on the Controller. > [!NOTE] > > This step is usually completed by the Aviatrix administrator. This endpoint > will be updated later on in the guide. At this step, this guide uses > placeholder values. Choose an endpoint name for your Aviatrix SAML endpoint which will be used throughout the guide. This guide will use `aviatrix_saml_copilot` as an example for the endpoint name. **CoPilot UI** 1. Log in to Aviatrix CoPilot. 2. Go to Administration > User Access > Access Management. 3. Under Login Authentication, click **+SAML Endpoint**. ![image3-1-1](https://mintcdn.com/aviatrix-14b37c43/k65bKcXF8onWe87v/images/guides/platform-administration/saml-image3-1-1.png?fit=max\&auto=format\&n=k65bKcXF8onWe87v\&q=85\&s=88eec40ab71421a143d595daba5ac0e9) The Create SAML Endpoint dialog displays. ![Create SAML Endpoint dialog](https://mintcdn.com/aviatrix-14b37c43/k65bKcXF8onWe87v/images/guides/platform-administration/saml-image3-1-2.png?fit=max\&auto=format\&n=k65bKcXF8onWe87v\&q=85\&s=10b947f5d031d37e98a1d60e326c0eee) 4. Enter the following information: > [!NOTE] > > You can create the Aviatrix SAML endpoint using a placeholder or invalid IdP Metadata URL as the Identity Provider Metadata type, and the system will accept it. However, when authentication is attempted, an error message will appear because the metadata cannot be parsed. > > To avoid this, make sure to update the SAML endpoint with a valid IdP Metadata > URL or XML from your Identity Provider before testing or enabling SAML login. > > Placeholder values such as `https://www.google.com` are intended for initial > setup or demonstration purposes only. | Field | Value | | - | - | | Name | Enter a unique identifier for the service provider. | | Identity Provider Metadata Type | Select **URL** or **Text**. This is copied from the SAML provider configuration. For now, put in a placeholder URL, such as "[https://www.google.com](https://www.google.com)". If you select **Text**, you must enter the text and then any related xml code. To avoid this, make sure to update the SAML endpoint with a valid IdP Metadata URL or XML from your Identity Provider before testing or enabling SAML login. | | Entity ID | You can select **Hostname** or **Custom**. Select Hostname for now. If you select **Custom** you must enter a Custom Entity ID. | | Access Set By | Select **Controller** or **SAML Identity Provider Attribute**. If you select Controller, you must select a Permission Group. If you select SAML Identity Provider Attribute, you can choose to Block Empty Profiles. This prevents users who do not have a profile assignment sent by the IdP from accessing CoPilot. | | Sign Auth Requests | Sign the certificate when requesting access to the IdP from the client. Turn this setting on to have Auth Requests sent from the Controller to the IdP signed by the Controller. The same Controller webserver certificate will be used to sign the request. The certificate is exported as part of the SP metadata. Turn this setting off if you do not need this setup for security or compliance. | | Custom SAML Request Template | For now leave blank. Depending on your specific IdP, you may have to check this option. If so, replace the sample template with your own template. | > [!NOTE] > > Each endpoint only supports one type of access. If you need admin and > read-only access, create two separate SAML apps. 5. Click **Save**. 6. Depending on your IdP provider, you may need to upload SP metadata. 1. After the temporary SAML endpoint is created, click the vertical ellipsis ![threedot](https://mintcdn.com/aviatrix-14b37c43/mOZ0lmP7pRPIryx3/images/guides/platform-administration/threedot.png?fit=max\&auto=format\&n=mOZ0lmP7pRPIryx3\&q=85\&s=6ed1c085429177f16d2735e2406d15c3) icon and select **Download SP Metadata** next to the SAML endpoint. 2. Copy the SP metadata as text. **Terraform** Create the SAML endpoint with the [aviatrix\_saml\_endpoint](https://registry.terraform.io/providers/AviatrixSystems/aviatrix/latest/docs/resources/aviatrix_saml_endpoint) resource. > [!NOTE] > > Unlike the CoPilot UI, which accepts a placeholder or invalid IdP Metadata URL > and only fails when authentication is later attempted, the Terraform provider > validates the metadata when the resource is created. Creating this resource > with a placeholder value such as `https://www.google.com` fails with > `[AVXERR-SAML-0011] The SAML endpoint ... cannot be enabled`. Retrieve real > metadata from your IdP (see [Creating a SAML App for Aviatrix CoPilot with the > IdP](#creating-a-saml-app-for-aviatrix-copilot-with-the-idp)) before creating > this resource with Terraform. **1. Create the SAML endpoint** Once you have real IdP metadata, add an `aviatrix_saml_endpoint` resource block to your Terraform configuration file. Using `idp_metadata` with a local metadata file: ```hcl theme={null} resource "aviatrix_saml_endpoint" "aviatrix_saml_copilot" { endpoint_name = "aviatrix_saml_copilot" idp_metadata_type = "Text" idp_metadata = file("idp_metadata.xml") } ``` Or, using `idp_metadata_url` pointing at your IdP's metadata endpoint: ```hcl theme={null} resource "aviatrix_saml_endpoint" "aviatrix_saml_copilot" { endpoint_name = "aviatrix_saml_copilot" idp_metadata_type = "URL" idp_metadata_url = "https://dev-xyzz.okta.com/app/exampleapp/sso/saml/metadata" } ``` `idp_metadata` and `idp_metadata_url` cannot be set at the same time. **2. Update the IdP metadata later, if needed** If your IdP rotates its signing certificate or metadata, update the `idp_metadata` or `idp_metadata_url` value and reapply. **3. Verify and Apply the Configuration** Run `terraform plan`, review the planned changes, and run `terraform apply`. ## Related Resources * [aviatrix\_saml\_endpoint Resource](https://registry.terraform.io/providers/AviatrixSystems/aviatrix/latest/docs/resources/aviatrix_saml_endpoint) ### Creating a SAML App for Aviatrix CoPilot with the IdP > [!NOTE] > > This step is usually done by the IdP administrator. This section shows only a > generalized process for creating a SAML application. Create a SAML 2.0 app in the IdP Provider (for example, Okta) with the following values from the SAML endpoint you created above: * Assertion Consumer Service URL: to obtain this, in Aviatrix CoPilot go to Administration > User Access > Access Management, click the vertical ellipsis ![threedot](https://mintcdn.com/aviatrix-14b37c43/mOZ0lmP7pRPIryx3/images/guides/platform-administration/threedot.png?fit=max\&auto=format\&n=mOZ0lmP7pRPIryx3\&q=85\&s=6ed1c085429177f16d2735e2406d15c3) next to the SAML endpoint, and click **Copy Assertion Consumer Service URL**. * Audience URI (Entity ID) * SP Metadata URL * SP Login URL * Default RelayState = `` The following SAML attributes are expected: * FirstName * LastName * Email (unique identifier for SAML) > [!NOTE] > > These values are case-sensitive. **IdP-specific SAML App Integration** > [!NOTE] > > You require administrator access to create IdP endpoints for SAML. These are guides with specific IdP's that were tested to work with Aviatrix SAML integration: * [Azure AD](/docs/enterprise/10.1/guides/uservpn/user-vpn-saml-authentication-entra-id) * [Okta](/docs/enterprise/10.1/guides/uservpn/user-vpn-saml-authentication-okta) * [OneLogin](/docs/enterprise/10.1/guides/uservpn/user-vpn-saml-authentication-onelogin) ### Retrieving IdP Metadata After creating the SAML app in the IdP as per the previous procedure, you need to retrieve IdP Metadata either in URL or text form from this SAML application. * Azure AD - provides IdP metadata URL and needs a custom SAML request template * Okta - provides IdP metadata URL * OneLogin - provides IdP metadata URL ### Updating the Aviatrix SAML Endpoint > [!NOTE] > > This step is usually completed by the Aviatrix administrator. Take note of the > IdP Metadata type along with Text/URL your IdP provides, and if you need a > custom SAML request template in the previous section. 1. In Aviatrix CoPilot go to Administration > User Access > Access Management. 2. Under SAML, click the Edit icon ![edit icon](https://mintcdn.com/aviatrix-14b37c43/k65bKcXF8onWe87v/images/guides/platform-administration/edit-icon.png?fit=max\&auto=format\&n=k65bKcXF8onWe87v\&q=85\&s=aee2b27eac6c0901d79c564f707689af) next to the SAML endpoint you created earlier. 3. Paste the IdP Metadata URL derived from the SAML provider application into the **Identity Provider Metadata URL** field. 4. Click **Save**. ### Validating the Integration 1. Log out of Aviatrix CoPilot. 2. Choose your SAML endpoint name from the dropdown box. 3. Log in to Aviatrix CoPilot by selecting the SAML Provider and clicking **Sign In with SAML**. ![SAML login dialog](https://mintcdn.com/aviatrix-14b37c43/k65bKcXF8onWe87v/images/guides/platform-administration/saml-image3-6.png?fit=max\&auto=format\&n=k65bKcXF8onWe87v\&q=85\&s=a71726fd8d05c11ae0f7643661265b90) 4. You should be redirected to the IdP. Log in with your test user credentials. > [!TIP] > > If everything is configured correctly, after you have authenticated, you will > be redirected to the CoPilot dashboard. # Use Azure IAM Custom Role Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/platform-administration/custom-role-azure When Aviatrix Controller uses Azure API to manage networking and gateway resources, an application must be first created in Azure AD with an identity of Service Principal. ## Use Azure IAM Custom Role When Aviatrix Controller uses Azure API to manage networking and gateway resources, an application must be first created in Azure AD with an identity of Service Principal. This service principal requires an Azure IAM role assignment together with a set of permissions required by the Aviatrix Controller to provide service. By default, we use the Azure built-in "Contributor" role. The Contributor role has access to all resources of the subscription. If you wish to limit the Controller access permissions, you can do so by creating a custom role with a set of permissions required by the Controller as shown below. This document describes how to accomplish this task through the Azure portal. ## Apply Azure Role-Based Access Control (RBAC) to an Aviatrix Azure Account The Aviatrix Controller is viewed as an application running on Azure. Since this application needs to create or program Azure resources, such as launching a gateway, modifying route entries in a route table, etc., the application requires a role with certain permissions. By default, this role is a pre-defined Azure built-in role called "Contributor." If you wish not to use the Contributor role and instead creating a custom role with Aviatrix-provided permission, you can do so via Azure portal or with via PowerShell. Below is guide on how to accomplish that via PowerShell. > [!NOTE] > > For security purposes, Aviatrix recommends you use a custom role rather than > the default role Aviatrix created. When you use a custom role name it is > important to make sure the AssumeRole policy and Trust policy are correct. The > AssumeRole policy is attached to the Azure Virtual Machine role and the Trust > policy is accessed on the APP role Trust Relationship tab. For replacing the Contributor role via Azure portal, refer to [Azure IAM Custom Role](#replacing-the-contributor-role). ### Step 1. Add a Custom Role through Powershell The custom role must have permission that meets the requirement for Aviatrix Controller to function. The permission is represented by the json file below. Remember to replace the subscription "11111111-1111-1111-1111-111111111111" with your own valid subscription ID. avx\_rbac\_role.json: ```json theme={null} { "Name": "Aviatrix Controller Custom Role", "IsCustom": true, "Description": "Custom role for Aviatrix Controller", "Actions": [ "Microsoft.MarketplaceOrdering/offerTypes/publishers/offers/plans/agreements/*", "Microsoft.Compute/*/read", "Microsoft.Compute/availabilitySets/*", "Microsoft.Compute/virtualMachines/*", "Microsoft.Compute/disks/*", "Microsoft.Network/*/read", "Microsoft.Network/publicIPAddresses/*", "Microsoft.Network/networkInterfaces/*", "Microsoft.Network/networkSecurityGroups/*", "Microsoft.Network/loadBalancers/*", "Microsoft.Network/routeTables/*", "Microsoft.Network/virtualNetworks/*", "Microsoft.Storage/storageAccounts/*", "Microsoft.Resources/*/read", "Microsoft.Resourcehealth/events/*", "Microsoft.Resources/deployments/*", "Microsoft.Resources/tags/*", "Microsoft.Resources/marketplace/purchase/action", "Microsoft.Resources/subscriptions/resourceGroups/*", "Microsoft.ContainerService/managedClusters/listClusterUserCredential/action", "Microsoft.ContainerService/managedClusters/read" ], "NotActions": [], "AssignableScopes": ["/subscriptions/11111111-1111-1111-1111-111111111111"] } ``` In Powershell, perform the following: Edit avx\_rbac\_role.json, copy and paste above Aviatrix RBAC role. Save the file. ```powershell theme={null} New-AzRoleDefinition -InputFile avx_rbac_role.json ``` > [!NOTE] > > The Microsoft.ContainerService lines in the above json file example allow the > Aviatrix Controller to connect to your Kubernetes clusters. ### Step 2. Add a Role Assignment in the Azure Portal In Azure portal > Subscriptions > Access Control (IAM) > Add > Add role assignment. At Role assignment, fill the fields as shown below. | Field | Value | | - | - | | Role | Aviatrix Controller Custom Role (this is the role created from above) | | Assign access to | User, group, or service principal | | Select | My-new-controller (this is the registered application name for the Controller) | Once the above step is complete, you have assigned the My-new-controller (as a service principal) the custom role called "Aviatrix Controller Custom Role." For more information on how to PowerShell to create custom role on Azure, refer to [this link](https://learn.microsoft.com/en-us/azure/role-based-access-control/custom-roles-powershell). ## Aviatrix Required Custom Role Permissions ```json theme={null} { "properties": { "roleName": "Aviatrix Controller Custom Role", "description": "Custom role for Aviatrix Controller", "assignableScopes": [], "permissions": [ { "actions": [ "Microsoft.MarketplaceOrdering/offerTypes/publishers/offers/plans/agreements/*", "Microsoft.Compute/*/read", "Microsoft.Compute/availabilitySets/*", "Microsoft.Compute/virtualMachines/*", "Microsoft.Compute/disks/*", "Microsoft.Network/*/read", "Microsoft.Network/publicIPAddresses/*", "Microsoft.Network/networkInterfaces/*", "Microsoft.Network/networkSecurityGroups/*", "Microsoft.Network/loadBalancers/*", "Microsoft.Network/routeTables/*", "Microsoft.Network/virtualNetworks/*", "Microsoft.Storage/storageAccounts/*", "Microsoft.Resources/*/read", "Microsoft.Resourcehealth/events/*", "Microsoft.Resources/deployments/*", "Microsoft.Resources/tags/*", "Microsoft.Resources/marketplace/purchase/*", "Microsoft.Resources/subscriptions/resourceGroups/*", "Microsoft.ContainerService/managedClusters/listClusterUserCredential/action", "Microsoft.ContainerService/managedClusters/read" ], "notActions": [], "dataActions": [], "notDataActions": [] } ] } } ``` For Azure China, remove "Microsoft.MarketplaceOrdering/offerTypes/publishers/offers/plans/agreements/" and "Microsoft.Resources/marketplace/purchase/" from "actions". > [!NOTE] > > The Microsoft.ContainerService lines in the above json file example allow for > the discovery of your AKS clusters. ## Creating a Custom Role Log in to the Azure portal. Go to Subscriptions. Select the subscription whose network already managed by Aviatrix Controller. Click **Access control (IAM)**. Click **Roles**, as shown below. ![iam-role](https://mintcdn.com/aviatrix-14b37c43/k65bKcXF8onWe87v/images/guides/platform-administration/iam-role.png?fit=max\&auto=format\&n=k65bKcXF8onWe87v\&q=85\&s=a1bbd99ae22c44c200903d139982d280) Click **+Add** and select **Add custom role**. Select **Start from scratch** and click **Next**, as shown below. ![start-from-scratch](https://mintcdn.com/aviatrix-14b37c43/mOZ0lmP7pRPIryx3/images/guides/platform-administration/start-from-scratch.png?fit=max\&auto=format\&n=mOZ0lmP7pRPIryx3\&q=85\&s=73c03bd00bcd385ac7a59f2aa1c5f807) Click **JSON**, and then click **Edit**. Remove the existing JSON template and copy and paste the above Aviatrix required permissions JSON into the Editor box, as shown below. ![click-json](https://mintcdn.com/aviatrix-14b37c43/k65bKcXF8onWe87v/images/guides/platform-administration/click-json.png?fit=max\&auto=format\&n=k65bKcXF8onWe87v\&q=85\&s=6b9aec703108030de164ecc251facaf6) Click **Save**. Click **Permissions**. You should see the permissions have been populated, as shown below. ![aviatrix-custom-role](https://mintcdn.com/aviatrix-14b37c43/k65bKcXF8onWe87v/images/guides/platform-administration/aviatrix-custom-role.png?fit=max\&auto=format\&n=k65bKcXF8onWe87v\&q=85\&s=eb8cd50309355d835275c7135b09a037) ![show-permission](https://mintcdn.com/aviatrix-14b37c43/mOZ0lmP7pRPIryx3/images/guides/platform-administration/show-permission.png?fit=max\&auto=format\&n=mOZ0lmP7pRPIryx3\&q=85\&s=ed4c85a43e6e09f278d7260996fdbf35) Click **Assignable scopes** and then **Add assignable scopes**. Select the subscription. Click **JSON**. You should say the subscription has been added to the assignable Scopes, as shown below. ![subscription-scope](https://mintcdn.com/aviatrix-14b37c43/mOZ0lmP7pRPIryx3/images/guides/platform-administration/subscription-scope.png?fit=max\&auto=format\&n=mOZ0lmP7pRPIryx3\&q=85\&s=5dcb6d19cb633020f2c8dc6f682bc583) Click **Review + create**, and then click **Create**. ## Replacing the Contributor Role This step is optional. It is only applicable if you have already assigned the "Contributor" role to the Aviatrix Controller service principal. If not, skip this step and proceed to the next step. Now that you have created a custom role called Aviatrix Controller Custom Role, go ahead replace the Contributor role, as shown below. ![remove\_contributor](https://mintcdn.com/aviatrix-14b37c43/k65bKcXF8onWe87v/images/guides/platform-administration/remove-contributor.png?fit=max\&auto=format\&n=k65bKcXF8onWe87v\&q=85\&s=0d92904fd309b06d80fb4a5eba82656f) Click **+Add**, select **Add role assignment**. Fill in the fields as shown below. ![replace\_role](https://mintcdn.com/aviatrix-14b37c43/k65bKcXF8onWe87v/images/guides/platform-administration/replace-role.png?fit=max\&auto=format\&n=k65bKcXF8onWe87v\&q=85\&s=3c53684882a346baae458cef976ecd4a) ## Multiple Custom Roles Approach The Aviatrix role permissions can be split into multiple custom roles each with a subset of permissions. Subscription permission must be at the subscription scope. The additional permission may have the scope of one or more Resource Groups. Below is an example where the "Aviatrix Custom Role for subscription" has the scope of subscription and the remaining permissions has the scope of Resource Group. ### Subscription Scope IAM Custom Role ```json theme={null} { "properties": { "roleName": "Aviatrix Custom Role for subscription", "description": "Aviatrix Custom role for gateway subscription permission", "assignableScopes": [], "permissions": [ { "actions": [ "Microsoft.MarketplaceOrdering/offerTypes/publishers/offers/plans/agreements/*" ], "notActions": [], "dataActions": [], "notDataActions": [] } ] } } ``` ## Resource Group Scope IAM Custom role Note when creating a custom role for a resource group on Azure portal, start at Subscription > Resource groups, select one resource group, and click **Access Control (IAM)**. Then, follow the role creation process with the permission described in the file below to create the role. When configuring Assignable scopes, select one or more resource groups (it is multi-selectable) for this role. After the role is created, assign the role to the Service principal of the Aviatrix Controller application. > [!NOTE] > > It takes a few minutes for the display to appear for the custom role just > created. Once it can be displayed, you can find it by going to Subscription > > Resource groups > select one resource group assigned to the role, then click > **Access Control (IAM)**, then click **Roles**. Then search for the role you > just created. ```json theme={null} { "properties": { "roleName": "Aviatrix Custom Role for services", "description": "Aviatrix Custom role for the network and gateway services", "assignableScopes": [], "permissions": [ { "actions": [ "Microsoft.Compute/*/read", "Microsoft.Compute/availabilitySets/*", "Microsoft.Compute/virtualMachines/*", "Microsoft.Compute/disks/*", "Microsoft.Network/*/read", "Microsoft.Network/publicIPAddresses/*", "Microsoft.Network/networkInterfaces/*", "Microsoft.Network/networkSecurityGroups/*", "Microsoft.Network/loadBalancers/*", "Microsoft.Network/routeTables/*", "Microsoft.Network/virtualNetworks/*", "Microsoft.Storage/storageAccounts/*", "Microsoft.Resources/*/read", "Microsoft.Resourcehealth/events/*", "Microsoft.Resources/deployments/*", "Microsoft.Resources/tags/*", "Microsoft.Resources/marketplace/purchase/*", "Microsoft.Resources/subscriptions/resourceGroups/*" ], "notActions": [], "dataActions": [], "notDataActions": [] } ] } } ``` If you wish to use Contributor role for the above part of the permission, ignore the JSON file listed above. Simply navigate to the Azure portal > Resource groups > select the resource group. Click **Access Control (IAM)** > **+Add** > **Add Role assignment**. Then, select **Contributor** as the Role and assign the Contributor role to the Aviatrix Controller service principal. ## Additional References To learn more on Azure custom role and how to configure it, see [Azure Custom Roles](https://learn.microsoft.com/en-us/azure/role-based-access-control/custom-roles). To view the complete Azure role permissions, refer to [Azure resource provider operations](https://learn.microsoft.com/en-us/azure/role-based-access-control/resource-provider-operations). # DNS Server Configuration for Aviatrix Services Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/platform-administration/dns-server-general This document provides an overview of how DNS (Domain Name System) servers are configured within Aviatrix services. DNS is a critical infrastructure component that translates domain names into IP addresses, enabling communication between Aviatrix components and external systems. This document provides an overview of how DNS (Domain Name System) servers are configured within Aviatrix services. DNS is a critical infrastructure component that translates domain names into IP addresses, enabling communication between Aviatrix components and external systems. Proper DNS configuration ensures reliable name resolution, system reachability, and integration with cloud and on-premises services. ## Supported DNS Options Aviatrix supports the following types of DNS configurations: * **Aviatrix Default DNS Server**: This option is suitable for most internet-based resolution needs. * **Cloud VPC/VNet DNS Server**: The Cloud VPC/VNet DNS server is automatically assigned through the VPC or VNet DHCP option sets and is used to resolve internal services within the same cloud network. * **Custom DNS Servers**: This option uses user-defined IPv4 addresses, typically used for resolving internal domains or applying network-specific DNS policies. ## Components That Require DNS The following Aviatrix components rely on DNS servers for their core operations: ### Aviatrix Controller The Aviatrix Controller requires DNS for licensing, CoPilot integration, telemetry reporting, and external API access. * Default Setting * The Aviatrix Default DNS server uses Google Public DNS servers. * Optional Setting * **Cloud VPC/VNet DNS Server**: This option uses DNS servers configured through the VPC/VNet DHCP option set. See [DNS Server Configurations for the Aviatrix Controller](/docs/enterprise/10.1/guides/platform-administration/copilot-dns-controller) for more information. ### Aviatrix CoPilot The Aviatrix CoPilot depends on DNS to reach API endpoints, support integration services, and perform connectivity checks to cloud resources. * Default Setting * **Cloud VPC/VNet DNS Server**: This option uses cloud-assigned DNS for consistent resolution within the environment. * Optional Setting * **Custom DNS Servers**: This option allows configuration of up to two custom DNS IPv4 addresses for resolving internal or private domains. See [DNS Server Configuration for the Aviatrix CoPilot](/docs/enterprise/10.1/reference/ui/settings/configuration#configure-advanced-settings) for more information. ### Aviatrix Gateways Gateways use DNS servers to resolve hostnames used in SmartGroups, logging, egress policies, and communication with cloud services. * Default Setting * The Aviatrix Default DNS server uses Google Public DNS servers. * Optional Setting * **Cloud VPC/VNet DNS Server**: The Cloud VPC/VNet DNS Server is automatically configured via cloud DHCP. See [DNS Server Configuration for Aviatrix Gateways](/docs/enterprise/10.1/reference/gateways/transit-spoke-dns) for more information. ### Hostname Resource-Type SmartGroups For SmartGroups where the resource type is set to `Hostname`, DNS is required to evaluate and resolve hostnames used in security and routing policies. * Default Setting * **Gateway's Management DNS Server**: Each gateway uses its own individually configured DNS setting. * Optional Setting * **Custom DNS Servers**: This option supports up to two DNS IPv4 addresses. These settings are applied to all gateways that enforce the SmartGroup. See [DNS Server for Hostname Resolution (Hostname SmartGroups)](/docs/enterprise/10.1/concepts-architectures/components/dcf/hostname-smartgroups-about#dns-server-configuration) for more information. ## DNS Setting Summary | Component | Default Setting | Option Settings | | - | - | - | | Controller | Aviatrix Default DNS | Cloud VPC/VNet DNS | | CoPilot | Cloud VPC/VNet DNS | Custom DNS Servers | | Gateways | Aviatrix Default DNS | Cloud VPC/VNet DNS, Custom DNS Servers | | Hostname Resource-Type SmartGroups | Gateway Management DNS Server | Custom DNS Servers | # GCP Account Onboarding Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/platform-administration/gcp-account-onboarding Before creating a cloud account for Google Cloud (GCP) on Aviatrix CoPilot, follow the steps below to make sure you have the credentials set up for API calls. ## GCP Credentials Before creating a cloud account for Google Cloud (GCP) on Aviatrix CoPilot, follow the steps below to make sure you have the credentials set up for API calls. 1. [Create a Google Cloud account](https://cloud.google.com/). Continue to the next step if you have already done so. > [!NOTE] > > CoPilot supports multiple accounts with each account associated with a > different Google Cloud project, but there needs to be at least one account > to start with. 2. [Create a Google Cloud Project](https://console.cloud.google.com/project). Continue to the next step if you have already created one. Note that the project ID will be used in referencing the project by Aviatrix CoPilot. (As an example, we created a project Aviatrix-UCC. The project ID is aviatrix-ucc-1214.) ## Required APIs Enable the following Google Cloud APIs on the project before you onboard the account: * **Compute Engine API** * **Cloud Resource Manager API** * **Cloud Logging API** The following APIs are optional and are required only if you use the corresponding feature: * **Kubernetes Engine API**: required only if you connect GKE clusters. * **Cloud Functions API**: required only if you use Cloud Functions. * **Cloud Run Admin API**: required only if you use Cloud Run. * **Serverless VPC Access API**: required only if you use Cloud Functions or Cloud Run with a VPC connector. ## Enable Compute Engine API on the Selected Project 1. Go to your Google Cloud Platform console, click on the dropdown menu in the top left, and select **APIs and Services**. At the Dashboard, click on **Enable APIs and Services**. ![Google Cloud Enable APIs and Services](https://mintcdn.com/aviatrix-14b37c43/k65bKcXF8onWe87v/images/guides/platform-administration/gcloud-enable-apis-and-services.png?fit=max\&auto=format\&n=k65bKcXF8onWe87v\&q=85\&s=648727ac973132ecdd32317b44bb7804) 2. On the Search box, enter "Compute Engine API" and select it from search results. ![Google Cloud API Library Search](https://mintcdn.com/aviatrix-14b37c43/k65bKcXF8onWe87v/images/guides/platform-administration/gcloud-api-library-search.png?fit=max\&auto=format\&n=k65bKcXF8onWe87v\&q=85\&s=fab9967885bb7cf288411f333eb39d03) 3. Click **Enable**. ## Create a Credential File When you create a cloud account on the Aviatrix Controller for Google Cloud, you will upload a Google Cloud Project Credentials file. Follow the steps below to download the credential file from the Google Developer Console. 1. Open the [Credential page](https://console.developers.google.com/project/_/apiui/credential). 2. Select the project you are creating credentials for. 3. At Credentials, click **Create credentials** and select **Service account** as shown below. ![service\_account](https://mintcdn.com/aviatrix-14b37c43/mOZ0lmP7pRPIryx3/images/guides/platform-administration/service-account.png?fit=max\&auto=format\&n=mOZ0lmP7pRPIryx3\&q=85\&s=65512ec5253c7ac132e45d294b25b31c) 4. At the Service Accounts, enter a service account name and click **Create**. For Service account permissions, select Project, Editor, as shown below. ![iam\_credential](https://mintcdn.com/aviatrix-14b37c43/k65bKcXF8onWe87v/images/guides/platform-administration/iam-credential.png?fit=max\&auto=format\&n=k65bKcXF8onWe87v\&q=85\&s=5fd85cfaecacb42f4d2608e118f05cc2) 5. Select a service account and then select the Keys tab 6. Click the **Add Key** dropdown menu, and select **Create new key**. 7. Select the **JSON** radio button and click **Create**. 8. Click **Create**. The credential file downloads to your local computer. 9. Upload the Project Credential file to the Aviatrix Controller at the Google Cloud Platform account create page. ## Create a Service Account with Restricted Access We recommend creating the service account with the Editor role, but in some cases an organization might want to further restrict permission for the service account. In such a situation, Aviatrix recommends having at least the following roles assigned to the service account. These roles allow Aviatrix to perform its functions properly, such as managing the compute resources, route tables, firewall rules, shared service VPC network, etc. 1. Compute Admin 2. Service Account User 3. Organization Administrator (required for GCP Shared VPC) 4. Project IAM Admin (required for GCP Shared VPC) ![Restricted Access](https://mintcdn.com/aviatrix-14b37c43/k65bKcXF8onWe87v/images/guides/platform-administration/restricted-access.png?fit=max\&auto=format\&n=k65bKcXF8onWe87v\&q=85\&s=eac736581ac770f950ec12704ca91465) If an organization is currently using GCP Shared VPC or planning to use it in it the future, then enabling Organization Administrator and Project IAM Admin is also required. In addition to restricting the GCP roles, you can restrict the rights for those roles. You can grant roles permission to perform the following tasks: ```text theme={null} compute.addresses.create compute.addresses.createInternal compute.addresses.delete compute.addresses.deleteInternal compute.addresses.get compute.addresses.list compute.addresses.use compute.addresses.useInternal compute.disks.create compute.disks.get compute.firewalls.create compute.firewalls.delete compute.firewalls.get compute.firewalls.list compute.firewalls.update compute.forwardingRules.create compute.forwardingRules.delete compute.forwardingRules.list compute.globalOperations.get compute.healthChecks.create compute.healthChecks.delete compute.healthChecks.useReadOnly compute.httpHealthChecks.get compute.httpHealthChecks.useReadOnly compute.images.create compute.images.list compute.images.useReadOnly compute.instanceGroups.create compute.instanceGroups.delete compute.instanceGroups.get compute.instanceGroups.update compute.instanceGroups.use compute.instances.create compute.instances.delete compute.instances.get compute.instances.list compute.instances.setDeletionProtection compute.instances.setMachineType compute.instances.setMetadata compute.instances.setTags compute.instances.start compute.instances.stop compute.instances.updateNetworkInterface compute.instances.use compute.licenses.get compute.licenses.list compute.machineTypes.list compute.networks.addPeering compute.networks.create compute.networks.delete compute.networks.get compute.networks.list compute.networks.listEffectiveTags compute.networks.removePeering compute.networks.updatePolicy compute.networks.use compute.projects.get compute.projects.setCommonInstanceMetadata compute.regionBackendServices.create compute.regionBackendServices.delete compute.regionBackendServices.get compute.regionBackendServices.list compute.regionBackendServices.update compute.regionBackendServices.use compute.regionOperations.get compute.routes.create compute.routes.delete compute.routes.list compute.subnetworks.create compute.subnetworks.delete compute.subnetworks.get compute.subnetworks.list compute.subnetworks.use compute.subnetworks.useExternalIp compute.targetPools.addInstance compute.targetPools.create compute.targetPools.delete compute.targetPools.get compute.targetPools.removeInstance compute.targetPools.use compute.zoneOperations.get compute.zones.list iam.serviceAccounts.actAs logging.logEntries.create pubsub.subscriptions.consume pubsub.subscriptions.create pubsub.subscriptions.delete pubsub.subscriptions.get pubsub.topics.attachSubscription pubsub.topics.create pubsub.topics.delete pubsub.topics.get pubsub.topics.publish resourcemanager.projects.get ``` If you want to monitor egress traffic on a GCP cloud account, ensure that the following APIs are enabled: * container: `container.googleapis.com` * cloud Resource Manager: `cloudresourcemanager.googleapis.com` For more information about requirements for monitoring egress traffic on GCP, see [Implementing Egress in an Aviatrix-Managed Network](/docs/enterprise/10.1/guides/security/egress/egress-introduction) . GCP requires the following permissions to discover and connect to GKE clusters: ```text theme={null} container.clusters.connect container.clusters.get container.clusters.list container.endpointSlices.list container.namespaces.list container.nodes.list container.pods.list container.services.list ``` For more information about Kubernetes Engine (GKE) requirements, see [Kubernetes Prerequisites and Permissions](/docs/enterprise/10.1/reference/dcf/kubernetes-prerequisites) . ## Troubleshooting Tips If the cloud account creation fails, check the error message on Aviatrix CoPilot and try again with the steps provided in this document. For additional support, please open a support ticket at the [Aviatrix Support Portal](https://support.aviatrix.com). # IP Whitelist AWS Azure Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/platform-administration/ip-whitelist-aws-azure This document describes how to configure IAM policies (AWS) and Conditional Access (Azure) to prevent a malfunctioning Controller mistakenly restored from a backup assuming control of onboarded CSP accounts. ## IP-based Whitelisting of Aviatrix Controller in AWS and Azure This document describes how to configure [IAM policies (AWS)](/docs/enterprise/10.1/reference/general/iam-role#iam-policies-aws) and [Conditional Access (Azure)](https://learn.microsoft.com/en-us/azure/active-directory/conditional-access/overview) to prevent a malfunctioning Controller mistakenly restored from a backup assuming control of onboarded CSP accounts. ### Whitelisting in AWS In your AWS Console, do the following to whitelist the Controller Elastic IP: 1. Navigate to Access Management > Policies under Identity and Access Management (IAM). ![image::controller/ip-whitelist-aws-1.png](https://mintcdn.com/aviatrix-14b37c43/k65bKcXF8onWe87v/images/guides/platform-administration/ip-whitelist-aws-1.png?fit=max\&auto=format\&n=k65bKcXF8onWe87v\&q=85\&s=7015c346a563cc0666db6a5f73e4a814) 2. Click *Create policy* and switch to the JSON editor. ![image::controller/ip-whitelist-aws-2.png](https://mintcdn.com/aviatrix-14b37c43/k65bKcXF8onWe87v/images/guides/platform-administration/ip-whitelist-aws-2.png?fit=max\&auto=format\&n=k65bKcXF8onWe87v\&q=85\&s=f601455f5dabdb2b0d3932db9320ed17) 3. Paste the following code into the JSON policy editor (replace the placeholder A.B.C.D/32 address with the Elastic IP of your Aviatrix Controller — for example, 34.216.134.10): ```json theme={null} { "Version": "2012-10-17", "Statement": [ { "Action": "*", "Resource": "*", "Effect": "Deny", "Condition": { "NotIpAddress": { "aws:SourceIp": ["A.B.C.D/32"] } } } ] } ``` 4. Click *Next: Tags*. 5. Click *Next: Review*. 6. On the Create policy page, name the policy (for example, aviatrix-controller-whitelist-policy) and give it a description (optional). ![image::controller/ip-whitelist-aws-3.png](https://mintcdn.com/aviatrix-14b37c43/k65bKcXF8onWe87v/images/guides/platform-administration/ip-whitelist-aws-3.png?fit=max\&auto=format\&n=k65bKcXF8onWe87v\&q=85\&s=a1b3cccaf63c7c9426ce1fcf5da476a6) 7. Click *Create policy*. 8. Switch to the Access management > Roles view and click the aviatrix-role-app. ![image::controller/ip-whitelist-aws-4.png](https://mintcdn.com/aviatrix-14b37c43/k65bKcXF8onWe87v/images/guides/platform-administration/ip-whitelist-aws-4.png?fit=max\&auto=format\&n=k65bKcXF8onWe87v\&q=85\&s=31f1b565454b7d12e0baa1ffd37d0c6d) 9. Select *Attach policies* from the Add permissions drop down menu. ![image::controller/ip-whitelist-aws-5.png](https://mintcdn.com/aviatrix-14b37c43/k65bKcXF8onWe87v/images/guides/platform-administration/ip-whitelist-aws-5.png?fit=max\&auto=format\&n=k65bKcXF8onWe87v\&q=85\&s=b59d2bd8d1adc26a8bbf98041faca2f8) 10. On the Add permissions page select the whitelist policy you just created and click *Attach policies*. ![image::controller/ip-whitelist-aws-6.png](https://mintcdn.com/aviatrix-14b37c43/k65bKcXF8onWe87v/images/guides/platform-administration/ip-whitelist-aws-6.png?fit=max\&auto=format\&n=k65bKcXF8onWe87v\&q=85\&s=4f84c919d86663d458cdf69e4eae6beb) 11. You should now see two policies attached to the aviatrix-role-app: ![image::controller/ip-whitelist-aws-7.png](https://mintcdn.com/aviatrix-14b37c43/k65bKcXF8onWe87v/images/guides/platform-administration/ip-whitelist-aws-7.png?fit=max\&auto=format\&n=k65bKcXF8onWe87v\&q=85\&s=7207f67fa086eb0246986d24b883a658) Your AWS account is now protected from unauthorized IP access and rogue Controllers. #### Updating the Whitelist Policy You should update the whitelist policy when you: * rebuild the Controller with a different Elastic IP * Invoke the Controller IP migration process You can append the Elastic IP of the new Controller to the existing policy as shown in the following code snippet (replacing the placeholder IP addresses): ```json theme={null} { "Version": "2012-10-17", "Statement": [ { "Action": "*", "Resource": "*", "Effect": "Deny", "Condition": { "NotIpAddress": { "aws:SourceIp": [ "A.B.C.D/32", <-- Old Controller EIP, note the trailing comma "W.X.Y.Z/32" <-- New Controller EIP ] } } } ] } ``` Delete the comments before saving the policy. > [!NOTE] > > Remove the old Controller Elastic IP from the policy after successful > migration or Controller rebuild. ### Whitelisting in Azure Before configuring the IP-based Controller whitelisting in Azure you must: * have the service principal set up as described in [Azure Account Credential Setup](/docs/enterprise/10.1/guides/platform-administration/account-azure). The service principal may already exist if the Azure account was previously onboarded in the Aviatrix Controller. If the service principal already exists, you can determine its name by navigating to Microsoft Entra ID > Manage > App Registrations in the Azure portal. * an [available Microsoft Entra ID Premium P1 license](https://azure.microsoft.com/en-us/pricing/details/active-directory/). Do the following to check if you already have this license: 1. Go to Microsoft Entra ID > Manage > Licenses. 2. On the Licenses page, go to Manage > All products. To configure whitelisting in Azure: 1. Navigate to Microsoft Entra ID > Conditional Access > Manage > Named Locations in the Azure portal and create a new IP ranges location with the current public IP address of the Controller. ![image::controller/ip-whitelist-azure-1.png](https://mintcdn.com/aviatrix-14b37c43/k65bKcXF8onWe87v/images/guides/platform-administration/ip-whitelist-azure-1.png?fit=max\&auto=format\&n=k65bKcXF8onWe87v\&q=85\&s=ed0dc951bfa8e7d0ea42a45f9ac5518a) 2. Navigate to Microsoft Entra ID Conditional Access > Policies in the Azure portal and create a new policy. ![image::controller/ip-whitelist-azure-2.png](https://mintcdn.com/aviatrix-14b37c43/k65bKcXF8onWe87v/images/guides/platform-administration/ip-whitelist-azure-2.png?fit=max\&auto=format\&n=k65bKcXF8onWe87v\&q=85\&s=f3bb6176b088ba28cb8bb094afa7477c) 3. Name the policy and click *Specific service principals included*. ![image::controller/ip-whitelist-azure-3.png](https://mintcdn.com/aviatrix-14b37c43/k65bKcXF8onWe87v/images/guides/platform-administration/ip-whitelist-azure-3.png?fit=max\&auto=format\&n=k65bKcXF8onWe87v\&q=85\&s=922db7640bbd1f1aa7d1ca7c24b02488) 4. In the resulting dialog, do the following: ![image::controller/ip-whitelist-azure-4.png](https://mintcdn.com/aviatrix-14b37c43/k65bKcXF8onWe87v/images/guides/platform-administration/ip-whitelist-azure-4.png?fit=max\&auto=format\&n=k65bKcXF8onWe87v\&q=85\&s=794008322754729be398c258dc7bec37) * Select *Workload identities* from the What does this policy apply to list * Select the Include tab and select the *Select service principals* radio button * Click the *None* hyperlink 5. In the resulting dialog, select the principal you created/identified at the beginning of this procedure. ![image::controller/ip-whitelist-azure-5.png](https://mintcdn.com/aviatrix-14b37c43/k65bKcXF8onWe87v/images/guides/platform-administration/ip-whitelist-azure-5.png?fit=max\&auto=format\&n=k65bKcXF8onWe87v\&q=85\&s=c798507646b30f570c1e86349431e296) 6. In the Conditional Access Policy configuration screen, click *All cloud apps* under Cloud apps or actions and then select the *All cloud apps* radio button. ![image::controller/ip-whitelist-azure-6.png](https://mintcdn.com/aviatrix-14b37c43/k65bKcXF8onWe87v/images/guides/platform-administration/ip-whitelist-azure-6.png?fit=max\&auto=format\&n=k65bKcXF8onWe87v\&q=85\&s=c42a65ef6a881893f4e2b8122f282052) 7. After the condition is created, click the hyperlink in the Conditions section and create a location-based exclusion by selecting the named location created in step 1. ![image::controller/ip-whitelist-azure-7.png](https://mintcdn.com/aviatrix-14b37c43/k65bKcXF8onWe87v/images/guides/platform-administration/ip-whitelist-azure-7.png?fit=max\&auto=format\&n=k65bKcXF8onWe87v\&q=85\&s=3919dc6717efc7131cf076eb9ec0a489) 8. To finalize the configuration, set the Access controls to *Block Access* and set Enable policy to *On*. ![image::controller/ip-whitelist-azure-8.png](https://mintcdn.com/aviatrix-14b37c43/k65bKcXF8onWe87v/images/guides/platform-administration/ip-whitelist-azure-8.png?fit=max\&auto=format\&n=k65bKcXF8onWe87v\&q=85\&s=8daaa4ec96ac590fcbbb691f44ea93a5) #### Verifying Conditional Access Policy To confirm that the Conditional Access policy has been applied successfully, you can view the logs in Microsoft Entra ID Conditional Access > Monitoring > Sign-in logs. ![image::controller/ip-whitelist-azure-9.png](https://mintcdn.com/aviatrix-14b37c43/k65bKcXF8onWe87v/images/guides/platform-administration/ip-whitelist-azure-9.png?fit=max\&auto=format\&n=k65bKcXF8onWe87v\&q=85\&s=d718136bd68bbf1bc12c28f98badf13e) # Admin Users and Duo Sign in Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/platform-administration/objective-duo This document describes a reference design using the Aviatrix Controller console's user management and Duo authentication capability to manage multiple users with admin privileges. ## Admin Users and Duo Sign in ### Objectives This document describes a reference design using the Aviatrix Controller console's user management and Duo authentication capability to manage multiple users with admin privileges. As the cloud Ops team continues to expand to manage more cloud deployments, each team member often needs their own username and password with admin privilege. In addition to a username and password for login credentials, a 2FA authentication can be added for enhanced security to manage cloud Controller. Duo authentication is one of the supported methods. When enabled, it requires the user to accept a push message on the user's mobile device from Duo service in addition to username and password at the user login time. The following diagram illustrates the user relationship in a typical cloud Ops department. In this example, the Ops team has created three secondary access accounts. An access account is associated with one or more distinct cloud provider's API credentials. Typically, a cloud account corresponds to an IAM account of a distinct AWS and/or Azure account with a credit card. A default user "admin" is created by the system. In the picture below, admin has created three secondary access accounts. Additional users in admin and access account are added by the admin or admin users. ![account-structure](https://mintcdn.com/aviatrix-14b37c43/k65bKcXF8onWe87v/images/guides/platform-administration/acct-structure.png?fit=max\&auto=format\&n=k65bKcXF8onWe87v\&q=85\&s=b58aa018d00fb983799a9088f4b88249) # Role-Based Access Control (RBAC) or User Access Overview Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/platform-administration/rbac-faq The Aviatrix Platform is a multicloud and multi-tenant enterprise platform. As such, the Aviatrix Platform manages multiple cloud accounts by requiring access by multiple administrators. ## Aviatrix User Access Management The Aviatrix Platform is a multicloud and multi-tenant enterprise platform. As such, the Aviatrix Platform manages multiple cloud accounts by requiring access by multiple administrators. This user access provides access controls to protect the security and integrity of the Aviatrix Platform while providing the ability to delegate and limit specific Aviatrix features to groups defined by the admin of the Controller. Aviatrix user access aims to achieve two objectives: * *Granular Access Control* An Aviatrix Platform administrator in a specific permission group can perform certain tasks for a subset of Aviatrix [Access Account](/docs/enterprise/10.1/guides/platform-administration/aviatrix-account) . For example, an Administrative user can be limited to perform on his own AWS account VPC attachment function. * *Self Service* An Aviatrix Platform administrator in a specific permission group can onboard its own cloud accounts on CoPilot and perform tasks. For example, a CoPilot administrator can be allowed to onboard his own AWS account in CoPilot and create a group of users for different tasks on this access account. Another use case is for developers to have a read\_only login permission to troubleshoot network connectivity issues. ## User Access in the Aviatrix Platform Role-Based Access Control (RBAC), or user access, allows you to create a hierarchy of administrators within the Aviatrix Platform. It has the flexibility to permutate based on your requirements. The best way to explain how Aviatrix Platform user access works is through examples. Below are a few deployment examples. ## User Access Deployment Example 1 In this example, the CoPilot admin creates a user, Bob, who has full responsibility to access account account-A and account-B. The CoPilot admin also creates another user, Alice, who has full responsibility to access account-C and account-D. ![rbac\_example\_1](https://mintcdn.com/aviatrix-14b37c43/k65bKcXF8onWe87v/images/guides/platform-administration/rbac-example-one.png?fit=max\&auto=format\&n=k65bKcXF8onWe87v\&q=85\&s=b7b8d786d49206c0767744bed5bf21b4) ## Tasks carried out by an Admin 1. The admin [creates an account admin group](https://legacy.docs.aviatrix.com/documentation/latest/platform-administration/accounts-and-users/users-and-permissions.html#start-help-create-permission-group). The admin gives the group a name, such as "account-admins." 2. The admin gives this group permission to create Access Accounts. 3. The admin creates a new user Bob, and adds Bob to the account\_admins group. The admin enters Bob in the name field and completes the other fields. For Permission Groups, the admin selects account-admins created in step 1. ### Tasks carried out by Bob 1. Bob should receive an email to invite him to access CoPilot. 2. Bob logs in and creates a new permission group with full access. He enters a permission group name, for example, "group-bob." 3. Bob associates himself with the Permission Group group-bob. 4. Bob grants group-bob All Write permissions. 5. Bob creates a new Access Account account-A. For Permission Groups, he selects group-bob. This creates an access account that associates a cloud account that Bob manages. For the Account Name field, Bob enters "account-A." 6. Bob can repeat the previous steps to create account-B. Now Bob has full functional access to both account-A and account-B. 7. Apply Step 3 to Step 7 for Alice to manage account-C and account-D. ### Can Bob assign a teammate with subset of functional privileges? Yes. The deployment is shown in the diagram below. ![rbac\_example\_2](https://mintcdn.com/aviatrix-14b37c43/k65bKcXF8onWe87v/images/guides/platform-administration/rbac-example-two.png?fit=max\&auto=format\&n=k65bKcXF8onWe87v\&q=85\&s=6fa627fb89126806fd6c072a58edc00f) Bob should perform the following tasks to set it up. 1. Bob creates a new permission group, such as "Site2Cloud-ops." 2. Bob assigns himself to the Site2Cloud-ops group. 3. Bob clicks **Manage permission** for Site2Cloud-ops group to select Site2Cloud permission for the group. 4. Bob clicks **Manage access accounts** for Site2Cloud-ops group to select account-A. 5. Bob creates a new user such as "Adam" and associates Adam to Site2Cloud-ops group. After the above tasks, Adam will be able to log in and perform Site2Cloud tasks for account-A. However, Adam cannot perform Site2Cloud tasks for Alice's account. ## Adding a Read Only User Read\_only user has visibility to all pages on the Aviatrix Platform and can perform troubleshooting tasks. A read\_only user cannot make modifications to any functions or accounts. In this example, Alice creates a read\_only user George. Alice performs the following steps. 1. Alice logs in and creates a new user named George. 2. Alice adds a User Name, User Email, and Password. For Permission Groups, she selects read\_only. ![rbac\_example\_3](https://mintcdn.com/aviatrix-14b37c43/k65bKcXF8onWe87v/images/guides/platform-administration/rbac-example-three.png?fit=max\&auto=format\&n=k65bKcXF8onWe87v\&q=85\&s=7fb366e12ad4c8e62d24169cbc0ba8bc) ## Adding Multiple Admin Users Yes. Only an admin can add more admin users. An admin user has the same privilege as the login admin with full access to all pages and accounts. In this example, an admin creates a new admin user, Jennifer. The admin performs the following steps. 1. The admin logs in and creates a new user, Jennifer. 2. The admin adds the User Name "Jennifer," User Email, Password. For Permission Groups, the admin selects **admin**. ![rbac\_example\_4](https://mintcdn.com/aviatrix-14b37c43/k65bKcXF8onWe87v/images/guides/platform-administration/rbac-example-four.png?fit=max\&auto=format\&n=k65bKcXF8onWe87v\&q=85\&s=6ad1ad701e01d09c5274fa581e81d5f0) ## Supporting Remote Authentications CoPilot User Access supports remote authentication against LDAP, Duo, and other SAML IDPs. For LDAP and Duo, user access for the Aviatrix Platform supports authentication only. The permissions are still validated locally on the Aviatrix Platform. For other SAML IDPs, you can configure profile attribute associated with the SAML user for permissions and avoid having to add users on the Aviatrix Platform. ## Setting up SAML Login for User Access The Aviatrix Platform login supports [SAML login.](/docs/enterprise/10.1/reference/ui/administration/user-access#access-management) You have the option of authorizing users by CoPilot configuration or through SAML IDP Attribute. See [CoPilot SAML Authentication](/docs/enterprise/10.1/guides/platform-administration/copilot-saml-authentication) . # Set Up SAML Login for CoPilot Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/platform-administration/saml-login-setup If you use an identity provider (IdP) to allow your organization's team members to log in to Aviatrix Controller via SAML, they can log in to Aviatrix CoPilot via SAML authentication also provided the following setup is in place. If you use an identity provider (IdP) to allow your organization's team members to log in to Aviatrix Controller via SAML, they can log in to Aviatrix CoPilot via SAML authentication also provided the following setup is in place. > [!NOTE] > > If you launch CoPilot from the Controller user interface, you must manually > add the CoPilot public IP address to the Controller's security group. > Otherwise, the SAML Provider option will not be visible to CoPilot users on > the CoPilot login page. To set up SAML login for CoPilot: 1. (Pre-requisite) Set up SAML login for Aviatrix Controller. This procedure assumes you have already set up your IdP configuration in the IdP application and associated the configuration in your Controller. In your IdP application, take note of the value specified for the *Single sign on URL* SAML setting to see if the SSO URL contains your Controller's public IP address or your Controller's FQDN (either one can be used but this value must match the value set in a later step). ![SAML Single Sign On URL configuration](https://mintcdn.com/aviatrix-14b37c43/0QYX2iFaLoliwUis/images/guides/platform-administration/samlsso.png?fit=max\&auto=format\&n=0QYX2iFaLoliwUis\&q=85\&s=e8b8df77575acb83ee87df48c9c6a21f) 2. In Aviatrix CoPilot > Settings > Configuration locate the *Controller Public IP/FQDN* field. ![Controller Public IP/FQDN field in CoPilot Settings](https://mintcdn.com/aviatrix-14b37c43/0QYX2iFaLoliwUis/images/guides/platform-administration/samlsso2.png?fit=max\&auto=format\&n=0QYX2iFaLoliwUis\&q=85\&s=d6c3e034cc981e7e49037ecaf14ca197) 3. Verify the value of the *Controller Public IP/FQDN* field matches the value of the public IP address or the FQDN of your Controller — depending on which one is set in the SSO URL in the SAML settings (these values must match). If needed, update the field and click *Save*. 4. Verify your Controller is associated with your CoPilot. In Controller > Settings > CoPilot > CoPilot Association, verify the slider is set to > *Enabled* and the correct IP address of the CoPilot instance is specified. * IP Address/Hostname field Enter the static IP address for your running CoPilot instance/virtual machine. This can be the private or public IP address of your CoPilot instance/virtual machine. The IP address specified here is used for connectivity between Controller and CoPilot for intra-platform communication (such as API message exchanges). If CoPilot is located in the same VPC/VNet as your Controller, specifying a private IP can increase operational bandwidth and potentially save on cost. If you enter the private IP address here, and you want to be able to open CoPilot in your web browser directly from your Controller, then specify the public IP address in the *Public IP (Optional)* field. * Public IP (Optional) field If you specified the private IP address of your CoPilot instance in the "IP Address/Hostname" field, enter the public IP address of your CoPilot instance here if you want to be able to open CoPilot in your web browser directly from your Controller. # Centralized Security Hub VPC (Early Access) Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/security/dcf/centralized-security-hub-vpc Deploy Aviatrix DCF gateways behind an AWS Gateway Load Balancer in a dedicated security hub VPC to enforce firewall policy across hub-and-spoke networks connected to a user-managed AWS Transit Gateway. > [!NOTE] > > Centralized Security Hub VPC is **Early Access** in 9.0 and is **API-only** in > this release. There is no CoPilot UI for creating or managing a security hub > VPC. To request Early Access, contact your Aviatrix account team. Enterprises running hub-and-spoke networks on AWS route internet-bound and east-west traffic through a centralized security hub VPC connected to an AWS Transit Gateway (TGW). The Centralized Security Hub VPC deploys Aviatrix Distributed Cloud Firewall (DCF) gateways into a dedicated VPC behind an AWS Gateway Load Balancer (GWLB), integrating directly with your existing user-managed TGW. This enables DCF policy enforcement without Aviatrix-managed transit gateways, reducing TGW data processing charges, inter-AZ transfer fees, and operational complexity. Aviatrix creates and manages all security hub VPC infrastructure: subnets, route tables, GWLB, GWLB endpoints (GWLBEs), internet gateway, and DCF gateways. You retain full ownership of your TGW and its route tables. ## Prerequisites * Aviatrix Controller 9.0 or later. * Gateway software 9.0 or later. * An AWS account onboarded to the Aviatrix Controller with IAM permissions to create VPCs, subnets, route tables, internet gateways, Gateway Load Balancers, GWLB endpoints, VPC endpoint services, and EC2 instances. * An existing user-managed AWS Transit Gateway with available attachment capacity. * Sufficient AWS service quotas for the target region. A 3-zone hub creates 12 subnets, 5 route tables, 3 gateways, 1 internet gateway, 3 GWLB endpoints, 1 Gateway Load Balancer, 1 VPC endpoint service, and 1 target group. ## How It Works ### Traffic Path **Egress (internet-bound):** Spoke workload → TGW → security hub VPC TGW attachment subnet → GWLBE → GWLB → DCF gateway (policy evaluation) → SNAT → internet **East-west (RFC 1918):** Spoke workload → TGW → security hub VPC → DCF gateway (policy evaluation, no NAT) → TGW → destination spoke VPC Internet-bound traffic is subject to SNAT. East-west traffic between spoke VPCs is inspected and returned directly to workloads through the TGW without NAT. ### Infrastructure Created Per Deployment For each Availability Zone you select, Aviatrix creates four subnets: | Subnet | Purpose | | - | - | | TGW attachment subnet | Hosts TGW ENIs. Default route points to the GWLBE in the same zone. | | GWLBE subnet | Hosts the Gateway Load Balancer endpoint. | | GWLB pool subnet | Hosts GWLB ENIs and the DCF gateway internal interface. Default route points to the TGW for return traffic. | | Public subnet | Hosts the DCF gateway external interface. Default route points to the internet gateway. | Aviatrix also creates one internet gateway, one Gateway Load Balancer (with cross-zone mode enabled), one VPC endpoint service, and one target group per deployment. GWLB cross-zone mode ensures symmetric session handling across three or more Availability Zones. ## Configure TGW Route Tables Aviatrix does not modify your TGW route tables. After the security hub VPC is provisioned, configure TGW routing manually to direct workload traffic to the security hub VPC. **1. Update workload VPC subnet route tables** In each workload VPC subnet route table, add a default route (`0.0.0.0/0`) pointing to the TGW attachment ENI. **2. Update the TGW egress route table** In the TGW route table associated with workload VPCs, add: * A default route (`0.0.0.0/0`) pointing to the security hub VPC TGW attachment (static route). * Propagated routes for each workload VPC CIDR pointing to their respective TGW attachments. **Example egress TGW route table after configuration:** | Destination | Target | Type | | - | - | - | | `10.1.0.0/16` | Workload VPC-A attachment | Propagated | | `10.2.0.0/16` | Workload VPC-B attachment | Propagated | | `10.3.0.0/16` | Workload VPC-C attachment | Propagated | | `0.0.0.0/0` | Security hub VPC attachment | Static | > [!NOTE] > > The security hub VPC's internal route tables (TGW attachment subnets, GWLB > pool subnets, and public subnets) are pre-configured by Aviatrix and do not > require manual changes. ## Limitations * **Early Access and API-only**: Security hub VPCs are created, modified, and deleted through the Aviatrix API in 9.0. No CoPilot UI is available in this release. * **AWS only**: Azure and GCP are not supported in this release. * **User-managed TGW route tables**: Aviatrix does not configure TGW route tables. You configure routing manually after the security hub VPC is provisioned. * **Greenfield VPC only**: The security hub VPC is provisioned by Aviatrix and cannot reuse an existing VPC that already contains third-party appliances. * **Cross-AZ traffic charges**: GWLB cross-zone mode ensures symmetric session routing but may result in cross-AZ data transfer charges. * **High AWS resource consumption**: Confirm sufficient AWS service quotas before deployment. See the prerequisites above for resource counts. ## Related Topics * [DCF Overview](/docs/enterprise/10.1/concepts-architectures/components/dcf/dcf-overview) * [DCF Policies and Rules](/docs/enterprise/10.1/guides/security/dcf/dcf-settings) # Creating Groups for Distributed Cloud Firewall Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/security/dcf/dcf-create-groups A Distributed Cloud Firewall (DCF) SmartGroup contains one or more filters to identify cloud endpoints that map to an app domain. A filter specifies resource matching criteria. ## SmartGroups A Distributed Cloud Firewall (DCF) SmartGroup contains one or more filters to identify cloud endpoints that map to an app domain. A filter specifies resource matching criteria. Matching criteria could be a cloud tag; a resource attribute (such as account name or region); a list of IP prefixes; or a Site2Cloud external connection. All conditions within the filter must be satisfied to be matched. A tag or resource attribute-based filter must be associated with a resource type (VPC/VNet, subnet, or VM). **CoPilot UI** For the steps to create SmartGroups in the CoPilot UI, see [Creating SmartGroups](/docs/enterprise/10.1/reference/ui/groups/index#create-smartgroup) . **Terraform** Define a SmartGroup with the [aviatrix\_smart\_group](https://registry.terraform.io/providers/AviatrixSystems/aviatrix/latest/docs/resources/aviatrix_smart_group) resource. Each `match_expressions` block is a filter; add more than one block to match on multiple criteria such as account name, region, tags, or a CIDR range. **1. Add the SmartGroup resource** Add the following `aviatrix_smart_group` resource block to your Terraform configuration file: ```hcl theme={null} resource "aviatrix_smart_group" "protected_vpcs" { name = "protected-vpcs" selector { match_expressions { type = "vpc" account_name = "prod-account" region = "us-east-1" tags = { env = "production" } } match_expressions { cidr = "10.0.0.0/16" } } } ``` **2. Verify and Apply the Configuration** Run `terraform plan`, review the planned changes, and run `terraform apply`. *** ## WebGroups A DCF WebGroup contains one or more domain names or URLs that assists in filtering (and providing security to) Internet-bound traffic. **CoPilot UI** For the steps to create WebGroups in the CoPilot UI, see [Creating WebGroups](/docs/enterprise/10.1/reference/ui/groups/index#create-web-group) . **Terraform** Define a WebGroup with the [aviatrix\_web\_group](https://registry.terraform.io/providers/AviatrixSystems/aviatrix/latest/docs/resources/aviatrix_web_group) resource. Use `snifilter` to match on a domain name, or `urlfilter` to match on a specific URL path. A `match_expressions` block cannot combine both filters. **1. Add the WebGroup resource** Add the following `aviatrix_web_group` resource block to your Terraform configuration file: ```hcl theme={null} resource "aviatrix_web_group" "trusted_domains" { name = "trusted-domains" selector { match_expressions { snifilter = "aviatrix.com" } } } ``` **2. Verify and Apply the Configuration** Run `terraform plan`, review the planned changes, and run `terraform apply`. *** ## ExternalGroups An ExternalGroup can contain countries, threat feeds, and SaaS-based services (Azure and GitHub). **CoPilot UI** For the steps to create ExternalGroups in the CoPilot UI, see [Managing the Relationship Between Feeds and ExternalGroups](/docs/enterprise/10.1/reference/ui/groups/index#create-external-group) . **Terraform** ExternalGroups use the same [aviatrix\_smart\_group](https://registry.terraform.io/providers/AviatrixSystems/aviatrix/latest/docs/resources/aviatrix_smart_group) resource as SmartGroups, with an `external` match expression instead of a cloud-resource filter. Set `external` to `geo`, `threatiq`, `azureips`, or `githubips`, and use `ext_args` for feed-specific arguments. **1. Add the ExternalGroup resource** Add the following `aviatrix_smart_group` resource block to your Terraform configuration file: ```hcl theme={null} resource "aviatrix_smart_group" "blocked_countries" { name = "blocked-countries" selector { match_expressions { external = "geo" ext_args = { country_iso_code = "US" } } } } ``` **2. Verify and Apply the Configuration** Run `terraform plan`, review the planned changes, and run `terraform apply`. *** ## Related Resources * [Aviatrix Terraform provider](https://registry.terraform.io/providers/AviatrixSystems/aviatrix/latest/docs) * [aviatrix\_smart\_group resource](https://registry.terraform.io/providers/AviatrixSystems/aviatrix/latest/docs/resources/aviatrix_smart_group) * [aviatrix\_web\_group resource](https://registry.terraform.io/providers/AviatrixSystems/aviatrix/latest/docs/resources/aviatrix_web_group) * [Disabling the Distributed Cloud Firewall Feature](/docs/enterprise/10.1/guides/security/dcf/dcf-disable-feature) * [Creating Distributed Cloud Firewall Rules](https://legacy.docs.aviatrix.com/documentation/latest/security/dcf-create-rules.html) # Disabling the Distributed Cloud Firewall Feature Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/security/dcf/dcf-disable-feature If you disable Distributed Cloud Firewall (DCF), the following occurs: If you disable Distributed Cloud Firewall (DCF), the following occurs: **Features are disabled:** * Enforcement on PSF Gateways (card is removed from the Settings tab) * Enforcement on External Connections (card is removed from the Settings tab) * Enforcement on Transit Egress (card is removed from the Settings tab) * DCF for Kubernetes SmartGroups * WebGroups * ExternalGroups **Tab behavior changes:** * The Policies tab and Monitor tab become unavailable and show the Distributed Cloud Firewall Begin splash screen. * The Detected Intrusions tab will only show historical data. * The Egress > Overview tab does not display (specific to Egress Security Score). * The Egress Security Score functionality is removed from the Egress VPC/VNets tab (if using Controller 8.0). To disable the Distributed Cloud Firewall (DCF) feature: 1. To disable the DCF feature, navigate to **Security > Distributed Cloud Firewall > Settings**. 2. Click **Disable** on the Distributed Cloud Firewall card. You are prompted that disabling the feature will remove all DCF-related features and functionality. 3. Click **Disable** to confirm. Clean up any remaining DCF rules: 1. On the **Security > Distributed Cloud Firewall > Settings** tab, click **Clean up Policies** on the Distributed Cloud Firewall card. 2. You are prompted that cleaning up policies means all rules will be deleted, and that you will not be able to recover them. 3. Select the **I understand that this action cannot be undone** checkbox and click **Clean Up** to confirm. * [Enabling the Distributed Cloud Firewall Feature](/docs/enterprise/10.1/guides/security/dcf/dcf-enable-feature) * [Creating Groups for Distributed Cloud Firewall](/docs/enterprise/10.1/guides/security/dcf/dcf-create-groups) # Distributed Cloud Firewall Setup and Default Action Rule Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/security/dcf/dcf-enable-feature Distributed Cloud Firewall (DCF) provides advanced traffic management capabilities to enforce zero trust principles across your network. This document focuses on enabling the DCF feature and configuring the Default Action Rule, ensuring secure and consistent traffic handling in the absence of... Distributed Cloud Firewall (DCF) provides advanced traffic management capabilities to enforce zero trust principles across your network. This document focuses on enabling the DCF feature and configuring the Default Action Rule, ensuring secure and consistent traffic handling in the absence of explicit rules. > [!NOTE] > > If you configured the ThreatIQ and/or Geoblocking features prior to Controller version 7.2.4820, in 7.2.4820 you automatically receive a free Distributed Cloud Firewall (DCF) license. > > If you did not configure the ThreatIQ and/or Geoblocking features prior to > Controller version 7.2.4820, you are expected to purchase a DCF license. This > will include the ExternalGroup feature (replaces the ThreatIQ and Geoblocking > features). ## Enabling the Distributed Cloud Firewall Feature Enabling the Distributed Cloud Firewall feature allows you to create and manage rules, rulesets, and policies to enforce zero trust principles. > [!NOTE] > > If you enabled Distributed Cloud Firewall in a previous Controller version, > you do not need to enable it again. **CoPilot UI** To enable the Distributed Cloud Firewall (DCF) feature, if it is not enabled already: 1. Go to the **Security > Distributed Cloud Firewall > Policies** tab. 2. Click **Begin Using Distributed Cloud Firewall**. 3. Click **Begin** again to confirm the action and enable the feature. **Terraform** Enable Distributed Cloud Firewall with the [aviatrix\_config\_feature](https://registry.terraform.io/providers/AviatrixSystems/aviatrix/latest/docs/resources/aviatrix_config_feature) resource, available as of provider version R9.0.0+. > [!NOTE] > > This resource requires provider version 9.0.0+ (Controller 9.0 or > later). If you are on Controller 8.2, use the CoPilot UI tab above > instead. **1. Enable the Distributed Cloud Firewall feature** Add the following `aviatrix_config_feature` resource block to your Terraform configuration file: ```hcl theme={null} resource "aviatrix_config_feature" "dcf" { feature_name = "microseg" is_enabled = true } ``` **2. Verify and Apply the Configuration** Run `terraform plan`, review the planned changes, and run `terraform apply`. *** ## Default Action Rule for Distributed Cloud Firewall Starting with Controller version 8.1, the Default Action Rule is automatically created when DCF is enabled. Legacy rules such as Greenfield and DefaultDenyAll are no longer generated; the Default Action Rule replaces their function. The Default Action Rule cannot be deleted. When you upgrade to Controller 8.1 or earlier, the existing DefaultDenyAll rule is converted to a non-system rule, which you can delete. If you use Controller 8.0 or earlier, both a [Greenfield Rule and a DefaultDenyAll rule](#greenfield-rule-and-defaultdenyall-rule) are deleted. After enabling the Distributed Cloud Firewall feature, a Default Action Rule is created. This is a system rule used to enforce [zero trust principles](/docs/enterprise/10.1/concepts-architectures/components/dcf/dcf-zero-trust) by controlling how traffic is handled in the absence of explicit rules. This rule must be set to Deny after it is created, to ensure that traffic is not permitted by default. The action of the Default Action Rule is enforced globally; it is part of the broader policy evaluation framework and is evaluated after all user-defined and system-defined rules/rulesets. **CoPilot UI** > [!NOTE] > > You may see the Manage Rules Better with Rulesets splash screen when you first > access the Policies tab. If so, click **Acknowledge** to continue. #### New DCF Users 1. On the **Security > Distributed Cloud Firewall > Policies** tab, click **Begin Using Distributed Cloud Firewall**. You may have to click **Begin** again to confirm the action. This Default Action Rule is then created on the Policies tab in the User Ruleset (formerly the V1 Policy List ruleset). 2. Change the action of the Default Action Rule to **Deny**. This is mandatory. 3. Continue with creating DCF rulesets and rules. #### Existing DCF Users 1. On the **Security > Distributed Cloud Firewall > Policies** tab, click **Acknowledge** on the splash screen that informs you about the Default Action Rule. The Default Action Rule is displayed on the Policies tab in the V1 Policy List ruleset. The Default Action Rule replaces the pre-existing DefaultDenyAll rule, if you used the DCF feature prior to Controller 8.1. You can delete the old DefaultDenyAll rule if it exists. 2. Change the action of the Default Action Rule to **Deny**. 3. You can delete the pre-existing Greenfield-Rule if it exists, assuming that you have reviewed traffic patterns first. #### Modifying the Default Action Rule You can do the following to the Default Action Rule: * Change the action from Permit to Deny * Change the name of the rule * Enable logging **Terraform** Manage the Default Action Rule with the [aviatrix\_distributed\_firewalling\_default\_action\_rule](https://registry.terraform.io/providers/AviatrixSystems/aviatrix/latest/docs/resources/aviatrix_distributed_firewalling_default_action_rule) resource. Terraform declares the desired state directly, so there is no separate path for new versus existing DCF users — set `action` to `DENY` regardless of how DCF was originally enabled. **1. Set the Default Action Rule to Deny** Add the following `aviatrix_distributed_firewalling_default_action_rule` resource block to your Terraform configuration file: ```hcl theme={null} resource "aviatrix_distributed_firewalling_default_action_rule" "this" { action = "DENY" logging = true } ``` **2. Verify and Apply the Configuration** Run `terraform plan`, review the planned changes, and run `terraform apply`. *** ## Greenfield Rule and DefaultDenyAll Rule In Controller 8.0 and lower, the placeholder Greenfield Rule prevents traffic from being dropped before you start configuring the rest of your rules. After you create additional rules you can move the Greenfield Rule where needed in your rule priority list. You can edit or delete the Greenfield Rule later, if desired. > [!NOTE] > > The Greenfield Rule is only enforced on gateways, and not on Security Groups > in the cloud. By default (if you selected the recommended **Permit All Traffic** option), the Greenfield Rule has the following attributes: * Source/Destination Groups: Anywhere (0.0.0.0/0) * Protocol: Any * Action: Permit * Logging: On The DefaultDenyAll Rule blocks traffic to any CIDR covered in Distributed Cloud Firewall rules. This rule is not editable. ## DCF-Related Features If the Distributed Cloud Firewall feature is enabled, these features are available: * Enforcement on PSF Gateways * Enforcement on External Connections * Enforcement on Transit Egress You must enable these features from the **Security > Distributed Cloud Firewall > Settings** tab. With these features, you can enforce [DCF on PSF Gateways](/docs/enterprise/10.1/reference/dcf/dcf-psf), External Connections, or [Transit Egress gateways](/docs/enterprise/10.1/reference/dcf/dcf-transit-egress). * DCF on Kubernetes Clusters from the Feature Previews list. To use this feature, you must enable it from the Discovery of Kubernetes Resources card on the **Groups > Settings** tab. ## Related Resources * [Aviatrix Terraform provider](https://registry.terraform.io/providers/AviatrixSystems/aviatrix/latest/docs) * [aviatrix\_config\_feature resource](https://registry.terraform.io/providers/AviatrixSystems/aviatrix/latest/docs/resources/aviatrix_config_feature) * [aviatrix\_distributed\_firewalling\_default\_action\_rule resource](https://registry.terraform.io/providers/AviatrixSystems/aviatrix/latest/docs/resources/aviatrix_distributed_firewalling_default_action_rule) * [Managing Distributed Cloud Firewall Rulesets](https://legacy.docs.aviatrix.com/documentation/latest/security/dcf-ruleset-manage.html) * [Disabling the Distributed Cloud Firewall Feature](/docs/enterprise/10.1/guides/security/dcf/dcf-disable-feature) # Distributed Cloud Firewall for Kubernetes Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/security/dcf/dcf-kubernetes Aviatrix Distributed Cloud Firewall (DCF) for Kubernetes extends Zero Trust security to containerized workloads across AWS EKS, Azure AKS, Google GKE, and self-managed clusters. ## Overview Aviatrix Distributed Cloud Firewall (DCF) for Kubernetes extends Zero Trust security to containerized workloads across AWS EKS, Azure AKS, Google GKE, and self-managed Kubernetes clusters. This integration provides identity-based security policies, secure egress control, and unified visibility for Kubernetes environments. > [!NOTE] > > DCF on Kubernetes Clusters is Generally Available (GA) as of Controller 10.1, > promoted from Feature Previews. ### Key Capabilities * **Identity-Based Security** — Enforce firewall policies based on Kubernetes identities (namespace, pod, service) rather than ephemeral IP addresses. Policies automatically follow workloads as they scale, move, or restart. * **Multicloud Kubernetes Security** — Unified security policies across AWS EKS, Azure AKS, Google GKE, and self-managed clusters. Define security once, enforce everywhere. * **Native Kubernetes Integration** — Define firewall policies using Kubernetes Custom Resource Definitions (CRDs). Security policies are managed with the same `kubectl` and YAML workflows your teams already use. * **Secure Egress Control** — Prevent unauthorized outbound traffic from Kubernetes workloads. Control egress at namespace, pod, and cluster levels with domain-based filtering. * **Advanced NAT and IP Management** — Resolve IP overlap and exhaustion issues across multiple Kubernetes clusters with advanced NAT capabilities. > [!NOTE] > > DCF currently enforces rules on traffic from Kubernetes clusters to > destinations outside the VPC/VNet only. ### Supported Kubernetes Distributions * AWS Elastic Kubernetes Service (EKS) * Azure Kubernetes Service (AKS) * Google Kubernetes Engine (GKE) * Self-managed and private Kubernetes clusters in cloud ## Getting Started To use DCF for Kubernetes: 1. Meet the [prerequisites](/docs/enterprise/10.1/reference/dcf/kubernetes-prerequisites) for your cloud provider. 2. Enable [Kubernetes Resource Discovery](/docs/enterprise/10.1/guides/security/dcf/kubernetes-resource-discovery) . 3. [Onboard your Kubernetes clusters](/docs/enterprise/10.1/guides/security/dcf/kubernetes-onboard) . For private clusters, see [Onboarding Private Kubernetes Clusters](/docs/enterprise/10.1/guides/security/dcf/private-kubernetes-cluster-onboard) . 4. [Enable DCF policies for Kubernetes](#enabling-dcf-policies-for-kubernetes) 5. Optionally, install CRDs for in-cluster policy management. 6. Create [SmartGroups](/docs/enterprise/10.1/concepts-architectures/components/dcf/smartgroups-about) and DCF rules. ## Enabling DCF Policies for Kubernetes > [!WARNING] > > The Aviatrix > [k8s-firewall helm chart](https://github.com/AviatrixSystems/k8s-firewall-charts) > must be installed in the cluster before onboarding so the CRDs and the > `avx-controller` ClusterRole are present. See the per-provider onboarding pages > for installation steps. > > CRD-based policy on **AKS and GKE is implemented in the Controller but does not > have end-to-end test coverage as of 9.0**. Coverage on EKS is complete. Report > any issues encountered on AKS or GKE to Aviatrix support. **CoPilot UI** To enable DCF policies for Kubernetes using the CoPilot UI, follow these steps: **1. Enable Discovery of Kubernetes Resources** On **Groups** > **Settings**, click **Enable** on the **Discovery of Kubernetes Resources** card. See [Discovery of Kubernetes Resources](/docs/enterprise/10.1/guides/security/dcf/kubernetes-resource-discovery) for more information. **2. Enable Enforcement on Kubernetes** On **Security** > **Distributed Cloud Firewall** > **Settings**, click **Enable** on the **Enforcement on Kubernetes** card. A notification appears to confirm the changes. **Terraform** To enable DCF policies for Kubernetes using Terraform, add the following to your Terraform configuration: ```hcl theme={null} resource "aviatrix_k8s_config" "this" { enable_k8s = true enable_dcf_policies = true } ``` *** ## Managing Policies with Kubernetes CRDs DCF policies can be defined directly inside Kubernetes clusters using Custom Resource Definitions (CRDs). This allows platform teams to manage firewall policies with the same `kubectl` and GitOps workflows used for application deployments. ### Installing the CRDs Register the Aviatrix CRDs to your Kubernetes cluster using the Helm chart: ```bash theme={null} helm install --repo https://aviatrixsystems.github.io/k8s-firewall-charts k8s-firewall k8s-firewall ``` Verify the CRD registration: ```bash theme={null} kubectl get crds ``` Expected output includes: ``` NAME CREATED AT firewallpolicies.networking.aviatrix.com 2025-11-14T07:19:03Z webgrouppolicies.networking.aviatrix.com 2025-11-14T07:19:05Z ``` ### Writing a Firewall Policy A `FirewallPolicy` CRD defines egress rules for pods matching a label selector. The following example permits pods with the label `app: dev-pods` in the `dev` namespace to reach `www.google.com` on any protocol: ```yaml theme={null} kind: FirewallPolicy apiVersion: networking.aviatrix.com/v1alpha1 metadata: name: test-firewall-policy namespace: dev spec: rules: - name: test logging: true selector: matchLabels: app: dev-pods action: permit protocol: any destinationSmartGroups: - name: anywhere webGroups: - name: checkip webGroups: - name: checkip domains: - "www.google.com" smartGroups: - name: anywhere selectors: - cidr: 0.0.0.0/0 ``` Apply the policy: ```bash theme={null} kubectl apply -f firewall-policy.yaml ``` ### Writing a WebGroup Policy A `WebgroupPolicy` CRD defines domain-based egress filtering as a standalone resource. The following example permits pods with the label `app: web-client` in the `default` namespace to reach specific domains: ```yaml theme={null} kind: WebgroupPolicy apiVersion: networking.aviatrix.com/v1alpha1 metadata: name: pod-to-web namespace: default spec: selector: matchLabels: app: web-client domains: - "api.example.com" - "cdn.example.com" ``` Apply the policy: ```bash theme={null} kubectl apply -f webgroup-policy.yaml ``` ### Verifying Policy Status Check policy events to confirm the policy was applied successfully: ```bash theme={null} kubectl get events -n dev ``` Example output for a successful FirewallPolicy: ``` LAST SEEN TYPE REASON OBJECT MESSAGE 57m Normal UpdatePolicyListSuccess firewallpolicy/test-firewall-policy Updated policy list for firewall policy with UUID e006b5fe-bf73-46f7-93b8-d76e7325e2eb ``` Example output for a successful WebgroupPolicy: ``` LAST SEEN TYPE REASON OBJECT MESSAGE 41s Normal CreateWebGroupSuccess webgrouppolicy/pod-to-web Created webgroup with name webgrouppolicy-default--pod-to-web--58c5fb5e 41s Normal CreateSmartGroupSuccess webgrouppolicy/pod-to-web Created smartgroup with name webgrouppolicy-target-default--pod-to-web--58c5fb5e 41s Normal CreatePolicySuccess webgrouppolicy/pod-to-web Updated policy webgrouppolicy-default-pod-to-web-58c5fb5e ``` If an event shows a `Warning` type or a `Reason` ending in `Failure`, check that the CRDs are installed correctly and that the Aviatrix Controller can reach the cluster. For the full schema of `.status` (including `ruleset`, `attachmentPoint`, `smartGroups[]`, and `lastReconciledGeneration`), see [Kubernetes CRD Reference](/docs/enterprise/10.1/reference/dcf/kubernetes-crd-reference#status-schema) . ## Rule Ordering Three things determine the order in which DCF evaluates rules generated from your custom resources. ### Within a Single FirewallPolicy Rules under `spec.rules[]` are evaluated top-down in the order they appear. To reorder rules, edit the resource and reapply. ### Between Different FirewallPolicy Resources Multiple `FirewallPolicy` resources have **no deterministic order between each other**. They are peers in DCF. If two rules in two different `FirewallPolicy` resources could match the same flow, the result is undefined. > [!TIP] > > When ordering between rules matters, place them in the same `FirewallPolicy` > resource. ### FirewallPolicy vs. WebgroupPolicy `FirewallPolicy` rules are always evaluated before `WebgroupPolicy` rules. Treat `WebgroupPolicy` as the catch-all egress allow-list. ### CR-Managed Rules vs. Admin-Authored DCF Rules All CR-managed rules live inside a single dedicated section of the DCF policy tree. Your Aviatrix administrator can move that section above or below other hand-authored DCF rules. You cannot interleave individual CR rules with hand-authored DCF rules. The placement is per-section. > [!TIP] > > For bulk allow or deny lists that should "frame" the Kubernetes rules, keep > them in admin-authored DCF rules and place the Kubernetes section between > them. ## Troubleshooting ### Common Pitfalls | Symptom | Likely cause | | - | - | | The CR exists but no objects appear in CoPilot. | The cluster is not registered with Aviatrix, or the Kubernetes feature is not enabled. | | `.status` never populates. | The Helm chart is not installed, or the Aviatrix Controller cannot authenticate to the API server. Check cluster status in CoPilot. | | `tlsProfile not found` events. | The named profile does not exist on the controller. Create it in CoPilot or Terraform first. | | Rules from different `FirewallPolicy` resources behave inconsistently. | They are peers. See [Rule Ordering](#rule-ordering). Consolidate or use non-overlapping selectors. | | Two CRs target the same workload. | Both rules apply, and the order between them is undefined. Avoid overlap. | ### Reading Reconciliation Events The Aviatrix Controller emits Kubernetes events on every reconciliation: ```bash theme={null} kubectl get events -n --field-selector involvedObject.name= ``` Look for `Warning` events with reasons such as: * **Spec is invalid**: fix the CR spec. * **app domain with name X not found**: a `name` reference does not match an inline definition or any controller-side AppDomain. Either define it inline or have your Aviatrix administrator create it. * **Failed to update k8s root attachment point**: usually a controller-side conflict. The controller retries automatically. ## Updating and Deleting Policies ### Updating a Policy Edit the resource and run `kubectl apply`. The Aviatrix Controller diffs the new spec against the previous reconciled state, updates the affected DCF objects, and removes any AppDomains that are no longer referenced. UUIDs of unchanged rules are preserved so that features such as bandwidth tracking continue to work. ### Deleting a Policy Run `kubectl delete`. A finalizer holds the resource until the controller has cleaned up the attachment point, policy list, and all AppDomains the resource created. ### Cluster Offboarding Caveat If you remove a cluster from Aviatrix while CRs still exist in it: 1. Aviatrix refuses the offboard if any SmartGroup elsewhere still references this cluster. Clean those references up first. 2. Once offboarded, the controller stops watching the cluster. CRs in the cluster are no longer reconciled, and the DCF objects they previously created remain in place. > [!WARNING] > > To offboard cleanly, delete the CRs first so that the finalizer-driven > teardown runs, then offboard the cluster. ## Related Topics * [Kubernetes CRD Reference](/docs/enterprise/10.1/reference/dcf/kubernetes-crd-reference) * [Onboarding Kubernetes Clusters](/docs/enterprise/10.1/guides/security/dcf/kubernetes-onboard) * [Onboarding Private Kubernetes Clusters](/docs/enterprise/10.1/guides/security/dcf/private-kubernetes-cluster-onboard) * [Kubernetes Prerequisites and Permissions](/docs/enterprise/10.1/reference/dcf/kubernetes-prerequisites) * [Creating SmartGroups](/docs/enterprise/10.1/concepts-architectures/components/dcf/smartgroups-about) * [Aviatrix Kubernetes Firewall](/docs/enterprise/10.1/concepts-architectures/components/dcf/kubernetes-overview) # Kubernetes Intra-Cluster Inspection Source: https://docs.aviatrix.com/docs/enterprise/10.1/guides/security/dcf/dcf-kubernetes-intra-cluster Enable Distributed Cloud Firewall (DCF) intra-cluster inspection to enforce pod-to-pod security policies inside Kubernetes clusters using native NetworkPolicy objects translated from DCF rules. Distributed Cloud Firewall (DCF) intra-cluster inspection extends DCF enforcement to **pod-to-pod traffic within a Kubernetes cluster**. The Controller translates DCF rules that use K8s SmartGroup selectors into native Kubernetes NetworkPolicy objects. The cluster's CNI plugin enforces these objects at the pod network level. This is fundamentally different from DCF enforcement at the spoke gateway (which inspects traffic entering or leaving the cluster). Intra-cluster inspection operates inside the cluster using Kubernetes-native enforcement. No Aviatrix gateway is in the data path for intra-cluster flows. A single DCF rule with K8s SmartGroups on both source and destination generates NetworkPolicy for every enabled cluster **and** still enforces at the Aviatrix gateway. Because NetworkPolicy selects pods by label, policy follows pods through restarts, rescheduling, and autoscaling without IP-based gaps. > [!WARNING] > > Kubernetes NetworkPolicy is **allow-list-based and union-based**. There is no > native DENY action. If any NetworkPolicy from any source (including > non-Aviatrix policies) permits a flow, a DCF DENY rule translated into an > isolation policy has no effect on that flow. Review your existing > NetworkPolicies carefully before enabling intra-cluster inspection. ## Supported Environments Intra-cluster inspection is a [Preview Feature](/docs/enterprise/10.1/reference/feature-modes/index) in Controller 10.1 and is supported on the following Kubernetes distributions: | Distribution | Support level | | - | - | | AWS EKS | Public Preview | | Azure AKS | Public Preview | | Self-managed clusters (with a supported CNI) | Public Preview | > [!NOTE] > > GKE is not supported for intra-cluster inspection in Controller 10.1. ### CNI Requirements The cluster's CNI plugin must enforce NetworkPolicy. The following CNIs are supported: * Calico * Cilium * AWS VPC CNI 1.14 or later * Weave Net * Antrea * Canal > [!WARNING] > > If the cluster's CNI does not enforce NetworkPolicy (such as Flannel, Kubenet, > or Kindnet), all translated NetworkPolicies are **silently ignored**. The > Controller and CoPilot do not raise an error or warning. Pods remain > unrestricted. Verify that your CNI enforces NetworkPolicy before enabling this > feature. ## Prerequisites Before you enable intra-cluster inspection, confirm the following: * **Distributed Cloud Firewall** is enabled (CoPilot > **Security** > **Distributed Cloud Firewall**). * **Kubernetes Resource Discovery** is enabled (CoPilot > **Groups** > **Settings** > **Discovery of Kubernetes Resources**). * The target cluster is [onboarded to Aviatrix](/docs/enterprise/10.1/guides/security/dcf/kubernetes-onboard) . * The Aviatrix K8s Helm chart is installed in the cluster with the `manageNetworkPolicies` flag set to `true`. This flag ships `false` by default, so you must explicitly enable it when running or upgrading the Helm chart. * The cluster's CNI plugin enforces Kubernetes NetworkPolicy (see [CNI Requirements](#cni-requirements)). * You have K8s SmartGroups defined that use Kubernetes selectors (`k8s_cluster_id`, `k8s_namespace`, `k8s_pod`, `k8s_service`, or `tags.