Aviatrix Controller and Gateway Software Release Notes

The following issue has been fixed in this release.

Aviatrix Transit Gateways with large number of tunnels and running for a long time could encounter an issue where in the IPSec process becomes unresponsive leading to all IPsec tunnels going into a DOWN state. The cause of this is an internal counter reaching its maximum value and overflowing. To recover, the transit gateway needs to be rebooted.

While it is not possible to specify the exact number of tunnels and length of time it would take for the internal counter to overflow, the few customers who encountered this issue had greater than 800 ipsec tunnels on the transit gateway and took three to four months to encounter this issue. The number of ipsec tunnels on the gateway can be seen from Copilot UI under Diagnostics > Cloud Routes > Gateway Routes.

Fixed an issue where GRPC traffic could be dropped when routed through Distributed Cloud Firewall (DCF) web filters using SNI-based web groups.

The fix improves support for HTTP/2-based traffic in scenarios where only H2 is present in the ALPN list, resolving a limitation in the Traffic Server tunneling logic.

Fixed an issue where Aviatrix Gateways could experience memory buildup and restarts during prolonged Layer 7 (L7) traffic. The system now properly clears outdated web filter data to prevent excessive memory usage.

When using Controller High Availability (HA) with Controllers version 8.0 and later, the standby Controller will fail to launch correctly. This is because the HA mechanism relies on a fixed software version specified in the Auto Scaling Group (ASG) launch template, but with Controllers version 8.0 and later now require the version to be passed dynamically through cloud-init during instance creation.

This issue occurs only in environments that use:

Workaround: Use the new CloudFormation template to enable AWS Controller High Availability. This template supports dynamic version injection and restores compatibility with Controllers version 8.0 and later in supported regions. For versions 7.x and earlier, use the existing CloudFormation script (without the v3 suffix).

Note: This solution is not available in AWS regions that do not support Lambda Function URLs.

Azure Standard_B1ms SNAT-enabled Egress Spoke Gateways may experience significant throughput drops under high connection loads. This limitation is caused by the Azure Standard_B1ms instance type, which has limited compute and network resources.

Affected Scenario:

Workaround:

Upsize the Spoke Gateway to a larger Azure instance type for workloads that require more than 10K concurrent connections or consistent network throughput.

Auto migration will not work from 7.2 to 8.0 when proxy is enabled. You must use a manual backup and restore process to perform the upgrade. Follow the steps below to back up and restore during the upgrade:

The Controller auto-migration and Gateway upgrade features do not function properly when the Aviatrix Controller has proxy settings enabled. In such environments, migration may fail, and you must follow a manual backup and restore process instead of using the standard auto-migration workflow. This limitation is due to current backend behavior that does not support migration through proxy-enabled setups.

Affected Scenario:

Check Whether You Are Affected:

If proxy configurations are present in either location, your deployment is affected.

Workaround:

Follow the manual backup and restore steps below to upgrade the Aviatrix Controller and Gateways:

When upgrading Aviatrix Gateways from version 7.2.x to 8.0.0 with TLS decryption enabled in Distributed Cloud Firewall (DCF), the Gateway automatically regenerates its TLS decryption certificate authority (CA). Because each Gateway maintains its own unique CA for security, the regenerated CA no longer matches the CA previously trusted by clients.

As a result, you may experience the following issues after the upgrade:

Affected Scenario:

Workaround:

When upgrading from Controller version 7.1 to 7.2 or 8.0, Spoke Gateways with routing through a Public Subnet Filtering (PSF) Gateway may fail to upgrade and become unreachable if the PSF Gateway has not been upgraded first. This issue affects AWS environments where Spoke Gateway route tables are configured to point to a PSF Gateway.

To avoid this issue, follow the correct upgrade sequence:

During a gateway software upgrade, traffic matching DCF WebGroup rules may be briefly dropped during the upgrade. This impacts both Layer 7 (HTTP/HTTPS) and Layer 4 traffic and occurs across all supported cloud providers (AWS, Azure, and GCP). The disruption typically lasts a few seconds but may vary depending on gateway load and policy complexity.

Workaround:

None

Recommendations:

In environments where Distributed Cloud Firewall (DCF) and customized SNAT are used together, DCF rules may fail to match traffic correctly when the same SmartGroups are specified in both the source and destination fields. This is because the system does not account for the translated source address during rule evaluation.

As a result, traffic may be unintentionally blocked by the DefaultDenyAll rule, and policies may not apply as expected—particularly in cross-cloud or cross-region scenarios.

Affected Configurations:

Workaround:

In earlier versions, avoid using 0.0.0.0/0 as the destination in SNAT rules. Instead, specify only the required destination CIDRs.

Distributed Cloud Firewall (DCF) is not officially supported on Edge gateways. Although DCF rules may appear to be deployed to Edge gateways, they are not fully validated and may not function correctly, especially in environments using NAT for overlapping IP address spaces.

DCF rules pushed to Edge gateways may not account for NAT translations, leading to incorrect rule behavior and potential traffic filtering issues.

Affected Deployments:

Workaround:

When recreating a policy-based Site-to-Cloud (S2C) VPN connection after deleting a previous one with the same remote CIDR, the system may incorrectly report a CIDR overlap error, even though the original connection has been removed. This occurs because the system does not fully clean up the remote CIDR information, causing it to believe the CIDR is still in use.

Affected Scenario:

Workaround:

Contact Aviatrix Support to manually clear the cached CIDR information.

The Distributed Cloud Firewall (DCF) policy writer writes approximately 40KB of data per gateway during each configuration snapshot, regardless of whether there are policy changes. In large deployments, this results in frequent and unnecessary write operations to the controller database.

Affected Scenario: - DCF enabled across many gateways - Frequent configuration snapshots (triggered automatically or during updates)

Impact: Increased system load and database write activity, which may affect controller performance and stability in large-scale environments.

Workaround:

There is no direct workaround at this time. Users operating at scale should monitor controller resource usage closely. Aviatrix is actively working to reduce unnecessary write operations in a future update. If performance issues are observed, contact Aviatrix Support for evaluation and potential tuning options.

In Aviatrix Controller version 8.0, Edge Gateway version numbers may be incorrectly updated in the Controller UI after the gateway comes back online from a down state. This occurs even when no new software installation has taken place.

Instead of preserving the actual version running on the Edge Gateway, the Controller may incorrectly overwrite it with its own version. This can lead to confusion during troubleshooting, upgrade planning, or compliance checks.

Affected Environments:

Workaround:

Note:

This issue only affects Edge Gateways. Cloud provider (CSP) Gateways in AWS, Azure, GCP, or OCI are not affected.

In Controller release 8.0, gateway software upgrades take longer to complete compared to earlier versions. On average, the upgrade rate drops from approximately 14 gateways per minute in version 7.2 to approximately 11 gateways per minute in 8.0, which is an increase of about 20% in execution time.

Affected Scenarios:

Impact:

Only the upgrade duration is affected. Gateway functionality remains unaffected after a successful upgrade.

Recommendations:

Aviatrix Edge Gateways deployed on Equinix Network Edge and certain VMware environments may experience issues with root disk resizing during initial setup. The root filesystem might not expand to utilize the full allocated disk space. This can prevent essential cloud-init modules from executing properly.

Affected Versions:

Workaround:

Customers running Aviatrix Edge Gateways on Equinix Edge or VMware environments with version 7.1.4191 should contact Aviatrix Support for assistance.

In versions prior to 8.0.0, the Public Internet SmartGroup includes the RFC6598 Shared Address Space (100.64.0.0/10). Starting in version 8.0.0, this range is excluded from new installations to improve Layer 7 (L7) traffic inspection and policy enforcement.

However, during upgrades to 8.0.0, the existing configuration is retained, and the 100.64.0.0/10 range is not automatically removed.

Impact:

Workaround:

Recommendation:

After upgrading to version 8.0.0, review your SmartGroup configuration if your deployment uses the 100.64.0.0/10 range in DCF rules.

In the CoPilot UI, Groups > SmartGroups and Groups > ExternalGroups with multiple filters may not appear as originally configured after being saved. This issue occurs when creating groups with multiple sets of any resource type. While policy enforcement is correct, the UI may display missing or merged filter sets, leading to ambiguity and confusion during review or editing.

Affected Scenario:

Workaround:

There is no workaround at this time. If possible, avoid using multiple filter sets in a single group until the issue is resolved.

In Aviatrix Controller version 8.0.0, you may encounter a problem when creating or modifying Distributed Cloud Firewall (DCF) rules using either the CoPilot UI or Terraform. In the CoPilot UI, the ruleset may not display correctly and the "Commit" button may be non-functional. When using Terraform, an error may occur indicating that the DCF policy API is unavailable.

This issue prevents you from applying new or updated DCF rules, impacting network security policy management.

Affected Scenario:

Workaround:

Contact Aviatrix Support. They can run a script to restore the missing policy list without requiring a full upgrade.

Jumbo Frame support cannot be enabled on BGPoLAN (BGP over LAN) connections for AWS HPE gateways. Attempts to enable this feature may result in an error indicating that Jumbo Frames are not supported.

This affects environments where high-throughput performance is critical, such as large-scale or latency-sensitive deployments.

Affected Scenario:

Limitation:

In version 8.0.0, Jumbo Frame support can only be enabled when creating a new BGPoLAN connection on AWS HPE gateways. Editing an existing connection to enable Jumbo Frames is not supported.

Workaround:

None.

To enable Jumbo Frame support, delete the existing connection and recreate it with the setting enabled.

IPSec diagnostics in the Controller UI do not display logs for non-Equinix Edge Gateways (such as AEP or self-managed Edge Gateways). When accessing the diagnostics page for these gateways, the IPSec log section may appear empty, even if IPSec tunnels are operating correctly.

This issue affects visibility into tunnel-level logs and may complicate troubleshooting efforts.

Affected Scenario:

Workaround:

Use tunnel status and statistics to verify IPSec operation.

Note:

This is a UI diagnostic issue only. IPSec tunnel functionality is not impacted.

When deploying Edge Gateways using images g3-202504251522 and g3-202504251525, the root disk may be incorrectly sized after the VM boots and the ZTP (Zero Touch Provisioning) process runs. Even if the VM is created with a 64GB disk, the root filesystem may be limited to only 12GB.

This may lead to insufficient storage for certain workloads or during upgrades.

Affected Scenario:

Workaround:

Manual resizing of the root partition and filesystem is required. Please contact Aviatrix Support for assistance, as this step cannot be performed independently.