Aviatrix Controller and Gateway Software Release Notes

AVX-59843

If you have BGP sessions over IPsec and the BGP peer IPs are non-APIPA (169.254.x.x) IP addresses, the sessions might not be able to be established or reestablished after experiencing tunnel flapping. The BGP peer subnet route is installed by the kernel and this route is synced to a special route table which is used by BGP to send BGP packets to a peer. There is a failure to sync this route, therefore BGP packets can not reach remote BGP peer.

AVX-50619

Removed misleading TUNNEL-STATUS-CHANGED messages in the event logs seen after a Controller upgrade. These messages could result in false positive tunnel down alerts.

AVX-50964

Fixed a problem with Terraform firewall deployment. When deploying firewalls using the aviatrix_firewall_instance Terraform resource, you can also provide CSP Tags. If you added or changed any of the tags of an already deployed aviatrix_firewall_instance resource and then ran a terraform apply, the firewall would be deleted and recreated. This replacement process no longer occurs.

AVX-51456

Corrected an issue with destination network address translation (DNAT). DNAT rules could be configured on Aviatrix Gateways using Terraform provider version 3.1.4. When setting up DNAT rules on standalone Gateways with policy-based tunnels configured, an error message indicated the interface for the connection could be found.

AVX-51937

An issue with the TGW Segmentation for Egress functionality in AWS Transit Gateway (TGW) FireNet was fixed. It was not working as expected when a 10.0.0.0/8 route was advertised from on-premises. Despite enabling the TGW Segmentation for Egress option to prevent communication between two network domains connected to the FireNet, communication remained possible. This issue affected customers using AWS TGW FireNet for egress and attempting to isolate network domains.

AVX-52016

(AWS GovCloud) Fixed an issue where a Gateway state change on Transit FireNet in GovCloud could cause the 0/0 route to AWS NAT Gateway to unexpectedly update. The updated route would point to a Spoke Gateway’s ENI instead of remaining with the AWS NAT Gateway.

AVX-52263

This release addresses an issue of incorrectly programming a CIDR that is configured on the transit using exclude learned CIDR to Spoke. When a CIDR was excluded via the Exclude Learned CIDRs to Spoke VPC option on the transit, the spoke did not learn this route. Also, the specific CIDR was removed from the spoke VPC private route tables. However, when Configure Private VPC Default Route was enabled and then disabledt, the spoke VPC route table was repopulated with the excluded CIDR.

AVX-53881

Fixed an issue that occurred when enabling the Firenet inspection policy for Azure subnet groups. Users could experience asymmetric routing issues if a subnet within the spoke consumed an entire address space added to the VNET. This led to missing VNET local routes for subnets within subnet groups, causing return traffic to be dropped at the firewall.

AVX-54175

Fixed an issue with Controller where customers could encounter an error stating ‘Command execution is not complete’ when attempting to view details or diagnostics on transit gateways. This issue was specific to transit gateways and did not affect spoke gateways.

AVX-54777

Addressed an issue with earlier releases where Aviatrix Gateways, in certain scenarios, could experience traffic blackholes when subjected to prolonged periods of heavy traffic (e.g., 10 Gbps). This could lead to a complete traffic stall, impacting all traffic streams including syslog traffic to CoPilot and other configured agents. This issue was observed with Gateways running on instances like t3.micro and could cause the Gateways to enter a ‘Configuration Not Up-To-Date’ state due to excessive logging, even if the Gateway status shown as UP on the controller.

For this release, this issue has been mitigated. Please review the workaround if it occurs in your environment.

To work around this issue, manually add the required route to the destination subnet’s route table via the Azure Portal to facilitate return traffic. However, this manual addition might not be handled correctly by the Controller in the event of a gateway failure. It is advisable to monitor the routing and manually intervene if necessary.

AVX-55015

An issue was fixed that could occur in handling Site2Cloud Mapped NAT connections when the local CIDR was set to 0.0.0.0/0. When a user edited or deleted a connection mapped to this CIDR, the corresponding IP table rule was not properly removed. This could cause incorrect routing behavior.

AVX-55069

Corrected an issue where traffic was routed correctly in a customer network, but the FlightPath feature could display an incorrect path. The routing algorithm used by FlightPath did not consider route specificity. Instead, it showed the path with the best metric regardless of specificity. This issue arose when there were multiple routes configured with varying levels of specificity and metric.

AVX-55080

This release addresses the issue observed In Controller version 7.1.3006, where Azure customers could experience frequent tunnel flapping between spoke and transit gateways. This issue occurred when the eth0 interface on the spoke gateway flapped, causing the route to the transit IP to be deleted. This behavior was observed particularly after a gateway reboot. The fix addresses the issue and preserves the underlay routes.

AVX-55333

Previously, customers were unable to use "Backup" as a tag key for Aviatrix Gateways. This restriction presented a blocker for some customers who needed this specific tag key for their operations. The restriction has now been removed, and customers can use "Backup" as a tag key.

AVX-55379

Fixed an issue that occurred when a remote BGP peered device initiated a graceful restart and stopped its BGP session. The BGP routes might not have withdrawn properly on Edge.

AVX-55905

Logging Terminology Consistency ImprovementInconsistent use of "DROP" and "DENY" in traffic logs for blocked connections caused confusion when interpreting Layer 7 and Layer 4 traffic logs. "DROP" and "DENY" were used interchangeably to indicate blocked connections. With this release, all Logs will be updated to consistently use "DENY" for all blocked traffic.

AVX-56088

Fixed an issue where restarting the networking service on the Controller caused all gateways and CoPilot to disconnect from the Controller.

AVX-57245

Corrected an issue where Single Availability Zone (AZ) HA feature enabled on an Aviatrix Gateway (it is enabled by default) could cause a reboot cycle when you rebooted the Gateway.

AVX-57551

Fixed an issue where performing a like-version image replacement of an HA Gateway on GCP might result in network disruption of up to 4 minutes. This applied to a Gateway running version 7.2.4820. This did not apply to an image upgrade of a gateway running a 7.1 build.

AVX-57922

Security Notice: CVE-2024-50603 has been permanently patched.

AVX-58109

Azure gateways are now able to migrate from “Basic” to “Standard” IP SKUs. However, if there is an HA gateway, both Azure gateways need an image upgrade on both gateways. This is an Azure limitation.

AVX-58286

An issue was fixed where gateways using Legacy Egress FQDN filtering could experience traffic interruptions when processing very large data packets (approximately 4000 bytes or larger). This could result in halting all network traffic through the affected gateway. The system would attempt to automatically restart the problematic process, but the issue could recur if the system continued sending large packets.

AVX-58682

This release addresses an issue where, with certain large deployments, a timeout communicating with the routing system could result in excessive CPU usage on gateways.

AVX-58757

Fixed an error condition whereby upon upgrading from 7.0 to 7.1, the Controller was not able to update gateway configuration, resulting in the affected gateway being flagged as not up-to-date.

AVX-58763

When hostname-filtering is deployed to a 7.2.b gateway and the gateway is rolled back to 7.2.a, the policy with the hostname filtering will be ignored and skipped.

AVX-59073

Resolved IP addresses are not displayed for hostname/domain-based groups in Distributed Cloud Firewall (DCF) Monitor Logs. Users can only see hostnames/domains, not corresponding IP addresses.

AVX-59149

This release addresses a regression in 7.1 with TCP maximum segment size (MSS) clamping support, which is now supported on Standalone gateways.

AVX-56595

SNAT/DNAT on Transit Edge peering is not supported in this release. Although the configuration is allowed, the routing with SNAT/DNAT is not currently working properly.

AVX-56827

Azure Security Group Orchestration can take up to 30 minutes to update the Network Security Group (NSG) to be applied after the SgO is enabled. This can happen on a large scale set up, for example with 2 VPC/VNets with 100 subnets and 1000 VMs in each VNet.

AVX-57110

When creating a custom GeoGroup using Terraform and downloading the configuration from the Aviatrix Controller, the "match_expressions" block defining the country codes is missing.

AVX-57153

AVX-57342

The DCF Enforcement on External Connections feature can cause high CPU usage and delays in updating gateway configurations when enabled in a very large-scale test environment.

This is a Preview Feature and not Ideal for GA environments.

Disabling DCF Enforcement on External Connections feature resolves the high CPU usage and configuration delays. This feature is accessed at Security > Distributed Cloud Firewall > Settings.

AVX-57382

When creating SmartGroups for Azure resources using Terraform, the region formatting is incorrect. Incorrect region filters will cause SmartGroups to not work as intended for Azure resources grouped by region.

This issue affects creating Azure SmartGroups via Terraform only. It does not impact other clouds or creating groups through the CoPilot UI.

To address this issue, the region format must be updated in the Terraform provider code manually. You can get the exact region name using a command like az account list-locations --output table.

AVX-59518

AVX-59843

If you have BGP sessions over IPsec and the BGP peer IPs are non-APIPA (169.254.x.x) IP addresses, the sessions might not be able to be established or reestablished after experiencing tunnel flapping. The BGP peer subnet route is installed by the kernel and this route is synced to a special route table which is used by BGP to send BGP packets to a peer. There is a failure to sync this route, therefore BGP packets can not reach remote BGP peer.

AVX-61310

In an Aviatrix multi-transit design with Transit Peering, where one of the Transit Gateways has BGP S2C enabled and learns a default route (0.0.0.0/0), an issue occurs following a controller upgrade where incorrect metrics are applied to PeerS2c routes.

AVX-34763

Fixed an issue where new AWS accounts showed "Pass" status immediately after being added, and even after Audit revealed IAM policy inconsistencies. Initial status now more accurately reflects account configuration and the Status updates properly after Audit to show any detected issues.Aviatrix recommends that you run the Audit tool in CoPilot (Administration > Audit) after adding new AWS accounts, to verify the configuration.

AVX-39609

When you upgraded the image of a VPN Gateway, a rare issue could cause the Gateway to fail. In this situation, VPN users might not have been able to connect. This is fixed.

AVX-41823

Fixed an issue where some routing tables were not properly updated when adding new network interfaces to Aviatrix Gateways. This could cause some network traffic to route inefficiently or experience connectivity issues when trying to reach the newly added interfaces. The system now automatically updates all related routing tables when new interfaces are added to a Gateway.

AVX-42076

Fixed an issue where FireNet management IP addresses displayed in the Aviatrix Controller did not match the actual IP addresses in Azure and firewall vendor interfaces. This was primarily a display issue and did not impact functionality. You will now see consistent IP information across the Aviatrix Controller, Azure portal, and firewall vendor interfaces.

AVX-43890

Corrected an issue seen while upgrading from a fresh AWS Controller that resulted in the error, "Exception ‘ValueError: None is not a valid UpgradePhase’’. This error appeared in logs when upgrading a new Aviatrix Controller launched from AWS Marketplace to version 7.1.1906.

AVX-45480

Distributed Cloud Firewall rules were not properly applied to non-encrypted, non-web traffic (Non-TLS and Non-HTTP traffic) when processed by the High Performance Encryption (HPE) enabled gateways.This issue was fixed to enable correct identification and Rule enforcement for all traffic types, regardless of Rule order.

AVX-46165

Fixed an issue where FlightPath did not correctly analyze all network traffic rules for certain AWS configurations. You no longer need to manually check network access control list (NACL) rules for accurate results. This fix provides more precise troubleshooting capabilities for AWS network configurations.

AVX-48675

Azure Intra-VPC Security Group Orchestration was not properly detecting all network resources on virtual networks. As a result, some network resources were omitted from the configuration. This fix ensures all resources are properly included in the configuration.

AVX-49421

Reduced the time needed to execute a Terraform plan by using caching.

AVX-49668

After a software upgrade, the Controller was unable to update the Aviatrix Gateway configuration, resulting in a Gateway that was marked as “not up-to-date”. This has been corrected.

AVX-50640

In single IP HA Site2Cloud because of CSP underlay NAT issue, the single IP HA IPsec connection may not come up after the failover of a gateway. To address this, an enhancement was made in the way Aviatrix loads the IPSec configuration to the standby gateway so that the CSP underlay NAT issue is avoided.

AVX-51412

AVX-52626

There was an issue when modifying the remote subnet CIDR range of an existing Site2Cloud (S2C) connection using Terraform provider version 3.1.4 with Aviatrix Controller versions 7.1.3696 and 7.0.2239. Instead of updating the remote subnet CIDR range as specified in the Terraform configuration, the change was incorrectly applied to the local subnet CIDR range.

AVX-53179

AVX-53878

Fixed an issue where Transit Gateway peering tunnels in AWS and Azure could incorrectly show "Unknown" status. This could occur when recreating peering using the API or automation and only affected High Performance Encryption (HPE) enabled transit peering connections.

AVX-53986

Fixed an issue where the Aviatrix Controller was using excessive memory when managing large numbers of access accounts. Customers managing large numbers of access accounts should see improved Controller stability and performance after upgrading.

AVX-54035

Fixed an issue where license renewal failures could prevent creation of new gateways and tunnels. This occurred when the system attempted to renew licenses within 10 days of expiration. Renewal failed and backup acquisition consumed the remaining licenses.

AVX-54732

Fixed an issue where Aviatrix Edge Platform (AEP) Gateway upgrade status displayed incorrectly.After a successful image upgrade, the Controller showed an empty upgrade status and the CoPilot interface displayed the status as "unknown". This was only a display issue with no impact on Gateway functionality.

AVX-54897

Resolved a routing conflict that could disrupt the connection between Aviatrix Edge Gateways and the Controller using private IP. The system now properly handles routes learned from on-premises connections (BGP over LAN, IPSec, GRE) to prevent interference with controller communication.

AVX-55012

After upgrading to version 7.1, certain Source Network Address Translation (SNAT) IP addresses were not properly advertised to connected networks when manual connection summaries were configured. This has been corrected. Outbound traffic using the affected SNAT IP addresses now connects properly.This issue only affected BGP over IPSec connections between Transit Gateways and on-premises devices.

AVX-55092

Resolved a race condition that prevented the Layer7 engine process from initializing properly during system startup. This potentially disrupted normal network traffic handling on affected gateways.

AVX-55434

Resolved an issue where attaching a virtual network with both IPv4 and IPv6 address spaces could cause invalid routes to be added to the network, losing management connectivity. Data traffic was not affected. The workaround was to avoid attaching virtual networks with IPv6 enabled.

AVX-55474

AVX-56022

After a Spoke Gateway reboot, including from a resize or upgrade, the default route (0.0.0.0/0) advertised by the Spoke Gateway was removed from other connected VNet route tables. This resulted in loss of expected network connectivity between VNets.

AVX-56466

Resolved an issue affecting Azure Transit Gateways (with FireNet, VNG, or BGP over LAN enabled) where upgrading to 7.1.4139 resulted in additional routes being added to the Gateways’ secondary network interfaces.

AVX-56779

Fixed an issue where restore from backup fails during controller image upgrades.

AVX-56921

Resolved an issue where, during Azure service outages, resource handling incorrectly deleted all Azure resources from its database. This could cause brief interruptions in expected network traffic.

AVX-51456

Destination network address translation (DNAT) rules cannot be configured on Aviatrix Gateways using Terraform provider version 3.1.4. When setting up DNAT rules on standalone Gateways with policy-based tunnels configured, an error message indicates the interface for the connection cannot be found. To work around this issue, configure DNAT rules through the Aviatrix CoPilot interface at Cloud Fabric > Gateways. See Enabling Gateway DNAT Settings.

AVX-52095

If your Controller is running 7.1.4101, 7.1.3956, or earlier release (older Linux OS), you cannot upgrade directly to 7.2 or later releases. Upgrade to a release running the newer Linux OS (7.1.4183, 7.1.4139, 7.1.4105, 7.1.3958) before proceeding to any 7.2 releases.

AVX-55015

An issue can occur in handling Site2Cloud Mapped NAT connections when the local CIDR is set to 0.0.0.0/0. When a user edits or deletes a connection mapped to this CIDR, the corresponding IP table rule is not properly removed. This can cause incorrect routing behavior.

AVX-55379

When a remote BGP peered device initiates a graceful restart and stops its BGP session, BGP routes might not withdraw properly on Edge. Depending on the polling timing, the current BGP polling logic can send stale routes to the Controller once graceful restart occurs. This is particularly likely to happen when polling timers are shorter than graceful restart timers.

To work around this issue, disable graceful restart at the neighbor BGP device when it stops its BGP.

AVX-56499

The maximum number of CIDRs that can be enforced in a SmartGroup is 10,000. This limit includes both CIDR and tag-based resources in a SmartGroup. Anything beyond 10,000 CIDRs will be ignored and not enforced.

AVX-56595

SNAT/DNAT on Transit Edge peering is not supported in this release. Although the configuration is allowed, the routing with SNAT/DNAT is not currently working properly.

AVX-56778

When rolling back Aviatrix gateways from version 7.2 to 7.1, the rollback process does not block the operation if Site-to-Cloud (S2C) SmartGroup rules are enabled on the Aviatrix Gateways. The rollback completes successfully, but the S2C SmartGroup rules remain enforced on the rolled back 7.1 Gateways, potentially leading to connectivity issues.

To prevent issues, disable or remove any S2C SmartGroup rules before attempting to roll back Gateways from version 7.2 to 7.1.

This issue does not impact HPE or Public Subnet Filter gateways.

AVX-56827

Azure Security Group Azure Security Group Orchestration can take up to 30 minutes to update the Network Security Group (NSG) to be applied after the SgO is enabled. This can happen on a large scale set up, for example with 2 VPC/VNets with 100 subnets and 1000 VMs in each VNet.

AVX-57110

When creating a custom GeoGroup using Terraform and downloading the configuration from the Aviatrix Controller, the "match_expressions" block defining the country codes is missing.

AVX-57153

AVX-57245

With Single Availability Zone (AZ) HA feature enabled on an Aviatrix Gateway (it is enabled by default), when you reboot the Gateway, it might run into a reboot cycle. This is a timing related issue and could be hit if the Controller is busy with many tasks.

To avoid the recurring reboot or to bring the Gateway out of the reboot cycle, disable the Single AZ HA feature on the Gateway.

AVX-57342

The DCF Enforcement on External Connections feature can cause high CPU usage and delays in updating gateway configurations when enabled in a very large-scale test environment.

This is a Preview Feature and not Ideal for GA environments.

Disabling DCF Enforcement on External Connections feature resolves the high CPU usage and configuration delays. This feature is accessed at Security > Distributed Cloud Firewall > Settings.

AVX-57382

When creating SmartGroups for Azure resources using Terraform, the region formatting is incorrect. Incorrect region filters will cause SmartGroups to not work as intended for Azure resources grouped by region.

This issue affects creating Azure SmartGroups via Terraform only. It does not impact other clouds or creating groups through the CoPilot UI.

To address this issue, the region format must be updated in the Terraform provider code manually. You can get the exact region name using a command like az account list-locations --output table.

AVX-57551

Performing a like-version image replacement of an HA Gateway on GCP might result in network disruption of up to 4 minutes. This applies to a Gateway running version 7.2.4820. This does not apply to an image upgrade of a gateway running a 7.1 build.

You would run a like-version image replacement when a Gateway needs significant repair. For information about image upgrades, see Upgrade Gateways for the Latest Aviatrix Supported Images (AWS and Azure Only).

AVX-59843

If you have BGP sessions over IPsec and the BGP peer IPs are non-APIPA (169.254.x.x) IP addresses, the sessions might not be able to be established or reestablished after experiencing tunnel flapping. The BGP peer subnet route is installed by the kernel and this route is synced to a special route table which is used by BGP to send BGP packets to a peer. There is a failure to sync this route, therefore BGP packets can not reach remote BGP peer.

AVX-62067

Aviatrix Transit Gateways with large number of tunnels and running for a long time could encounter an issue where in the IPSec process becomes unresponsive leading to all IPsec tunnels going into a DOWN state. The cause of this is an internal counter reaching its maximum value and overflowing. To recover, the transit gateway needs to be rebooted.

While it is not possible to specify the exact number of tunnels and length of time it would take for the internal counter to overflow, the few customers who encountered this issue had greater than 800 tunnels and took three to four months to encounter this issue.