Corrected Issues in Aviatrix Release 10.0.1
| Issue | Description |
|---|---|
| AVX-78190 | Fixed an issue where GCP controller upgrades from 8.2.x to 9.0.10 could fail during the security group configuration migration step. GCP controller upgrades that previously failed with a JSON unmarshalling error now complete successfully. |
| AVX-78616 | Fixed an issue where the error message for an ActiveMesh HA pair drain failure was unclear. Starting in Controller 10.0.1, the error message is updated to clearly state that the primary gateway and its HA peer must be submitted in separate requests. The underlying restriction is unchanged: gateway image-upgrade or delete requests that include both members of an ActiveMesh HA pair in a single call can still partially fail, so the two members should continue to be submitted in separate requests. |
Known Issues in Aviatrix Release 10.0.1
| Issue | Description |
|---|---|
| AVX-62299 | When upgrading from Controller version 7.1 to 7.2 or 8.0, Spoke Gateways with routing through a Public Subnet Filtering (PSF) Gateway may fail to upgrade and become unreachable if the PSF Gateway has not been upgraded first. This issue affects AWS environments where Spoke Gateway route tables are configured to point to a PSF Gateway. To avoid this issue, follow the correct upgrade sequence:1. Upgrade the PSF Gateway first. 2. Wait for the PSF Gateway upgrade to complete successfully. 3. Then upgrade the dependent Spoke Gateways. |
| AVX-64868 | In some scenarios involving rapid VRRP state transitions, the keepalived VRRP state may not be reported accurately to the Controller. This can result in temporary discrepancies between the actual VRRP status and what is displayed in the Controller UI, leading to confusion and difficulties during troubleshooting. Impact:
|
| AVX-66631 | Transit gateways with large-scale tunnel deployments (1300+ tunnels) may experience extended traffic loss during image upgrades. Although the image upgrade completes successfully, traffic may remain down for several minutes afterward due to delayed tunnel reconfiguration. Workaround:
|
| AVX-67126 | Dry-run validation may fail when upgrading the Controller from version 8.0.10 to 8.1.0 due to a gateway version mismatch error. This occurs when the upgrade path starts from 8.0.0, progresses to 8.0.10 successfully, but encounters a dry-run failure when proceeding to 8.1.0. |
| AVX-70543 | When HA-enabled spoke gateways have DPI/IDS or Layer7 policies configured with “Destination: Anywhere” and the destination smart group contains private CIDRs, the policies become invalid and cause traffic drops. Affected Scenario: Spoke gateways with HA enabled using DPI/IDS or Layer7 policies where destination smart groups include private CIDR ranges. Impact:
|
| AVX-70958 | When clients use HTTP/2 connections, TrafficServer incorrectly reuses origin connections, potentially causing connection handling issues in MITM SNI verification scenarios. Impact:
Use both IP address and SNI instead of IP alone to ensure proper connection isolation. |
| AVX-71320 | Source IP translation (SNAT) does not function correctly for policy-based Site-to-Cloud (S2C) connections when traffic flows from cloud to site. The translated source IP is not applied, and the original source IP is used instead. Impact:
No workaround is available for this issue. Upgrade to Controller 10.1.0 or later, where this issue is fixed. |
| AVX-71559 | When performing a batch gateway image upgrade in Azure, some gateways may fail during the upgrade process. Impact:
Retry the gateway image upgrade for the failed gateways individually rather than as a batch operation. Contact Aviatrix Support for assistance. |
| AVX-71820 | When deploying a load balancer–enabled VPN gateway with an overlapping VPN CIDR on Controller versions 8.0, 8.1, or 8.2, the gateway creation fails. Impact:
Workaround: Ensure that the VPN CIDR does not overlap with existing gateways behind the load balancer before deployment. Contact Aviatrix Support for assistance. |
| AVX-72940 | Creating a new gateway with the same name as an existing gateway may cause local files of the existing gateway to be deleted when the creation fails. The existing gateway name disappears from the Controller CLI when this issue occurs. This can break SSH access (sshgw) for the existing gateway. Impact:
|
| AVX-73334 | After upgrading a VPN gateway image to Controller 9.0 with FIPS 140-3 enabled, the Aviatrix VPN Client (AVPNC 2.17.7, which uses OpenVPN 2.4.9) cannot connect to the gateway and returns AUTH_FAILED — client incompatible with this server. FIPS 140-3 on the gateway side requires OpenVPN 2.6.0 or later (RFC 5705 TLS Keying Material Exporters), which AVPNC 2.17.7 does not support.Impact:
|
| AVX-73436 | When using the update_spoke_vpc_route_table API to onboard an Azure route table, the default route (0.0.0.0/0) is not programmed in the spoke VNET route table if the Spoke Gateway has learned the default route from any of the following sources:
Impact:
Manually add the default route to the Azure route table. Contact Aviatrix Support for assistance. |
| AVX-75586 | When using Terraform to create gateways and immediately attach them to transit gateways, a race condition can occur where the Terraform provider attempts to create the spoke-to-transit attachment before the gateway has fully transitioned to the “Up” state. The gateway creation API returns before the gateway is operationally ready, causing subsequent attachment operations to fail. Impact:
Add a delay or polling mechanism in Terraform configurations between gateway creation and spoke-to-transit attachment resources. Use depends_on with a time_sleep resource to allow the gateway to reach the “Up” state before attempting attachment. |
| AVX-75599 | The TrafficServer (ATS) process on gateways may crash with a signal 6 (Aborted) when a custom Suricata rule feed containing a large number of rules (20,000+) is applied to a DCF intrusion analysis profile. The crash occurs due to an assertion failure in the IDS inspection queue handling, causing traffic interruption on affected gateways. Impact:
Reduce the number of custom rules to under 1,000, or remove the custom rule feed from the intrusion analysis profile and use the system default IPS profile until a fix is available in a future release. Contact Aviatrix Support for assistance. |
| AVX-76132 | Unable to configure more than one OpenVPN gateway behind a UDP Load Balancer. Only the first gateway is retained while additional gateways are incorrectly excluded. This issue affects versions 6.9, 7.x, 8.x, and 9.0.0. Impact:
Contact Aviatrix Support for assistance in applying the workaround. |
| AVX-76413 | Description: When the controller’s gateway-status tracking restarts and finds gateways recorded as “Down,” the controller reprograms the network as if those gateways are down, without allowing time for the gateways to reconnect and confirm they are up. This can occur when the controller has previously lost connectivity to gateways (for example, during a transient network issue) and the tracking service then restarts while the gateways themselves remain healthy and continue forwarding traffic. Impact: A temporary data plane disruption occurs while the controller has the gateways marked as down. The controller automatically detects the gateways are up and reprograms the network correctly immediately afterward. In a large-scale user deployment (~1,400 gateways), full recovery completed in approximately 6 minutes. The underlying behavior has existed for 4+ years and has been observed in a user environment only once. Workaround: Not applicable. The system recovers automatically, no user action required. Recovery typically completes within minutes (~6 min observed in a ~1,400-gateway deployment; smaller deployments recover faster). |
| AVX-77135 | When the Use SSL to connect option is enabled for LDAP configuration, VPN gateway configuration updates may not apply correctly. Impact:
Contact Aviatrix Support for assistance. |
| AVX-77490 | The Controller UI displays a “Syntax Error” message when creating an HA gateway for a spoke or transit gateway. The Controller UI is the only affected surface. CoPilot and Terraform both create HA gateways successfully on Controller 10.0.1. Impact: HA cannot be enabled on spoke or transit gateways through the Controller UI on Controller 10.0.1. CoPilot and Terraform are not affected. Workaround: Use one of the following on Controller 10.0.1:
|
| AVX-77618 | Running Controller diagnostics may report NTP-related errors that do not reflect the actual NTP synchronization status of the Controller. Impact:
Verify Controller time synchronization through other means before acting on NTP-related diagnostic errors. Contact Aviatrix Support for assistance. |
| AVX-77714 | When creating a Smart Group with the Kubernetes workload resource type, attempting to use Region, Account Name, or Account ID as a match criterion fails with error AVXERR-SMARTGROUP-0007 (“invalid selector combination for k8s resource type”). Region, Account Name, and Account ID are not supported selectors for the Kubernetes workload resource type. Impact:
Use one of the supported Kubernetes workload selector combinations (cluster ID, namespace, service, or pod, alone or in supported groupings) when creating Smart Groups. Contact Aviatrix Support for assistance. |
| AVX-78500 | When Distributed Cloud Firewall (DCF) is disabled on a gateway, DCF policy enforcement may not be fully removed from all gateway interfaces. As a result, traffic can continue to be evaluated against the previously applied DCF policy even though DCF appears disabled. This can occur when DCF is disabled without any accompanying policy change. Impact: * After DCF is disabled, a gateway may continue enforcing the last applied DCF policy on some interfaces. * This residual enforcement is not reflected in the DCF status shown for the gateway and may persist until DCF is re-enabled or the gateway configuration changes. Workaround: When disabling DCF, also make a minor DCF policy change (for example, add and then remove a rule) so enforcement is fully cleared from all interfaces. If you suspect DCF policy is still being enforced after disabling DCF, contact Aviatrix Support. |
| AVX-78524 | Due to delays in the Azure Marketplace publishing process, the Azure China Gateway VM image (Azure China North 3, publisher cbcnetworks, offer aviatrix-gateway) will not be available in Azure China when Controller 10.0.1 is released. Attempting to launch a new Azure China Gateway on Controller 10.0.1 fails with a marketplace-offer-not-found error (AVXERR-TRANSIT-0067).Impact: Customers with Azure China (China North 3) gateways cannot deploy new Azure China Gateways on Controller 10.0.1. Workaround: Customers with Azure China gateways should remain on the Controller 9.0 series until the Azure Marketplace offer is published and this Known Issue is resolved. |
| AVX-78671 | Enabling Logging on the Default Action Rule and then disabling it can cause the Monitor tab to continue showing log entries for that rule for up to one hour after Logging is turned off. Impact: * After you disable Logging on the Default Action Rule (following a period when Logging was enabled and traffic was flowing), the Monitor tab continues to display log entries for that rule for up to one hour. * Excess log volume during that hour can make it harder to review logs for other Distributed Cloud Firewall (DCF) rules. Workaround: No action is required. Logging for the Default Action Rule stops automatically once the one-hour period elapses. Until then, you can filter Monitor tab entries by rule name to distinguish residual Default Action Rule logs from current rule activity. |
| AVX-79012 | Some gateways may become stuck in a failed configuration-push state for an extended period after an Image Upgrade to Aviatrix 10.0.1 (from 9.0.10). This has been observed primarily on lower-memory gateway instance types (for example, AWS t3.micro). Impact: Affected gateways may show a “Not Up-To-Date” config status and a down backup tunnel hours after the image upgrade completes. The root cause is a Distributed Cloud Firewall (DCF) configuration push failure caused by insufficient available memory on memory-constrained instance types. Workaround: Retry the gateway Image Upgrade. If the gateway remains stuck in this state, increase the gateway’s instance size for additional memory headroom, or contact Aviatrix Support. |
| AVX-79173 | During a Controller-initiated image upgrade, an Aviatrix gateway can intermittently be left in a partially configured state because of a timing-related issue in the gateway’s post-upgrade configuration sequence. Impact: The affected gateway’s image upgrade is reported as only partially successful. The gateway remains in a partially configured state until an administrator intervenes to restore it to the UP state. Workaround: Retry the image upgrade on the affected gateway. Because the issue is timing-related, a subsequent retry typically succeeds without further action. |
| AVX-79521 | On Controller versions prior to 10.1.0, it is possible to onboard multiple Aviatrix access accounts that map to the same underlying cloud service provider (CSP) account. Because several Controller features assume a one-to-one relationship between an Aviatrix access account and a CSP account, duplicate access accounts can lead to unpredictable configuration and operational behavior. Duplicate is defined per CSP as follows:
Impact:
Onboard only one Aviatrix access account per underlying CSP account. If duplicate access accounts have already been onboarded and are causing issues, contact Aviatrix Support for assistance with identifying and removing the duplicates. |
| AVX-79855 | When Distributed Cloud Firewall is disabled on a transit gateway with Site2Cloud (S2C) or other external connections landing on it, and no policy change is queued at the time, existing DCF filtering rules on the interface are not removed. The disable path is skipped when there is no policy difference to push. Impact:
Restart the Aviatrix gateway to remove the stale DCF filter rules. Alternatively, contact Aviatrix Support to have the stale DCF filter rules manually removed. |
| AVX-80347 | When a packet capture is started and then stopped from the CoPilot UI (Gateway Diagnostics > Packet Capture) on a gateway, the Controller can continue processing the stopped capture in the background even after the UI shows it as stopped. Impact:
Start a packet capture on a different gateway to clear the stuck background process for the previously affected gateway. If the Controller becomes unresponsive, contact Aviatrix Support for assistance restoring access. A fix is planned for a future maintenance release. |
| AVX-80983 | When Azure Security Group Management reads the current state of a security group rule before adding a new gateway’s IP address to it during gateway launch, an Azure API throttling response on that read can be misinterpreted as the rule not existing. In that case, the existing rule is deleted and a new rule is created instead of the existing rule being updated. Impact:
Contact Aviatrix Support if you observe unexpected security group rule changes on gateway launch in an Azure environment. |
| AVX-80977 | A VPN user’s profile assignment could appear as N/A immediately after the user was attached to a profile through the API, until the user’s next reconnection. Impact:
Have the affected user reconnect to update the displayed profile assignment. Contact Aviatrix Support for assistance. |
| AVX-81892 | On gateways enforcing Distributed Cloud Firewall (DCF) policies that include SmartGroups with FQDN selectors, an internal policy-update path can deadlock in rare cases, causing the gateway data-plane process to stop forwarding traffic while remaining alive. The deadlock is more likely when the FQDN selector resolves to a set of addresses that rotate frequently (short DNS TTL), because each resolution change triggers a policy reload that races with an internal hit-counter update. Impact:
|