Skip to main content

Corrected Issues in Aviatrix Release 8.0.40

IssueDescription
AVX-64447Fixed an issue where toggling between Active/Active and Active/Standby modes in Site2Cloud connections was not working properly. Users can now successfully switch between these HA modes as expected.
AVX-66324Fixed an issue where bell notifications were missing for Distributed Cloud Firewall (DCF) L7 rules between Kubernetes pods and VMs when using HA gateways. Previously, traffic would work intermittently when DCF L7 rules were applied between Kubernetes services and VMs in different VPCs with HA gateways. The system now properly generates notifications when these rules are applied.
AVX-67530Fixed an issue where the traffic count displayed in the Controller interface could be inaccurate when using Distributed Cloud Firewall (DCF) with external groups that include multiple IP ranges. The Controller now reports traffic statistics correctly for DCF rules involving external groups, providing accurate visibility for monitoring, analysis, and validation of firewall policy behavior.
AVX-69733Resolved an issue where the ESTABLISHED rule disappeared after a Public Subnet Filtering (PSF) gateway image upgrade. This issue affected PSF gateways using the legacy stateful firewall on Controller versions 7.1 and later, and could result in traffic disruption after the upgrade. The rule is now preserved during PSF gateway image upgrades.
AVX-70253Fixed an issue where FireNet deployments with bootstrap enabled could fail in Google Cloud due to changes in how GCP credentials were handled during the bootstrap process. The bootstrap workflow has been updated to correctly retrieve and use GCP credentials, ensuring FireNet deployments with bootstrap complete successfully in Google Cloud environments.
AVX-71087Fixed an issue where the default access control rules did not properly allow ICMP traffic used for debugging. The updated rules ensure ICMP-based troubleshooting continues to work after upgrades.
AVX-71217Resolved an issue where the VRRP state file could become empty when upgrading AEP and self-managed Edge-as-Spoke gateways in active-active HA configurations from version 7.2 to 8.0.30. With this fix, VRRP primary and backup state information is preserved during the upgrade, and newly created Edge-as-Spoke gateways with VRRP configuration no longer remain in the Initializing state.

Known Issues in Aviatrix Release 8.0.40

IssueDescription
AVX-62003Azure gateway image upgrades may fail when the Controller does not have the required Azure image subscription access. During the upgrade, the system deletes the existing gateway before validating subscription availability, which can result in gateway deletion without a replacement being created. This leaves dangling gateways in the Controller and can cause potential service outages. Impact: Existing gateways may be deleted during image upgrade. Replacement gateway creation fails due to missing subscription. Customers may experience connectivity loss and dangling gateway entries in the Controller. Manual intervention required, leading to support escalations. Workaround: None. To avoid outages, ensure the Controller subscription includes access to the required Azure image before attempting upgrades.
AVX-62299When upgrading from Controller version 7.1 to 7.2 or 8.0, Spoke Gateways with routing through a Public Subnet Filtering (PSF) Gateway may fail to upgrade and become unreachable if the PSF Gateway has not been upgraded first. This issue affects AWS environments where Spoke Gateway route tables are configured to point to a PSF Gateway. To avoid this issue, follow the correct upgrade sequence: 1) Upgrade the PSF Gateway first. 2) Wait for the PSF Gateway upgrade to complete successfully. 3) Then upgrade the dependent Spoke Gateways.
AVX-62506During a gateway software upgrade, traffic matching DCF WebGroup rules may be briefly dropped during the upgrade. This impacts both Layer 7 (HTTP/HTTPS) and Layer 4 traffic and occurs across all supported cloud providers (AWS, Azure, and GCP). The disruption typically lasts a few seconds but may vary depending on gateway load and policy complexity. Workaround: None. Recommendations: Schedule gateway upgrades during maintenance windows or low-traffic periods. Use HA deployments and upgrade gateways one at a time in HA pairs. Monitor logs for “Failed to load policy” messages to confirm when policies are reloaded.
AVX-63224In Controller release 8.0, gateway software upgrades take longer to complete compared to earlier versions. On average, the upgrade rate drops from approximately 14 gateways per minute in version 7.2 to approximately 11 gateways per minute in 8.0, which is an increase of about 20% in execution time. Affected Scenarios: Upgrading from version 7.2.x to 8.0.x. Upgrading between 8.0.x versions. Impact: Only the upgrade duration is affected. Gateway functionality remains unaffected after a successful upgrade. Recommendations: Allocate approximately 20% more time for gateway upgrades. For large environments (for example, 1,000+ gateways), plan for 90–120 minutes of upgrade time. Schedule upgrades during maintenance windows to accommodate the longer duration.
AVX-64502On Azure gateways with High Performance Encryption (HPE) enabled, an underlay network issue may cause the eth0 interface to drop, bringing the interface flap. When this occurs, the DHCP-assigned primary IP address may be released while the static IP remains, resulting in one of the static IPs being promoted as the primary address. This can impact gateway operations. Impact: The gateway and its associated tunnels may go down, resulting in traffic disruption. Workaround: Stop and start the affected gateway from the cloud service provider console.
AVX-64868In some scenarios involving rapid VRRP state transitions, the keepalived VRRP state may not be reported accurately to the Controller. This can result in temporary discrepancies between the actual VRRP status and what is displayed in the Controller UI, leading to confusion and difficulties during troubleshooting. Impact: Controller UI may show incorrect VRRP status such as both peers reporting Primary or Initializing. No impact on actual VRRP traffic handling or failover behavior. Workaround: Use diagnostic logs to verify actual VRRP state.
AVX-65016In some environments, the Firewall state may not recover from Unaccessible after the first vendor integration failure. This issue has been observed when integrating with third-party firewall vendors, leaving the gateway firewall state stuck even after the environment stabilizes. Impact: Firewall integration appears stuck in Unaccessible state. Recovery does not occur automatically after initial failure. May require manual intervention to restore proper firewall state reporting. Workaround: Contact Aviatrix Support for manual correction.
AVX-66631When performing image upgrades on Transit gateways with a large number of tunnels (1300+ tunnels), traffic loss occurs after the upgrade completion. Affected Scenario: Transit gateways with scale number of tunnels during image upgrade operations. Impact: Traffic disruption lasting approximately 9 minutes after upgrade completion. Affects high-scale Transit gateway deployments. Service interruption during critical upgrade windows. Workaround: Schedule image upgrades during maintenance windows to minimize business impact. Consider upgrading Transit gateways with fewer tunnels first to reduce exposure time.
AVX-66696When DCF processes high volumes of logging messages, rsyslogd rate-limiting may cause message loss. The system drops messages exceeding 500 per 5-second interval, with rsyslogd logging “messages lost due to rate-limiting” notifications. Affected Scenario: High-traffic environments generating intensive logging activity. Impact: Log messages may be dropped during peak traffic periods. Potential gaps in audit trails and monitoring data. Reduced visibility into network events and troubleshooting information. Workaround: Monitor rsyslogd logs for rate-limiting messages and consider implementing log aggregation strategies to distribute message processing load across multiple collection points.
AVX-67126Dry-run validation may fail when upgrading the Controller from version 8.0.10 to 8.1.0 due to a gateway version mismatch error. This occurs when the upgrade path starts from 8.0.0, progresses to 8.0.10 successfully, but encounters a dry-run failure when proceeding to 8.1.0.
AVX-67571In Oracle Cloud Infrastructure (OCI) environments, OpenVPN clients cannot connect to VPN gateways configured with DUO multi-factor authentication (MFA). Connection attempts fail with ECONNREFUSED errors during tunnel establishment, preventing authentication from completing. Impact: VPN tunnels cannot be established to DUO-enabled OCI gateways. Only affects OCI deployments with DUO MFA. Other authentication methods (OKTA, LDAP) work normally. Workaround: No current workaround. Users may temporarily switch to OKTA or LDAP authentication if feasible.
AVX-68561When DCF S2C is enabled in large scale deployments with 1300+ gateways, all gateway configurations become out of sync with the Controller. Affected Scenario: Large scale environments with DCF S2C enabled and high gateway counts (1300+ gateways). Impact: Gateway configurations show as out of sync in the Controller UI. Controller CPU utilization increases significantly. Conduit process consumes high CPU resources. Workaround: Disable DCF S2C feature in large scale deployments until the issue is resolved. Monitor gateway sync status and Controller CPU usage when re-enabling the feature.
AVX-68606During Gateway Software Upgrade operations involving large numbers of gateways, AEP EaS gateways may experience Charon service restarts that cause temporary traffic disruption. The strongSwan Charon process stops and restarts during the upgrade window, creating connectivity gaps for IPsec tunnels. Affected Scenario: Large-scale gateway upgrades (1000+ gateways) in testbed environments with AEP EaS configurations. Impact: Temporary traffic loss through AEP EaS during upgrade operations. IPsec tunnel connectivity interruption during Charon restart. Service disruption lasting approximately 30-60 seconds per affected gateway. Workaround: Schedule gateway upgrades during maintenance windows and upgrade gateways in smaller batches to minimize simultaneous impact across the network infrastructure.
AVX-68726On Azure Controllers with Controller Security Group Management enabled, gateway deployments may fail when multiple gateways or HA gateways are created. In this scenario, network security group rules may be overwritten or duplicated, which can cause Azure to return duplicate rule name errors. As a result, new gateways may fail to launch, and the Controller may automatically disable Security Group Management. Impact: Gateway deployments may fail in Azure. Security group rules may not be updated correctly. Controller Security Group Management may be disabled unexpectedly. Workaround: Disable Controller Security Group Management and manage Azure network security group rules manually, or contact Aviatrix Support for assistance.
AVX-68887When attaching VPN users to profiles using the attach_vpn_user_to_profile API, the CoPilot or Controller UI may continue to display the user profile as N/A even though the attachment operation completes successfully. In some cases, users later reappear as active but still show no profile association in the UI. This results in a display inconsistency between the UI and the backend state. Impact: VPN user profile assignments may appear unsuccessful in the UI, which can cause confusion during profile management. There is no functional impact: the VPN profile is correctly assigned in the backend, and users can connect to the VPN as expected. Affected Scenario: OpenVPN profile management operations that use API-based user-to-profile attachment. Workaround: None.
AVX-71489When the Controller has many accounts and inventory types configured, the public.inventory table in the database grows excessively large due to duplicate entries being inserted for each inventory operation instead of updating existing records. Impact: Database performance degradation due to table size growth. Increased storage requirements and maintenance overhead. Potential system slowdowns during inventory operations. Affected Scenario: Controllers with multiple accounts and various inventory types configured. Workaround: Monitor database size and consider periodic cleanup of old inventory records through database maintenance during low-usage periods.
AVX-71494When CoPilot Asset Inventory (CAI) performs queries on the inventory table, the existing database indexes are not utilized effectively, causing performance degradation during inventory operations. Affected Scenario: CAI inventory queries searching across cloud service providers, account names, subdomains, and resource counts experience slower response times. Impact: Delayed inventory data retrieval and display. Increased database load during CAI operations. Slower performance when viewing asset inventory reports. Workaround: None.
AVX-71630On Azure Aviatrix gateways with accelerated networking enabled and using Distributed Cloud Firewall (DCF) features, intermittent traffic drops may occur after upgrading from a version earlier than 7.2.2994 to 7.2.2994 or later. This issue is caused by incorrect eBPF filters being applied to the slave eth1 interface during the upgrade process. Affected Scenario: Azure gateways with accelerated networking enabled. DCF features enabled. Upgrading from a version prior to 7.2.2994 to 7.2.2994 or later. Impact: Intermittent traffic drops across affected gateways. Workaround: Contact Aviatrix Support for assistance.
AVX-71672When upgrading the Controller to version 8.1, the database migration may fail if the tunnel rtt_avg field contains None values. The migration logic expects either a numeric value or the string "N/A", and encountering a None value causes the upgrade to stop. Impact: Upgrade to 8.1 cannot complete. Controller remains on the previous version. Workaround: Contact Aviatrix Support for assistance in correcting the database values before retrying the upgrade.
AVX-71686Azure controllers using default P6 disk tier (240 IOPS) may experience performance issues, particularly with 8.x containerized deployments. This limitation can cause system instability and processing delays during high I/O operations. Affected Scenario: Controllers launched from Azure marketplace AMI with default disk configuration. Impact: System instability during high I/O operations. Processing delays and performance degradation. Potential service disruptions in production environments. Workaround: Upgrade the Azure controller disk tier from default P6 to minimum P10 (500 IOPS) through Azure portal disk configuration settings.
AVX-71820When deploying a load balancer–enabled VPN gateway with an overlapping VPN CIDR on Controller versions 8.0, 8.1, or 8.2, the gateway creation fails. Impact: VPN gateway deployment fails. Error message does not clearly indicate the root cause. Affected Scenario: Load balancer–enabled VPN gateway deployments on Controller versions 8.0, 8.1, and 8.2. Workaround: Ensure that the VPN CIDR does not overlap with existing gateways behind the load balancer before deployment. Contact Aviatrix Support for assistance.
AVX-72847The avx-gw-state-sync service may leak D-Bus connections on gateways which can gradually exhaust available D-Bus connections on the gateway. Impact: In environments with long running gateways, the leak may eventually prevent systemd services from progressing and could lead to resource exhaustion during gateway operations or upgrades. Workaround: Restart the avx-gw-state-sync service on the gateway to release the leaked connections. Or perform a gateway software upgrade to 7.2.5105 or 8.0.50 to solve this issue permanently.
AVX-73036In some environments running 8.0.30 or later builds, duplicate iptables mangle table MARK rules may remain on gateways during mapped Site-to-Cloud tunnel failover, gateway image upgrade, or rollback scenarios. These rules may accumulate due to incomplete cleanup during tunnel role transitions. Impact: No traffic impact has been observed. The issue only affects residual rule cleanup on the gateway. Workaround: No workaround is required. The extra rules do not affect traffic forwarding.
AVX-73589In some high-traffic environments using FQDN filtering, the NFQ process may stall due to a deadlock. If the signal interrupts a thread that is already executing a non-reentrant function, the signal handler may attempt to acquire the same lock, causing a deadlock. Impact: The avx-nfq process may stall and stop processing traffic until the service is restarted. Workaround: Restart the instance to continue processing traffic.
AVX-73836In environments where Duo Authentication is enabled for Client VPN, Duo-authenticated users may intermittently fail to connect to the VPN Gateway. The gateway may log the following error message: Duo OpenVPN: Received 403 Client duo_openvpn version 2.4 is deprecated and no longer supported. This occurs because the gateway uses an older Duo OpenVPN client library version that is no longer supported by the Duo service. Impact: Users configured with Duo authentication may fail to establish VPN connections. In some cases, bypass users may connect intermittently. Workaround: Update the Duo OpenVPN client version on the gateway by modifying the version in the duo_openvpn.py file from 2.4 to 3.0.