8.0.50 Release Notes

Resolved an issue where incorrect eBPF filters could be applied to the eth1 interface on Azure gateways with Accelerated Networking enabled during upgrades from versions earlier than 7.2.2994.

The upgrade process now properly handles interface filter configuration to prevent unintended traffic drops.

Resolved an issue where the avx-gw-state-sync service leaked D-Bus connections on gateways.

The implementation has been updated to use a shared D-Bus connection that is properly managed and cleaned up, preventing connection exhaustion during gateway operations.

Resolved an issue where duplicate iptables mangle table MARK rules and remote gateway route tables were not properly cleaned up during Site-to-Cloud mapped tunnel failover, gateway image upgrade, or rollback scenarios.

This issue could occur when migrating environments between releases or during tunnel role transitions. The cleanup logic has been corrected to ensure redundant rules are removed and route tables are properly maintained.

Resolved an issue where the NFQ process could stall due to a deadlock when flushing IP data nightly.

The implementation has been updated to prevent the deadlock condition and ensure the avx-nfq process continues to process traffic normally under high load.

Azure gateway image upgrades may fail when the Controller does not have the required Azure image subscription access. During the upgrade, the system deletes the existing gateway before validating subscription availability, which can result in gateway deletion without a replacement being created. This leaves dangling gateways in the Controller and can cause potential service outages.

Impact:

Workaround:

None. To avoid outages, ensure the Controller subscription includes access to the required Azure image before attempting upgrades.

When upgrading from Controller version 7.1 to 7.2 or 8.0, Spoke Gateways with routing through a Public Subnet Filtering (PSF) Gateway may fail to upgrade and become unreachable if the PSF Gateway has not been upgraded first. This issue affects AWS environments where Spoke Gateway route tables are configured to point to a PSF Gateway.

To avoid this issue, follow the correct upgrade sequence:

During a gateway software upgrade, traffic matching DCF WebGroup rules may be briefly dropped during the upgrade. This impacts both Layer 7 (HTTP/HTTPS) and Layer 4 traffic and occurs across all supported cloud providers (AWS, Azure, and GCP). The disruption typically lasts a few seconds but may vary depending on gateway load and policy complexity.

Workaround:

None

Recommendations:

In Controller release 8.0, gateway software upgrades take longer to complete compared to earlier versions. On average, the upgrade rate drops from approximately 14 gateways per minute in version 7.2 to approximately 11 gateways per minute in 8.0, which is an increase of about 20% in execution time.

Affected Scenarios:

Impact:

Only the upgrade duration is affected. Gateway functionality remains unaffected after a successful upgrade.

Recommendations:

In some scenarios involving rapid VRRP state transitions, the keepalived VRRP state may not be reported accurately to the Controller. This can result in temporary discrepancies between the actual VRRP status and what is displayed in the Controller UI, leading to confusion and difficulties during troubleshooting.

Impact:

Workaround:

In some environments, the Firewall state may not recover from Unaccessible after the first vendor integration failure. This issue has been observed when integrating with third-party firewall vendors, leaving the gateway firewall state stuck even after the environment stabilizes.

Impact:

Workaround:

Contact Aviatrix Support for manual correction.

When performing image upgrades on Transit gateways with a large number of tunnels (1300+ tunnels), traffic loss occurs after the upgrade completion.

Affected Scenario: Transit gateways with scale number of tunnels during image upgrade operations.

Impact:

Workaround: Schedule image upgrades during maintenance windows to minimize business impact. Consider upgrading Transit gateways with fewer tunnels first to reduce exposure time.

When DCF processes high volumes of logging messages, rsyslogd rate-limiting may cause message loss. The system drops messages exceeding 500 per 5-second interval, with rsyslogd logging "messages lost due to rate-limiting" notifications.

Affected Scenario: High-traffic environments generating intensive logging activity

Impact:

Workaround: Monitor rsyslogd logs for rate-limiting messages and consider implementing log aggregation strategies to distribute message processing load across multiple collection points.

In Oracle Cloud Infrastructure (OCI) environments, OpenVPN clients cannot connect to VPN gateways configured with DUO multi-factor authentication (MFA). Connection attempts fail with ECONNREFUSED errors during tunnel establishment, preventing authentication from completing.

Impact:

Workaround:

No current workaround. Users may temporarily switch to OKTA or LDAP authentication if feasible.

When DCF S2C is enabled in large scale deployments with 1300+ gateways, all gateway configurations become out of sync with the Controller.

Affected Scenario: Large scale environments with DCF S2C enabled and high gateway counts (1300+ gateways).

Impact:

Workaround: Disable DCF S2C feature in large scale deployments until the issue is resolved. Monitor gateway sync status and Controller CPU usage when re-enabling the feature.

During Gateway Software Upgrade operations involving large numbers of gateways, AEP EaS gateways may experience Charon service restarts that cause temporary traffic disruption. The strongSwan Charon process stops and restarts during the upgrade window, creating connectivity gaps for IPsec tunnels.

Affected Scenario: Large-scale gateway upgrades (1000+ gateways) in testbed environments with AEP EaS configurations.

Impact:

Workaround: Schedule gateway upgrades during maintenance windows and upgrade gateways in smaller batches to minimize simultaneous impact across the network infrastructure.

On Azure Controllers with Controller Security Group Management enabled, gateway deployments may fail when multiple gateways or HA gateways are created.

In this scenario, network security group rules may be overwritten or duplicated, which can cause Azure to return duplicate rule name errors. As a result, new gateways may fail to launch, and the Controller may automatically disable Security Group Management.

Impact:

Workaround:

Disable Controller Security Group Management and manage Azure network security group rules manually, or contact Aviatrix Support for assistance.

When attaching VPN users to profiles using the attach_vpn_user_to_profile API, the CoPilot or Controller UI may continue to display the user profile as N/A even though the attachment operation completes successfully.

In some cases, users later reappear as active but still show no profile association in the UI. This results in a display inconsistency between the UI and the backend state.

Impact: VPN user profile assignments may appear unsuccessful in the UI, which can cause confusion during profile management. There is no functional impact: the VPN profile is correctly assigned in the backend, and users can connect to the VPN as expected.

Affected Scenario: OpenVPN profile management operations that use API-based user-to-profile attachment.

Workaround: None.

When the Controller has many accounts and inventory types configured, the public.inventory table in the database grows excessively large due to duplicate entries being inserted for each inventory operation instead of updating existing records.

Impact:

Affected Scenario: Controllers with multiple accounts and various inventory types configured.

Workaround: Monitor database size and consider periodic cleanup of old inventory records through database maintenance during low-usage periods.

When CoPilot Asset Inventory (CAI) performs queries on the inventory table, the existing database indexes are not utilized effectively, causing performance degradation during inventory operations.

Affected Scenario: CAI inventory queries searching across cloud service providers, account names, subdomains, and resource counts experience slower response times.

Impact:

Workaround: None.

When upgrading the Controller to version 8.1, the database migration may fail if the tunnel rtt_avg field contains None values. The migration logic expects either a numeric value or the string "N/A", and encountering a None value causes the upgrade to stop.

Impact:

Workaround: Contact Aviatrix Support for assistance in correcting the database values before retrying the upgrade.

When deploying a load balancer–enabled VPN gateway with an overlapping VPN CIDR on Controller versions 8.0, 8.1, or 8.2, the gateway creation fails.

Impact:

Affected Scenario: Load balancer–enabled VPN gateway deployments on Controller versions 8.0, 8.1, and 8.2.

Workaround:

Ensure that the VPN CIDR does not overlap with existing gateways behind the load balancer before deployment. Contact Aviatrix Support for assistance.

In environments where Duo Authentication is enabled for Client VPN, Duo-authenticated users may intermittently fail to connect to the VPN Gateway.

The gateway may log the following error message:

This occurs because the gateway uses an older Duo OpenVPN client library version that is no longer supported by the Duo service.

Impact:

Users configured with Duo authentication may fail to establish VPN connections. In some cases, bypass users may connect intermittently.

Workaround:

Update the Duo OpenVPN client version on the gateway by modifying the version in the duo_openvpn.py file from 2.4 to 3.0.