Controller and Gateway Release Notes

To review the latest Release Notes, please visit this page: https://docs.aviatrix.com/documentation/latest/release-notes/controller-software-release-notes/controller-software-release-notes.html

Aviatrix strongly recommends you perform the tasks in the operations checklist including a dry run upgrade before upgrading your deployment of the Aviatrix network platform. Taking the time to perform dry runs and backing up your Aviatrix Platform configuration reduces the potential for issues during the upgrade and allows you to easily restore your configuration if there are issues after the upgrade. Correct any issues you find during your preparation before proceeding with an Aviatrix upgrade. For more information, see Upgrading the Aviatrix Platform.

If you cannot resolve all issues after following the preparation and dry run procedures, please open a ticket with Aviatrix Support.

This page provides release specific information including known and corrected issues. For information about new and enhanced features and behaviors see What’s New.

7.0.2250 Release Notes

Release Date: 21 March 2024

Issues Corrected in Aviatrix Release 7.0.2250

Issue	Description
AVX-50895	Customers using an Aviatrix Controller to orchestrate AWS Transit Gateways (TGWs) can encounter a software defect wherein the Aviatrix software might raise a false positive warning about duplicated CIDRs, which could impact route propagation. This issue occurs when you have two or more AWS TGWs and have TGW peering between them. The false positive warning can be raised on unrelated VPCs. If there are duplicated CIDRs in any TGW attachments in peered TGWs, routing propagation could be impacted.

Issue

Description

AVX-50895

Customers using an Aviatrix Controller to orchestrate AWS Transit Gateways (TGWs) can encounter a software defect wherein the Aviatrix software might raise a false positive warning about duplicated CIDRs, which could impact route propagation.

This issue occurs when you have two or more AWS TGWs and have TGW peering between them. The false positive warning can be raised on unrelated VPCs. If there are duplicated CIDRs in any TGW attachments in peered TGWs, routing propagation could be impacted.

7.0.2239 Release Notes

Release Date: 10 January 2024

Issues Corrected in Aviatrix Release 7.0.2239

Issue	Description
AVX-26567	A FireNet Egress FQDN gateway was dropping traffic.
AVX-36054	A gateway created in a newer Controller software version might have been rolled back to an older software version if: A gateway with the same name existed in an older Controller version. That gateway was deleted before upgrading to the present Controller version.
AVX-36996	(Azure) After an Azure FireNet FQDN Egress gateway image upgrade, the gateway goes into the “config_fail” state. To resolve this issue, try restarting the gateway. If the gateway state does not change, please contact Aviatrix Support.
AVX-38843	In a Site2Cloud connection with a single IP HA Gateway, the standby gateway continuously sent out an IPsec connection requests to a remote peer even though the connection would never get established. This issue created a stale state in the CSP underlay, which may have caused IPsec tunnel flaps between the active gateway and remote peer. This issue has been resolved by blocking the HA gateway from sending IPsec connection requests to remote peers for these Site2Cloud connections.
AVX-39477	When you tried to do an image upgrade or a software rollback for a BGP-enabled gateway on which you applied the “remove-unnecessary-packages-from-gateway” software patch, the operation may have failed.
AVX-39662	(GCP) Upgrading a GCP Transit Gateway with BGPoLAN and Firenet features enabled might have resulted in the loss of direct connectivity to the on-site firewall appliance.
AVX-41223	At the early stage of the gateway initialization, if you configured SSM agents to patch your Ubuntu servers automatically, the gateway initialization process may have failed.
AVX-41361	If a domain name used in an Egress FQDN tag had a long DNS record, attaching that FQDN tag to a gateway could fail. The error given was “command hostname_filter failed due to exception errors invalid IPNetwork.” An email notification was sent.
AVX-41555	When a Controller was in Private Mode and you opened the Firewall page and tried to select a subnet, nothing appeared in the dropdown menu.
AVX-41693	Linux auditd logs filled the disk space of some instances.
AVX-41976	Upgrading a Controller using older Controller images may fail with a zstd package installation failure.
AVX-42269	(GCP) In GCP, if the gateway deployment fails due to CSP (Cloud Service Provider) errors, the rollback fails due to the configuration being in an inconsistent state.
AVX-42706	FortiGate Firewalls failed to launch due to using an incorrect template on the Controller.
AVX-42789	Increased the length of time before an attempt to Encrypt Existing Gateways times out. This improvement helps avoid an encryption failure you would receive if the encryption timed out too soon. The warning message for that encryption failure was: Encryption failed. Waiter SnapshotCompleted failed: Max attempts exceeded.
AVX-43028	On a newly registered CloudN, users could not create attachments to multiple transits from a single CloudN Gateway.
AVX-43362	Aviatrix’s Single AZ HA (Availability Zone High Availability HA) feature would restart a gateway if it found that the gateway had gone down. Due to a timing issue between the process handling the feature and detecting the gateway state, it was possible for the gateway to go into a repeated stop-and-start loop. Disabling the Single AZ HA feature would break the loop.
AVX-43663	There was a memory leak in a firewall monitoring task. The memory leak was proportional to the number of firewalls in the network.
AVX-44255	When you tried to do a dry run for a Controller software version upgrade with more than one version in the pending list for upgrading and chose “latest” as the default version for the upgrade, the Controller incorrectly ran the dry run for the last version to upgrade to instead of the next upgrade version. For example, if you ran a dry run for 6.9 > 7.0 > 7.1, the Controller ran the dry run for 7.1 instead of 7.0.
AVX-44526	VPN NAT for gateway traffic didn’t work as expected due to a NAT-related misconfiguration in the Iptable rules. Action required: If you experience a VPN NAT issue after upgrading to this software version, disable and reenable your UserVPN NAT configuration.
AVX-44812	Deployments with a Utility license were unable to view some license details.
AVX-44974	(Azure) When Transit Gateways had Active-Standby enabled and the Active Transit Gateway was down, the attached Azure Native Spoke VNet route tables failed to switchover routes.
AVX-45873	When you used a link local address as an IPSec peer address, a Controller upgrade to release 6.8.1148 would drop traffic.
AVX-45897	On the Site2Cloud Details page in the Controller, the message “Authentication Type: null” was displayed for Site2Cloud connections even though there was a PSK authentication. Now, the page correctly displays “Authentication Type: psk” where PSK is the Authentication Type.
AVX-46098	When an Egress Filtering Gateway had a base Stateful Firewall policy of DENY, the gateway added the DROP rule from the base policy instead of letting the packets flow to the egress filter. The Egress Filtering Gateway should not have the DROP rule from the Stateful Firewall base policy. Instead, the packets should be allowed to flow to the egress filter.
AVX-46462	An HPE gateway resize could fail if the gateway had a peering with a gateway from release 6.7.1148 or earlier, as the new peering had additional fields in the structure.
AVX-46788	The Controller would not disable the Access Security feature during a Controller restore if the feature was not enabled in the backup configuration.
AVX-46873	(Azure) If any VNet that uses Intra-VNet micro-segmentation does not have the correct user permissions to access the Azure account, there would be issues with Security Group orchestration across all the VNets. To avoid similar permission issues, make sure all VNets on which Intra-VNet micro-segmentation is enabled have all the necessary read permissions to access your Azure account.
AVX-47027	(OCI Gov) OCI Gov gateways failed to launch.
AVX-47234	Previously, the S2C RX Balancing feature was supported only on AWS C5 and C5n gateway sizes. S2C RX Balancing now supports AWS C6in instances. Now, you can upgrade your gateway instance size to C6in and enable S2C RX Balancing. See xref:bgp-connection-settings.adoc#s2c-rx-balancing.
AVX-47361	(AWS) A rare race condition caused the AWS TGW (AWS Transit Gateway) data migration in the Controller database to fail during a gateway upgrade. This issue caused a problem with route programming.
AVX-47486	(AWS) Starting with software release 7.0.1307, AWS Gateways enabled tags in the instance metadata service. As a result, the tag keys used on the instance had to match this pattern: ([0-9a-zA-Z\\-_+=,.@:]{1,255}), and could not be a reserved name ('.', ‘..', '_index'). Image upgrades and new gateway creations would fail if tag keys in the instance metadata did not manage the requirements above.
AVX-47764	(AWS) When a VPC was attached to an AWS Transit Gateway (TGW), if you deleted one of the Spoke VPC Advertised CIDRs, the routes in associated Transit Gateways were not correctly updated.
AVX-47795	An issue with reading the Controller time zone caused the Controller to send false alerts about an expired PKI agent certificate on gateways.
AVX-48193	When a Transit Gateway had a Stateful Firewall policy configured that uses tags, creating or deleting BGP connections on the Transit Gateway could fail. The BGP connection change may have appeared to have completed successfully, but the updated configuration was not applied on the gateway.
AVX-48457	(AWS) AWS Gateways with tags that did not match new AWS requirements caused metadata service to fail to turn on.
AVX-48931	When you detached and reattached a CloudN attachment to an Aviatrix Transit Gateway that had any Stateful Firewall rules that used Stateful Firewall Tags, the BGP configuration incorrectly remained on the gateways.
AVX-49236	(OCI) After an OCI gateway image upgrade, several routing tables within several VCNs were missing the default route, 0.0.0.0/0.

Issue

Description

AVX-26567

A FireNet Egress FQDN gateway was dropping traffic.

AVX-36054

A gateway created in a newer Controller software version might have been rolled back to an older software version if:

A gateway with the same name existed in an older Controller version.
That gateway was deleted before upgrading to the present Controller version.

AVX-36996

(Azure) After an Azure FireNet FQDN Egress gateway image upgrade, the gateway goes into the “config_fail” state.

To resolve this issue, try restarting the gateway. If the gateway state does not change, please contact Aviatrix Support.

AVX-38843

In a Site2Cloud connection with a single IP HA Gateway, the standby gateway continuously sent out an IPsec connection requests to a remote peer even though the connection would never get established. This issue created a stale state in the CSP underlay, which may have caused IPsec tunnel flaps between the active gateway and remote peer.

This issue has been resolved by blocking the HA gateway from sending IPsec connection requests to remote peers for these Site2Cloud connections.

AVX-39477

When you tried to do an image upgrade or a software rollback for a BGP-enabled gateway on which you applied the “remove-unnecessary-packages-from-gateway” software patch, the operation may have failed.

AVX-39662

(GCP) Upgrading a GCP Transit Gateway with BGPoLAN and Firenet features enabled might have resulted in the loss of direct connectivity to the on-site firewall appliance.

AVX-41223

At the early stage of the gateway initialization, if you configured SSM agents to patch your Ubuntu servers automatically, the gateway initialization process may have failed.

AVX-41361

If a domain name used in an Egress FQDN tag had a long DNS record, attaching that FQDN tag to a gateway could fail. The error given was “command hostname_filter failed due to exception errors invalid IPNetwork.” An email notification was sent.

AVX-41555

When a Controller was in Private Mode and you opened the Firewall page and tried to select a subnet, nothing appeared in the dropdown menu.

AVX-41693

Linux auditd logs filled the disk space of some instances.

AVX-41976

Upgrading a Controller using older Controller images may fail with a zstd package installation failure.

AVX-42269

(GCP) In GCP, if the gateway deployment fails due to CSP (Cloud Service Provider) errors, the rollback fails due to the configuration being in an inconsistent state.

AVX-42706

FortiGate Firewalls failed to launch due to using an incorrect template on the Controller.

AVX-42789

Increased the length of time before an attempt to Encrypt Existing Gateways times out. This improvement helps avoid an encryption failure you would receive if the encryption timed out too soon. The warning message for that encryption failure was: Encryption failed. Waiter SnapshotCompleted failed: Max attempts exceeded.

AVX-43028

On a newly registered CloudN, users could not create attachments to multiple transits from a single CloudN Gateway.

AVX-43362

Aviatrix’s Single AZ HA (Availability Zone High Availability HA) feature would restart a gateway if it found that the gateway had gone down. Due to a timing issue between the process handling the feature and detecting the gateway state, it was possible for the gateway to go into a repeated stop-and-start loop. Disabling the Single AZ HA feature would break the loop.

AVX-43663

There was a memory leak in a firewall monitoring task. The memory leak was proportional to the number of firewalls in the network.

AVX-44255

When you tried to do a dry run for a Controller software version upgrade with more than one version in the pending list for upgrading and chose “latest” as the default version for the upgrade, the Controller incorrectly ran the dry run for the last version to upgrade to instead of the next upgrade version. For example, if you ran a dry run for 6.9 > 7.0 > 7.1, the Controller ran the dry run for 7.1 instead of 7.0.

AVX-44526

VPN NAT for gateway traffic didn’t work as expected due to a NAT-related misconfiguration in the Iptable rules.

Action required: If you experience a VPN NAT issue after upgrading to this software version, disable and reenable your UserVPN NAT configuration.

AVX-44812

Deployments with a Utility license were unable to view some license details.

AVX-44974

(Azure) When Transit Gateways had Active-Standby enabled and the Active Transit Gateway was down, the attached Azure Native Spoke VNet route tables failed to switchover routes.

AVX-45873

When you used a link local address as an IPSec peer address, a Controller upgrade to release 6.8.1148 would drop traffic.

AVX-45897

On the Site2Cloud Details page in the Controller, the message “Authentication Type: null” was displayed for Site2Cloud connections even though there was a PSK authentication. Now, the page correctly displays “Authentication Type: psk” where PSK is the Authentication Type.

AVX-46098

When an Egress Filtering Gateway had a base Stateful Firewall policy of DENY, the gateway added the DROP rule from the base policy instead of letting the packets flow to the egress filter. The Egress Filtering Gateway should not have the DROP rule from the Stateful Firewall base policy. Instead, the packets should be allowed to flow to the egress filter.

AVX-46462

An HPE gateway resize could fail if the gateway had a peering with a gateway from release 6.7.1148 or earlier, as the new peering had additional fields in the structure.

AVX-46788

The Controller would not disable the Access Security feature during a Controller restore if the feature was not enabled in the backup configuration.

AVX-46873

(Azure) If any VNet that uses Intra-VNet micro-segmentation does not have the correct user permissions to access the Azure account, there would be issues with Security Group orchestration across all the VNets.

To avoid similar permission issues, make sure all VNets on which Intra-VNet micro-segmentation is enabled have all the necessary read permissions to access your Azure account.

AVX-47027

(OCI Gov) OCI Gov gateways failed to launch.

AVX-47234

Previously, the S2C RX Balancing feature was supported only on AWS C5 and C5n gateway sizes. S2C RX Balancing now supports AWS C6in instances. Now, you can upgrade your gateway instance size to C6in and enable S2C RX Balancing. See xref:bgp-connection-settings.adoc#s2c-rx-balancing.

AVX-47361

(AWS) A rare race condition caused the AWS TGW (AWS Transit Gateway) data migration in the Controller database to fail during a gateway upgrade. This issue caused a problem with route programming.

AVX-47486

(AWS) Starting with software release 7.0.1307, AWS Gateways enabled tags in the instance metadata service. As a result, the tag keys used on the instance had to match this pattern: ([0-9a-zA-Z\\-_+=,.@:]{1,255}), and could not be a reserved name ('.', ‘..', '_index').

Image upgrades and new gateway creations would fail if tag keys in the instance metadata did not manage the requirements above.

AVX-47764

(AWS) When a VPC was attached to an AWS Transit Gateway (TGW), if you deleted one of the Spoke VPC Advertised CIDRs, the routes in associated Transit Gateways were not correctly updated.

AVX-47795

An issue with reading the Controller time zone caused the Controller to send false alerts about an expired PKI agent certificate on gateways.

AVX-48193

When a Transit Gateway had a Stateful Firewall policy configured that uses tags, creating or deleting BGP connections on the Transit Gateway could fail. The BGP connection change may have appeared to have completed successfully, but the updated configuration was not applied on the gateway.

AVX-48457

(AWS) AWS Gateways with tags that did not match new AWS requirements caused metadata service to fail to turn on.

AVX-48931

When you detached and reattached a CloudN attachment to an Aviatrix Transit Gateway that had any Stateful Firewall rules that used Stateful Firewall Tags, the BGP configuration incorrectly remained on the gateways.

AVX-49236

(OCI) After an OCI gateway image upgrade, several routing tables within several VCNs were missing the default route, 0.0.0.0/0.

Known Issues in Aviatrix Release 7.0.2239

Issue	Description
AVX-45598	(AWS) When you add a UserVPN Load Balancer to the UserVPN User Accelerator in the Aviatrix Controller before the Load Balancer state becomes active in the Cloud Service Provider, the Controller may throw an exception: “command vpn_user_xlr failed due to exception errors 'HealthState'<p></p>. An email notification with exception reason and trace log has been sent to exceptions@aviatrix.com for troubleshooting. Please feel free to contact Aviatrix Support. To resolve this issue: Delete the endpoint group associated with the Load Balancer from the Global Accelerator configuration through the Cloud Service Provider console. Then, re-associate the Load Balancer with the Global Accelerator through the Aviatrix Controller UI.
AVX-48903	(AWS and OCI) In all 7.0 Controller software versions, when you upgrade an image for a non-HPE Transit Gateway with a Site2Cloud GRE connection and Jumbo Frame, the upgrade can fail. To resolve this issue, Turn off Jumbo Frame on the gateway and then run the image upgrade.
AVX-49375	When you try to create a GCP Palo Alto firewall instance using a certain version of a Palo Alto image, the instance creation fails. The affected versions are versions of the Palo Alto Networks Next-Generation Firewall BUNDLE that contain the letter “h,” such as “8.1.25-h1.” If you experience this issue, choose a Palo Alto Networks image version that does not contain the letter “h.” New Check Point and FortiGate Fortinet instance deployments are unaffected.
AVX-50076	The Aviatrix Controller now only displays the metrics for the last hour, in Dashboard > Controller Metrics or Gateway Metrics. For detailed Gateway metrics, please use Aviatrix CoPilot.

Issue

Description

AVX-45598

(AWS) When you add a UserVPN Load Balancer to the UserVPN User Accelerator in the Aviatrix Controller before the Load Balancer state becomes active in the Cloud Service Provider, the Controller may throw an exception: “command vpn_user_xlr failed due to exception errors 'HealthState'<p></p>. An email notification with exception reason and trace log has been sent to exceptions@aviatrix.com for troubleshooting. Please feel free to contact Aviatrix Support.

To resolve this issue: Delete the endpoint group associated with the Load Balancer from the Global Accelerator configuration through the Cloud Service Provider console. Then, re-associate the Load Balancer with the Global Accelerator through the Aviatrix Controller UI.

AVX-48903

(AWS and OCI) In all 7.0 Controller software versions, when you upgrade an image for a non-HPE Transit Gateway with a Site2Cloud GRE connection and Jumbo Frame, the upgrade can fail.

To resolve this issue, Turn off Jumbo Frame on the gateway and then run the image upgrade.

AVX-49375

When you try to create a GCP Palo Alto firewall instance using a certain version of a Palo Alto image, the instance creation fails. The affected versions are versions of the Palo Alto Networks Next-Generation Firewall BUNDLE that contain the letter “h,” such as “8.1.25-h1.”

If you experience this issue, choose a Palo Alto Networks image version that does not contain the letter “h.” New Check Point and FortiGate Fortinet instance deployments are unaffected.

AVX-50076

The Aviatrix Controller now only displays the metrics for the last hour, in Dashboard > Controller Metrics or Gateway Metrics. For detailed Gateway metrics, please use Aviatrix CoPilot.

7.0.2004 Release Notes

Release Date: 29 August 2023

Issues Corrected in Aviatrix Release 7.0.2004

Issue	Description
AVX-39662	(GCP) Upgrading a GCP Transit Gateway with BGPoLAN and Firenet features enabled might have resulted in the loss of direct connectivity to firewall appliance management.
AVX-43545	When you updated the credentials of your cloud access accounts, the Aviatrix Controller could no longer get the latest status of the resources (for example, instances or VPCs) in your Cloud Service Providers: AWS, Azure, or GCP.
AVX-43547	On a newly registered CloudN, users could not create attachments to multiple transits from a single CloudN Gateway.
AVX-43550	A Stateful Firewall rule allowing reverse-path traffic flows was temporarily removed during a software upgrade.
AVX-43552	A previous method for adding new metrics to interface RRD files caused unnecessary delay and decreased performance. Resolved this issue so that the new metrics are available without the extra expense of time and performance. You must upgrade to software version 7.0.2004 or 7.1.2131 or later to access the new metrics.
AVX-43863	(GCP) A tag issue prevented the Global VPC feature for Spoke Gateways from being enabled or disabled properly.
AVX-44818	Bootstrap configuration for a firewall took longer than expected, causing traffic loss from the Transit Gateway. Use the following two attributes in Terraform to provide sufficient time for the firewalls to be configured via Bootstrap so that the configuration is applied to the firewalls. Note that the specific values for these attributes number_of_retries - (Optional) Number of retries for save or synchronize. (Set to at least 1, default is 0) retry_interval - (Optional) Retry interval in seconds for save or synchronize. Example: 900. Default value: 300. Recommended: 900.
AVX-45566	VPN NAT for gateway traffic didn’t work as expected because one of the NAT-related chains is missing in the iptables. Action required: Upgrade your gateway image.
AVX-45567	(GCP) Simultaneous, multiple GCP gateway image upgrades in the same GCP project might fail. To resolve this issue, try upgrading the gateway images again.
AVX-45569	Linux auditd logs filled the disk space of some instances.
AVX-45571	(Azure) After an Azure FireNet-enabled gateway image upgrade, the gateway went into the “config_fail” state.
AVX-48199	(GCP) Controllers that manage GCP resources may run into errors when a new Controller instance is started (via Controller High Availability or Controller Migration) or when a new GCP account is onboarded.

Issue

Description

AVX-39662

(GCP) Upgrading a GCP Transit Gateway with BGPoLAN and Firenet features enabled might have resulted in the loss of direct connectivity to firewall appliance management.

AVX-43545

When you updated the credentials of your cloud access accounts, the Aviatrix Controller could no longer get the latest status of the resources (for example, instances or VPCs) in your Cloud Service Providers: AWS, Azure, or GCP.

AVX-43547

On a newly registered CloudN, users could not create attachments to multiple transits from a single CloudN Gateway.

AVX-43550

A Stateful Firewall rule allowing reverse-path traffic flows was temporarily removed during a software upgrade.

AVX-43552

A previous method for adding new metrics to interface RRD files caused unnecessary delay and decreased performance. Resolved this issue so that the new metrics are available without the extra expense of time and performance. You must upgrade to software version 7.0.2004 or 7.1.2131 or later to access the new metrics.

AVX-43863

(GCP) A tag issue prevented the Global VPC feature for Spoke Gateways from being enabled or disabled properly.

AVX-44818

Bootstrap configuration for a firewall took longer than expected, causing traffic loss from the Transit Gateway.

Use the following two attributes in Terraform to provide sufficient time for the firewalls to be configured via Bootstrap so that the configuration is applied to the firewalls. Note that the specific values for these attributes

number_of_retries - (Optional) Number of retries for save or synchronize. (Set to at least 1, default is 0)
retry_interval - (Optional) Retry interval in seconds for save or synchronize. Example: 900. Default value: 300. Recommended: 900.

AVX-45566

VPN NAT for gateway traffic didn’t work as expected because one of the NAT-related chains is missing in the iptables.

Action required: Upgrade your gateway image.

AVX-45567

(GCP) Simultaneous, multiple GCP gateway image upgrades in the same GCP project might fail. To resolve this issue, try upgrading the gateway images again.

AVX-45569

Linux auditd logs filled the disk space of some instances.

AVX-45571

(Azure) After an Azure FireNet-enabled gateway image upgrade, the gateway went into the “config_fail” state.

AVX-48199

(GCP) Controllers that manage GCP resources may run into errors when a new Controller instance is started (via Controller High Availability or Controller Migration) or when a new GCP account is onboarded.

Known Issues in Aviatrix Release 7.0.2004

Issue	Description
AVX-45156	On an AEP Dell device, when you configure a Transit Gateway attachment with HPE (High Performance Encryption) mode, you could not set the tunnel count to more than 2. If you have a higher bandwidth/performance requirement which requires more tunnels, please contact Aviatrix Support for help.
AVX-45682	A rare issue with a gateway software upgrade may cause the BGP neighbor status to go down. To resolve this issue, restart the gateway.
AVX-45684	This issue occurs when you try to do a dry run for a Controller software version upgrade with more than one version in the pending list for upgrading. When you choose “latest” as the default version for the upgrade, the Controller incorrectly runs the dry run for the last version to upgrade to instead of the next upgrade version. For example, if you are running a dry run for 6.8 > 6.9 > 7.0 > 7.1, the Controller ran the dry run for 7.1 instead of 6.9. To resolve this issue, when you do a dry run, make sure to manually enter the next upgrade version instead of leaving the default, “latest.” For example, when you upgrade from 6.8 > 6.9 > 7.0 > 7.1, enter “6.9” as the version for the dry run.
AVX-45685	If an Egress FQDN gateway has an HA (High Availability) pair added, the HA gateway will not have Egress Control enabled. To enable Egress Control on both gateways, temporarily remove the Egress FQDN Filter tag from the primary gateway and then re-add it. To avoid this issue, create the HA gateway before assigning an Egress FQDN Filter tag.

Issue

Description

AVX-45156

On an AEP Dell device, when you configure a Transit Gateway attachment with HPE (High Performance Encryption) mode, you could not set the tunnel count to more than 2.

If you have a higher bandwidth/performance requirement which requires more tunnels, please contact Aviatrix Support for help.

AVX-45682

A rare issue with a gateway software upgrade may cause the BGP neighbor status to go down. To resolve this issue, restart the gateway.

AVX-45684

This issue occurs when you try to do a dry run for a Controller software version upgrade with more than one version in the pending list for upgrading. When you choose “latest” as the default version for the upgrade, the Controller incorrectly runs the dry run for the last version to upgrade to instead of the next upgrade version. For example, if you are running a dry run for 6.8 > 6.9 > 7.0 > 7.1, the Controller ran the dry run for 7.1 instead of 6.9.

To resolve this issue, when you do a dry run, make sure to manually enter the next upgrade version instead of leaving the default, “latest.” For example, when you upgrade from 6.8 > 6.9 > 7.0 > 7.1, enter “6.9” as the version for the dry run.

AVX-45685

If an Egress FQDN gateway has an HA (High Availability) pair added, the HA gateway will not have Egress Control enabled. To enable Egress Control on both gateways, temporarily remove the Egress FQDN Filter tag from the primary gateway and then re-add it.

To avoid this issue, create the HA gateway before assigning an Egress FQDN Filter tag.

7.0.1927 Release Notes

Release Date: 11 July 2023

Issues Corrected in Aviatrix Release 7.0.1927

Issue	Description
AVX-43137	If your Aviatrix Controller image was from 2022 or newer, a dependency caused an upgrade failure to 7.0.1724 or 7.1.1187.

Issue

Description

AVX-43137

If your Aviatrix Controller image was from 2022 or newer, a dependency caused an upgrade failure to 7.0.1724 or 7.1.1187.

7.0.1768 Release Notes

Release Date: 04 May 2023

Issues Corrected in Aviatrix Release 7.0.1768

Issue	Description
AVX-38158	(Alibaba) With CoPilot Security Group Management enabled, when you brought up gateways in Alibaba, they would be missing Security Group rules on CoPilot. This issue meant there would be no visibility of netflow and syslog data from the gateways.
AVX-38161	If a Spoke Gateway has multiple Custom Mapped or Mapped Site2Cloud connections, Forward Traffic to Transit configuration enabled, and the same virtual destination CIDRs are configured in other Site2Cloud connections, a failover in one connection will cause TCP sessions belonging to the other connections to drop.
AVX-39023	Gateway Diagnostics would encounter an error and not display the results.

Issue

Description

AVX-38158

(Alibaba) With CoPilot Security Group Management enabled, when you brought up gateways in Alibaba, they would be missing Security Group rules on CoPilot. This issue meant there would be no visibility of netflow and syslog data from the gateways.

AVX-38161

If a Spoke Gateway has multiple Custom Mapped or Mapped Site2Cloud connections, Forward Traffic to Transit configuration enabled, and the same virtual destination CIDRs are configured in other Site2Cloud connections, a failover in one connection will cause TCP sessions belonging to the other connections to drop.

AVX-39023

Gateway Diagnostics would encounter an error and not display the results.

7.0.1726 Release Notes

Release Date: 17 April 2023

Important Notices in Aviatrix Release 7.0.1726

Log Collection Export and Formatting Change

The process the Gateways and Controller use for exporting their log files to a remote log collection entity has changed. The log format has also changed. You must change your syslog collectors and any related automation to accept the new log format. For details, see Field Notice 42.

UTC Timezone Required when Upgrading to Controller 7.0.1726

Starting with Aviatrix Controller software version 7.0.1726, the syslog format of log files exported by Aviatrix gateways no longer includes timezone information. As a result, when upgrading to Controller 7.0.1726 or later, you must set the timezone to "UTC" on your Controller. CoPilot software versions 3.9 and later require the timezone to be set to "UTC so that timestamp information can be interpreted correctly.

To set the timezone to "UTC":

In Controller, open the Settings > Controller > Time > Change Timezone dialog.
Select UTC from the list.
Click OK.

Note the following:

The ability to set custom timezones is deprecated and will eventually be removed from the Controller when the Controller supports only UTC as the timezone for backend logs.
The CoPilot audit logs (shown in CoPilot > Administration > Audit) always assume a UTC timezone regardless of the custom timezone that may be set.

Issues Corrected in Aviatrix Release 7.0.1726

Issue	Description
AVX-34872	On a newly-deployed Controller or gateways, if multiple syslog profiles were configured, data was only forwarded on the most recently saved profile.
AVX-36249	In Private Mode, when the Controller’s proxy was set up, gateway diagnostics and an upgrade dry run would incorrectly show a status failure.
AVX-36387	(AWS) You received a gateway error message, “Missing account or VPC,” when you tried to bring up a gateway.
AVX-36913	(GCP) GCP gateways may experience CPU spikes every 10 minutes.
AVX-37066	Under certain conditions, when you tried to download Egress FQDN logs or stats, the download failed and you received an error message: … 'utf-8' codec can’t decode byte …
AVX-37120	Editing the Stateful Firewall policy for a gateway could fail when a large amount of rules were added to the policy.
AVX-37394	(Azure) An Azure FireNet route table would fill up and not allow any more gateways after you attached more than 400 non-HPE Spoke Gateways.
AVX-37801	(Azure) Deleting an Azure Spoke Gateway incorrectly deleted user-created RFC1918 routes in the VNet route table.
AVX-38409	A gateway credential could be doubly encrypted.
AVX-38469	Unnecessary or irrelevant threat rules for gateways were not successfully deleted.
AVX-38471	If the quagga bgp Debian packages were not installed properly, the Aviatrix Controller would try to reinstall the package instead of failing the gateway configuration.
AVX-38682	(GCP) When you selected the CheckPoint BYOL image as the third-party firewall option, the CheckPoint PAYG image came up instead.
AVX-38954	A Controller bug could lead to gateway crashes and traffic disruption.
AVX-39037	If you added policy rules to Distributed Firewalling, additional and unnecessary code could always run, even if the rules were deleted.
AVX-39040	Gateways reconnecting to the Controller could cause a resource leak on the gateway.

Issue

Description

AVX-34872

On a newly-deployed Controller or gateways, if multiple syslog profiles were configured, data was only forwarded on the most recently saved profile.

AVX-36249

In Private Mode, when the Controller’s proxy was set up, gateway diagnostics and an upgrade dry run would incorrectly show a status failure.

AVX-36387

(AWS) You received a gateway error message, “Missing account or VPC,” when you tried to bring up a gateway.

AVX-36913

(GCP) GCP gateways may experience CPU spikes every 10 minutes.

AVX-37066

Under certain conditions, when you tried to download Egress FQDN logs or stats, the download failed and you received an error message: … 'utf-8' codec can’t decode byte …

AVX-37120

Editing the Stateful Firewall policy for a gateway could fail when a large amount of rules were added to the policy.

AVX-37394

(Azure) An Azure FireNet route table would fill up and not allow any more gateways after you attached more than 400 non-HPE Spoke Gateways.

AVX-37801

(Azure) Deleting an Azure Spoke Gateway incorrectly deleted user-created RFC1918 routes in the VNet route table.

AVX-38409

A gateway credential could be doubly encrypted.

AVX-38469

Unnecessary or irrelevant threat rules for gateways were not successfully deleted.

AVX-38471

If the quagga bgp Debian packages were not installed properly, the Aviatrix Controller would try to reinstall the package instead of failing the gateway configuration.

AVX-38682

(GCP) When you selected the CheckPoint BYOL image as the third-party firewall option, the CheckPoint PAYG image came up instead.

AVX-38954

A Controller bug could lead to gateway crashes and traffic disruption.

AVX-39037

If you added policy rules to Distributed Firewalling, additional and unnecessary code could always run, even if the rules were deleted.

AVX-39040

Gateways reconnecting to the Controller could cause a resource leak on the gateway.

Known Issues in Aviatrix Release 7.0.1726

Issue

Description

AVX-36492

When single-IP HA (High Availability) is enabled on Aviatrix Gateways and the HA gateway goes up, a bug may cause the security group to not be added to the gateway. To resolve this issue, manually add the security group to the HA gateway.

AVX-37895

(Azure) Gateway deployment in Azure can fail if the Network Security Group (NSG) is not applied on the Controller’s Network Interface Card (NIC). If this happens, use one of two methods to resolve the issue:

Disable and re-enable the Controller Security Group management. This requires a disruption in traffic. In Azure, locate the NSG, which uses the format AVX-SG-<Pubic -IP>, and attach this NSG manually to the Controller’s NIC. This method does not require disruption in traffic.

7.0.1601 Release Notes

Release Date: 24 March 2023

Issues Corrected in Aviatrix Release 7.0.1601

Issue	Description
AVX-36971	The Monitor Gateway Subnet feature could shut down gateways during their initialization phase but not in subsequent phases.
AVX-36794	If a Spoke Gateway has multiple Custom Mapped or Mapped Site2Cloud connections, Forward Traffic to Transit configuration enabled, and the same virtual destination CIDRs are configured in other Site2Cloud connections, a failover in one connection will cause TCP sessions belonging to the other connections to drop.
AVX-37020	(Azure) Upgrading certain older Azure gateways was unsuccessful because they did not have the “gw_subnet_cidr” attribute.

Issue

Description

AVX-36971

The Monitor Gateway Subnet feature could shut down gateways during their initialization phase but not in subsequent phases.

AVX-36794

AVX-37020

(Azure) Upgrading certain older Azure gateways was unsuccessful because they did not have the “gw_subnet_cidr” attribute.

7.0.1577 Release Notes

Release Date: 07 March 2023

Issues Corrected in Aviatrix Release 7.0.1577

Issue	Description
AVX-32921	Some VPN user traffic to certain destinations was dropped on the VPN Gateway. This issue could occur when the VPN Gateway was rebooted and old VPN profile rules were not cleaned up from the system iptables.
AVX-33510	(GCP) All GCP gateways reached 100% CPU Utilization at the same time.
AVX-34540	When you configured NAT and Netflow on a gateway and rebooted it, the NAT rules were accidentally removed.
AVX-34487	A gateway upgrade may have failed if the gateway could not reach the Internet and install the Linux sysstat package.
AVX-34401	After the Controller was updated to the 6.7.1376 software version with the AVX-25632 bug fix, you could not attach a CloudN as a Gateway (CaaG) to an Azure Transit Gateway.
AVX-34163	When your Controller was deployed in private mode, enabling Netflow on a gateway failed. The iptables rules associated with Netflow would not be installed, and the gateway configuration failed.
AVX-34823	(AWS and Azure) In AWS accounts in the Controller that were onboarded using a key and secret instead of IAM Roles, an error occurred when you tried to bring up an Azure gateway.
AVX-34845	Removed a file from managed CloudN or the CaaG device during an upgrade to improve security.
AVX-35077	(Azure) If the Azure Spoke Gateways were down and a Transit Gateway propagated to an Azure Spoke Gateway with the default route, the Spoke VNet could not program route table default routes.
AVX-35096	(Azure) An API error may have caused the Controller to become unresponsive.
AVX-35728	If an incorrect passphrase was entered when attempting to enable SSH access to your Controller, a bug was causing all the keys for on-prem managed CloudN or CaaG devices to be removed.
AVX-35646	Previously, the gateway name reported in logs generated by the HTTP/HTTPS FQDN enforcer was “NA.” Now, the gateway name is correctly reported for newly created gateways.
AVX-35613	When the Controller’s timezone was set to any other time zone than UTC (Coordinated Universal Time), a software upgrade became stuck at 99% progress.
AVX-35773	During vendor integration with Panorama, increased the wait time for a Panorama commit to 1 minute. Because it can take some time for Panorama to commit template changes, doing a device push before that commit is ready could cause incomplete routes being pushed to devices. The increased wait time ensures that the Panorama commit is complete before the device push.
AVX-35844	(AWS) When you had a Transit Gateway attached to an AWS TGW and many Site2Cloud connections, the TGW list and plan page loaded slowly.
AVX-35958	The primary and HA gateway shared the same remote IP configuration.
AVX-36147	Configuring customized SNAT policies on a Spoke Gateway via Terraform failed.
AVX-36546	FlightPath may have incorrectly shown Spoke and Transit Gateway routes as inactive if the Controller and Gateways were using the following software versions: 7.0.1373, 7.0.1383, 6.9.308, or 6.8.1483.
AVX-36893	A Controller restore may have failed if the Controller had some dangled files.

Issue

Description

AVX-32921

Some VPN user traffic to certain destinations was dropped on the VPN Gateway. This issue could occur when the VPN Gateway was rebooted and old VPN profile rules were not cleaned up from the system iptables.

AVX-33510

(GCP) All GCP gateways reached 100% CPU Utilization at the same time.

AVX-34540

When you configured NAT and Netflow on a gateway and rebooted it, the NAT rules were accidentally removed.

AVX-34487

A gateway upgrade may have failed if the gateway could not reach the Internet and install the Linux sysstat package.

AVX-34401

After the Controller was updated to the 6.7.1376 software version with the AVX-25632 bug fix, you could not attach a CloudN as a Gateway (CaaG) to an Azure Transit Gateway.

AVX-34163

When your Controller was deployed in private mode, enabling Netflow on a gateway failed. The iptables rules associated with Netflow would not be installed, and the gateway configuration failed.

AVX-34823

(AWS and Azure) In AWS accounts in the Controller that were onboarded using a key and secret instead of IAM Roles, an error occurred when you tried to bring up an Azure gateway.

AVX-34845

Removed a file from managed CloudN or the CaaG device during an upgrade to improve security.

AVX-35077

(Azure) If the Azure Spoke Gateways were down and a Transit Gateway propagated to an Azure Spoke Gateway with the default route, the Spoke VNet could not program route table default routes.

AVX-35096

(Azure) An API error may have caused the Controller to become unresponsive.

AVX-35728

If an incorrect passphrase was entered when attempting to enable SSH access to your Controller, a bug was causing all the keys for on-prem managed CloudN or CaaG devices to be removed.

AVX-35646

Previously, the gateway name reported in logs generated by the HTTP/HTTPS FQDN enforcer was “NA.” Now, the gateway name is correctly reported for newly created gateways.

AVX-35613

When the Controller’s timezone was set to any other time zone than UTC (Coordinated Universal Time), a software upgrade became stuck at 99% progress.

AVX-35773

During vendor integration with Panorama, increased the wait time for a Panorama commit to 1 minute. Because it can take some time for Panorama to commit template changes, doing a device push before that commit is ready could cause incomplete routes being pushed to devices. The increased wait time ensures that the Panorama commit is complete before the device push.

AVX-35844

(AWS) When you had a Transit Gateway attached to an AWS TGW and many Site2Cloud connections, the TGW list and plan page loaded slowly.

AVX-35958

The primary and HA gateway shared the same remote IP configuration.

AVX-36147

Configuring customized SNAT policies on a Spoke Gateway via Terraform failed.

AVX-36546

FlightPath may have incorrectly shown Spoke and Transit Gateway routes as inactive if the Controller and Gateways were using the following software versions: 7.0.1373, 7.0.1383, 6.9.308, or 6.8.1483.

AVX-36893

A Controller restore may have failed if the Controller had some dangled files.

Known Issues in Aviatrix Release 7.0.1577

Issue	Description
AVX-25000	(AWS) A private mode gateway may not have Internet access, in which case it cannot directly upload a gateway tracelog to the S3 bucket. Instead, when you need to upload a gateway tracelog to an S3 bucket, upload the gateway tracelog to the Controller. Then, your Controller uploads the gateway tracelog to the S3 bucket.
AVX-27704	When a gateway has too many routes, the CoPilot Cloud Routes page does not display anything.
AVX-30776	(Azure) Avoid upgrading your Azure gateway image on gateways with “unmanaged disks” when the Companion Gateway version is “aviatrix-companion-gateway-v8” or an earlier Companion Gateway version. Azure and Aviatrix have prepared some special images with unmanaged disk support so you can upgrade a gateway image with an unmanaged disk. These are the Companion Gateway versions you can safely upgrade with an unmanaged disk: Controller version 6.7 - aviatrix-companion-gateway-v10u Controller version 6.8, 6.9 - aviatrix-companion-gateway-v13u Controller version 7.0, 7.1 - aviatrix-companion-gateway-v14u
AVX-34872	On a newly-deployed Controller or gateways, if multiple syslog profiles are configured, data is only forwarded on the most recently saved profile.
AVX-34997	If you deploy your Aviatrix Controller using proxy configuration and Private mode, the SMTP port does not open. In this situation, because Aviatrix accounts do not have an SMTP relay, your Controller will email Aviatrix Support about the error using port 443 via API.
AVX-36138	Gateway initialization, including Cloud Gateway creation, Cloud Gateway Image Upgrade, or Cloud Gateway Software Rollback fails if you completed both of the operations below (regardless of order): Changing the Controller time zone to those ahead of UTC/GMT. For example, for Australia/Sydney (AEST), the offset UTC is UTC+11:00. PKI re-bootstrap (including Certificate Domain Update and Gateway CA Certificate Upload) If you’ve already completed the actions above, try your gateway initialization again after X hours where X is the time zone difference between your Controller and the UTC/GMT. For example, if you change the Controller time zone to Australia/Sydney (AEST) and then upload the Gateway CA Certificate at 09:00, you need to wait until 20:00 (09:00 plus the 11:00-hour offset) to successfully create/replace/rollback any cloud gateway.
AVX-36796	Using SSH to access Checkpoint Firewall can fail. If this issue occurs, try this SSH command: ssh -oKexAlgorithms=+diffie-hellman-group-exchange-sha1 -i <private-key> admin@<fw_public_ip>

Issue

Description

AVX-25000

(AWS) A private mode gateway may not have Internet access, in which case it cannot directly upload a gateway tracelog to the S3 bucket. Instead, when you need to upload a gateway tracelog to an S3 bucket, upload the gateway tracelog to the Controller. Then, your Controller uploads the gateway tracelog to the S3 bucket.

AVX-27704

When a gateway has too many routes, the CoPilot Cloud Routes page does not display anything.

AVX-30776

(Azure) Avoid upgrading your Azure gateway image on gateways with “unmanaged disks” when the Companion Gateway version is “aviatrix-companion-gateway-v8” or an earlier Companion Gateway version.

Azure and Aviatrix have prepared some special images with unmanaged disk support so you can upgrade a gateway image with an unmanaged disk. These are the Companion Gateway versions you can safely upgrade with an unmanaged disk:

Controller version 6.7 - aviatrix-companion-gateway-v10u
Controller version 6.8, 6.9 - aviatrix-companion-gateway-v13u
Controller version 7.0, 7.1 - aviatrix-companion-gateway-v14u

AVX-34872

On a newly-deployed Controller or gateways, if multiple syslog profiles are configured, data is only forwarded on the most recently saved profile.

AVX-34997

If you deploy your Aviatrix Controller using proxy configuration and Private mode, the SMTP port does not open. In this situation, because Aviatrix accounts do not have an SMTP relay, your Controller will email Aviatrix Support about the error using port 443 via API.

AVX-36138

Gateway initialization, including Cloud Gateway creation, Cloud Gateway Image Upgrade, or Cloud Gateway Software Rollback fails if you completed both of the operations below (regardless of order):

Changing the Controller time zone to those ahead of UTC/GMT. For example, for Australia/Sydney (AEST), the offset UTC is UTC+11:00.
PKI re-bootstrap (including Certificate Domain Update and Gateway CA Certificate Upload)

If you’ve already completed the actions above, try your gateway initialization again after X hours where X is the time zone difference between your Controller and the UTC/GMT. For example, if you change the Controller time zone to Australia/Sydney (AEST) and then upload the Gateway CA Certificate at 09:00, you need to wait until 20:00 (09:00 plus the 11:00-hour offset) to successfully create/replace/rollback any cloud gateway.

AVX-36796

Using SSH to access Checkpoint Firewall can fail. If this issue occurs, try this SSH command:

ssh -oKexAlgorithms=+diffie-hellman-group-exchange-sha1 -i <private-key> admin@<fw_public_ip>

7.0.1400 Release Notes

Release Date: 16 Feb 2023

Issues Corrected in Aviatrix Release 7.0.1400

Issue	Description
AVX-32921	Some VPN user traffic to certain destinations was dropped on the VPN Gateway. This issue could occur when the VPN Gateway was rebooted and old VPN profile rules were not cleaned up from the system iptables.
AVX-34845	Removed a file from managed CloudN or the CaaG device during an upgrade to improve security.
AVX-35077	Issues Corrected (Azure) If the Azure Spoke Gateways were down and a Transit Gateway propagated to an Azure Spoke Gateway with the default route, the Spoke VNet could not program route table default routes.
AVX-35613	When the Controller’s timezone is set to any other time zone than UTC (Coordinated Universal Time), a software upgrade was stuck at 99% progress.
AVX-35728	If an incorrect passphrase was entered when attempting to enable SSH access to your Controller, a bug was causing all the keys for on-prem managed CloudN or CaaG devices to be removed.

Issue

Description

AVX-32921

AVX-34845

Removed a file from managed CloudN or the CaaG device during an upgrade to improve security.

AVX-35077

Issues Corrected (Azure) If the Azure Spoke Gateways were down and a Transit Gateway propagated to an Azure Spoke Gateway with the default route, the Spoke VNet could not program route table default routes.

AVX-35613

When the Controller’s timezone is set to any other time zone than UTC (Coordinated Universal Time), a software upgrade was stuck at 99% progress.

AVX-35728

If an incorrect passphrase was entered when attempting to enable SSH access to your Controller, a bug was causing all the keys for on-prem managed CloudN or CaaG devices to be removed.

7.0.1383 Release Notes

Release Date: 01 Feb 2023

Issues Corrected in Aviatrix Release 7.0.1383

Issue	Description
AVX-34823	(AWS and Azure) In AWS accounts in the Controller that were onboarded using a key and secret instead of IAM Roles, an error occurred when you tried to bring up an Azure gateway.
AVX-34487	A gateway upgrade may have failed if the gateway could not reach the Internet and install the Linux sysstat package.

Issue

Description

AVX-34823

(AWS and Azure) In AWS accounts in the Controller that were onboarded using a key and secret instead of IAM Roles, an error occurred when you tried to bring up an Azure gateway.

AVX-34487

A gateway upgrade may have failed if the gateway could not reach the Internet and install the Linux sysstat package.

Known Issues in Aviatrix Release 7.0.1383

Issue	Description
AVX-30776	(Azure) Avoid upgrading your Azure gateway image on gateways with “unmanaged disks” when the Companion Gateway version is “aviatrix-companion-gateway-v8” or an earlier Companion Gateway version. Azure and Aviatrix have prepared some special images with unmanaged disk support so you can upgrade a gateway image with an unmanaged disk. These are the Companion Gateway versions you can safely upgrade with an unmanaged disk: Controller version 6.7 - aviatrix-companion-gateway-v10u Controller version 6.8, 6.9 - aviatrix-companion-gateway-v13u Controller version 7.0, 7.1 - aviatrix-companion-gateway-v14u
AVX-27704	When a gateway had too many routes, the CoPilot Cloud Routes page did not display anything.

Issue

Description

AVX-30776

Controller version 6.7 - aviatrix-companion-gateway-v10u Controller version 6.8, 6.9 - aviatrix-companion-gateway-v13u Controller version 7.0, 7.1 - aviatrix-companion-gateway-v14u

AVX-27704

When a gateway had too many routes, the CoPilot Cloud Routes page did not display anything.

7.0.1377 Release Notes

Release Date: 24 Jan 2023

Issues Corrected in Aviatrix Release 7.0.1377

Issue	Description
AVX-34401	After the Controller was updated to the 6.7.1376 software version with the AVX-25632 bug fix, you could not attach a CloudN as a Gateway (CaaG) to an Azure Transit Gateway.
AVX-34887	BGP learned routes have been optimized to handle 10K routes with long AS Path lengths from multiple neighbors. This update helps you scale your network successfully.

Issue

Description

AVX-34401

After the Controller was updated to the 6.7.1376 software version with the AVX-25632 bug fix, you could not attach a CloudN as a Gateway (CaaG) to an Azure Transit Gateway.

AVX-34887

BGP learned routes have been optimized to handle 10K routes with long AS Path lengths from multiple neighbors. This update helps you scale your network successfully.

7.0.1373 Release Notes

Release Date: 09 Jan 2023

Issues Corrected in Aviatrix Release 7.0.1373

Issue

Description

AVX-27499

Resetting the configuration on EaaC/CaaG gateways failed to delete all connections with Transit Gateways.

AVX-27704

When a gateway had too many routes, the CoPilot Cloud Routes page did not display anything.

AVX-31614

When the Cloud VPC/VNet route table was full, new routes were not programmed when old routes were withdrawn.

AVX-32283

Certain web operations related to the Egress FQDN feature stalled due to fragmented TLS handshake packets. As a solution, the Aviatrix team coupled handling of these fragmented packets with the handling of packets with no SNI. To allow connections with fragmented client hellos to go through, enable your Controller’s FQDN configuration to allow packets with no SNI to go through.

AVX-32351

During Packet Capture, if you clicked Download multiple times, you received an error message: “Failed to open file.” Now, you can download successfully even if you click Download multiple times.

AVX-32730

You could not modify a UserVPN LDAP configuration and upload a CA certificate when more than one VPN Gateway was deployed behind a load balancer.

AVX-32904

If the Edge node could not access the Aviatrix release server because of a firewall setting or because the Management was over a private network, enabling the FIPS caused the Edge gateway to fail. The gateway could not be recovered.

7.0.1307 Release Notes

Release Date: 08 December 2022

Important Notices in Aviatrix Release 7.0.1307

AWS Gateways Enable Tags in Instance Metadata Service

(AWS) Starting with software release 7.0.1307, AWS Gateways will enable tags in the instance metadata service. As a result, the tag keys used on the instance must match this pattern: ([0-9a-zA-Z\\-_+=,.@:]{1,255}), and must not be a reserved name ('.', ‘..', '_index').

Software upgrades for gateways that have tag keys that don’t match the pattern will succeed, but after those upgrades, tag keys in the instance metadata service will not be enabled. Tag keys in instance metadata are used for Controller IP migration in post-7.0 releases in AWS gateways.

Image upgrades and new gateway creations will fail if tag keys in the instance metadata do not manage the requirements above.

Upgrading CloudN

CloudN users: Before you can upgrade to version 7.0.1307: * Make sure that your CloudN hardware is version 2.1 or a later version. If your hardware is 2.0 or earlier, you will need a hardware refresh.

Replace CloudN hardware version prior to 2.1 with CloudN hardware version 2.1 or later. You could also migrate to Aviatrix Edge.

To check which CloudN hardware version you are currently using, check your server. A server with a single SSD is running HW version 2.0 or a prior version and needs an update. A server with dual SSD Hard Disk drives is HW 2.1 or a later version and does not need an update.

Upgrade Prerequisites for CoPilot Users

When you migrate your Controller to 7.0.1307 from an earlier version, your Controller’s private IP address will change. Depending on which version of the Controller you are migrating from, you must perform the prerequisite tasks described in Migrating Your Aviatrix Controller before you start the migration.

Prerequisites for CoPilot Users

(Aviatrix CoPilot users) When you migrate your Controller, your Controller’s private IP address will change. Depending on which version of the Controller you are migrating from, perform the following tasks before you start the migration. This is to avoid being unable to log in to your CoPilot after the migration:

If you are migrating a Controller earlier than version 6.8.1088 or earlier than version 6.9.161:

In Copilot Home > Settings > Configuration, click Reset Controller IP. This will bring you to the CoPilot login page where you will enter your new Controller’s IP address once it’s available after the Controller migration.

If you are migrating a Controller version 6.8.1088 or later or 6.9.161 or later:

In the CSP environment of your CoPilot, confirm that your old Controller’s IP address (the Controller you are migrating from) is set in your CoPilot’s security group inbound rule for port 443.

Updating AWS Policy

(AWS) AWS access accounts have new required permissions. If you do not update your account permissions, you may not be able to access CoPilot or use the Distributed Cloud Firewall feature. You can update your policy from the Controller at Accounts > Access Accounts. See Auditing and Updating AWS IAM Policies in the Aviatrix Controller for details.

If you do not wish to use the default Aviatrix-managed policy, you can also manually add the new permissions to your AWS account.

Change Alert Configurations to Memory Available

(CoPilot users) The system metric Memory Free (memory_free) used in CoPilot for configuring alerts had been using a definition that was inconsistent with the operating system (OS) definition. Starting in 7.0.1307 the metrics memory_available and memory_free are consistent with the OS definition.

Due to this change, after upgrading your Controller, you may receive many Memory Free alerts indicating the memory free dropped. To mitigate this issue, you can change your alert configurations from using mem_free to using memory_available instead.

Post-Upgrade Tasks for Release 7.0.1307

Certain Transit Peering features, such as Excluded Network CIDRs, Excluded TGW Connections, and Connection AS Path Prepend, will not work when upgrading from the 6.9.128 Controller software version to the 7.0.1307 version. To resolve this issue, after upgrading to 7.0.1307, reconfigure these features.

Post-Upgrade Tasks for the 7.0.1307 Controller Software Version

6.9.128 Controller Software Version

After upgrading from the 6.9.128 Controller software version to the 7.0.1307 version, you must reconfigure Transit Peering features, including the following:

Excluded Network CIDRs
Excluded TGW Connections
Connection AS Path Prepend

Issues Corrected in 7.0.1307

Issue

Description

AVX-20197

After the Single AZ HA setting on a gateway was enabled and the GSO/GRO setting was disabled, the gateway may have auto-restarted multiple times.

AVX-20868

The Controller did not display bell notifications, or notifications that appear from the bell icon in the top right, when gateways went up or down. Now, your Controller displays bell notifications for Gateway and Tunnel up/down events as well as BGP connection up/down events.

Aviatrix also added an option to enable/disable bell alerts for each of the categories through your Controller under Settings > Controller > Alert Bell.

AVX-25209

The Aviatrix rsyslog may have unexpectedly stopped forwarding logging packets to remote server(s).

AVX-26004

Resolved an issue involving AWS accounts and permissions. If you onboarded an AWS account to your Controller, but your Controller didn’t have permission for some regions in that account, your account would print traceback logs, sometimes in large amounts. These logs did not affect performance but were unhelpful for managing your accounts. This fix suppressed those logs.

AVX-26505 (Azure)

In the Aviatrix template for launching new Palo Alto Firewall instances in Azure, the Accelerate Networking feature was automatically enabled. However, Accelerated Networking was not supported on the management interface for Palo Alto Firewall instances in Azure. Now, to avoid confusion, this Aviatrix template automatically disables Accelerated Networking on the management interface. You can still enable or disable Accelerated Networking out of band and use Accelerated Networking for WAN/LAN interfaces.

Note that this fix only affects the launching of new instances. Existing firewall instances are not affected.

AVX-27215

When you have a large network with FireNet gateways, applying Terraform took a long time and may have overused the Controller CPU.

AVX-27653, AVX-27869

Resolved two issues that could cause gateways to crash:

The conduit binary could become overwhelmed by Linux kernel netlink messages.
IP fragmented packets could trigger a kernel crash if the packet fragment was smaller than the UDP header.

This fix included releasing a new kernel driver. Important: If you experienced this issue, restart your gateway to use the new kernel driver.

AVX-27657

A full memory would cause the gateway’s tunnels to flap.

AVX-27732

FIPS 140-2 is neither supported nor required for Edge devices. Previously, if you tried to enable FIPS on the Controller, the edge gateway configuration would fail. Now, if you try to enable user-vpn in FIPS mode silently, the Edge gateways will bypass the request.

AVX-27820

Resolved an issue that sometimes caused a Controller to read the VPC CIDR of a gateway incorrectly. This issue caused an error message when OpenVPN was enabled: “Failed to initialize GlobalConfigDB: Error while trying to migrate from MongoDB to Etcd: Invalid IP address 1.”

AVX-27924

Launching HA Spoke gateways would cause the tags to be copied improperly from the main gateway.

AVX-27991

An IPset issue could cause a Transit Gateway memory issue or even crash when a network domain was enabled.

AVX-28175

If you created an Azure Transit Gateway of size Dv4 and Dsv4 with BGP over LAN interfaces and HPE, you experienced an error: [AVXERR-TRANSIT-0173] FireNet and BGP over LAN features require at least 4 interfaces.

AVX-28242

Fixed an issue which prevented OpenVPN users from connecting to their VPN after adding a second search domain separated by a comma (Controller > Edit Config > Modify Split Tunnel). Now, OpenVPN users can enter multiple search domain names separated by commas in split tunnel configuration.

AVX-28821

When you changed a Controller’s time zone to any time zone other than UTC, CoPilot did not display host information under Performance > Network Metrics for the Last Hour.

To resolve this issue in versions older than release 6.9.118, restart cloudxd in your Controller by going to Diagnostics > Services > CloudXD > Actions > Restart.

AVX-28898

A large number of Site2Cloud connections degraded your Controller’s responsiveness.

AVX-28938 (AWS)

The Controller Security Access Control feature overcomes the 1000-rule limitation of AWS security group rules per instance. Instead of using AWS Security groups to control access to the Controller, the Controller itself manages incoming TCP 443 access. You can configure this feature using API 2.5. Please contact Aviatrix Support for more information.

AVX-29002

If you mapped a Site2Cloud configuration a Spoke Gateway and then upgraded your gateway image with version 6.8 software, traffic to your Site2Cloud remote would break.

AVX-29016

When you registered a CAAG or Edge Gateway while your LAN/WAN interface was down, the CloudN list would fail to display. You could not perform basic actions like Diag, Deregister, or Reset Configuration.

AVX-29691

Under scale setups with thousands of tunnels, when micro-segmentation was disabled, micro-segmentation could still run and consume an entire CPU core.

AVX-30545

A gateway using a Linux kernel version older than 4.20 will see a configure route failure with an error message: Failed to get real route: protocol not available. To avoid this issue, upgrade your gateways to the latest image.

AVX-30621 (AWS)

A large number of access accounts experienced excessive memory usage.

AVX-31471

In Controller release 7.0.1307, you cannot deploy new Edge 1.0 gateways. Existing Edge 1.0 gateway (deployed prior to release 7.0.1307) will continue to run in release 7.0.1307.

AVX-32807

Resolved an asymmetric traffic flow issue with the rxhash network setting. Note that this fix is essential for customers who are upgrading Azure Gateway images from v8 to v13.

Deprecated Features in Aviatrix Release 7.0.1307

Transitive Peering Deprecated

The Transitive Peering feature is deprecated. This feature’s functionality will be replaced by Aviatrix Multicloud Transit. Aviatrix recommends deleting Transitive Peerings from your account, and then upgrading your Controller.

AVX-30507 - NAT and DNAT Sync to HA Gateway functionality has been removed. Previously, when configuring SNAT and DNAT on the primary gateway and Sync to HA Gateway was enabled, the SNAT and DNAT rules were automatically synchronized to the HA gateway instance. You now need to configure SNAT and DNAT rules on the primary and HA gateway instances separately.

In Terraform, the following gateway resources have been changed:

aviatrix_gateway_dnat: sync_to_ha argument is removed.
aviatrix_gateway_snat: sync_to_ha argument is removed.

Known Issues in 7.0.1307

Issue	Description
AVX-31857	In Azure, NSG (Network Security Group) rules are ordered based on priority number. In the Aviatrix Controller, the policies of DFW (Distributed Firewalling) are not ordered by priority, which means two Aviatrix policies can have the same priority. When the Controller tries to program the Azure NSG rules, since both Aviatrix Rules have the same priority, the Controller will not honor the order in which they appear in the AVX Policies. The order of the rules in NSG might get reversed if they share the same priority. Solution: To prevent this confusion, it is better to not have the same priority for more than one AVX rule. Having a unique priority for each rule will ensure that the Controller honors the order of rules of each subnet’s NSG on Azure.
AVX-32273	Known Aviatrix CSP gateway base images launched in release 6.3, 6.4, and 6.5 with default python 2.7.17 are not compatible with python 3.6.9 in the versions (6.8 and newer) of Aviatrix software. To avoid this issue, upgrade your Controller to the latest version and upgrade all gateways images launched in 6.5 or older to the latest version.
AVX-32968 (AWS China)	During the image upgrade or software rollback of an AWS China Gateway, the EBS volume may not be deleted after the old gateway instance is deleted. If this issue occurs in your account, manually delete the EBS volume from your AWS China account.
AVX-35490	After a Controller software upgrade or a CloudXD restart, the Controller migrates BGP routes, automatically triggering an “Approve New Routes” email for existing pending CIDRs on gateways with learned CIDRs approval enabled. This issue has no functional impact. Approved CIDRs remain intact and no routes are changed.
AVX-43180	If your Controller is using an outdated image, a software upgrade may fail. If your Controller software upgrade fails, please contact the Aviatrix Support team for assistance.
AVX-45386	On a gateway with multiple mapped Site2Cloud connections with Forward Traffic to Transit Gateway enabled, after a successful gateway image upgrade, some of these connections may not work. To resolve this issue, go to Controller > Site2Cloud > Setup and disable and re-enable "Forward Traffic to Transit Gateway" for each impacted Site2Cloud connection.

Issue

Description

AVX-31857

In Azure, NSG (Network Security Group) rules are ordered based on priority number.

In the Aviatrix Controller, the policies of DFW (Distributed Firewalling) are not ordered by priority, which means two Aviatrix policies can have the same priority.

When the Controller tries to program the Azure NSG rules, since both Aviatrix Rules have the same priority, the Controller will not honor the order in which they appear in the AVX Policies. The order of the rules in NSG might get reversed if they share the same priority.

Solution: To prevent this confusion, it is better to not have the same priority for more than one AVX rule. Having a unique priority for each rule will ensure that the Controller honors the order of rules of each subnet’s NSG on Azure.

AVX-32273

Known Aviatrix CSP gateway base images launched in release 6.3, 6.4, and 6.5 with default python 2.7.17 are not compatible with python 3.6.9 in the versions (6.8 and newer) of Aviatrix software. To avoid this issue, upgrade your Controller to the latest version and upgrade all gateways images launched in 6.5 or older to the latest version.

AVX-32968 (AWS China)

During the image upgrade or software rollback of an AWS China Gateway, the EBS volume may not be deleted after the old gateway instance is deleted. If this issue occurs in your account, manually delete the EBS volume from your AWS China account.

AVX-35490

After a Controller software upgrade or a CloudXD restart, the Controller migrates BGP routes, automatically triggering an “Approve New Routes” email for existing pending CIDRs on gateways with learned CIDRs approval enabled. This issue has no functional impact. Approved CIDRs remain intact and no routes are changed.

AVX-43180

If your Controller is using an outdated image, a software upgrade may fail. If your Controller software upgrade fails, please contact the Aviatrix Support team for assistance.

AVX-45386

On a gateway with multiple mapped Site2Cloud connections with Forward Traffic to Transit Gateway enabled, after a successful gateway image upgrade, some of these connections may not work. To resolve this issue, go to Controller > Site2Cloud > Setup and disable and re-enable "Forward Traffic to Transit Gateway" for each impacted Site2Cloud connection.