Field Notices
These field notices are provided as a service to our customers to proactively update them on major issues. This service is provided without any changes in our SLA. The information in this field notice will be updated as we learn more.
49. Field Notice
Date: 6 November 2024
Who is impacted?
Customers using Controller versions lower than 7.1.4105 and attempting to resize their HPE-enabled Gateways.
Issue Description
When attempting to resize a gateway on anAviatrix Controller running versions older (lower) than 7.1.4105 with HPE enabled, the controller will fail to restore peering tunnels if a resize operation fails. For example, due to insufficient CSP resources to support the resize.
Symptoms
The Aviatrix Controller will display a notification indicating that the resize operation failed, along with error messages. As a result, routes to the peer will be missing from the gateway’s routing table.
Impact
Traffic connectivity will be disrupted, as tunnels on the gateway that failed to resize are not restored.
Trigger
Resizing of HPE enabled gateways.
This issue can occur when running a Controller version older (lower) than 7.1.4105 with HPE enabled gateways and the gateway resize fails due to any errors, such as a CSP resource issue, not enough IPs available, etc.
Workaround
No workaround available. Contact the Aviatrix Support team prior to performing a Gateway resize activity if you are running a release that is impacted.
Recommendations
Contact the Aviatrix Support team prior to performing a Gateway resize activity if you are running a release that is impacted.
If you have any further questions, please contact the Aviatrix Support team.
48. Field Notice
Date: 1 August 2024
Attention: Deprecation of V1 Controller API
Who is impacted: This deprecation notice is only relevant to users who directly consume the Aviatrix Controller API. Terraform and UI usage is not affected.
The V1 Controller API is being deprecated with the announcement of this field notice, and it will be removed with the release of Controller version 8.0 expected early in 2025. You can find out if you are using the V1 Controller API by searching for "https://<controller_ip>/v1/api/" in your configuration.
Action Required
-
Adopt V2 API Before Upgrading: Anyone currently using the V1 API should transition to the V2 API before upgrading to Controller v8.0 to ensure uninterrupted service.
-
V2 API Availability: The V2 API is already available for organizations that are running Controller version 7.0 or later, and we encourage you to start the migration process as soon as possible.
Support and Information
-
Migration method: The V2 API provides a superset of the V1 API capabilities published in the Postman collection, which should make the migration relatively straightforward. The migration can generally be achieved by changing the base URL from https://<controller_ip>/v1/api/ to https://<controller_ip>/v2/api/.
-
Testing and Reporting Issues: We recommend thoroughly testing your integration with the V2 API as part of your migration and reporting any issues to the Aviatrix Support team promptly.
-
API Documentation: For access to the V1 and V2 API Postman collections, please contact the Aviatrix Support team.
-
Migration Support: For further information and assistance regarding the V2 API, please reach out to the Aviatrix Support team.
-
Continued Use of V1 API: You can continue using the V1 API until you are ready to upgrade to Controller version 8.0, but you must migrate to API V2 before upgrading to Controller version 8.0.
-
V1 API Availability on 7.2: The V1 API will remain available on Controller version 7.2 and will be supported until version 7.2 reaches its end of support cycle.
Migrating from Controller API V1 to V2
When migrating from Controller API V1 to API V2, in most instances there is a direct equivalent API operation in V2 and simply changing the base URL from /v1/ to /v2/ is sufficient for a successful migration. There are a few exceptions where there is no equivalent V2 API operation. In some cases, there is no direct equivalent operation available. In other cases, a different operation must be used to achieve the same outcome. These exceptions are listed in the table below.
| API V1 Operation | API V2 Operation | Comments |
|---|---|---|
get_gateway_supported_size |
list_transit_gw_supported_sizes, list_spoke_gw_supported_sizes |
get_gateway_supported_size still works, but requires an additional mandatory parameter (cloud_type) and returns only cloud specific results, rather than an object with all clouds. |
create_transit_gw |
create_multicloud_primary_gateway |
Requires “transit” parameter to be set to “yes”. Required parameter changes: region → vpc_region public_subnet → gw_subnet |
enable_transit_ha |
create_multicloud_ha_gateway |
Required parameter changes: public_subnet → gw_subnet |
create_spoke_gw |
create_multicloud_primary_gateway |
Requires “transit” parameter to be set to “no” (default). Required parameter changes: region → vpc_region public_subnet → gw_subnet |
enable_spoke_ha |
create_multicloud_ha_gateway |
Required parameter changes: public_subnet → gw_subnet |
enable_firenet_fail_close |
None |
Not available (feature deprecated) |
disable_firenet_fail_close |
None |
Not available (feature deprecated) |
list_firenet_display_fields |
None |
Not available (Should only be used by Aviatrix controller UI.) |
list_firewall_manager_vendor_display_fields |
None |
Not available (Should only be used by Aviatrix controller UI.) |
list_profile_policies_and_base_policy |
list_profile_policies |
Does not provide a description with the base policy. |
resume_dmz_firewall |
None |
Not available (feature deprecated) |
bypass_dmz_firewall |
None |
Not available (feature deprecated) |
switchover_dmz_firewall |
None |
Not available (feature deprecated) |
If you have any further questions, please contact the Aviatrix Support team.
47. Field Notice
Date: 27 June 2024
Advisory: Explore capabilities with Aviatrix CoPilot
Aviatrix CoPilot offers many features to simplify Day-0 and Day-2 operations, including:
-
Automation of Controller Migration to Ubuntu 22 Image.
-
Improved Gateway Management Page with the ability to download gateway configuration details and metrics (CPU and memory usage) in .csv or .xlsx format.
-
The Upgrade Plan & Upgrade Groups feature allows you to better plan gateway upgrades based on Region, Cloud or Environment, etc.
-
Simplification of Spoke & Transit Gateway deployment with fewer steps.
-
Up to 15 high availability (HA) spoke gateway instances for each Spoke Gateway.
|
Spoke Gateways with HA enabled do not support BGP, Site2Cloud, SNAT, DNAT, or FQDN. |
-
Aviatrix Network Insights API allows you to retrieve network metric and status data across your Aviatrix data plane for integration with third-party tools for data analysis and visualization of the performance and health of your Aviatrix-managed resources.
-
Practical and useful tools to view Gateway Performance and to configure Alert Notifications for Gateway health.
Please reach out to your Account Representative if you need more information on CoPilot OR contact Aviatrix Support by opening a support ticket for help to get started with Planning and Deploying Aviatrix CoPilot.
46. Field Notice
Date: 02 Nov 2023, revised 26 June 2024
Issue Description:
Extension of End-of-Support and End-of-Life Dates for Aviatrix Software Releases
To better serve our customers, Aviatrix has decided to provide a one-time extension to the End-of-Support (EOS) and End-of-Life (EOL) dates for Controllers running release 6.9, 7.0, and 7.1.
Related to this extension, we will also be releasing an additional patch for releases 6.9 and 7.0 before they reach EOL. This patch will provide extended time outside of the 2023 holiday window to support our customers in upgrading to our latest releases.
The previous EOS/EOL timeline was as follows:
| Release | GA Date | EOL Date | EOS Date |
|---|---|---|---|
6.8 |
AUG 09 2022 |
AUG 09 2023 |
NOV 09 2023 |
6.9 |
SEP 09 2022 |
SEP 09 2023 |
DEC 09 2023 |
7.0 |
DEC 08 2022 |
DEC 08 2023 |
MAR 08 2024 |
7.1 |
MAY 11 2023 |
MAY 11 2024 |
AUG 11 2024 |
The updated EOS/EOL Timeframe is as follows:
| Release | GA Date | EOL Date | EOS Date |
|---|---|---|---|
6.8 |
AUG 09 2022 |
AUG 09 2023 |
NOV 09 2023 |
6.9 |
SEP 09 2022 |
JAN 09 2024* |
APR 09 2024* |
7.0 |
DEC 08 2022 |
APR 08 2024* |
JUL 08 2024* |
7.1 |
MAY 11 2023 |
To be announced** |
To be announced** |
* Dates have been adjusted by this Field Notice.
** 7.1 EOL has been deferred and will be announced upon the availability of our next major release. After 7.1 EOL Aviatrix will provide at least an additional 6 months of support before EOS.
Always reference the Aviatrix EOS/EOL policy for the most up-to-date information. The EOS/EOL policy is here.
For any clarifications on this Field Notice or for upgrade assistance, please contact Aviatrix Support.
45. Field Notice
Date: 28 October 2023
Severity: High
Image upgrade and new gateway deployment fails.
A gateway state on an Aviatrix Controller might change to "Config_fail" when it is created or just after the image is upgraded. This could occur if the setup had previously applied a patch named "Remove unnecessary packages from gateway" under the Controller’s Software patches section. This issue does not always occur in all regions and clouds, but out of an abundance of caution we recommend all users who have applied the "Remove unnecessary packages from gateway" patch take action.
What is the impact?
Customers who applied the "Remove unnecessary packages from gateway" patch need to update the patches before deploying or upgrading gateways. Otherwise, the gateways might move into a "Config_fail" state.
Who is impacted?
To encounter this problem, the below conditions should be met:
The "Remove unnecessary packages from gateway" patch should show as Patched or Partly Patched on the Controller. To verify the status, go to Aviatrix Controller > Settings > Maintenance > Software Patches.
What is the recommendation?
Aviatrix Systems has updated the patch, and it is now available as a Software Patch.
|
Aviatrix strongly recommends not to attempt a gateway image upgrade or to deploy a new gateway until you update the available patches. |
How to detect the issue?
The gateway will display the config_fail state in the gateway page of the Controller:
In addition, the following log entries appear on the Controller under Aviatrix Controller > Troubleshoot > Logs > Display Aviatrix Command Log > DISPLAY.
Log entry text.
2023-10-26T23:03:05.846367+00:00 ip-10-61-86-154 CLOUDX[25238 26208]:
WARNING email_notifications_manager get_status_change_notif_email_address 216 : Status change
notification email address cannot be retrieved.'NoneType' object has no attribute 'get'
tools.packages.PackageUpdateError: Command '['apt-get', 'purge', '-y', 'apport', 'aptitude',
'apt-show-versions', 'apt-xapian-index', 'man-db', 'mlocate', 'nginx-core', 'nginx-common',
'nginx', 'ubuntu-advantage-tools', 'update-notifier-common']' returned non-zero exit status
100.How to fix and avoid the issue?
Aviatrix has updated the "Remove unnecessary packages from gateway" patch.
-
Prior to performing an image upgrade or deploying a gateway in the current release, please go to Aviatrix Controller > Settings > Maintenance > Software Patches and click on UPDATE AVAILABLE PATCHES. Once the patches are updated, a gateway image upgrade or gateway deployment can be performed. The update of the available patches is required to be done one time per Controller unless a Controller upgrade or a Controller migration is performed.
-
Whenever a Controller software upgrade (Platform Upgrade) is performed, you are required to Update Available Patches again before performing an image upgrade or gateway deployment. Please go to Aviatrix Controller > Settings > Maintenance > Software Patches and click on UPDATE AVAILABLE PATCHES.
-
Whenever a Controller Migration is performed, once the backup restore completes on the new Controller and all gateways are connected to it, you are required to Update Available Patches again before performing an image upgrade or gateway deployment. Please go to Aviatrix Controller > Settings > Maintenance > Software Patches and click on UPDATE AVAILABLE PATCHES.
44. Field Notice
Date: 25 October 2023
Who is impacted: Customers modifying rules in the Egress FQDN Filtering feature
Issue Description:
A critical issue identified within the Aviatrix Controller may impact your rule modifications in the Egress FQDN Filtering feature.
In versions 7.1.1710, 7.1.1794, and 7.1.1906, when you attempt to edit an existing FQDN Egress rule under a specific tag and click Save and Update, the Controller removes other rules with the same tag. This unexpected behavior can lead to an outage that may impact your business operations.
Workaround
To avoid encountering this issue, we recommend adding or deleting rules instead of modifying existing rules.
If you do need to modify an existing rule, use the following workaround:
-
Export the rules.
-
Modify the rules as needed in the text file.
-
Make sure no filters are in use in the Edit screen for the rules. Then, import the file with the modified rules.
-
Click Save and Update.
If you have already encountered the issue, please follow the above workaround. Our team highly recommends upgrading to version 7.1.2131, where this issue has been resolved, or a later release.
42. Field Notice
Date: 13 April 2023
(The content of this field notice was revised for clarity on 04/17/2023.)
Issue Description:
For all current Controller software versions (all versions earlier than 7.0.1726), Aviatrix gateways are exporting files to a remote log collection entity. Starting in Controller software version 7.0.1726, instead of exporting files to a remote log collection entity, the Aviatrix Controller and gateways will start streaming the log lines being written to “Syslog” and “Auth.log”.
When you use the default rsyslog server configuration suggested in Aviatrix Documentation, the logs streamed from the Controller and gateways will now have multiple files. Each file will be named with the application that generated the log.
For example: All logs generated by the avx-gw-state-sync application would be re-directed to a file named "avx-gw-state-sync" on the log server.
There will be a change in log format. You must change your syslog collectors and any related automation to accept the new log format.
Old format:
Mar 23 19:17:50 GW-UdpGateway-50.17.41.173 syslog 2023-03-05T19:17:50+00:00 GW-UdpGateway-50.17.41.173 avx-gw-state-sync[11249]: warn#011gateway_launcher/gateway_launcher.go:212#011daemon exited
New format:
Mar 23 19:17:50 GW-UdpGateway-50.17.41.173 avx-gw-state-sync[11249]: warn#011gateway_launcher/gateway_launcher.go:212#011daemon exited
Prefix of old format: Mar 23 19:17:50 GW-gg-aws-usw2-s127-35.162.124.66 syslog 2023-03-05T19:17:50+00:00
Prefix of new format: Mar 23 19:17:50 GW-gg-aws-usw2-s127-35.162.124.66
41. Field Notice
Date: 28 Nov 2022
Change in Default Behavior
The latest 7.0 version of Aviatrix controller introduces a token verification to Aviatrix’s private API.
Please take notice of a change in behavior beginning with Aviatrix Controller version 7.0. The 7.0 version introduces token-based Controller API operations that binds Aviatrix’s private API usage by Aviatrix API Legal Terms of Use*.
To allow time for customers to make necessary changes in their infrastructure to support token-based API operations, we will not enforce a strict check for the token in the 7.0 release. Therefore, Aviatrix’s private API will continue to work for your existing use cases while running 7.0. However, token checking will be enforced in a later release.
Who is impacted?
Direct users of Aviatrix’s private API would be impacted by this change. There is no impact to users of Aviatrix Terraform Provider, Aviatrix CoPilot and Aviatrix Controller UI. Customers who have a Controller HA set up would also be affected. After upgrading to the release with token enforcement enabled, recreate your Controller HA configuration. Use HA script 2.01 or above. For details on HA script version, refer to Controller HA.
Recommended Solution:
To insulate customers from our evolving private API, Aviatrix strongly recommends you switch to Aviatrix Terraform Provider for all operations involving automation.
If you have special need to still use Aviatrix’s private API, please reach out to Aviatrix Support by opening a ticket at Support Portal at https://support.aviatrix.com for guidance on Aviatrix’s private API token generation.
Please mention your Aviatrix private API use case(s) in your ticket for us to better understand your automation needs, thereby enhancing our Terraform Support.
*Aviatrix API Legal Terms of Use:
Use of Aviatrix API software (“Developer Software”) is governed by the Customer Terms of Use. We reserve the right to rescind any license to the Developer Software at our sole discretion without prior notice. DEVELOPER SOFTWARE IS MADE AVAILABLE BY US TO YOU ON AN “AS IS” AND “AS AVAILABLE” BASIS, (I) WITHOUT ANY REPRESENTATION OR WARRANTY OF ANY KIND, WHETHER EXPRESS, IMPLIED OR STATUTORY TO THE FULLEST EXTENT PERMITTED BY LAW AND (II) WITHOUT ANY OBLIGATION OF US TO PROVIDE TECHNICAL SUPPORT OR ANY INDEMNITY FOR YOUR ACCESS TO, AND USE OF, THE DEVELOPER SOFTWARE.