Migrate CoPilot to Appliance V3 (AWS)

The CoPilot migration to Appliance v3 on AWS is a necessary and essential step to enhance the security, availability, and performance of your CoPilot deployment. As the old version of CoPilot images will soon be deprecated, it is highly recommended to migrate your CoPilot to Appliance V3.

This migration involves updating the CoPilot image, enabling a seamless transition CoPilot Appliance (v1 or v2) to the latest version (v3), or from CoPilot Appliance v3 to the latest version of Appliance v3. Additionally, it provides support for migrating existing CoPilot deployments running on Appliance v3 to the latest version of Appliance v3.

We only support simple-deployment to simple-deployment migration and migrations within the same region. Migrations to a fault-tolerant (clustered) deployment or to a different region is not supported.

Overview

The migration of CoPilot to Appliance v3 (AWS) covers all aspects of the CoPilot system, including the CoPilot configuration and CoPilot data. This migration enhances security hardening, expands availability in new regions and improves database performance. As a result, if you are migrating CoPilot to Appliance v3, there is no need to perform a separate data migration from Settings > Maintenance > Copilot Management > MigrateMigration.

Improvements of Appliance V3

  • Enhanced Security Hardening: The migration to Appliance v3 includes additional security hardening measures. It is a mandatory security update that helps protect your data and infrastructure from potential vulnerabilities.

  • Expanded Availability: With the migration, CoPilot will be released in new regions, expanding its availability to a wider user base. Furthermore, the Appliance v3 will become the new default option in all regions, ensuring that you have access to the latest features and improvements.

  • Improved Database Performance: The migration introduces significant database improvements, resulting in faster queries and cached results. This means you can expect improved performance and efficiency when analyzing and visualizing data using CoPilot.

Overview of the Migration Phases

The CoPilot migration process includes the following phases:

Phase 1: Backup

In this phase, the existing CoPilot, including the CoPilot configuration and CoPilot data, is backed up to ensure no data loss during the migration.

Phase 2: Deploy New CoPilot

The program deploys a new CoPilot instance using the new Appliance v3 image. This updated version provides enhanced security, availability, and performance.

Phase 3: EIP Switch

The Elastic IP (EIP) associated with the CoPilot is switched from the old instance to the new instance. This ensures that the new CoPilot is accessible using the existing IP address.

Phase 4: Restore Configuration and Data

The program restores the CoPilot configuration and data to the new CoPilot instance. This step transfers all the settings and data from the old CoPilot to the new environment.

Phase 5: Restore CoPilot Data

The program initiates the restoration of CoPilot data to the new CoPilot instance. This ensures that all the data is available in the new CoPilot environment.

After completing Phase 4, you will be directed to the new CoPilot login page. You can use the same credentials that you used for the old CoPilot to log in.

Before CoPilot Migration

Before proceeding with the migration of CoPilot to Appliance v3 (AWS), you must upgrade your Controller to the latest patch version of the release you are currently using. This prerequisite task ensures a smooth and successful migration process.

If you are are unable to migrate to a new CoPilot image on 6.8.1148, 6.9.128, 7.0.1307 or 7.1.1710, please upgrade to the latest minor release or higher.

Before proceeding with the CoPilot migration, make sure to complete the following preparation tasks:

  1. Upgrade your Controller. You must upgrade your Controller to the latest patch version of the release you are currently using.

  2. Back Up CoPilot Data. It is highly recommended to perform a pre-migration backup of your CoPilot data that you want to migrate. This backup can help shorten the migration process and reduces the amount of data that might be lost during the migration.

  3. Adjust Maximum Session Duration. Change the maximum session duration of the AWS App Role to 12 hours on AWS > IAM > Roles. See Specify the Maximum Session Duration of an IAM Role on AWS.

  4. Check Current CoPilot Region. During the migration, you will need to select a region for your new CoPilot. Ensure that you choose the same region as your current CoPilot.

Check CoPilot Migration Notes

Read through the migration notices before you perform the CoPilot migration:

  • Configuration Backup: If you had CoPilot configuration backup enabled on the old CoPilot, it will be disabled after the migration. You’ll need to manually re-enable it on the new CoPilot if you want to continue backing up your configuration.

  • Simple to Simple Migration: We only support simple-deployment to simple-deployment migration and migrations within the same region. Make sure to perform the migration accordingly.

  • Data Backup: To speed up the migration process, it is highly recommended to perform a data backup of the date range that you want to migrate beforehand. This allows us to deduplicate the data and make the migration faster. In addition, it reduces the amount of data that might be lost during the migration.

  • Security Groups (SG): The new CoPilot’s security groups are managed automatically through CoPilot SG Management feature. If you had custom SG settings and was not using SG management on the old CoPilot, you’ll need to manually copy your old CoPilot’s SG to the new CoPilot. This is a known issue.

  • FlowIQ and Performance Page Availability: After deploying the new CoPilot, the FlowIQ and Performance page might not be immediately accessible as netflow/perfmon data restoration is in progress. This is a temporary issue.

  • EIP Switch and Login: If you are on the old CoPilot UI when the EIP is switched, you might see a "Session timed out" message on the login page. In that case, do not log in. The new CoPilot will initialize within a minute, and you’ll see a loading page. This is a known issue.

  • Potential Data Loss: During the migration, there is a possibility of a small amount of incoming data (such as performance monitoring and netflow data) being lost while the data restore process is ongoing.

General Data Backup/Restore Notice

It is recommended to set a more frequent backup frequency (daily or weekly). This provides greater granularity when selecting which backup to restore. Remember that increasing the backup frequency will not result in additional data stored in S3, as deduplication is applied.

Migrate CoPilot to Appliance V3 (AWS)

Currently, we only support simple-deployed to simple-deployment CoPilot migration.

Procedures of Migrating CoPilot to Appliance V3 (AWS)

Perform the following steps to migrate your CoPilot to Appliance v3.

  1. Log in to your CoPilot.

  2. From Settings > Maintenance > CoPilot Management, click CoPilot Management.

  3. On the Migrate to Appliance v3 card, click Migrate. You can migrate from Appliance v1 to v3, v2 to v3 or from Appliance v3 to v3.

  4. In the Migrate to Appliance v3 window, click Simple Deployment.

  5. Click Next to use the default deployment settings if your Controller and CoPilot are in the same region. Otherwise, you need to turn on the Customize Deployment Option to change the region to your current CoPilot region.
    To customize the deployment, toggle the Customize Deployment option to On, then perform the following steps:

    1. Choose AWS as the cloud provider.

    2. Choose your access Account.

    3. Choose a region as the deployment region. This region must be the same region as your current CoPilot.

    4. Choose a VPC/VNet.

    5. Choose a Subnet.

    6. Choose the VM size for CoPilot. See CoPilot Requirements for details. The VM size must be equal to or larger than the VM size of your current CoPilot., which is displayed as the default value for the CoPilot VM Size.

    7. Specify a disk size of your CoPilot deployment. The disk size must be equal to or larger than the disk size of your current CoPilot. of your Current CoPilot, which is displayed as the default value for the Data Disk Size.

  6. Specify the time period, the starting date, and end date for the data restore from drp-down box and the date picker. You can see the estimated data size and estimated migration time. If you have previously backed up data, it can save time during this migration phase.

  7. Click Start Migration to start the migration process. The migration takes some time to complete. You can choose to continue the migration in the background by clicking Continue in Background and close this window.

During the CoPilot migration, avoid performing any tasks on CoPilot to prevent the loss of configurations or settings in the new CoPilot after migration.

View CoPilot Migration Progress

Once you have initiated the CoPilot migration, you can close the migration window and continue with other tasks. The migration process will continue running in the background. However, you may want to periodically check the progress of the migration.

To view the CoPilot migration progress, follow these steps:

  1. From Settings > Maintenance > CoPilot Management, click CoPilot Management.

  2. Click View Progress to view the migration progress page.

After Deploying the New CoPilot

Click the View Progress to view the migration progress page. Keep an eye on the progress until you see the message "Finish deploying CoPilot, waiting for it to upgrade to the latest version." This indicates that the new CoPilot has been successfully deployed.

Add 0.0.0.0/0 to Security Group

The new CoPilot has the security group enabled by default. If you had custom security group configurations on the old CoPilot or have more than 500 gateways on the old CoPilot, you need to add the 0.0.0.0/0 rule to the security group of the new CoPilot instance as soon as the new CoPilot has been deployed:

  1. Log in to the Controller UI.

  2. From SETTINGS > CoPilot > CoPilot Security Group Management, toggle the settings to Disabled. Then click SAVE.

  3. Log into your AWS console. Go to EC2 > Security group.

  4. Edit the inbound rules of the security group associated with the new CoPilot instance.

  5. Add a rule to allow incoming traffic from 0.0.0.0/0 for UDP port 31283 (NetFlow) and UDP port 5000 (Syslog).

Log into the New CoPilot

Once you receive a message indicating that your CoPilot webpage will be refreshed and the new CoPilot UI will be loaded, your browser window will automatically refresh. You will then be directed to the login page of the new CoPilot UI. Use your old CoPilot credentials to log in.

Please ignore any "Failed to retrieve CoPilot data backup status" messages you may receive during this process. These messages are due to the EIP switch taking place.

Check the Ongoing CoPilot Migration Task

To check the status of the ongoing CoPilot migration task, follow these steps:

  1. Click on the Task icon located on the top-right menu of your new CoPilot UI.

  2. From the dropdown menu, click Tasks > Show All Tasks.

  3. Look for the task labeled "CoPilot Migration: Restore CoPilot Data".

  4. Check the status of this task to determine if it is completed or still in progress.

You can also go to Settings > Maintenance > Copilot Management > View Progress to check a more detailed progress of log for the data restore process.

Once the "CoPilot Migration: Restore CoPilot Data" task is marked as completed, it indicates that the CoPilot migration to Appliance v3 has been successfully completed.

After migration, Aviatrix automatically stops the task server and update process on the old CoPilot. No manual action is needed. If you restart the old CoPilot after the migration, the task server and update process will resume. To avoid potential issues, do not restart the old CoPilot VM after migration.

Troubleshoot the CoPilot Migration

If you encounter a "Session timeout. Please log in again" error message on the new CoPilot login page, follow these steps:

  1. Wait for 1-2 minutes before attempting to log in again.

  2. After the waiting period, log into the new CoPilot using your credentials.

For more details about troubleshooting the CoPilot Migration, see Troubleshoot the CoPilot Migration for more details.