> ## Documentation Index
> Fetch the complete documentation index at: https://docs.aviatrix.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Migrating CoPilot Data
> This section discusses migrating CoPilot data from old CoPilot to new CoPilot. It includes:
This section discusses migrating CoPilot data from old CoPilot to new CoPilot.
It includes:
* [About Migrating CoPilot Data](#about-migrating-copilot-data)
* [Supported CoPilot Data Migration Paths](#supported-copilot-data-migration-paths)
* [CoPilot Data Migration Preparations](#copilot-data-migration-preparations)
* [Perform Prerequisite Tasks for Migrating CoPilot Data](#perform-prerequisite-tasks-for-migrating-copilot-data)
* [CoPilot Data Migration Procedure](#copilot-data-migration-procedure)
* [Verify CoPilot Data Migration](#verify-copilot-data-migration)
* [Post CoPilot Migration Tasks](#post-copilot-migration-tasks)
* [Troubleshoot CoPilot Data Migration](#troubleshoot-copilot-data-migration)
* [Cancel CoPilot Data Migration](#cancel-copilot-data-migration)
## About Migrating CoPilot Data
Starting with CoPilot release 2.0.3, you can migrate data from one (source)
CoPilot instance to another (destination) CoPilot instance.
Data migration is supported across regions, availability zones, and VPCs/VNets
within the same CSP. Migration is not supported across CSPs.
The migrated data includes the indexes shown in Settings > Index Management. The
indexes are migrated from the data disk (volume) of the source instance to the
data disk (volume) of the destination instance. Configuration data for CoPilot
functions are also migrated, including but not limited to data for
notifications, alerts, network behavior analytics configurations, threat IP
configurations, and Geoblocking configurations.
You would migrate CoPilot data when upgrading to a new CoPilot *image release*
version. After launching a new CoPilot instance based on the new image release
version, you would migrate data from the old instance to the new instance. You
would also migrate CoPilot data any time you want to deploy a new instance of
CoPilot and retain your data from the old instance. You can launch the new
instance through the same CSP marketplace portal as your existing instance, by
using the controller UI (starting from Controller 6.7.1185 for AWS Cloud only),
or by using Terraform scripts.
For instructions on migrating CoPilot data from a source to a destination
instance, see
[Migrate data from one CoPilot instance to another](#migrate-data-from-one-copilot-instance-to-another).
## Supported CoPilot Data Migration Paths
CoPilot supports
[Migrating data from one CoPilot instance to another](#migrate-data-from-one-copilot-instance-to-another)
### Data Migration Flow for CoPilot Instance(s)
#### Migrate Data from One CoPilot Instance to Another
Data will be migrated from the old CoPilot data disk to the new CoPilot data
disk.
## CoPilot Data Migration Preparations
These instructions for migrating CoPilot data apply to the migration paths that
are listed in
[Supported Migration Types](#supported-copilot-data-migration-paths).
The following terms are used in these instructions:
* *old copilot* — Refers to your current (source) CoPilot instance that you want
to migrate data from.
* *new copilot* — Refers to your newly deployed (destination) CoPilot instance
that you want to migrate data to.
### Notes about Data Migration Process
Please consider the following points about the data migration process in the
current release:
* Migration of CoPilot data is not supported across CSPs. Data migration is
supported across regions, availability zones, and VPCs/VNets of the same CSP.
* Aviatrix has tested data migration for infrastructures with up to a total of
500 GB of data. If you have a much larger infrastructure, please contact
Aviatrix Support for more information about how to migrate your data.
* If data migration fails and you want to retry the migration, please contact
Aviatrix Support for assistance.
* If the data migration utility fails to migrate all indexes, you cannot revert
the migration but the data remains intact on the old copilot.
* During the migration process, you cannot make configuration changes to the
*old copilot*.
* Upon starting the data migration, the netflow and syslog data sent by your
Aviatrix gateways is automatically switched to be sent to your *new copilot*.
If the migration fails, and you decide to terminate your *new copilot*, the
netflow and syslog data that was directed to the *new copilot* during
migration will be lost.
* You can pick or customize the restore time period. However, we will restore
the available data if the chosen time period does not have data.
* If data migration fails and you decide to cancel the data migration, you can
terminate your *new copilot* and continue to use your *old copilot* (by
following the instructions indicated in the procedure).
CoPilot data may take a few hours for small environments and a few days for
large environments. Environments with 500 GB of data may take 3 to 4 days to
complete data migration.
The migration process will not cause down-time in your data plane but your
CoPilot will not be available to receive new data for about 10 minutes after
migration begins and 5 minutes after migration succeeds or fails. It is best
practice to plan the migration during a maintenance window.
You can perform prerequisite tasks outside your maintenance window to save
valuable time during the maintenance window. Please note that while you can use
your *old copilot* during the migration process, any changes you make during the
migration will NOT be reflected on the *new copilot*.
## Perform Prerequisite Tasks for Migrating CoPilot Data
Before you begin the data migration process, perform the following tasks.
Prerequisite tasks can be performed outside your maintenance window. When
logging in to CoPilot, use a controller user account that has full admin
permissions. To confirm that the user account has full admin permissions, log in
to your Controller, go to Accounts > Account Users, and verify the "Permissions
Groups" column is set to *admin* for the account in question.
### Obtain the Required Information
Obtain the following information:
* The IP address of your Aviatrix Controller.
* The IP address of your *old copilot*. The *old copilot* IP address can be the
private IP, public IP, or Elastic IP address (EIP) used for reachability of
the instance.
* The amount of *storage used* on your *old copilot* for the data volume
containing CoPilot data. To look up the storage used, navigate to CoPilot >
Settings > Resources and refer to the Used column of the volume. If your
CoPilot is based on CoPilot image release version 1.5.1, you will see a
*cpltLV* volume listed on the Resources page. In that case, refer to the Used
column of the cpltLV volume to take note of the storage used.
### Launch Your New Copilot
Make sure your new CoPilot meets the following:
* The size of the disk/volume you specify for the Instance is the same size or
larger than the *storage used* in your *old copilot*. If you deploy in AWS
using the controller UI deploy process, you specify the size in the "Data
Disk" column for the instance.
* Take note of your *new copilot* IP address. The new copilot IP address can be
the private IP, public IP, or Elastic IP address (EIP) used for reachability
of the instance. It is highly recommended that a persistent IP is used such as
an EIP or statically assigned private IP.
* After launch, your *new copilot* will take about an hour to automatically
update to the latest software release version. Your *new copilot* and *old
copilot* must be the same software version before starting data migration. You
will verify this later when following the data migration procedure.
### Open Required Ports on Each CoPilot Instance
At the applicable CSP portal, on the **new copilot** VM:
After initial deployment, your *new copilot* ports 31283 and 5000 will be open
for any IP (0.0.0.0/0) . It is strongly recommended to remove the 0.0.0.0
entry from the inbound access rules for these ports and add entries for all
your gateway IP addresses.
* Open port 443 to receive TCP traffic from the *old copilot* (*old copilot* IP
address).
* Open port 31283 to receive UDP traffic from each of your Aviatrix gateways.
* Open port 5000 to receive UDP traffic from each of your Aviatrix gateways.
*For private mode*, you open *TCP* port 5000.
* Open port 31284 to receive TCP traffic from the Aviatrix Controller and each
of your Aviatrix gateways (OpenTelemetry metrics and logs).
If you launched your *new copilot* from the Controller UI starting from
Controller release 6.8, the above security group configurations will be
automatically applied.
At the applicable CSP portal, on the **old copilot** VM:
* Open port 9200 to receive TCP traffic from the *new copilot* (new copilot IP
address).
* Open port 443 to receive TCP traffic from the *new copilot* (new copilot IP
address).
### Verify the aviatrix-app-policy Permission has been Added to Your Access Account
1. Verify that your Controller instance (Access
Account) has the IAM role `aviatrix-role-ec2` attached to it. Make sure
the `aviatrix-app-policy` policy has been added to the `aviatrix-role-ec2`
role. See details about IAM roles in AWS IAM Overview.
2. If the `aviatrix-app-policy` policy has not been added yet, perform the
following steps:
1. Go to your Controller UI.
2. From *Access Accounts*, choose your account name.
3. Click the *EDIT* button.
4. On the editing page, click the *LAUNCH CLOUDFORMATION SCRIPT* button to
create the `aviatrix-app-policy`.
It is important to note that the "aviatrix-app-policy" policy is a critical
component in order to ensure that you will be able to migrate your CoPilot
data. See more detail about IAM policies in{" "}
IAM Roles and Policies
.
## CoPilot Data Migration Procedure
The data migration steps listed below apply to the migration paths that are
listed in [Supported CoPilot Data Migration
Paths](#supported-data-migration-types).
To migrate CoPilot data from your *old copilot* to your *new copilot*:
1. Verify you have performed all prerequisite tasks, including launching your
new CoPilot and opening required ports as described in
[Perform Prerequisite Tasks for Migrating CoPilot Data](#perform-prerequisite-tasks-for-migrating-copilot-data)
2. Open the *new copilot* application in your web browser at
`https://`, where `` is the IP address of your *new
copilot* instance. For AWS clouds, this would be your Elastic IP (EIP)
address.
3. Go through the initial setup process to enter information about your *new
copilot* when prompted (see "Initial Setup of CoPilot" for information about
each prompt). When prompted to enter the IP address of your controller, make
sure you enter the IP address of the same controller your *old copilot* is
pointing to.
4. Verify the software version of your *new copilot* and *old copilot* are the
same (they should both be software version 2.0.3 or later).
5. Verify the Performance feature version on your *new copilot* and *old
copilot* are the same (they should both use Performance v2 or v1).
It is likely your *old copilot* is already using Performance v2 (the *Switch
to v2* option in the Performance page was activated). In this case, on your
*new copilot*, select the Performance sidebar option, and when prompted,
click *Switch to v2*. If your *old copilot* is using Performance v1 and your
*new copilot* is using v2, click *Switch to v2* on your *old copilot*.
6. Go to the *new copilot* UI. From *Settings*, click *Migration*.
7. Enter the IP address of your *old copilot* for the *Old CoPilot IP* field.
8. Pick a time period for the data that you want to migrate from your *old
copilot* to your *new copilot*.
* You can choose *All Available Data* to restore all available data.
Alternatively, you can choose a specific period from the drop-down list of
the *Time Period* field.
* You can also provide a custom time period for the migration data. However,
if the custom time period does not exist, we use the available data under
the available time period for data migration.
9. Check the estimated size and estimated time of the data that you want to
restore.
10. Click *Migrate*.
Migrating CoPilot data may take a few hours for small environments and a few
days for large environments. Environments with 500 GB of data may take 3 to
4 days to complete data migration. If you use your *old copilot* during the
migration process, changes you make during migration will NOT be reflected
in the *new copilot*. Upon starting the data migration, the netflow and
syslog data sent by your Aviatrix gateways is automatically switched to be
sent to your *new copilot*.
11. *If data migration succeeds (Migration Complete):*
If all data indices migrate successfully, you will get a Migration Complete
message. See these sections to
[Verify the Data Migration](#verify-the-data-migration) and
[Post CoPilot Migration Tasks](#post-copilot-migration-tasks).
12. *If data migration fails* (*Error: Failed to migrate. Please check the log
below.*), see
[Troubleshoot CoPilot Data Migration](#troubleshoot-copilot-data-migration)
## Verify the Data Migration
To verify the data are migrated on the *new copilot*:
* Go to *Settings > Index Management*. Verify all indexes are visible.
* Navigate to the CoPilot functional screens and verify you can see your
configuration data for notifications, alerts, anomalies, threat IPs,
Geoblocking, and other configurations.
## Post CoPilot Migration Tasks
After your *new copilot* is running with your migrated data intact, you can
perform the following tasks:
* On your *new copilot*, if you use your own SSL certificate (rather than the
CoPilot self signed certificate), update the DNS servers associated with your
certificate in Settings > Configuration.
* Remove your *old copilot* and associated cloud resources:
* EIP
* Access/security groups
* Disks/Volumes
* VM/Instance
* On your Aviatrix Controller, verify that the CoPilot association (Controller >
Settings > CoPilot) is pointing to your *new copilot* IP. If you used the
controller UI to launch your *new copilot*, this was automatically done for
you. Additionally, verify that the Remote Syslog and Netflow Agent
(Controller > Settings > Logging) are pointing to your *new copilot* IP.
## Troubleshoot CoPilot Data Migration
If one or more data indices do not migrate, you will get an *Error: Failed to
migrate.* message followed by a list of failed indices in the error message. Try
the following methods to troubleshoot.
* Retry the migration at least once by doing the following:
1. Click the *Clear Migration* button. This clears the indexes in the status
page of the user interface. It does not delete migrated indices.
2. In the *Migrate Data from Old CoPilot* dialog, re-enter the IP address of
your *old copilot* and tick the checkbox for acknowledging prerequisites.
3. Click *Migrate*. The migration process will continue to migrate data from
where it left off.
* If the migration process continues to fail, you have the following options:
* If you decide you want to continue to retry the data migration, please
contact Aviatrix Support.
* If you decide the indices that failed to migrate are not important (for
example, they are very old records you no longer need), you can choose to
use the *new copilot*. In this case, go to
[Verify Data Migration](#verify-the-data-migration) to verify the important
data you want is indeed in your *new copilot* before deleting your *old
copilot*.
## Cancel CoPilot Data Migration
If you decide to cancel the data migration and continue to use your *old
copilot*, do the following:
1. At the applicable CSP portal for the *new copilot*, stop the instance/virtual
machine.
2. Turn on the Task Server on your *old copilot* (CoPilot > Settings >
Services > Task Server).
3. Configure your controller to send netflow data to your *old copilot*
(Controller > Settings > Logging > Netflow Agent. See "Enable Netflow for
CoPilot FlowIQ Data" for details.)
4. Configure your controller to send syslog data to your *old copilot*
(Controller > Settings > Logging > Remote Syslog. See "Enable Syslog for
CoPilot Security Audit Data" for details.)
5. At the applicable CSP portal for the *old copilot* VM, you can remove the
access rules that were added to open TCP ports 9200 and 443 from the *new
copilot* source IP.
6. Remove your new copilot and its associated cloud resources.