Insane Mode Encryption Performance

This document discusses Aviatrix Insane Mode (or InsaneMode) performance test benchmarks, parameters that affect performance and how you can tune your environment for best performance.

For more information on Aviatrix Insane Mode, check out this link.

Insane Mode Performance Test Setup

The iperf3 test is performed between a Spoke VPC instance and on-prem VM. The test setup is shown in the diagram below. The encryption is end to end: between on-prem CloudN and Transit GW and between Spoke gateway and Transit GW.


Key variables that affect performance are:

  • MTU size of all devices in the data path.
  • Latency between on-prem and the Transit VPC.
  • Client TCP window size.
  • The number of TCP streams.

Performance Test Results

The performance test is conducted between a c5n.4xlarge instance in the Spoke VPC and on-prem host machine over a 10Gbps Direct Connect between the Transit VPC and an on-prem datacenter (Equinix co-lo). The physical latency is 5ms. Additional latency is injected into the data path to simulate the latency impact to the end to end throughput.

1. MTU = 9000 Bytes, C5n.4xlarge


2. MTU = 1500 Bytes, C5n.4xlarge, on-prem to spoke instance through Transit instance

The test result below is for the end to end encryption performance. Note the Transit gateway does encryption and decryption twice, hence the performance is around 1Gbps.


3. MTU = 1500 Bytes, C5n.4xlarge, instance to instance directly

The test result below is for the instance to instance over AWS Peering.


How to Tune Performance

1. Check MTU size

Use Trace Path. Go to Troubleshoot -> Diagnostics -> Network. Select a gateway and destination IP address, click Trace Path. It should display MTU of the devices along the path.

2. Tune TCP window size

For Linux machine, follow the instructions here to tune TCP window size.