Insane Mode Encryption Performance

This document discusses Aviatrix Insane Mode (or InsaneMode) performance test benchmarks, parameters that affect performance and how you can turn your environment for best performance.

For more information on Aviatrix Insane Mode, check out this link.

Insane Mode Performance Test Setup

The iperf3 test is performed between a Spoke VPC instance and on-prem VM. The test setup is shown in the diagram below. The encryption is end to end: between on-prem CloudN and Transit GW and between Spoke gateway and Transit GW.

insane_perf_setup

Key variables that affect performance are:

  • MTU size of all devices in the data path.
  • Latency between on-prem and the Transit VPC.
  • Client TCP window size.
  • The number of TCP streams.

Performance Test Results

The performance test is conducted between a c5.4xlarge instance in the Spoke VPC and on-prem host machine over a 10Gbps Direct Connect between Transit VPC and on-prem datacenter (Equinix co-lo). The physical latency is 5ms. Additional latency is injected into the data path to simulate the latency impact to the end to end throughput.

Aviatrix gateways at the Spoke VPC and Transit VPC are c5.4xlarge instance size. Both Spoke VPC gateway and Transit VPC gateway have Insane Mode enabled.

Additional performance tests were done for the new c5n.4xlarge. The c5n.4xlarge has significant performance improvements.

1. MTU = 9000 Bytes, C5.4xlarge

For Jumbo Frame Size of 9000 bytes, the TCP throughput against different end-to-end latencies and the number of TCP streams is shown as below.

insane_perf_jumbo

2. MTU = 1500 Bytes, C5.4xlarge

throughput_1500_25ms

3. MTU = 1500 Bytes, C5n.4xlarge

c5n_throughput_1500B

4. MTU = 9000 Bytes, C5n.4xlarge

c5n_throughput_9000B

How to Tune Performance

1. Check MTU size

Use Trace Path. Go to Troubleshoot -> Diagnostics -> Network. Select a gateway and destination IP address, click Trace Path. It should display MTU of the devices along the path.

2. Tune TCP window size

For Linux machine, follow the instructions here to tune TCP window size.