Connecting On-Prem with Overlapping CIDRs Using Customized SNAT and DNAT on Spoke Gateway

The Problem

Organizations usually plan out cloud network address ranges for building non-overlapping connectivity to on-prem, but there are times when a cloud network CIDR conflicts with an on-prem network address range. Moreover, there might be a constraint that neither source NAT nor destination NAT can be performed in the on-prem network but still requires connectivity to on-prem.

Aviatrix Solution

This document illustrates performing source NAT and destination NAT on an Aviatrix Spoke gateway where a customer needs connectivity between certain on-prem hosts and certain cloud instances, but the on-prem network range overlaps with the cloud network CIDR as shown in the diagram below. Additionally, traffic can be initiated from either side.

Topology - Aviatrix Global Transit HA Network with Active Mesh:

Transit-ActiveMesh-Spoke

In this example, the on-prem network address range is 10.3.0.0/16 and all other Spoke VPCs connect to on-prem via Aviatrix Global Transit HA Network with Active Mesh. There is one Spoke VPC with an identical CIDR of 10.3.0.0/16.

This example supports:

  1. Specific on-prem hosts and cloud instances which means no identical IP on each side including IPs of Aviatrix Spoke primary/HA gateway.

  2. Bi-directional traffic.

  3. Both on-prem and cloud network allow only the RFC 1918 CIDR.

This document provides a step-by-step configuration on Aviatrix CoPilot that addresses the following requirements:

  1. Deploying Aviatrix Global Transit HA Network with ActiveMesh.

  2. Deploying a virtual CIDR within the RFC 1918 range to solve the overlapping CIDR between the on-prem network and the cloud network.

Scenario:

  1. Traffic which is initiated from on-prem to cloud Spoke network sends to a virtual IP of the cloud instance. In addition, the cloud instance views a virtual IP of the on-prem host.

  2. Traffic initiated from the cloud Spoke network to on-prem sends to a virtual IP of on-prem host. In addition, on-prem views a virtual IP of cloud instance.

  3. All virtual IPs belong to the RFC 1918 range.

Follow the steps below to set up for the scenario.

Prerequisites

  • Upgrade the Aviatrix Controller to the latest version.

  • Prepare a Real/Virtual CIDR mapping table for the on-prem network and the cloud network.

  • One of the key steps to solve overlapping network issues is to route a non-overlapping CIDR. Therefore, please prepare a virtual route table CIDR for your on-prem and spoke network. In this example, a Virtual CIDR within the RFC 1918 range is created.

    Real/Virtual CIDR mapping table example:

==============  ==============  ================
                Real CIDR       Virtual CIDR
==============  ==============  ================
Spoke VPC       10.3.0.0/16     10.203.0.0/16
On-Prem VPC     10.3.0.0/16     10.103.0.0/16
==============  ==============  ================

    Find out the Real IPs of certain on-prem hosts and certain cloud instances to build a Real/Virtual IPs mapping table.

  • Since this solution is to tackle a specific use case where a customer needs a connectivity between certain on-prem hosts and certain cloud instances in overlapping CIDR, please find out those IPs and plan a Real/Virtual IPs mapping table for routing advertisement and NAT configuration.

    Real/Virtual IPs mapping table example:

================    ==============  ================
                    Real IP         Virtual IP
================    ==============  ================
Cloud instance      10.3.0.86/32    10.203.0.86/32
On-Prem host        10.3.0.85/32    10.103.0.85/32
================    ==============  ================

Build Aviatrix Global Transit HA Network with Active Mesh

Deploy the topology by following steps 1, 2, 3, 4, and 5 in Multicloud Network Transit Workflow.

  • Make sure HA is deployed for both Aviatrix Transit Gateway and Spoke Gateway.

  • Make sure the on-prem router advertises only the Real IP with /32 of on-prem host not the whole Real CIDR or Virtual IP/CIDR.

Example: on-prem router advertises 10.3.0.85/32 which is the Real IP of On-prem host

This action is to advertise the Virtual CIDR of cloud spoke network to the on-prem via BGP session so that on-prem is able to route the Virtual IP of the Cloud instance. Please refer to Customizing Spoke Advertised VPC CIDRs.

To configure:

  1. On the Cloud Fabric > Gateways > Spoke Gateways tab, click the name of the Aviatrix Spoke Gateway.

  2. Click the Settings tab.

  3. On the Settings tab, expand the Routing area.

  4. Under Customize Spoke Advertised VPC/VNet CIDRs, enter the Virtual CIDR of the cloud Spoke VPC that on-prem is able to route. Make sure to advertise the Virtual CIDR of the cloud Spoke VPC, not the virtual IP of specific cloud instance.

Example: Aviatrix Spoke gateway advertises 10.203.0.0/16 which is the Virtual CIDR of cloud spoke VPC

Attach Aviatrix Spoke Gateway to Aviatrix Transit Network

Attach Spoke Gateway to Transit.

Configure Aviatrix DNAT function on Aviatrix Spoke Gateway

This is for the traffic which is initiated from on-prem to Cloud Spoke Network. This action instructs the Spoke gateway to translate a destination address from a Virtual IP of cloud instance to a Real IP of cloud instance in cloud spoke VPC. Refer to the Aviatrix DNAT function document.

The Mark funtion in NAT is a unique number that is associated with specific packets. In this document, we leverage it for the purpose of tracking the session identified by the Source CIDR of DNAT and then utilizing it for the SNAT IPs of customized SNAT. It is an advanced option. Alternatively, you still can configure DNAT and customized SNAT rule without mark.

If using AWS or Azure, you add the VPC/VNet route table IDs in the Exclude Route Table field to prevent extra routes from being injected into the cloud routing table.

To configure:

  1. Go to the Settings sub-tab for the Spoke Gateway (Cloud Fabric > Gateways > select Spoke Gateway).

  2. Expand the Network Address Translation (NAT) area.

  3. Scroll down to Destination NAT and turn it On.

  4. Click +Rule.

  5. Fill in the fields as per the below example.

    Field Value

    Source CIDR

    Real IP of on-prem host (i.e. 10.3.0.85/32)

    Source Port

    Leave it blank

    Destination CIDR

    Virtual IP of cloud instance (i.e. 10.203.0.86/32)

    Destination Port

    Leave it blank

    Protocol

    all

    Interface

    eth0

    Connection

    Select the connection to Transit Gateway

    Mark

    A rule field to mark this traffic session (i.e. use 103085 to track source 10.3.0.85/32)

    DNAT IPs

    Real IP of cloud instance (i.e. 10.3.0.86)

    DNAT Port

    Leave it blank

    Exclude Route Table

    Collect all your cloud routing table IDs and add them here (only supported for AWS and Azure)
    DNAT_SPOKE_ONPREM_TO_CLOUD

  1. Click Save.

  2. Repeat steps 4-6 for multiple entries.

  3. Slide Apply Route Entry On to commit the rule.

Configure Aviatrix Customized SNAT function on Aviatrix Spoke Gateway and Spoke HA Gateway

This if for the traffic which is initiated from on-prem to cloud Spoke network. This action changes the packet’s source IP address from a Real IP of the on-prem host to a Virtual IP representing the on-prem host. Refer to Aviatrix Customized SNAT.

To configure:

  1. Go to the Settings sub-tab for the Spoke Gateway (Cloud Fabric > Gateways > select Spoke Gateway).

  2. Expand the Network Address Translation (NAT) area.

  3. Scroll down to Source NAT and turn it On if it is not turned On already.

  4. Click Customized SNAT.

  5. Select the Spoke Gateway instance.

  6. Click +Rule.

  7. Fill in the fields as per the below example.

    Field Value

    Source CIDR

    Leave it blank

    Source Port

    Leave it blank

    Destination CIDR

    Leave it blank

    Destination Port

    Leave it blank

    Protocol

    all

    Connection

    Select None

    Mark

    Fill the number that we configure in the previous DNAT step 5 (i.e. 103085)

    SNAT IPs

    Virtual IP of on-prem host (i.e. 10.103.0.85)

    SNAT Port

    Leave it blank

    Exclude Route Table

    Collect all your cloud routing table IDs and add them here (only supported for AWS and Azure)

  1. Click Save.

    SNAT_SPOKE_PRIMARY_ONPREM_TO_CLOUD

  1. Slide Apply Route Entry On to commit the rule.

  2. Repeat steps 5-9 for the Spoke HA Gateway.

    SNAT_SPOKE_HA_ONPREM_TO_CLOUD

Configure Aviatrix DNAT function on Aviatrix Spoke Gateway

This is for the traffic which is initiated from the cloud Spoke network to on-prem. This action instructs the Spoke gateway to translate a destination address from a Virtual IP of on-prem host to a Real IP of on-prem host. Refer to the Aviatrix DNAT function document.

To configure:

  1. Go to the Settings sub-tab for the Spoke Gateway (Cloud Fabric > Gateways > select Spoke Gateway).

  2. Expand the Network Address Translation (NAT) area.

  3. Click +Rule.

  4. Enter the fields as per the below example.

    Field Value

    Source CIDR

    Real IP of cloud instance (i.e. 10.3.0.86/32)

    Source Port

    Leave it blank

    Destination CIDR

    Virtual IP of on-prem host (i.e. 10.103.0.85/32)

    Destination Port

    Leave it blank

    Protocol

    all

    Interface

    eth0

    Connection

    Select None

    Mark

    A rule field to mark this traffic session (i.e. use 103086 to track source 10.3.0.86/32)

    DNAT IPs

    Real IP of on-prem host (i.e. 10.3.0.85/32)

    DNAT Port

    Leave it blank

    Exclude Route Table

    Collect all your cloud routing table ids and fill them here (only supported for AWS and Azure)

  1. Click Save.

    DNAT_SPOKE_CLOUD_TO_ONPREM

  2. Repeat steps 3-5 for multiple entries.

  3. Slide Apply Route Entry On to commit the rule.

Configure Aviatrix Customized SNAT function on Aviatrix Spoke Gateway and Spoke HA Gateway

This is for the traffic which is initiated from the cloud Spoke network to on-prem. This action changes the packet’s source IP address from a Real IP of cloud instance to a Virtual IP representing cloud instance. Please refer to Aviatrix Customized SNAT.

To configure:

  1. Go to the Settings sub-tab for the Spoke Gateway (Cloud Fabric > Gateways > select Spoke Gateway).

  2. Expand the Network Address Translation (NAT) area.

  3. Scroll down to Source NAT and turn it On if it is not turned On already.

  4. Click Customized SNAT.

  5. Select the Spoke Gateway instance.

  6. Click +Rule.

  7. Enter the fields as per the below example.

    Field Value

    Source CIDR

    Leave it blank

    Source Port

    Leave it blank

    Destination CIDR

    Leave it blank

    Destination Port

    Leave it blank

    Protocol

    all

    Connection

    Select the connection to Transit Gateway

    Mark

    Fill the number from the previous DNAT procedure (i.e. 103086)

    SNAT IPs

    Virtual IP of cloud instance (i.e. 10.203.0.86)

    SNAT Port

    Leave it blank

    Exclude Route Table

    Collect all your cloud routing table ids and add them here (only supported for AWS and Azure)

  1. Click Save.

    SNAT_SPOKE_PRIMARY_CLOUD_TO_ONPREM

  2. Repeat steps 6-8 for the Spoke HA Gateway. You can also repeat the steps to add more entries.

  3. Slide Apply Route Entry On to commit the rule.

Verify Traffic Flow

Traffic from on-prem to cloud spoke network

  • Issue ICMP traffic from on-prem host to a Virtual IP of cloud instance

    ONPREM_HOST_TO_CLOUD_INSTANCE

  • Execute packet capture on the cloud instance

    CLOUD_INSTANCE_PACKET_CAPTURE

Traffic from cloud spoke network to on-prem

  • Issue ICMP traffic from cloud instance to a Virtual IP of on-prem

    CLOUD_INSTANCE_TO_ONPREM_HOST

  • Execute packet capture on the on-prem host

    ONPREM_HOST_PACKET_CAPTURE