Background

IBM Cloud for VMware Solutions deploys VMware vCenter Server (VCS) environments using a network architecture consisting of three VLANs: one private VLAN used for management traffic and for NSX VTEPs, a second private VLAN used for storage traffic and vMotion, and a public VLAN.

Initially a sample NSX configuration is deployed for your use, including a distributed logical router (DLR), and an edge services gateway (ESG) that provides NAT service outbound from a logical switch (VXLAN) to both the IBM Cloud private network (10.0.0.0/8 addresses) and the public Internet.

The simple case is to deploy your virtual machines onto the logical switch and take advantage of the ESG to access the private and public networks. (Note that the ESG is initially configured with the sample NAT rule disabled, so you will need to enable it.) However, in our case study we want to deploy a virtual machine that will be used as part of the management stack to manage vCenter, ESXi hosts, and deploy workloads into vCenter. As a result, we prefer to have our virtual machine live directly on the private management network, but it will still need access to the public network, for example to download updates. This means we will need both to assign a private IP to the VM, and also to reconfigure the ESG to provide NAT from the private network to the public network.

Additional Details

You can discover the management VLAN on which your VCS instance is deployed by logging into the IBM Cloud infrastructure portal, displaying details for your bare metal servers, and identifying the Private interface. This information is important if you later need to order additional private portable IP addresses for your use. IBM Cloud infrastructure provides two different kinds of IP addresses: (1) primary subnets whose allocation IBM Cloud manages for bare metal servers and virtual servers, and (2) portable subnets whose allocation is typically managed by you and not by IBM Cloud. Note however that IBM Cloud for VMware Solutions orders and manages several portable subnets for your VCS instance. The only portable subnets associated with the VCS that are available for your use are those that are attached to the private and public interfaces of the sample ESG deployed in your instance. We will use one of these addresses for our VM’s deployment.

Procedure

1. Establish connectivity to your VCS environment (e.g., using the IBM Cloud VPN)
2. Login to your vCenter web client UI
3. Click the Home icon and navigate to Networking & Security
4. Select NSX Edges and double click on the customer-nsx-edge
5. Select the Manage tab, Settings item, and view the list of Interfaces. Note the interface with a 10.x.x.x/26 address. This represents the private portable subnet available to you for your use. One IP address is used by the ESG but the remaining addresses (excluding the network address, gateway address = network+1, broadcast address) are available to you for your use. The ESG can be configured to serve as a NAT for any address in the same subnet as itself. Note well that you will be responsible to manage the assignment of addresses within this subnet to prevent conflict!
6. Configure the ESG firewall to allow outbound traffic from the 10.x.x.x/26 network
   1. Select the Firewall tab and add a new rule after the “All outgoing customer VMs” rule
   2. Configure this rule to allow outgoing traffic from the management network; the source IP specification should be the same subnet as the ESG, for example 10.123.171.128/26
   3. Click to Publish Changes
7. Configure the ESG to NAT traffic from the private to the public network
   1. Select the NAT tab and add a new SNAT rule
   2. Configure this rule to operate on the Public Uplink, for all protocols, for the source IP range matching the ESG subnet (e.g., 10.123.171.128/26), and with a translated IP address matching the public IP address for the ESG (use the same address as the existing NAT rule). Ensure that the rule is enabled.
   3. Click to Publish Changes
8. Deploy and configure your virtual machine
   1. IBM Cloud maintains a mirror of many popular Linux distributions, available only on the private network.
   2. Ensure that your VM is attached to the management network. Attach its adapter to the SDDC-DPortGroup-Mgmt port group.
   3. Configure the network adapter using an address from the ESG subnet. Set its default gateway to point to the ESG rather than to the IBM Cloud backend customer router (BCR). Identify the DNS server(s) for your instance by viewing one of your hosts’ TCP/IP configuration in vCenter. For example, if using RHEL:

      # ifcfg-ens192
      HWADDR=00:50:56:b0:88:39
      NAME=ens192
      GATEWAY=10.123.171.132
      DNS1=10.123.158.32
      DOMAIN=example.com
      DEVICE=ens192
      ONBOOT=yes
      USERCTL=no
      BOOTPROTO=static
      NETMASK=255.255.255.192
      IPADDR=10.123.171.133
      NETWORK=10.123.171.128
      BROADCAST=10.123.171.191

   4. Configure the adapter’s static routes to point to the BCR (i.e., the subnet gateway address) for all private network addresses. Note that IBM Cloud uses both subnets 10.0.0.0/8 and 161.26.0.0/16 for internal traffic. For example, if using RHEL:

      # route-ens192
      10.0.0.0/8 via 10.123.171.129 dev ens192
      161.26.0.0/16 via 10.123.171.129 dev ens192

   5. Configure NTP to point to time.service.networklayer.com

The result is that we can access both the private and public networks from our VM:

[root@localhost ~]# ### Ping vCenter
[root@localhost ~]# ping -c 1 10.123.170.130 | fgrep transmitted
1 packets transmitted, 1 received, 0% packet loss, time 0ms
[root@localhost ~]# ### Ping Google DNS
[root@localhost ~]# ping -c 1 8.8.8.8 | fgrep transmitted
1 packets transmitted, 1 received, 0% packet loss, time 0ms