Sunday, November 18, 2012

VMware Networking Basics

Networking Basics


VMware Workstation provides several ways you can configure a virtual machine for virtual networking.

Bridged networking configures your virtual machine as a unique identity on the network, separate and unrelated to its host.

Network address translation (NAT) configures your virtual machine to share the IP and MAC addresses of the host. The virtual machine and the host share a single network identity that is not visible outside the network. NAT can be useful when you are allowed a single IP address or MAC address by your network administrator. You might also use NAT to configure separate virtual machines for handling http and ftp requests, with both virtual machines running off the same IP address or domain.

Host-only networking configures your virtual machine to allow network access only to the host. This can be useful when you want a secure virtual machine that is connected to the host network, but available only through the host machine.

Custom networking lets you configure your virtual machine’s network connection manually.

If you select the Typical setup path in the New Virtual Machine Wizard when you create a virtual machine, the wizard sets up bridged networking for the virtual machine. You can choose any of the common configurations — bridged networking, network address translation (NAT) and host-only networking — by selecting the Custom setup path. The wizard then connects the virtual machine to the appropriate virtual network.

You can set up more specialized configurations by choosing the appropriate settings in the virtual machine settings editor, in the virtual network editor (on Windows hosts) and on your host computer.

On a Windows host, the software needed for all networking configurations is installed when you install VMware Workstation. On a Linux host, when you install Workstation, you can choose whether to have bridged and host-only networking available to your virtual machines: you must choose both options during the Workstation installation to make all networking configurations available for your virtual machines.




Bridged Networking

Bridged networking connects a virtual machine to a network using the host computer’s Ethernet adapter.

Bridged networking is set up automatically if you select Use bridged networking in the New Virtual Machine Wizard or if you select the Typical setup path. This selection is available on a Linux host only if you enable the bridged networking option when you install VMware Workstation.

If your host computer is on an Ethernet network, this is often the easiest way to give your virtual machine access to that network. Linux and Windows hosts can use bridged networking to connect to both wired and wireless networks.

If you use bridged networking, your virtual machine needs to have its own identity on the network. For example, on a TCP/IP network, the virtual machine needs its own IP address. Your network administrator can tell you whether IP addresses are available for your virtual machine and what networking settings you should use in the guest operating system. Generally, your guest operating system may acquire an IP address and other network details automatically from a DHCP server, or you may need to set the IP address and other details manually in the guest operating system.

If you use bridged networking, the virtual machine is a full participant in the network. It has access to other machines on the network and can be contacted by other machines on the network as if it were a physical computer on the network.

Be aware that if the host computer is set up to boot multiple operating systems and you run one or more of them in virtual machines, you need to configure each operating system with a unique network address. People who boot multiple operating systems often assign all systems the same address, since they assume only one operating system will be running at a time. If you use one or more of the operating systems in a virtual machine, this assumption is no longer true.

If you make some other selection in the New Virtual Machine Wizard and later decide you want to use bridged networking, you can make that change in the virtual machine settings editor (VM > Settings).

The VM-1 can communicate to the HOST operating system, and connect to the outside LAN/INTERNET, but cannot communicate to the NAT and HostOnly networks.

Host-Only Networking
Host-only networking creates a network that is completely contained within the host computer.

A host-only network is set up automatically if you select Use Host-only Networking in the New Virtual Machine Wizard. On Linux hosts, this selection is available only if you enabled the host-only networking option when you installed VMware Workstation.

Host-only networking provides a network connection between the virtual machine and the host computer, using a virtual Ethernet adapter that is visible to the host operating system. This approach can be useful if you need to set up an isolated virtual network.

If you use host-only networking, your virtual machine and the host virtual adapter are connected to a private Ethernet network. Addresses on this network are provided by the VMware DHCP server.

If you make some other selection in the New Virtual Machine Wizard and later decide you want to use host-only networking, you can make that change in the virtual machine settings editor (VM > Settings).

The VM-5 and VM-4 can communicate to each other, and cannot communicate to the HOST operating system and other VM’s in the HostOnly network.

The VM-6 and VM-7 cannot communicate each other, Bridged and to the HOST network also.


Network Address Translation (NAT)

NAT gives a virtual machine access to network resources using the host computer’s IP address.

A network address translation connection is set up automatically if you follow the Custom path in the New Virtual Machine Wizard and select Use network address translation.

If you want to connect to the Internet or other TCP/IP network using the host computer’s dial-up networking or broadband connection and you are not able to give your virtual machine an IP address on the external network, NAT is often the easiest way to give your virtual machine access to that network.

NAT also allows you to connect to a TCP/IP network using a Token Ring adapter on the host computer.

If you use NAT, your virtual machine does not have its own IP address on the external network. Instead, a separate private network is set up on the host computer. Your virtual machine gets an address on that network from the VMware virtual DHCP server. The VMware NAT device passes network data between one or more virtual machines and the external network. It identifies incoming data packets intended for each virtual machine and sends them to the correct destination.

If you select NAT, the virtual machine can use many standard TCP/IP protocols to connect to other machines on the external network. For example, you can use HTTP to browse Web sites, FTP to transfer files and Telnet to log on to other computers. In the default configuration, computers on the external network cannot initiate connections to the virtual machine. That means, for example, that the default configuration does not let you use the virtual machine as a Web server to send Web pages to computers on the external network.

If you make some other selection in the New Virtual Machine Wizard and later decide you want to use NAT, you can make that change in the virtual machine settings editor (VM > Settings).

In the above diagram, the VM-2 & VM-3 can communicate to each other and other VM’s in the NAT network, and also to the HOST operating system

If we looked in to the host services, we can see the below main services for the Virtual networking.
DHCP Server

The DHCP (dynamic host configuration protocol) server provides IP network addresses to virtual machines in configurations that are not bridged to an external network — for example, host-only and NAT configurations.
NAT Device

The NAT (network address translation) device allows you to connect your virtual machines to an external network when you have only one IP network address on the physical network, and that address is used by the host computer. You can, for example, use NAT to connect your virtual machines to the Internet through a dial-up connection on the host computer, through the host computer’s Ethernet adapter, or through a wireless Ethernet adapter.

The NAT device is set up automatically when you install VMware Workstation. (On a Linux host, you must choose to make NAT available to your virtual machines.)

Networking Best Practices 


Traffic Types:


  • Management (High Availability)
  • vMotion
  • Fault Tolerance (Not in all cases)
  • VM Networks
  • Backup (Not in all cases)
  • Storage/NAS (Depends on the type)

Note: backup and Storage say depends because in some cases you may or may not have iSCSI/NAS storage or be running backups for your virtual machines, Especially if you use a product like Veeam or CommVault. Fault tolerance isn’t really used and I believe that even when it does get better it still may not be worth it, considering all the bigger workloads and cost in licensing as well. Here are my recommendations and best practices I follow for dedicating traffic:

Management: If possible: VLAN it, Separate the traffic (to a different switch), Use teaming or a single Nic (if you set up a MGMT kernel on another port group), You can run/share traffic with vMotion, Fault Tolerance, Backup, and Storage NAS. If you do share traffic use some sort of QOS or Network I/O control. BE mindful that running management with all this traffic isn’t recommended but this would provide you a way to run all this traffic over a separate switch a part from production VM traffic. If you have plenty of NICs then you can run it over the VM production network (but you don’t want to expose it to that network) but you must somehow separate it with a different subnet or VLAN. Most cases I see vMotion and MGMT being shared with Fault Tolerance (FT with big 10GB networks). Your NIC teaming should use explicit failover and over-ride so your vMotion/FT traffic will go over a seperate interface then your management traffic.
vMotion-FT-Backup-Storage-NAS: L2 traffic, hopefully doesn’t have to be routed, in most cases I see this and management traffic being shared, especially with 10GB. vMotion+FT+Backup+NAS if you don’t have a ton of connections. On this particular set up it would be good to setup Jumbo Frames. This traffic you wouldn’t want running over production if possible so a dedicated switch would be really good, also VMware recommends using a dedicated storage switch anyways.
VM Networks: I usually dedicate two NICs for VM production traffic and usually create separate port groups for each type of VM related traffic. In some cases you may have a customer who requires separating this out over different NICs. Again this is just one of those you have to look at based on requirements at that time. Normally the ladder is good enough.
Storage/NAS and Backup: In most cases businesses may have their own backup network. You could run storage and backup traffic over those switches if you choose. In that case, you mines of well also run vMotion and FT.



HP Virtual Connect (VC) Network Traffic Layout


A scenario where the entire network traffic which is going inside the HP blades and HP VC in a HP Blade system C7000


Hardware Details
- HP Enclosure = BladeSystem C7000 Enclosure G2
- HP Blades = BL680c G7
- HP Virtual Connect Flex Fabric
- Dual Port FlexFabric 10Gb Converged Network Adapter
- Dual Port FlexFabric 10Gb Network Adapter

Network Traffic Details
- VMware Vcenter
- FT(Fault Tolerance) and VMotion


The above network traffic classification, to be separated due to the huge network load and considering
security aspect also. This is one of the scenario designing VSphere 5 with HP 3PAR and HP C-Class Blade Center

The Network Traffic Design

More details -

Each blade is having 3 x Dual port 10Gig FlexFabric adapter on board. So total there is 6 x 10Gig ports, they are called LOM (Lan On Motherboard) ports. That is LOM1 to LOM6, and each LOM is internally further divided in to 4 adapters, and these 4 adapters share a common bandwidth that is they can have maximum of 10G. And we can divide the traffic inside for each LOM, that is the beauty of the FlexFabric adapters.
Here LOM1 to LOM4 are 10G FlexFabric Converged adapters (FCOE), so each LOM have one FC port and this is used for the SAN traffic. LOM5 and LOM6 are normal 10G FlexFabric adapters.
There are 2 HP Virtual Connect (VC) modules in the enclosure, they are connected to the BAY1 and BAY2, for redundancy LOM1, LOM3 and LOM5 is internally connected to the BAY1 and LOM2, LOM4, LOM6 to BAY2. There will one uplink for network and one uplink for FC Switch (SAN) for each VC, this will give the redundancy, HA, and load balancing and both VC are in Active/Active mode. So each traffic will have at least 2 adapters 1 from each LOM, this will give the redundancy, HA, Load balancing etc.
The VC is simply a Layer 2 network device, it wont do the routing.
NOTE-
Here the VMotion and FT traffic flow is happening inside the blade center Back Plane it self, and is not going to the VC or external core switch.
This is a specific scenario, here each blades inside the enclosure is configured together as one ESXi Cluster and so there is no need to do VMotion or FT outside the Blade. Here the advantage is that the VMotion and FT traffic wont overload the VC or core switch.

Optimizing iSCSI software initiator performance and NIC teaming


There are several factors involved in optimizing iSCSI performance when using the software iSCSI initiator
The only area where iSCSI performance can be easily optimized on an ESX host is in the configuration of the network
separate the network traffic

On an ESX server where you have virtual machine traffic, VMotion traffic and iSCSI traffic 
- vSwitch1 = for virtual machines
- vSwitch2 = for VMotion
- vSwitch3 = for iSCSI

Segregating and isolating the iSCSI network traffic by  VLAN configuration on virtual switch or physical switch 
is required to ensure iSCSI data integrity

VMware Multiprotocol Design 








Using 8 nics

Vswitch0 - Vmotion and management and some virtual machine port group you can create.
use 2 Pnics, each Pnic to each Pswitch, teaming policy = Route based on the originating port ID

management traffic is less
maximum vmotion speed you will get is around below 150 mb/sec because the vmotion buffer size is 256Kb
so you can accomodate some vm traffic also


Vswitch1 - ISCSI

use 2 Pnics, each Pnic to each Pswitch, teaming policy = Route based on the originating port ID
- 2 vmkernel portgroup, for ISCSI, on the each port group, in the NIC teaming, select the "override Failover Order", select on pnic acitve and other pnic as unused adapters, So each port group will have one dedicated pNICS

- use round robin as mutipathing policy in the esxi and also check the netapp, which multipathing is to be used. IF active/active use RR and if ALUA use Fixed path with Array Preference or MRU, check with the netapp.

- USE JUMBO frames, and set to 6000,

Vswitch2 - NFS

- 2 vmkernel portgroup, for NFS, on the each port group, in the NIC teaming, select the "override Failover Order", select on pnic acitve and other pnic as standby adapters, So each port group will have one dedicated pNICS
 create 2 NFS shares, there will be one NFS IP for each controller (you teamed the storage nics), and divide the vms to these nfs datastores, so the entire load will be distributed.
For NFS the vmkernel, will do an one to one mapping between the esx pnic to the storage pnic. So in an instant it can use only one pnic so the max network speed for one path is 1gig

- USE JUMBO frames, and set to 6000,

vSWITCH3 - Virtual machine traffic

use 2 Pnics, each Pnic to each Pswitch, teaming policy = Route based on the originating port ID
and create VM port groups.

Here advantages,
- dedicated nics for NFS and ISCSI so best performance for the entire vm storage
- dedicated nics for VM traffic