Moving a VM between hosts without vCenter

I was recently given a task of moving a VM used by our application development team from it's own little host to our production VMware infrastructure. This VM in particular has 12 disks, totaling 665Gb of data. Since this was sort of a chore, I figured I'd lay out the possible approaches for the next admin out there who isn't sure how do this.

First, the basics. An option for the VM is to migrate using vMotion. You can migrate a VM between hosts and/or datastores with either vMotion or svMotion (Storage vMotion), respectively. This is really your best option if both of your hosts or datastores reside within the same vCenter environment. The main reason I'm writing this document is for the time(s) when you have to move a VM from a standalone host or from a host within a vCenter environment to a different standalone host or to a host within a different vCenter environment.

There's at least 4 ways to go about it and I'll discuss each below.

Exporting as an OVF Template

Exporting a VM as an OVF template is a relatively straight-forward process. In the fat client, you select the VM, click "File", select "Export", and then select "Export OVF Template". What is an OVF Template?

An OVF template is a way to package a VM, set of VMs or a vApp (more on that in another post). The OVF (Open Virtual Format) filetype has been a common way to package an appliance that you just throw into your environment that requires relatively minimal configuration. My VBAs that I talked about in a previous post regarding my experience with Unitrends Virtual Backup utilizes Linux-based appliances (called Virtual Backup Assistants) as an OVF template. I've also posted about the application we use as the backbone of my university's emergency notification system. It runs on a Linux-based appliance (sensing a theme yet?) that deploys as an OVF template. I'm planning on playing with an ELK (Elastic search, Logstash, Kabana) appliance that also deploys as an OVF template. 

In my scenario, the VM I had to move had 12 disks attached to it and exporting the VM as an OVF didn't seem to capture the additional disks in the VM. I haven't found out if exporting as an OVF doesn't capture additional disks normally or if I just did something wrong. Onto the next method.

Joining the standalone host and using vMotion

Another way to move a VM off of a host is to join the host to your vSphere instance. Once joined, you can leverage vMotion and svMotion to move the VM from the host's local storage to the shared CPU, RAM & Disk of your (or one of your many, depending on the size of your environment) vSphere instance. 

Joining a host to a vCenter server isn't very difficult. The jist of it is simply right-clicking on a datacenter object in vCenter and selecting the "Add Host" option. The only things you have to think about is that the licensing and virtual networking match up.

Without going into too much detail on VMware licensing and networking (each of which could deserve its own post), the vMotion feature to live-migrate a VM between hosts with no shared storage requires using the web-clients on vSphere 5.1 or higher with at least an Essentials Plus or higher license. From a networking standpoint, each host needs a vmkernel port group for vmotion, destination and source hosts must have access to the same subnets and lastly, the network labels must match up if you're using standard vSwitches. 

In my case, the networking on the lone host with the VM I had to move didn't match up to the networking on the hosts in my production vSphere infrastructure. Onto the next method. 

There have been enhancements in vCenter 6 that make cross-vCenter vMotion migrations easier but that too still has its prerequisites. Both vCenters must be running v6.0 or higher under the same SSO domain, have at least a 250Mbps network connection for vMotion, and have layer 2 connectivity between port groups as the IP address doesn't change in vMotion migrations. 

Copying the VMDK & VMX files (aka The Dirty Way)

You might not know this (which I doubt) but your VMs are actually just special files sitting on a special OS that knows how to execute them. Server Virtualization is very cool but it's certainly not magic. Let's take a look at the file types that comprise a Virtual Machine. 

Nvram file - This file contains the BIOS (Basic Input/Output System) for the VM the same way a physical comport has a small memory chip on the motherboard that contains it's BIOS. 

VMX file - This file contains the configuration information for the VM. When you make changes to a VM's hardware such as adding a disk or additional NICs, that information is kept in the VMX file. 

VMDK file - This file is basically the hard drive of your VM. Every VM has at least one but can have many more. If you assigned a 40Gb drive to your VM, the VMDK file will be about 40Gb, even if that virtual disk doesn't have 40Gb of data on it (provided your disk is thick-provisioned as opposed to thin-provisioned.) It also contains a descriptor file that provides information on drive sectors, similar to the MFT (Master File Table) on a physical disk. There are 3 types of VMDK files. 

-Flat.VMDK - In a thick-provisioned disk (which is the most common), this is just like I explained above. This is a regular VMDK, which is a virtual disk file that sits in a VMFS (more on that in another post) datastore and is seen as a hard drive to a VM. If you give a VM a 50Gb disk, you'll see a 50Gb VMDK file, even if that VMDK doesn't actually contain 50Gb of data. 

-Delta.VMDK - This type of VMDK is very similar to the flat.vmdk as described above. The major difference is that the delta.vmdk is created when a snapshot is taken. The original flat.vmdk is made read only and the delta is the data written since the snapshot. 

-RDM.VMDK- When you give a VM direct access to a LUN (Logical Unit Number) or disk, this is referred to as a Raw Device Mapping, or RDM. When you give a VM and RDM, the VMDK associated with this VM will be an RDM.VMDK. This virtual disk file contains a descriptor file just like a flat VMDK, but instead of containing MFT information (what file is in what block & sector) it contains location information regarding where the LUN or disk is, how big it is, etc. The RDM.VMDK file will appear to be as large as the LUN or disk itself but it's actually pretty small. 

VSWP file - This is the swap file setup for the host to use for a specific VM if the host becomes overcommitted on RAM. You might be familiar with the idea if you've run Linux before and have had to setup the swap partition. Windows has a similar feature known as the Page File. The general idea is to use the a bit of hard drive space to store data that would normally be stored in RAM in situations where a host doesn't have enough RAM to handle all of it's VMs. It's a lot slower than RAM but it works. The VSWP file will typically be sized by the amount of RAM given to a VM minus its memory reservation. For example, if you give a VM 8Gb of RAM with a 2Gb reservation, your VSWP file will be approximately 6Gb. 

VMSS file - When a VM is suspended, a VMSS file is created to store the memory contents do that the VM can pick up right where it left off. When the VM is brought out if suspension, its memory contents are written back to the memory on the physical host but the VMSS file will hang around until the VM is powered off. 

VMSD file - This file is created to store snapshot metadata. A vmsd file is always present in the VM files but will be a 0 byte file until a snapshot is taken of that VM. Only one of these files exists, even if you take several sequential snapshots of a VM, so when a snapshot is taken, this file gets updated. It contains the name of the VMDK, the name of the VMSN file made from the snapshot, the display name and description of the VM and the UID of the snapshot to name a few things. 

VMSN file - This file has to do with, you guessed it, snapshots. When a snapshot is taken of a VM, a VMSN file is created to store the state of the VM. It's akin to a Restore Point in Windows. The size of the VMSN file will vary on a number of things, particularly if you choose to include the memory contents along with the snapshot. If chosen, the VMSN file will be whatever size plus slightly more than the contents of the RAM. (Similar to the VMSS file we discussed above but that's for the memory contents of a suspended VM, not a snapshotted one.) 

LOG file - This last file is exactly what it sounds like; It contains logging data for the VM. Each VM will contain a current vmware.log file as well as numbered log files going back to the last 6 log files. This information is used primarily for troubleshooting issues with the VM itself. 

(While there are several other VM file types (such as .ctk, .vmem or .vmtm files to name a few), I just wanted to discuss the common files you'll see when moving a VM manually.)

The jist of moving a VM with just the file is to explore the source datastore and download the files. Once the files have been downloaded from the source, you can upload those files to the destination datastore. When the files have finished uploading, you explore the destination datastore, right-click on the VMX file and select "Add to Inventory". After that, you'll see the VM show up in your inventory and you can go ahead and power it up. I like to watch the console of the VM after moving it to make sure it spears to boot up correctly.

First, click on datastores > manage > files to view the VM folders in a datastore.

Then, right-click and select "Download from Datastore". In the web-client, you'll need to download each file individually. In the fat-client, it'll download all vm files in one operation.

When you're ready to upload the files to the new datastore, click the icon to the right of the cursor in the picture above. 

When the files have finished uploading, right-click on the VMX file and select "Register VM..."

In my specific scenario, this was the method that worked for moving my VM with 12 attached disks totaling 665Gb. It took forever (about 25 hours when all was said and done) but it worked successfully. 

One-off Backup N' Restore (VeammZip & Others)

This the final way that I know of to move a VM. If you've ever restored a VM with products from Unitrends or Veamm, you know you can restore a VM to whatever host or datastore you want as long as the backup software can access it. The makers of the backup softwares have better documentation than I could produce quickly so I'll have you refer to them if the 3 above methods don't work for you.

I hope this post helps someone with a similar task who hasn't done this before.