Sunday, December 14, 2014

vMotion Fails at 14% - Operation Timed Out

While working on a 3 host VMware cluster I was trying to vMotion some servers around and begin maintenance and updates.  I hit this error for the first time "Operation Timed Out" and the dreaded red X.

As it turns out there was a vmx-****.vswp file inside the VM folder that apparently was left over from a failed DRS migration.  After doing some research I found out that during a DRS migration the VMX file is started on both nodes.  Once the migration is completed normally one of these gets removed.  In this case it did not.

If you browse the datastore of the VM that will not move and you open up the folder of the VM you should see two vmx-****.vswp files.  If you see only one then you've got another issue and this isn't the fix.  If you do see two they will be .vswp-1 and .vswp-2.  There could be a .vswp for the VM itself.  DON'T DELETE THAT ONE!  I can't stress that enough.

To know which one to get rid of just look at the time stamps.  If the VM is running it may be hard to tell which is which so make sure you turn the VM off before you begin this process.  Once it's off then whichever file remains in the folder is the one you need to delete.

You can try deleting the file using the datastore browser but if that doesn't work then here's how you can remove it.

1. Start the SSH daemon on the host which has the VM registered and then login via SSH.  I recommend Putty.  You can download it HERE along with many other great tools for network and systems maintenance.
2. # cd /vmfs/volumes/{datastore name}
3. # cd {VM folder name}
4. # ls -lash *.vswp (this will show all the files and timestamps just to verify)
5. # rm vmx-{VM name}-[1-2].vswp

That's all there is to it.  Remember to disable the SSH daemon on your host and now you should be able to vMotion the VM again with it running.

1 comment:

1964CLOUD said...

I cant get this to work, I can see the rogue swvp file when the vms are on but cant remove when off I cant see them neither can i remove them and all this is via via cmd line. Please am i missing something or can anyone help? Am on vcenter 5.5. am able to list the correct datastore and cd to the vm folder and list all the vm file but can see any swap files.