VMWare: Recovering from running out of disk space on the Host machine

If you've ever run out of disk space you know what a pain that is. However when you run out of diskspace on the host machine running VMware, things get even more interesting.

When disk space runs out on the Host VMs are plunged into a “stuck” state, which means they cannot be restarted or started normally. Generally the best way to manage the state of the server is to:

1. Get a list of VMs:
# vmware-cmd -l

2. Check the state:

# vmware-cmd "path-to-the-vm.vmx" getstate

If this second command returns getstate() = stuck, grab a cup of coffee and know this will take a little while to fix. At this point, you’ll need to kill all the VM processes manually:

1. Find out the ids for the processes:

# ps -ef | grep vm

2. Kill the process based on the id returned:

# kill idgoeshere

Repeat for all the VMs that are stuck. This puts the VM in stopped mode. You can verify this by getting the VM state (above). To start the VM, run the following:

# vmware-cmd "path-to-the-vm.vmx" start

This should get your VM back up and running. Keep in mind this actually restarts the machine. In my experience it takes a few minutes to get the machine to respond to anything (ping, ssh, etc) and if you have any services that are not set to start on boot up you will have to do that manually.