Virtual machines can be executed on any execution site with VMware, Xen (via libvirt), or KVM. To do this, HTCondor must be informed of some details of the virtual machine installation, and the execution machines must be configured correctly. This permits the execution of vm universe jobs.
What follows is not a comprehensive list of the options that help set up to use the vm universe; rather, it is intended to serve as a starting point for those users interested in getting vm universe jobs up and running quickly. Details of configuration variables are in section 3.3.27.
Begin by installing the virtualization package on all execute machines, according to the vendor's instructions. We have successfully used VMware, Xen, and KVM. If considering running on a Windows system, a Perl distribution will also need to be installed; we have successfully used ActivePerl.
For VMware, VMware Server 1 must be installed and running on the execute machine. HTCondor also supports using VMware Workstation and VMware Player, version 5. Earlier versions of these products may also work. HTCondor will attempt to automatically discern which VMware product is installed. If using Player, also install the VIX API, which is freely available from VMware.
For Xen, there are three things that must exist on an execute machine to fully support vm universe jobs.
For KVM, there are two things that must exist on an execute machine to fully support vm universe jobs.
There are configuration variables related to the virtual machines for vm universe jobs. Some options are required, while others are optional. Here we only discuss those that are required.
First, the type of virtual machine that is installed on the execute machine must be specified. For now, only one type can be utilized per machine. For instance, the following tells HTCondor to use VMware:
VM_TYPE = vmware
The location of the condor_vm-gahp and its log file must also be specified on the execute machine. On a Windows installation, these options would look like this:
VM_GAHP_SERVER = $(SBIN)/condor_vm-gahp.exe VM_GAHP_LOG = $(LOG)/VMGahpLog
To use VMware, identify the location of the Perl executable on the execute machine. In most cases, the default value should suffice:
VMWARE_PERL = perl
This, of course, assumes the Perl executable is in the path of the condor_master daemon. If this is not the case, then a full path to the Perl executable will be required.
If using VMware Player, which does not support snapshots, configure the START expression to reject jobs which require snapshots. These are jobs that do not have vmware_snapshot_disk set to False. Here is an example modification to the START expression.
START = ($(START)) && (!(TARGET.VMPARAM_VMware_SnapshotDisk =?= TRUE))
The final required configuration is the location of the VMware control script used by the condor_vm-gahp on the execute machine to talk to the virtual machine hypervisor. It is located in HTCondor's sbin directory:
VMWARE_SCRIPT = $(SBIN)/condor_vm_vmware
Note that an execute machine's EXECUTE variable should not contain any symbolic links in its path, if the machine is configured to run VMware vm universe jobs. Strange behavior has been noted when HTCondor tries to run a vm universe VMware job using a path to a VMX file that contains a symbolic link. An example of an error message that may appear in such a job's event log:
Error from starter on master_vmuniverse_strtd@nostos.cs.wisc .edu: register(/scratch/gquinn/condor/git/CONDOR_SRC/src/con dor_tests/31426/31426vmuniverse/execute/dir_31534/vmN3hylp_c ondor.vmx) = 1/Error: Command failed: A file was not found/( ERROR) Can't create snapshot for vm(/scratch/gquinn/condor/g it/CONDOR_SRC/src/condor_tests/31426/31426vmuniverse/execute /dir_31534/vmN3hylp_condor.vmx)To work around this problem:
Once the configuration options have been set, restart the condor_startd daemon on that host. For example:
> condor_restart -startd leovinus
The condor_startd daemon takes a few moments to exercise the VM capabilities of the condor_vm-gahp, query its properties, and then advertise the machine to the pool as VM-capable. If the set up succeeded, then condor_status will reveal that the host is now VM-capable by printing the VM type and the version number:
> condor_status -vm leovinus
After a suitable amount of time, if this command does not give any output, then the condor_vm-gahp is having difficulty executing the VM software. The exact cause of the problem depends on the details of the VM, the local installation, and a variety of other factors. We can offer only limited advice on these matters:
For Xen and KVM, the vm universe is only available when root starts HTCondor. This is a restriction currently imposed because root privileges are required to create a virtual machine on top of a Xen-enabled kernel. Specifically, root is needed to properly use the libvirt utility that controls creation and management of Xen and KVM guest virtual machines. This restriction may be lifted in future versions, depending on features provided by the underlying tool libvirt.