This is an outdated version of the HTCondor Manual. You can find current documentation at http://htcondor.org/manual.

2.12 Docker Universe Applications

A docker universe job instantiates a Docker container from a Docker image, and HTCondor manages the running of that container as an HTCondor job, on an execute machine. This running container can then be managed as any HTCondor job. For example, it can be scheduled, removed, put on hold, or be part of a workflow managed by DAGMan.

The docker universe job will only be matched with an execute host that advertises its capability to run docker universe jobs. When an execute machine with docker support starts, the machine checks to see if the docker command is available and has the correct settings for HTCondor. Docker support is advertised if available and if it has the correct settings.

The image from which the container is instantiated is defined by specifying a Docker image with the submit command docker_image. This image must be pre-staged on a docker hub that the execute machine can access.

After submission, the job is treated much the same way as a vanilla universe job. Details of file transfer are the same as applied to the vanilla universe. One of the benefits of Docker containers is the file system isolation they provide. Each container has a distinct file system, from the root on down, and this file system is completely independent of the file system on the host machine. The container does not share a file system with either the execute host or the submit host, with the exception of the scratch directory, which is volume mounted to the host, and is the initial working directory of the job.

Therefore, the submit description file should contain the submit command

  should_transfer_files = YES

With this command, all input and output files will be transferred as required to and from the scratch directory mounted as a Docker volume.

If no executable is specified in the submit description file, it is presumed that the Docker container has a default command to run.

When the job completes, is held, evicted, or is otherwise removed from the machine, the container will be removed.

Here is a complete submit description file for a sample docker universe job:

  universe                = docker
  docker_image            = debian
  executable              = /bin/cat
  arguments               = /etc/hosts
  should_transfer_files   = YES
  when_to_transfer_output = ON_EXIT
  output                  = out.$(Process)
  error                   = err.$(Process)
  log                     = log.$(Process)
  request_memory          = 100M
  queue 1

A debian container is the HTCondor job, and it runs the /bin/cat program on the /etc/hosts file before exiting.