11.3 Upgrading from the 8.4 series to the 8.6 series of HTCondor

Upgrading from the 8.4 series of HTCondor to the 8.6 series will bring new features introduced in the 8.5 series of HTCondor. These new features include the following (note that this list contains only the most significant changes; a full list of changes can be found in the version history: 11.5):

• condor_q-related changes:
  • condor_q now defaults to showing only the current user’s jobs. (Ticket #5271). Similarly, condor_qedit defaults to editing only jobs owned by the current user. (Ticket #5889). (The previous behavior of both commands can be restored by setting CONDOR_Q_ONLY_MY_JOBS to False – see  3.5.9.)
  • condor_q now defaults to batch mode, which produces a single line of output summarizing a batch of jobs (see 2026). (Ticket #5708). (The previous behavior can be restored by setting CONDOR_Q_DASH_BATCH_IS_DEFAULT to False – see  3.5.9.)
  • condor_q (and condor_history and condor_status) can now read and write JSON, XML, and new ClassAd formats (see 2025,  1953, and  2154). (Ticket #5688). (Ticket #5844). (Ticket #5820).
• Job submission-related changes:
  • Added the ability for the condor_schedd to transform job ClassAds upon job submission (see section 3.7.2).
  • Added the ability to group jobs into batches, and assign names to the batches, using the new -batch arguments to condor_submit and condor_submit_dag.
  • Added support in the submit language for retrying jobs if they fail (see 2211).
• condor_dagman-related changes:
  • Added the ability to define SCRIPTS, VARS, etc., for all nodes in a DAG with a single command (see section 2.10.9). (Ticket #5729).
  • Simplified how DAG node priorities work (see section 2.10.9). This means that existing DAGs that use the node priority feature will run differently than they have in the past. (Ticket #4024). (Ticket #5749).
  • Added the new splice connection feature (see section 2.10.9), which allows more flexible dependencies between splices. (Ticket #5213).
• HTCondor can now use IPv6 interfaces; it prefers IPv4 if both IPv4 and IPv6 are available. (Ticket #5104).
• HTCondor now has initial support for Singularity containers (see section 3.17). (Ticket #5828).
• condor_status can now display a single line of output for each machine (rather than a line per slot). (Ticket #5596).
• A number of improvements to the Python bindings including: submission (Ticket #5666). (Ticket #4916).; draining (Ticket #5507).; per-thread security contexts (Ticket #5632).; Computing-On-Demand support (Ticket #5130).; and multiple query support (Ticket #5187).
• Jobs can now be submitted to the Slurm batch scheduling system via the new slurm type in the grid universe. (Ticket #5515).
• Numerous improvements to Docker support, including (Ticket #5680).; (Ticket #5760).; (Ticket #5761).; (Ticket #5750).; (Ticket #5740).; (Ticket #5609).; (Ticket #5456).

Upgrading from the 8.4 series of HTCondor to the 8.6 series will also introduce changes that administrators and users of sites running from an older HTCondor version should be aware of when planning an upgrade. Here is a list of items that administrators should be aware of.

• Shared port (see section 3.9.2) is now enabled by default; set USE_SHARED_PORT to False to disable it. Note that this configuration macro does not control the HAD or replication daemon’s use of shared port; use HAD_USE_SHARED_PORT or REPLICATION_USE_SHARED_PORT instead. See section  3.13.2 for more details on how to configure HAD (and/or the replication daemon) to work with shared port, since just activating shared port without any other configuration change will not work. (Ticket #3813). (Ticket #5103).
• To mitigate performance problems, LOWPORT and HIGHPORT no longer restrict outbound port ranges on Windows. To re-enable this functionality, set OUT_LOWPORT and OUT_HIGHPORT (see  3.5.4 and  3.5.4). (Ticket #4711).
• Cgroups (see section 3.14.13) are now enabled by default. This means that if you have partitionable slots, jobs need to get request_memory correct. (Ticket #5936).
• By default, condor_q queries only the current user’s jobs, unless the current user is a queue superuser or the CONDOR_Q_ONLY_MY_JOBS configuration macro is set to False. (Ticket #5271).
• Added support for immutable and protected job attributes, which makes SUBMIT_REQUIREMENTS more useful (see section 3.5.9). (Ticket #5065).
• By default, the condor_schedd no longer changes the ownership of spooled job files (they remain owned by the submitting user). (Ticket #5226).
• When SEC_ENABLE_MATCH_PASSWORD_AUTHENTICATION is set to True, the related authorizations are now automatically enabled. (Ticket #5304). (See  3.5.24 for details.)
• The master can now run an administrator-defined script at shutdown; see section 3.5.7 for details. (Ticket #5590).

⇐ ↙ ↑ ⇑ ⇒ Contents Index