This is an outdated version of the HTCondor Manual. You can find current documentation at http://htcondor.org/manual.

10.6 Stable Release Series 7.8

This is a stable release series of HTCondor. As usual, only bug fixes (and potentially, ports to new platforms) will be provided in future 7.8.x releases. New features will be added in the 7.9.x development series.

The details of each version are described below.

Version 7.8.8

Release Notes:

• HTCondor version 7.8.8 released on March 28, 2013.

New Features:

• When using glexec, HTCondor now automatically retries each glexec operation if glexec exits with an error code that is likely to be caused by a transient error, such as a communication error with the mapping service. Previously, any glexec error would cause the job to be put on hold. Now, the job will only go on hold if the maximum number of glexec retries is exceeded. (Ticket #2415).

Configuration Variable and ClassAd Attribute Additions and Changes:

• The new configuration variable GLEXEC_RETRIES is an integer value that specifies the maximum number of times to retry a call to glexec when glexec exits with status 202 or 203, error codes that indicate a possible transient error condition. The default number of retries is 3. (Ticket #2415).

• The new configuration variable GLEXEC_RETRY_DELAY is an integer value that specifies the minimum number of seconds to wait between retries of a failed call to glexec. The default is 5 seconds. The actual delay to be used is determined by a random exponential backoff algorithm that chooses a delay with a minimum of the value of GLEXEC_RETRY_DELAY and a maximum of 100 times that value. (Ticket #2415).

Bugs Fixed:

• Fixed a bug that caused the condor_gridmanager to frequently delegate the X.509 proxy for jobs of the condor grid type to the remote condor_schedd when the delegated proxy's lifetime is not limited. This occurred when configuration variable DELEGATE_JOB_GSI_CREDENTIALS_LIFETIME or job ClassAd attribute DelegateJobGSICredentialsLifetime was set to 0. (Ticket #3395).

• Fixed a bug in condor_advertise that could cause failure to publish ClassAds to condor_collector daemons other than the first one in the list of condor_collector daemons. (Ticket #3404).

• (This bug was fixed in HTCondor version 7.8.4.) condor_userlog no longer ignores ULOG_JOB_RECONNECT_FAILED events. (Ticket #3215).

• Fixed a bug that caused cream grid jobs to become held with a HoldReason attribute stating

  "CREAM error: Transfer failed: globus_ftp_client: 
    an invalid value for url was used"
  

  when the jobs did not have any input or output files to transfer. (Ticket #3415).

• Fixed a bug that could cause HTCondor daemons to abort on condor_reconfig when the value of configuration variable STATISTICS_WINDOW_SECONDS was reduced. (Ticket #3443).

• Fixed a bug that could cause daemons using CCB to fail to reconnect to the CCB server after becoming disconnected. This condition would cause the daemon to become unreachable via CCB until the daemon was restarted. (Ticket #3476).

• If condor_shared_port was using a dynamic port and the condor_master was using the shared port, then if condor_shared_port died, all subsequent attempts to restart it on a different port failed. (Ticket #3478).

• The nordugrid_gahp no longer exits if the job ClassAd contains a bad NordugridRSL attribute value. (Ticket #3495).

• Fixed a bug that caused the command-line arguments of grid universe jobs of grid type cream to be ignored if specified using the new quoting syntax in the submit description file. (Ticket #3473).

• Reduced the likelihood of a problem that caused the condor_master to restart some of its children after a recent reconfiguration, because the condor_master incorrectly concluded that the children were hung. (Ticket #3510).

Known Bugs:

• None.

Additions and Changes to the Manual:

• None.

Version 7.8.7

Release Notes:

• HTCondor version 7.8.7 released on December 18, 2012.

New Features:

• None.

Configuration Variable and ClassAd Attribute Additions and Changes:

• None.

Bugs Fixed:

• Fixed a bug wherein running the condor_suspend command on a scheduler universe job would cause the condor_schedd to crash. (Ticket #3259).

• HTCondor now places EC2 jobs on hold when they fail to authenticate, rather than leaving them and other jobs with the same authentication tokens idle indefinitely. (Ticket #2274).

• For EC2 resource jobs within the grid universe, HTCondor now only destroys the user's half of the SSH keypair when sure that the instance has terminated. This prevents transient problems from rendering an instance permanently inaccessible. (Ticket #3289).

• For EC2 resource jobs within the grid universe, HTCondor no longer generates an SSH keypair if the user did not request one. (Ticket #3061).

• HTCondor no longer generates SSH keypair names that are incompatible with OpenStack v4 (Essex). (Ticket #3060).

• Jobs that were submitted with condor_submit -spool and failed during submission were left indefinitely in the queue in the Hold state. This has been fixed, such that these jobs are removed from the queue. In addition, the condor_schedd daemon will periodically check for jobs that have been in Hold state due to failed file transfer for at least twelve hours; these jobs will be removed from the queue. (Ticket #3200).

• Fixed a problem where an ssh_to_job or an interactive job session would be terminated prematurely if the execute machine was configured to track process trees via a dedicated login. That is, when configuration variable DEDICATED_EXECUTE_ACCOUNT_REGEXP is being used. (Ticket #3232).

• ClassAd functions int() and real() now ignore trailing characters within a string argument that contains a valid number. (Ticket #3102).

• Contrary to the intended behavior, jobs run via glexec did not get put on hold shortly before their proxy expired. (Ticket #3283).

• Starting in version 7.8.0, when using glexec, the job was put on hold shortly after the user's proxy was refreshed. The incorrect, but stated hold reason was, "Proxy about to expire." (Ticket #3280).

• The configuration variable DELEGATE_JOB_GSI_CREDENTIALS_REFRESH had no effect, as it was implemented using the incorrect variable name of DELEGATE_JOB_GSI_CREDENTIALS_RENEWAL. The implementation now uses the correct name. (Ticket #3282).

• When using privilege separation, jobs would be put on hold after they finished running if the working directory contained links to files that were not globally readable. (Ticket #2904).

• The configuration variable ENABLE_ADDRESS_REWRITING was not correctly applied to the condor_schedd address when claiming a slot. The incorrect behavior observed was always as though ENABLE_ADDRESS_REWRITING was False. This could result in communication errors for jobs running from multi-homed submit machines. (Ticket #3330).

• When using condor_defrag or condor_drain, a rare sequence of events could result in the condor_startd exiting with the following error message:

  ERROR "match_info() called with unexpected state (Drained)"
  

  (Ticket #3331).

• The condor_gridmanager no longer crashes when a CREAM grid job is submitted with an X.509 proxy that does not have VOMS attributes. (Ticket #3356).

• condor_chirp fetch could only transfer text files on Windows. It would truncate or corrupt binary files. (Ticket #3355).

• Fixed a bug in the condor_gridmanager that prevented the use of grid-type batch to submit jobs into an HTCondor pool. The condor_gridmanager would attempt to use the wrong GAHP server. (Ticket #3364).

• The default for the undocumented configuration variable X_RUNS_HERE was inverted from True to False starting with the release of version 7.7.3. Its default has been reset to True. When False, the condor_master will not start the condor_kbdd. (Ticket #3343).

• Fixed a bug in the init script for Red Hat derived Linux systems that prevented the Condor service from being stopped during system shutdown. (Ticket #3368).

• The condor_master would sometimes crash on reconfiguration when the High Availability configuration had changed. It no longer crashes. (Ticket #3292).

• Fixed a bug that caused the condor_starter to crash when condor_chirp is used and there is a configuration variable setting of USE_NFS = True or USE_AFS = True. This will happen with parallel universe jobs, because the MPI scripts invoke condor_chirp. (Ticket #3361).

Known Bugs:

• None.

Additions and Changes to the Manual:

• The manual incorrectly identified configuration variable COLLECTOR_PERSISTENT_AD_LOG as PERSISTENT_AD_LOG. This has now been corrected throughout the manual. (Ticket #3205).

Version 7.8.6

Release Notes:

• Condor version 7.8.6 released on October 25, 2012.

• Security Item: This version contains an important security bug fix. See below for details of this and other bugs fixed.

Bugs Fixed:

• Security Item: Fixed a bug which allowed jobs submitted to the standard universe to escalate privilege on the submit machine and execute code as root. (CVE-2012-5390)

Version 7.8.5

Release Notes:

• Condor version 7.8.5 released on October 22, 2012.

New Features:

• Condor now contains a tool called accountant_log_fixer, that can fix the damage to the file Accountantnew.log caused by a bug in the Condor version 7.8.4 condor_negotiator. (Ticket #3221).

Configuration Variable and ClassAd Attribute Additions and Changes:

• None.

Bugs Fixed:

• Fixed a problem with jobs that are submitted with

    noop_job = true
  

  These jobs would, in rare cases, cause the condor_schedd daemon to crash. (Ticket #3156).

• The condor_startd daemon crashed if a job failed to match a partitionable slot after the application of configuration variables of the MODIFY_REQUEST_EXPR_ category. (Ticket #3260).

• The condor_schedd daemon would mark scheduler universe jobs as completed and remove them from the job queue, even when they should have been requeued, according to policy. This caused condor_dagman jobs to fail to restart. This bug exists in all Condor versions 7.8.0 through 7.8.4. Upon upgrading from these Condor versions, users will need to intervene in order to restart their condor_dagman jobs. condor_dagman should not need such intervention when upgrading from Condor version 7.8.5. To restart a condor_dagman job, the simplest solution is to issue the command

    condor_submit <DAGFile>.condor.sub
  

  where the original DAG was submitted with

    condor_submit <DAGFile>
  

  (Ticket #3207).

• The Condor version 7.8.4 condor_negotiator daemon wrote corrupt resource entries to the file Accountantnew.log, which it would then not be able to read. The Condor version 7.9.0 condor_negotiator daemon will abort when trying to read these corrupted resource entries. The condor_negotiator will now correct these corrupt resource entries over time. (Ticket #3221).

• Fixed a bug in which the condor_schedd statistics ClassAd attributes JobsAccumExecuteTime and JobsAccumPostExecuteTime were sometimes much too large for jobs that had been vacated and then restarted. Note that these currently undocumented attributes would only appear in the ClassAd if the verbosity level for condor_schedd statistics was set at the high value of 2 by the configuration variable STATISTICS_TO_PUBLISH. (Ticket #3227).

Known Bugs:

• None.

Additions and Changes to the Manual:

• None.

Version 7.8.4

Release Notes:

• Condor version 7.8.4 released on September 19, 2012.

• This release contains several important security fixes and all users should upgrade as soon as possible.

New Features:

• None.

Configuration Variable and ClassAd Attribute Additions and Changes:

• The new configuration variable GSI_AUTHZ_CONF fixes a bug in which an instance of Condor may utilize the wrong Globus mapping. The configuration variable defines a path and file name to the file that contains the Globus mapping library. See the complete definition at  3.3.26. (Ticket #2103).

Bugs Fixed:

• Security Item: Some code that was no longer used was removed. The presence of this code could expose information which would allow an attacker to control another user's job. (CVE-2012-3493)

• Security Item: Some code that was no longer used was removed. The presence of this code could have lead to a Denial-of-Service attack which would allow an attacker to remove another user's idle job. (CVE-2012-3491)

• Security Item: Filesystem (FS) authentication was improved to check the UNIX permissions of the directory used for authentication. Without this, an attacker may have been able to impersonate another submitter on the same submit machine. (CVE-2012-3492)

• Security Item: Although not user-visible, there were multiple updates to remove places in the code where potential buffer overruns could occur, thus removing potential attacks. None were known to be exploitable.

• Security Item: Although not user-visible, there were updates to the code to improve error checking of system calls, removing some potential security threats. None were known to be exploitable.

• Fixed the condor_schedd daemon; it would crash when a submit description file contained a malformed $$() expansion macro that contained a period. (Ticket #3216).

• Fixed a case in which a daemon could crash and leave behind a log file owned by root. This root-owned file would then cause subsequent attempts to restart the daemon to fail. (Ticket #2894).

• Fixed a special case bug in which configuration variables defined utilizing initial substrings of $(DOLLAR), for example $(D) and $(DO), were not expanded properly. (Ticket #3217).

• The command condor_q -run now displays correct HOST field information for local universe jobs. (Ticket #3150).

Known Bugs:

• None.

Additions and Changes to the Manual:

• None.

Version 7.8.3

Release Notes:

• Condor version 7.8.3 released on September 6, 2012.

New Features:

• The libcondorapi library for reading and writing job event logs is again available as a shared library on Linux and Mac OS platforms. Since Condor 7.5.x, it had only been available as a static library. (Ticket #3047).

Configuration Variable and ClassAd Attribute Additions and Changes:

• To avoid the output of an unnecessary DAGMan error message, the value of DAGMAN_LOG_ON_NFS_IS_ERROR is ignored when both CREATE_LOCKS_ON_LOCAL_DISK and ENABLE_USERLOG_LOCKING are True. (Ticket #3087).

Bugs Fixed:

• Fixed a bug in which usage of cgroups incorrectly included the page cache in the maximum memory usage. This bug fix is also included in Condor version 7.9.0. (Ticket #3003).

• Jobs from a hook to fetch work, where the hook is defined by configuration variable <Keyword>_HOOK_FETCH_WORK, now correctly receive dynamic slots from a partitionable slot instead of claiming the entire partitionable slot. (Ticket #2819).

• Fixed a bug in which a slot might become stuck in the Preempting state when a condor_startd is configured with a hook to fetch work, as defined by <Keyword>_HOOK_FETCH_WORK . (Ticket #3076).

• Fixed a bug that caused Condor to transfer a job's input files from the execute machine back to the submit machine as if they were output files. This would happen if the job's input files were stored in Condor's spool directory; occurred if the job was submitted via Condor-C or via condor_submit with the -spool or -remote options. (Ticket #2406).

• Fixed a bug that could cause the first grid-type cream jobs destined for a particular CREAM server to never be submitted to that server. This bug was probably introduced in Condor version 7.6.5. (Ticket #3054).

• Fixed several problems with the XML parsing class ClassAdXMLParser in the ClassAds library:
  • Several methods named ParseClassAd() were declared, but never implemented. (Ticket #3049).
  • The parser silently dropped leading white space in string values. (Ticket #3042).
  • The parser could go into an infinite loop or leak memory when reading a malformed ClassAd XML document. (Ticket #3045).

• Fixed a bug that prevented the -f command line option to condor_history from being recognized. The -f option was being interpreted as -forward. At least four letters are now required for the -forward option (-forw) to prevent ambiguity. (Ticket #3044).

• The implementation of the condor_history -backwards option, which is the default ordering for reading the history file, in the 7.7 series did not work on Windows platforms. This has been fixed. (Ticket #3055).

• Fixed a bug that caused an invalid proxy to be delegated when refreshing the job's X.509 proxy when configuration variable DELEGATE_JOB_GSI_CREDENTIALS_LIFETIME was set to 0. (Ticket #3059).

• Fixed a bug in which DAGMan did not account properly for jobs being suspended and then unsuspended. (Ticket #3108).

• condor_dagman now takes note of job reconnect failed events (event code 24) in the user log, for counting idle jobs. (Ticket #3189).

• Job IDs generated by NorduGrid ARC 12.05 and above are now properly recognized. (Ticket #3062).

• Fixed a bug in which Condor would not mark grid-type nordugrid jobs as Running due to variation in the format of the job status value. NorduGrid ARC job statuses of the form INLRMS: ? are now properly recognized both with and without the space after the colon. (Ticket #3118).

• The condor_gridmanager now properly handles X.509 proxy files that are specified in the job ClassAd with a relative path name. (Ticket #3027).

• Fixed a bug that caused daemon names, as set in configuration variables such as STARTD_NAME, containing a period character to be ignored. (Ticket #3172).

• Fixed a bug that prevented the condor_schedd from removing old execute directories for local universe jobs on start up. (Ticket #3176).

• The condor_defrag daemon sometimes scheduled fewer draining attempts than specified. (Ticket #3199).

• Fixed a bug that could cause the condor_gridmanager to crash if a grid universe job's X.509 user certificate did not contain an e-mail address. (Ticket #3203).

• Fixed a bug introduced in Condor version 7.7.5 that caused multiple condor_schedd daemons running on the same machine to share the job queue with each other due to way in which the default value of configuration variable JOB_QUEUE_LOG was set. (Ticket #3196).

• Fixed a bug that could cause condor_q to not print all jobs when it thought it was querying an old condor_schedd daemon. (Ticket #3206).

• Fixed a bug that could cause a job's standard output and standard error files to be written in the job's initial working directory, despite the submit description file's specification to write them to a different directory. This would happen when the file transfer mechanism was used, the execution machine was running Condor version 7.7.1 or earlier, and either Condor's security negotiation was disabled or the configuration variable SEC_ENABLE_MATCH_PASSWORD_AUTHENTICATION was set to True. (Ticket #3208).

• The log message generated when the EXECUTE directory is missing is now more helpful. (Ticket #3194).

• The load average was incorrect for non-English versions on Windows platforms. This has been fixed for Windows Vista and more recent versions. (Ticket #3182).

Known Bugs:

• None.

Additions and Changes to the Manual:

• There is now documentation for the submit description file commands encrypt_input_files, encrypt_output_files, dont_encrypt_input_files, and dont_encrypt_output_files in the condor_submit manual page. These commands have been available since Condor version 6.7.2, but were never documented. See descriptions starting at  11. (Ticket #3174).

Version 7.8.2

Release Notes:

• Condor version 7.8.2 released on August 14, 2012.

• Security Item: Fixed a critical problem with DNS handling.

New Features:

• None.

Configuration Variable and ClassAd Attribute Additions and Changes:

• None.

Bugs Fixed:

• Security Item: Fixed a critical problem with DNS handling.

Known Bugs:

• None.

Additions and Changes to the Manual:

• None.

Version 7.8.1

Release Notes:

• Condor version 7.8.1 released on June 15, 2012.

New Features:

• None.

Configuration Variable and ClassAd Attribute Additions and Changes:

• (Added in 7.8.0.) The new configuration variable ENABLE_DEPRECATION_WARNINGS causes condor_submit to issue warnings when a job requests features that are no longer supported. (Ticket #2968).

• (Added in 7.7.6) The new configuration variable BATCH_GAHP should be used instead of PBS_GAHP , LSF_GAHP and SGE_GAHP . These older configuration variables are still recognized, but their use is now discouraged. (Ticket #2670).

• The default value for GROUP_SORT_EXPR was changed so that the <none> group would always negotiate last when using hierarchical group quotas. Associated with that, the default value for NEGOTIATOR_ALLOW_QUOTA_OVERSUBSCRIPTION was changed to True. These changes were made to make negotiation behave more like it did in the stable 7.4 series of Condor, before hierarchical group quotas were added. (Ticket #3040).

Bugs Fixed:

• Fixed a bug that caused events to not be written to the job event log when the log is written in XML and a job policy expression triggering the event contains any double quote marks. (Ticket #3048).

• Fixed a bug in the Condor init script that would cause the init script to hang if Condor was not running. (Ticket #2872).

• Fixed a bug that caused parallel universe jobs using Parallel Scheduling Groups (see section 3.12.8) to occasionally stay idle even when there were available machines to run them. (Ticket #3017).

• Fixed a bug that caused the condor_gridmanager to crash when attempting to submit jobs to a local PBS, LSF, or SGI cluster. (Ticket #3014).

• Fixed a bug in the handling of local universe jobs which caused the condor_schedd to log a spurious ERROR message every time a local universe job exited, and then further caused the statistics for local universe jobs to be incorrectly computed. (Ticket #3008).

• Changed the internally used condor_ckpt_probe executable to link statically, which should make the checkpoint signature more resistant to non-significant changes in the system configuration. (Ticket #2901).

• Restored Globus and VOMS support for the Mac OS X platform. (Ticket #2991).

• Fixed a bug when Condor runs under the PrivSep model, in which if a job created a hard link from one file to another, Condor was unable to transfer the files back to the submit side, and the job was put on hold. (Ticket #2987).

• When configuration variables MaxJobRetirementTime or MachineMaxVacateTime were very large, estimates of machine draining badput and completion time were sometimes nonsensical because of integer overflow. (Ticket #3001).

• Fixed a bug where per-job subdirectories and their contents in $(SPOOL) would not be removed when the associated job left the queue. (Ticket #2942).

• Fixed a bug that could cause the condor_schedd to occasionally crash due to a race condition when running local universe jobs. Associated with the bug would be the error message

  No local universe jobs were expected to be running, but one just exited!
  

  (Ticket #3009).

Known Bugs:

• None.

Additions and Changes to the Manual:

• Submit description file commands introduced in Condor version 7.7.1 have now been documented. See the condor_submit manual page at 11 for the newly added definitions of

  ec2_availability_zone

  ec2_ebs_volumes

  ec2_elastic_ip

  ec2_keypair_file

  ec2_vpc_ip

  ec2_vpc_subnet

• There is now a manual page for condor_router_rm, a script that provides additional features convenient for removing jobs managed by the Condor Job Router.

• Documentation not completed for the 7.7.6 release is now available. The use of configuration variable BATCH_GAHP, as well as the use of the new grid_resource of type batch for local submission of PBS, LSF, and SGE jobs is documented. See section 5.3.5 for details. (Ticket #2670).

Version 7.8.0

Release Notes:

• Condor version 7.8.0 released on May 10, 2012.

New Features:

• (Added in 7.7.6.) The new -_condor_relocatable argument may be given as part of the invocation of a program that uses standalone checkpointing. This allows checkpointed programs to restart without attempting to change to their original directory. (Ticket #2877).

• (Added in 7.7.5.) Added the -absent flag to condor_status, which displays absent ClassAds. (Ticket #2690).

• (Added in 7.7.5.) Implement absent ads, which help track pool membership in a persistent way. (Ticket #2608).

Configuration Variable and ClassAd Attribute Additions and Changes:

• The job ClassAd attribute RemotePool is now saved in LastRemotePool when the job finishes running.

Bugs Fixed:

• (Fixed in 7.7.6.) Fix -absent, -vm, and -java flags to condor_status so that they work with the -long option. (Ticket #2943).

• Support glob() on Scientific Linux 6 and others using the new Linux system call fstatat(), but only when not using remote system calls. (Ticket #2945).

• Fixed potential startd crash introduced in v7.7.5 when claiming a partitionable slot that was in the Owner state. (Ticket #2936).

• When ClassAd function stringListMember() is called with an empty string as the second argument, it now evaluates to False. Previously, it incorrectly evaluated to Undefined. (Ticket #2953).

• Format tags %v and %V for the -format option now properly print all ClassAd value types. Previously, True and False were printed as integers, and new ClassAd types like lists and nested ClassAds could not be printed. (Ticket #2960).

• Restored RCS keyword strings CondorVersion and CondorPlatform to the Condor binaries. These strings are found and printed by the ident program on Unix. They were missing in Condor versions 7.7.3 and later. (Ticket #2932).

• condor_job_router failed to route spooled source jobs. (Ticket #2955).

• Fixed a bug on Debian 6 and RHEL 6 that could cause standard universe jobs to never checkpoint. This would happen if the job triggered a call to NSCD (Name Service Caching Daemon) but NSCD wasn't running. Calls to NSCD can be triggered by a look up of a user account or resolving a machine hostname to an IP address. Now, NSCD is never consulted by a standard universe job (this was already the behavior on other platforms). (Ticket #2973).

Known Bugs:

• None.

Additions and Changes to the Manual:

• None.