This is an outdated version of the HTCondor Manual. You can find current documentation at http://htcondor.org/manual.

10.5 Stable Release Series 8.0

This is a stable release series of HTCondor. As usual, only bug fixes (and potentially, ports to new platforms) will be provided in future 8.0.x releases. New features will be added in the 8.1.x development series.

The details of each version are described below.

Version 8.0.7

Release Notes:

• HTCondor version 8.0.7 released on June 5, 2014.

New Features:

• The new configuration variable ALWAYS_USE_LOCAL_CKPT_SERVER forces all standard universe job checkpoints to be read from a checkpoint server running on the same machine where the job runs. This is only useful when all the checkpoint servers are using a common shared file system. (Ticket #4265).

• The new configuration variable JOB_EXECDIR_PERMISSIONS defines the permissions on a job's scratch directory. It defaults to setting permissions as 0700. (Ticket #4208).

• An experimental new feature allows the specification of icc on the condor_compile command line, instead of cc or gcc, to build standard universe programs with the Intel C compiler. (Ticket #4234).

Bugs Fixed:

• Fixed a bug which caused the memory used by the condor_procd grow arbitrarily in size when cgroup-based tracking was in effect. (Ticket #4343).

• Fixed a bug in which any job running on an execute node with a process name that had a space character caused the condor_procd to log a short read error repeatedly to the ProcLog. (Ticket #4235).

• Fixed a bug which would cause the condor_schedd to repeatedly crash. This occurred when parallel universe jobs were running as the condor_schedd was restarted, and if the condor_startd daemons in use by running jobs were also shut down while the condor_schedd was not running. (Ticket #4305).

• Fixed the parsing of the condor_q command line, such that it looks for the entire option -stream-results, instead of only the portion -stream. The code now matches what was previously documented in the manual page and in the usage output. (Ticket #4205).

• Fixed a bug in the grid universe that could allow a removed job to leave the queue without cleaning up all job state on the remote server. (Ticket #4216).

• Fixed the printing of nested ClassAds, so that the nested ClassAds can be read back properly. (Ticket #3772).

• Fixed a bug in the condor_starter that could cause a crash if the job is evicted while a condor_ssh_to_job command is active. (Ticket #4251).

• HTCondor executables are now linked on RHEL 5 platforms using both GNU and System V hash styles to support older Linux distributions, such as SUSE Enterprise Linux 10. (Ticket #4278).

• Fixed a bug in which condor_dagman did not correctly update the node status file at the end of the run for any DAGs containing a FINAL node. (Ticket #4209).

• Fixed a bug in which condor_dagman sometimes printed incorrect node status counts to the node status file. Note that this fix also included changing one of the strings used as a node status from "STATUS_UNREADY" to "STATUS_NOT_READY". (Ticket #4248).

• Fixed a bug that caused the nordugrid_gahp to crash if the NorduGrid ARC server returned an unexpected response during job submission. (Ticket #4318).

• Fixed a bug that caused the condor_startd to crash when NUM_SLOTS was set to 0 and a condor_reconfig command was issued. (Ticket #4327).

• Under limited and unusual error conditions, the condor_schedd and the condor_shadow could treat a failed file transfer as if it had succeeded. As a result, the condor_schedd could try to run a job whose input files were not available, or a job could leave the queue as completed, despite its output files not being transferred back from the executing machine. (Ticket #4328).

• The condor_shadow no longer performs a DNS query while holding the lock for the job event logs. (Ticket #4377).

Version 8.0.6

Release Notes:

• HTCondor version 8.0.6 released on February 11, 2014.

New Features:

• A full port, which includes support for the standard universe, is available for the Red Hat Enterprise Linux 7.0 Beta release on the x86_64 architecture. (Ticket #3629).

• HTCondor now forces proxies that it delegates to be a minimum of 1024 bits. (Ticket #4168).

Bugs Fixed:

• Fixed a rare bug in the condor_negotiator where completely disabled preemption achieved with PREEMPTION_REQUIREMENTS = False might still cause a newly started job to be preempted. This bug was much more likely when configuration variable NEGOTIATOR_CYCLE_DELAY was set lower than the default value. (Ticket #4185).

• Fixed a bug in the condor_schedd which would cause it to crash when running remotely submitted parallel universe jobs. (Ticket #4163).

• Fixed a bug with concurrency limits and partitionable slots, in which the partitionable slot would consume a concurrency token, even though the slot never runs jobs. This would cause fewer jobs than desired to run. (Ticket #4145).

• Fixed a bug in which the Job Exit work fetch hook was prematurely killed. (Ticket #3669).

• When the Windows installer was told to use VMware, it configured a requirement for a condor_vmware_local_settings file that it did not provide. This caused the condor_vm-gahp to fail to start, unless the user created this file. This configuration has been removed, so that the file is no longer required. (Ticket #4109).

• Fixed a crash of the condor_shadow, triggered when a disconnect from the condor_starter occurs just as the job terminates. (Ticket #4127).

• Modified the output of condor_q, such that when invoked with options -userlog and -af, a blank line and a totals summary line are not displayed. (Ticket #4045).

• For hierarchical group quotas, fixed an incorrect count of the number of jobs when applying a quota. (Ticket #4117).

• Fixed the HTCondor module send_command() python binding; an incorrect argument was only the first character of the daemon's name, instead of the full name. This affected the ability to turn off specific daemons, as sent to the condor_master. (Ticket #4160).

• Fixed the transfer of files larger than 4Gbytes, such that they no longer stop before the transfer is completed. This bug presented itself on all Windows systems and on all platforms running on a 32-bit architecture. (Ticket #4150).

• Fixed a bug that caused condor_submit_dag to crash on very large DAG input files, such as those larger than 2 Gbytes. The new configuration variable DAGMAN_USE_OLD_DAG_READER, as detailed in section 3.3.25, allows disabling the new file reader code put in place to fix the bug. (Ticket #4171).

• Fixed a bug that caused attempts to set the CPU affinity on Windows platforms to be quietly ignored. (Ticket #4131).

Version 8.0.5

Release Notes:

• HTCondor version 8.0.5 released on December 12, 2013.

• Helper script condor_ckpt_probe has been missing from HTCondor releases since version 7.8.1, and it is once again in this release. As a result, the machine ClassAd attribute CheckpointPlatform will change for standard universe jobs upon upgrade to HTCondor version 8.0.5. This will prevent standard universe jobs started before the upgrade from continuing, because the attribute change eliminates a match with upgraded machines. To work around this issue, change the LastCheckpointPlatform attribute to be current, such that all jobs that have produced a checkpoint will qualify to continue on the upgraded machines. Make the change by using condor_qedit. For example, use

  condor_qedit -constraint 'LastCheckpointPlatform == "LINUX INTEL 2.6.x normal N/A"'
      LastCheckpointPlatform "LINUX INTEL 2.6.x normal N/A ssse3 sse4_1 sse4_2"
  

  where CheckpointPlatform after upgrade shows as LINUX INTEL 2.6.x normal N/A ssse3 sse4_1 sse4_2. (Ticket #4025).

New Features:

• None.

Bugs Fixed:

• Fixed a bug that resulted in incorrect values for ClassAd attributes RemoteGroupQuota and SubmitterGroupQuota. These attributes are commonly used in an expression that defines PREEMPTION_REQUIREMENTS. (Ticket #4093).

• Fixed a permissions bug that prevented Java universe jobs from running. (Ticket #4087).

• Fixed a bug in which job output files may not have been properly truncated at start up on Windows platforms. (Ticket #4097).

• Fixed bug with condor_submit when invoked with the -interactive option and cgroups where enabled. The shell was terminated immediately after it started. (Ticket #4028).

• Prevent illegal values from being written into the job queue or accounting recovery logs, in order to minimize the chance of errors when the condor_schedd or condor_negotiator are restarted. Also have condor_qedit validate attribute names and values. (Ticket #3616).

• The condor_starter now writes to the log file StarterLog.slotX when running work fetch jobs, mirroring the behavior when running jobs that come from a condor_schedd. Previously, log file StarterLog was used for all work fetch jobs. (Ticket #3091).

• Fixed a Python binding crash due to a segmentation fault, when evaluating an expression tree with an undefined reference. (Ticket #3910).

• Fixed the cURL file transfer plugin such that it now works on Windows platforms. (Ticket #3979). (Ticket #4012).

• Fixed a bug that caused cream grid universe jobs to fail, if the submit description file contained submit commands of environment or cream_attributes. (Ticket #4037).

• Fixed a bug that could cause the condor_schedd to crash when starting a local universe job. (Ticket #4088).

• Fixed a bug that caused stdout and stderr of nordugrid grid universe jobs to be lost when the remote NorduGrid ARC server was using HTCondor as its local batch scheduler. (Ticket #4017).

• condor_status with the -t option now consistently specifies the -total option. The -target option will now be distinguished, as it requires at least -targ in its specification. (Ticket #4096).

• Restored the omitted HTCondor Perl module. (Ticket #4098).

• For RPM installations, the post-install script now checks to make sure that the SELinux unconfined_execmem_exec_t type exists before trying to add it to the /usr/sbin/condor_startd file context. (Ticket #4034).

• Fixed a bug in which both daemons and tools may have crashed due to the inability to resolve a host name defined by configuration variable COLLECTOR_HOST to an IP address. (Ticket #3946).

• Changed the algorithm for calculating the value of attribute RecentDaemonCoreDutyCycle such that the published value will never be negative. (Ticket #4052).

• For VMware vm universe jobs, the condor_vm-gahp removed from the vm description file any cdrom, floppy drive, serial or parallel devices. Now, only devices that do not refer to image files are removed, as the devices may be useful to the virtual machine. (Ticket #4002).

Version 8.0.4

Release Notes:

• HTCondor version 8.0.4 released on October 24, 2013.

• A clipped version of HTCondor is now provided for Ubuntu 12.04 on the x86_64 architecture. (Ticket #3972).

New Features:

• None.

Configuration Variable and ClassAd Attribute Additions and Changes:

• The new configuration variable DYNAMIC_RUN_ACCOUNT_LOCAL_GROUP permits an administrator of a Windows machine to specify a local group other than the default Users for the condor-reuse-slot<X> run account. (Ticket #3998).

Bugs Fixed:

• Added a retry to work around problems with slow DHCP servers. The result of the problem was that daemons would not be able to determine their own host names, and, among other problems, machine ClassAds would appear in the output of condor_status with blank host names. (Ticket #3956).

• A spurious warning in the log of the condor_startd about undefined PREEMPT_VANILLA expressions is no longer generated. (Ticket #3984).

• EC2 grid universe jobs may provide data in files. File contents of null-terminated strings worked correctly, but binary data did not. This bug fix makes sure that both kinds of data are read correctly. (Ticket #3924).

• EC2 grid universe jobs on OpenNebula work better for user data specified with ec2_user_data, where the size of the data is larger than 2Kbytes. (Ticket #3923).

• Fixed a bug preventing HTTP file transfers from following redirects. HTTP file transfers also now fail if the file does not exist, but the server does exist. (Ticket #3904).

• Fixed a rare bug in which the condor_schedd would exit with an ERROR message pertaining to select(), when running with a very heavy load and many condor_shadow daemons time out. (Ticket #3947).

• Fixed a bug in condor_ssh_to_job that caused it to kill the job when the ssh session exited, if cgroups were enabled. (Ticket #3921).

• If a standard universe job exited without producing any checkpoints and no checkpoint server was used, two spurious error messages would be logged to the SchedLog, as it tried to remove the old checkpoint images from the non-existent checkpoint server. These error messages are no longer logged. (Ticket #3919).

• The configuration file for Windows erroneously had two entries for configuration variable JAVA_CLASSPATH_SEPARATOR, with the second entry specifying the separator used on Unix platforms, which overrode the first entry. The Unix separator no longer affects this configuration file for Windows platforms. (Ticket #3957).

• The NorduGrid GAHP no longer consumes all of the CPU when run with threaded Globus libraries. (Ticket #3958).

• Fixed a bug that could cause ClassAd function quantize() to not evaluate properly when its second argument was a list and ClassAd caching was enabled. (Ticket #3967).

• Fixed a bug that caused the configuration variable setting STARTD_CRON_AUTOPUBLISH = If_Changed to not work correctly. Updates were incorrectly sent to the condor_collector in many cases when no attribute value had changed. (Ticket #3983).

• Fixed a bug that could cause condor_q -analyze or -better-analyze to sometimes crash on an Illegal Instruction, when the Requirements expression contained a function. (Ticket #3985).

• The configuration variables UPDATE_COLLECTOR_WITH_TCP and TCP_UPDATE_COLLECTORS are now respected when forwarding ClassAds to an HTCondorView server and setting CONDOR_VIEW_HOST. (Ticket #3986).

• condor_who now prints an error message when passed an invalid argument. (Ticket #3987).

• Changed the script that finds the VMware tools to use the standard environment variables to find Program Files, instead of using a hard coded path. This fixes installation for both 64-bit and non-English language Windows platforms. (Ticket #272).

Known Bugs:

• None.

Additions and Changes to the Manual:

• None.

Version 8.0.3

Release Notes:

• HTCondor version 8.0.3 released on September 23, 2013.

New Features:

• None.

Configuration Variable and ClassAd Attribute Additions and Changes:

• None.

Bugs Fixed:

• When expressions for SUSPEND, CONTINUE, PREEMPT, and KILL were evaluated by the condor_startd, a resulting value of UNDEFINED or ERROR caused an exception in the condor_startd. These values are now treated as False, eliminating the exception. This fix addresses CVE-2013-4255. (Ticket #3869).

• Fixed a bug that prevented the use of simple host names to identify machines when using the command-line tools. Before the fix, condor_status bar.foo.org would show machine ClassAds, but condor_status bar would not. Both now show machine ClassAds. (Ticket #3694).

• Fixed a bug with condor_ssh_to_job that occurs when USE_PID_NAMESPACES is enabled. The bug presented itself as condor_ssh_to_job running in a private pid namespace, and the user running condor_ssh_to_job not being able to see their job with ps or gdb. As a fix, condor_ssh_to_job now runs in the global namespace, so that it can see the processes in the user's job. (Ticket #3872).

• Fixed a performance problem with the condor_qedit command that would cause the condor_schedd to run very slowly when condor_qedit is run on a large number of jobs. condor_qedit no longer writes an event to the job event log. (Ticket #3827).

• Fixed a problem where the classad python module would return incorrect results when ClassAd caching is enabled. (Ticket #3879).

• DAGMan's updating of its job ClassAd with DAG status attributes no longer causes extra events to be written to the job event log and event log. (Ticket #3863).

• Fixed a bug that caused the command

    condor_submit -
  

  to be interpreted as an interactive submit, rather than a request to read input from stdin. condor_qsub was also modified to be immune to this bug, such that it will still work with other versions of HTCondor containing the bug. (Ticket #3902).

• Value 032 of the job ClassAd attribute HoldReasonCode was being used for two different reasons. Now, value 032 identifies that the maximum total input file transfer size was exceeded. Value 034 identifies that memory usage exceeds a memory limit. (Ticket #3858).

• The values of the job ClassAd attributes RemoteSysCpu and RemoteUserCpu are sometimes impossibly large. This bug rarely occurs and is not well understood. Code changes attempt to fix this problem. (Ticket #3814).

• Fixed a bug that caused DAG recovery mode to fail on Pegasus-generated sub-DAGs. (Recovery mode is invoked, for example, when a condor_dagman is held and released, or when a schedd is restarted, as happens on a version upgrade.) (Ticket #3882).

Known Bugs:

• None.

Additions and Changes to the Manual:

• None.

Version 8.0.2

Release Notes:

• HTCondor version 8.0.2 released on August 22, 2013.

• Debian 5 is past its end of life. Starting with this release, we no longer provide native packages or tarballs for Debian 5. (Ticket #3852).

New Features:

• None.

Configuration Variable and ClassAd Attribute Additions and Changes:

• The default value of ENABLE_DEPRECATION_WARNINGS has been changed to False. (Ticket #3848).

Bugs Fixed:

• Implemented a workaround to avoid triggering a Linux kernel defect when using cgroups and suspending jobs. (Ticket #3847).

• Fixed a python bindings problem of missing converters by providing pyclassad as a shared library. (Ticket #3780).

• Fixed a file permission bug introduced in HTCondor version 7.9.2 that prevented vm universe jobs from working when using the Xen or KVM hypervisor. (Ticket #3781).

• Fixed a bug that could cause the condor_collector to become unresponsive if the remote HTCondorView server, specified with configuration variable CONDOR_VIEW_HOST, becomes unavailable. (Ticket #3758).

• Code to publish Linux distribution attributes in the machine ClassAd is now more robust in the event that the /etc/issue file was edited. (Ticket #3854).

• Fixed a bug that could cause jobs to be incorrectly placed on hold upon completion with a hold reason claiming an out-of-memory event. (Ticket #3824).

• Fixed a bug that prevented work fetch scripts from running on systems where cgroup based tracking and management was enabled. (Ticket #3819).

• Fixed a bug that could cause the condor_negotiator to give out the same slot twice, and result in a scary entry in the NegotiatorLog file with the wording:

    INSANE: bestCached != bestSoFar
  

  (Ticket #2245).

• Fixed a bug introduced in HTCondor version 7.9.3, in which concurrency limits were not respected across negotiation cycles when NEGOTIATOR_CONSIDER_PREEMPTION was False. (Ticket #3815).

• Fixed a bug from HTCondor version 7.9.6. The bug exhibited itself when using CCB to connect to the condor_startd; the condor_negotiator and condor_schedd would sometimes crash and then be restarted with the following error message in the log:

  ERROR "Selector::add_fd(): fd -1 outside valid range 0-1023"
  

  A workaround for the problem is relevant to HTCondor versions 7.9.6 through 8.0.1. Configure

    SERVICE_COMMAND_SOCKET_MAX_SOCKET_INDEX = -1
  

  (Ticket #3801).

• Fixed a bug in the condor_qsub script that caused it to exit with a syntax error when a job with a memory requirement was submitted. (Ticket #3808).

• Fix a bug causing security groups for EC2 jobs to be ignored. Also, the code respects the use of commas, as documented, to separate the items in the list of security groups specified by the submit description file command ec2_security_groups. (Ticket #3787).

• When invoking glexec, environment variable GLEXEC_TARGET_PROXY is now set to /dev/null. In some situations, it was previously set to a nonexistent path, which caused errors in some configurations. (Ticket #3794).

• HTCondor daemons are now less vulnerable to long connection delays when attempting to connect to hosts that are off-line. A specific case where this helps is when condor_schedd is using a high availability configuration, and the primary machine running the condor_collector is off-line. (Ticket #3828).

• Fixed a bug that could cause condor_dagman to hang due to a rarely seen event ordering. This bug could have been triggered when using the configuration variable DAGMAN_MAX_JOBS_IDLE, or its equivalent command line option -maxidle. (Ticket #3834).

• Fixed a bug that caused job submission from Windows platforms using condor_submit with the -spool option to always fail. (Ticket #3791).

Known Bugs:

• DAGMan recovery mode does not work for Pegasus-generated SUBDAGs. For SUBDAGs, doing condor_hold or condor_release on the condor_dagman job, or stopping and re-starting the condor_schedd with the DAGMan job in the queue will result in failure of the DAG. This can be avoided by doing a condor_rm of the DAGMan job, which produces a Rescue DAG, and re-submitting the DAG; the Rescue DAG is automatically run. This bug was introduced in HTCondor version 8.0.1. (Ticket #3882).

Additions and Changes to the Manual:

• None.

Version 8.0.1

Release Notes:

• HTCondor version 8.0.1 released on July 17, 2013.

New Features:

• HTCondor now provides the Debian Linux 7.0 (wheezy) platform, including support for the standard universe. (Ticket #3665).

Configuration Variable and ClassAd Attribute Additions and Changes:

• None.

Bugs Fixed:

• Fixed a bug that prevented per-slot settings of the STARTD_ATTRS configuration variable from being set correctly for partitionable slots named with a SLOTX_ prefix. (Ticket #3726).

• Fixed a bug that caused condor_status -submitters to report twice as many jobs running as were actually running. This bug appeared in HTCondor versions 7.9.6 and 8.0.0. (Ticket #3713).

• Fixed a bug with hierarchical group quotas in the condor_negotiator in which group hierarchies with parent groups that set configuration variable GROUP_ACCEPT_SURPLUS to False would be assigned allocations above their quota. (Ticket #3695).

• Fixed a bug in which scheduler universe jobs that have an on_exit_hold expression that evaluates to True could have duplicate hold messages in the user log. (Ticket #3651).

• Fixed a bug in which condor_dagman would submit multiple copies of the same job, fail, write a Rescue DAG, and leave the jobs in the queue. This was due to a warning from condor_submit that the submit description file was not using lines containing the string "cluster". As a fix, condor_dagman will search for the string " submitted to cluster ". This will generate fewer false alarms. If the submission succeeds and condor_dagman gets confused, the jobs will be removed when condor_dagman writes a Rescue DAG. (Ticket #3658).

• Added libdate-manip-perl as a dependency to the Debian packages. It is required in order to run the condor_gather_info script. (Ticket #3692).

• Configuration variable CCB_ADDRESS did not correctly support a list of CCB servers. Only the first one in the list was used. (Ticket #3699).

• Fixed a bug that caused some communication layer log messages to end with binary characters. (Ticket #3706).

• Fixed a bug that can cause the condor_procd to erroneously exit on Mac OS X when many processes are created in a short period of time. (Ticket #3725).

• Removed a bug that caused condor_dagman to have problems restarting after an upgrade from HTCondor version 7.8. (Ticket #3707).

• Fixed a bug that caused the command

    condor_q -dag -run
  

  to print garbage. (Ticket #3578).

• Fixed a bug that prevented jobs with an invalid ec2_keypair from being removed. (Ticket #3485).

• Fixed a memory leak and potential crash in the condor_gridmanager when requests to an EC2 service fail. (Ticket #3701).

• Fixed a bug in the condor_gridmanager that can cause EC2 jobs to be submitted a second time during recovery. (Ticket #3705).

• Fixed a memory leak in the condor_gridmanager that was triggered when submitting EC2 grid universe jobs. (Ticket #3720).

Known Bugs:

• None.

Additions and Changes to the Manual:

• None.

Version 8.0.0

Release Notes:

• HTCondor version 8.0.0 released on June 6, 2013.

New Features:

• The condor_chirp write command now accepts an optional numbytes parameter following the local file name. This allows the write to be limited to the specified number of bytes. (Ticket #3548).

• The HTCondor Python bindings now build on Mac OS X. (Ticket #3584).

• Updated the sample condor.plist file to work better with current versions of Mac OS X. (Ticket #3624).

Configuration Variable and ClassAd Attribute Additions and Changes:

• The new configuration variable DEDICATED_SCHEDULER_WAIT_FOR_SPOOLER permits the specification of a very strict execution order for parallel universe jobs handed to a remote scheduler. (Ticket #2946).

Bugs Fixed:

• Fixed a bug that happened with partitionable slots, jobs that requested more than one cpu, and a negotiator with NEGOTIATOR_CONSIDER_PREEMPTION was false. This would cause the negotiator to incorrectly assume that each slot had a slot weight of one. (Ticket #3737).

• The redundant configuration variable CheckpointPlatform has been removed and the configuration variable CHECKPOINT_PLATFORM documented. (Ticket #3544).

• A standard universe job will no longer crash, and it will no longer cause the condor_shadow to crash if the job calls mmap() with an unsupported combination of flags. (Ticket #3565).

• Support for VMware Workstation and VMware Player under the vm universe now works properly on Windows platforms. (Ticket #3627).

• For grid universe jobs intended for an EC2 grid resource, errors which have no response body now report the HTTP code. (Ticket #3541).

• condor_chirp put would experience an assertion failure when used on an empty file. This bug has been fixed, and put can now be used on an empty file. (Ticket #3542).

• The 32-bit condor_starter could fail to execute jobs when the initial working directory of the job was on a subsystem containing 64-bit metadata, such as inode numbers. (Ticket #3605).

• condor_dagman failed to react correctly if a nested DAG file did not exist. It now reacts correctly and prints a more helpful error message. (Ticket #3623).

• Fixed a bug that caused the condor_master daemon on Windows platforms to think there were new binaries when changing to and from daylight savings time. The condor_master daemon would then kill and restart itself, as well as all of the daemons, if configuration variable MASTER_NEW_BINARY_RESTART was set to its default value of GRACEFUL. (Ticket #3572).

• Fixed a bug that caused redundant lines to show up in the user log at the end of the partitionable resource usage summary. (Ticket #3621).

• Fixed several bugs that can cause the condor_procd to fail on Mac OS X and not be restartable. (Ticket #3617). (Ticket #3618). (Ticket #3620).

• The condor_procd now ignores process id 0 on Mac OS X. (Ticket #3516).

• Fixed memory leaks in the condor_shadow and the condor_startd; fixed a file descriptor leak in the standard universe condor_starter. (Ticket #3590).

• Fixed a bug in which condor_dagman would miscount the number of held jobs when multiple copies of hold events were written to the user log. (Ticket #3650).

Known Bugs:

• The following obsolete binaries have not yet been removed from the HTCondor tarballs:
  • classad_functional_tester
  • classad_version
  • condor_test_match
  • condor_userlog_job_counter
  (Ticket #3670).

• condor_status -submitters reports twice as many jobs running as were actually running. (Ticket #3713).

Additions and Changes to the Manual:

• Fixed the condor_configure man page and added a corresponding condor_install man page. (Ticket #3619).

• Added stub man pages for the Bosco commands. (Ticket #3634).