HTCondor
™
Version 8.7.10 Manual
Center for High Throughput Computing, University of Wisconsin–Madison
October 31, 2018
Contents
1
Overview
1.1
High-Throughput Computing (HTC) and its Requirements
1.2
HTCondor’s Power
1.3
Exceptional Features
1.4
Current Limitations
1.5
Availability
1.6
Contributions and Acknowledgments
1.7
Contact Information
1.8
Privacy Notice
2
Users’ Manual
2.1
Welcome to HTCondor
2.2
Introduction
2.3
Matchmaking with ClassAds
2.4
Running a Job: the Steps To Take
2.5
Submitting a Job
2.6
Managing a Job
2.7
Priorities and Preemption
2.8
Java Applications
2.9
Parallel Applications (Including MPI Applications)
2.10
DAGMan Applications
2.11
Virtual Machine Applications
2.12
Docker Universe Applications
2.13
Time Scheduling for Job Execution
2.14
Special Environment Considerations
2.15
Potential Problems
3
Administrators’ Manual
3.1
Introduction
3.2
Installation, Start Up, Shut Down, and Reconfiguration
3.3
Introduction to Configuration
3.4
Configuration Templates
3.5
Configuration Macros
3.6
User Priorities and Negotiation
3.7
Policy Configuration for Execute Hosts and for Submit Hosts
3.8
Security
3.9
Networking (includes sections on Port Usage and CCB)
3.10
The Checkpoint Server
3.11
DaemonCore
3.12
Monitoring
3.13
The High Availability of Daemons
3.14
Setting Up for Special Environments
3.15
Java Support Installation
3.16
Setting Up the VM and Docker Universes
3.17
Singularity Support
3.18
Power Management
4
Miscellaneous Concepts
4.1
HTCondor’s ClassAd Mechanism
4.2
HTCondor’s Checkpoint Mechanism
4.3
Computing On Demand (COD)
4.4
Hooks
4.5
Logging in HTCondor
5
Grid Computing
5.1
Introduction
5.2
Connecting HTCondor Pools with Flocking
5.3
The Grid Universe
5.4
The HTCondor Job Router
6
Cloud Computing
6.1
Introduction
6.2
HTCondor Annex User’s Guide
6.3
Using
condor_annex
for the First Time
6.4
HTCondor Annex Customization Guide
6.5
HTCondor Annex Configuration
7
Application Programming Interfaces (APIs)
7.1
Python Bindings
7.2
Chirp
7.3
The HTCondor User and Job Log Reader API
7.4
The Command Line Interface
7.5
The DRMAA API
8
Platform-Specific Information
8.1
Linux
8.2
Microsoft Windows
8.3
Macintosh OS X
9
Frequently Asked Questions (FAQ)
10
Contrib and Source Modules
10.1
Introduction
10.2
Using HTCondor with the Hadoop File System
10.3
The HTCondorView Client Contrib Module
10.4
Job Monitor/Log Viewer
11
Version History and Release Notes
11.1
Introduction to HTCondor Versions
11.2
Development Release Series 8.7
11.3
Upgrading from the 8.4 series to the 8.6 series of HTCondor
11.4
Stable Release Series 8.6
11.5
Development Release Series 8.5
11.6
Stable Release Series 8.4
12
Command Reference Manual (man pages)
bosco_cluster
bosco_findplatform
bosco_install
bosco_ssh_start
bosco_start
bosco_stop
bosco_uninstall
condor_advertise
condor_annex
condor_check_userlogs
condor_checkpoint
condor_chirp
condor_cod
condor_compile
condor_config_val
condor_configure
condor_continue
condor_convert_history
condor_dagman
condor_dagman_metrics_reporter
condor_drain
condor_fetchlog
condor_findhost
condor_gather_info
condor_gpu_discovery
condor_history
condor_hold
condor_install
condor_job_router_info
condor_master
condor_now
condor_off
condor_on
condor_ping
condor_pool_job_report
condor_power
condor_preen
condor_prio
condor_procd
condor_q
condor_qedit
condor_qsub
condor_reconfig
condor_release
condor_reschedule
condor_restart
condor_rm
condor_rmdir
condor_router_history
condor_router_q
condor_router_rm
condor_run
condor_set_shutdown
condor_ssh_to_job
condor_sos
condor_stats
condor_status
condor_store_cred
condor_submit
condor_submit_dag
condor_suspend
condor_tail
condor_top
condor_transfer_data
condor_transform_ads
condor_update_machine_ad
condor_updates_stats
condor_urlfetch
condor_userlog
condor_userprio
condor_vacate
condor_vacate_job
condor_version
condor_wait
condor_who
gidd_alloc
procd_ctl
A
ClassAd Attributes
A.1
ClassAd Types
A.2
Job ClassAd Attributes
A.3
Machine ClassAd Attributes
A.4
DaemonMaster ClassAd Attributes
A.5
Scheduler ClassAd Attributes
A.6
Negotiator ClassAd Attributes
A.7
Submitter ClassAd Attributes
A.8
Defrag ClassAd Attributes
A.9
Collector ClassAd Attributes
A.10
ClassAd Attributes Added by the
condor_collector
A.11
DaemonCore Statistics Attributes
B
Codes and Other Needed Values
B.1
condor_shadow
Exit Codes
B.2
Job Event Log Codes
B.3
Well-known Port Numbers
B.4
DaemonCore Command Numbers
B.5
DaemonCore Daemon Exit Codes
Index
[
⇒
]