HTCondor Week 2016

University of Wisconsin — Madison, Wisconsin — May 17–20, 2016

picture of Madison

Wednesday, May 18

 8:00 am  9:00 am Coffee and Registration
coffee, tea, ice water
Session Moderators: Greg Thain, Erin Grasmick
 9:00 am  9:05 am Welcome
Miron Livny
Center for High Throughput Computing
 9:05 am  9:20 am What's New in HTCondor? What's Coming Up?
Todd Tannenbaum
Center for High Throughput Computing
 9:25 am  9:45 am NEOS and HTCondor: Optimizing Your World
The NEOS Server, an internet-based service for solving numerical optimization problems, provides free access to more than 60 state-of-the-art solvers. Our dedicated HTCondor pool at UW-Madison delivers a daily average of 3,000 results to people from dozens of countries around the world.
Michelle Craft
NEOS/Wisconsin Institute for Discovery
 9:50 am 10:10 am Imagining a Future of Agricultural Connections to Water Resources Considering Uncertainty
An important future change to groundwater systems is the use of groundwater for irrigated agriculture. In Western Wisconsin, a county is interested in defining the maximum impact of streams that may occur if agriculture reaches a maximum level. Working with stakeholders, we determined criteria defining suitable irrigated agricultural land. These results are stochastic, so rather than choosing one possible outcome, we performed Monte Carlo analysis to explore a range of outcomes. HTCondor was central to coordinating this effort. Along the way, two-factor authentication became required on our Windows resources, so we worked with the HTCondor team to enable HTCondor to work in an environment with two-factor authentication.
Michael Fienen
USGS
10:15 am 10:35 am HTCondor, Docker, iRODS, and HPC for Every Scientist with the CyVerse Discovery Environment
Ian McEwen
U of Arizona/CyVerse
10:35 am 10:55 am Break
yogurt with granola, whole fruit, assorted house baked cookies, coffee, tea, ice water, soda
Session Moderators: Zach Miller, Matyas Selmeci
10:55 am 11:15 am From Distributed Systems to Data Science
This talk will give a high-level overview of Apache Spark, a framework for distributed data processing. We will compare and contrast the compute models and use cases for Spark and HTCondor. Finally, we will share lessons and best practices we have learned while establishing data science initiatives and present some analytic applications developed in the Emerging Technologies group at Red Hat.
William Benton
Red Hat, Inc.
11:20 am 11:40 am Research Data Depot - Data Services at Purdue
Preston Smith
Purdue
11:45 am 12:05 pm Integrating New Workloads and HTCondor at BNL
William Strecker-Kellogg
BNL
12:10 pm 12:30 pm HTCondor in the Enterprise
This talk discusses the ways Cycle Computing’s customers are using HTCondor to do business-critical work. This includes the ways they’ve moved their HTCondor workloads to the cloud and taken advantage of scale and flexibility to improve their results.
Ben Cotton
Cycle Computing
12:30 pm  1:30 pm Lunch
sliced prime rib with horseradish cream, chicken roulade, mixed green salad with vinaigrette, soup (vegetarian), eggplant and mozzarella napoleon with roasted tomato sauce (vegetarian), bread and butter, brownies and dessert bars, coffee, tea, ice water, soda
Session Moderators: Jaime Frey, Tim Cartwright
 1:30 pm  1:50 pm Hiding All the Details: Running Grid Jobs Inside Docker Containers on the OSG
For distributed computing resources, there’s a natural push and pull over operating systems and environments. Scientific codes, such as the CMS analysis software, are slow to move to new operating platforms. However, system administrators enjoy using new platforms for additional security, stability, and features. Other, opportunistic communities, may want to run jobs in an entirely different Linux distribution. In the past decade, each side sacrificed a little bit of happiness to get on a common platform. Virtual machines appeared as a distant choice but have never become widely used in scientific computing. Recently, Docker offers a lightweight solution to compose environments (with minimal efficiency loss) and got first-class support in HTCondor. In this work, we show one way to integrate Docker universe into a HTCondor-based grid site such that each stakeholder - owner of the resources, system administrator, or opportunistic user - can get their preferred runtime environment.
Derek Weitzel
University of Nebraska Holland Computing Center
 1:55 pm  2:15 pm Putting Idle Campus Central IT Resources to Work Using HTCondor
Michael Layde
UW-Madison DoIT
 2:20 pm  2:40 pm Once More, with Feeling! A Monitoring Feedback Loop for HTC Jobs with Unknown Resources
Ben Tovar
Notre Dame University
 2:45 pm  3:05 pm Computing Betti Tables Using HTCondor
A Betti table is an array of algebraic invariants that are used to understand solutions of polynomial equations. Yet there is also a huge gap in our ability to compute examples of Betti tables. Classically, we compute these tables via Grobner bases, however this technique quickly becomes impractical. For instance, for the projective plane, the degree 3 embedding can be computed by hand, and the degree 4 case can be computed by Grobner bases, but beyond this, computation fails to terminate. We are using an alternate, and highly parallelizable algorithm, to try and compute further cases. Using HTCondor and numerical linear algebra, we have recently computed the degree 5 case, and we expect to go further.
Jay Yang
UW-Madison
 3:05 pm  3:35 pm Break
raw vegetables with assorted dips, tortilla chips with salsa and guacamole, peanut butter filled pretzels, snack mix, coffee, tea, ice water, soda
Session Moderators: John "TJ" Knoeller, Aaron Moate
 3:35 pm  3:55 pm Estimating Long-Term Trends on the Upper Mississippi River
We used HTCondor to compare different statistical models and to develop new models to estimate trends from long-term fish and vegetation data from the Upper Mississippi River.
Richard Erickson
USGS
 4:00 pm  4:20 pm Provisioning EC2 with HTCondor
John Hover
RHIC/ATLAS Computing Facility, Brookhaven National Laboratory
 4:25 pm  4:45 pm Provisioning Cloud-Based Computing Resources via a Dynamical Systems Approach
We aim to build a software service for provisioning cloud-based computing resources that can be used to augment users’ existing, fixed resources and meet their batch job demands. This service must be designed to automate the delivery of compute resources (HTCondor execute nodes) to match user job demand in such a way that cloud-based resource utilization is high and, thus, cost per cpu-hour is low. In addition, since this provisioning service will acquire resources on behalf of its users, acting as a third-party buyer for them, it is also our fiduciary responsibility to ensure the system is stable or, at least, that stability can be maintained. In order to assess if stable resource utilization is possible, a dynamical systems approach is developed to provide a framework for understanding how the provisioning service will respond to user job demand.
Marty Kandes
University of California, San Diego
 4:50 pm  5:10 pm Scalable Submit Host
Brian Bockelman
University of Nebraska-Lincoln

Reception sponsored by Cycle Computing

Courtesy of Cycle Computing we will have a reception following the Wednesday session. It will be at Lucky's Bar and Grille, 1421 Regent Street from 6pm to 7pm. Drinks and appetizers will be served. It is a short walk from the meeting location. Walking directions

Specific talks and times are subject to change.