Skip to main content.


The MW Homepage

This page contains information, documentation, and source code for the MW project.

What is MW?

MW is a tool for making a master-worker style application that works in the distributed, opportunistic environment of HTCondor. MW applications use HTCondor as a resource management tool, and can use either HTCondor-PVM or MW-File a file-based, remote I/O scheme for message passing. Writing a parallel application for use in the HTCondor system can be a lot of work. Since the workers are not dedicated machines, they can leave the computation at any time. Machines can arrive at any time, too, and they can be suspended and resume computation. Machines can also be of varying architechtures and speeds. MW will handle all this variation and uncertainty in the opportunistic environment of HTCondor.

The master, or MWDriver, is basically a layer that sits above the program's resource management and message passing mechanisms and below your application. It handles things like nodes going up/down, suspending/resuming, and the starting and stopping of jobs. It does this by managing a set of user-defined tasks and a pool of workers. The workers do user-defined work on the tasks that they're given. The MWDriver simply matches tasks to workers.

To learn more, read the users guide, or look at the overview. You can refer to several papers that we have written that explain the rationale behind our design choices. Other information can be found in overview talks given in December 99 , June 99, February 99, March 2000, and most recently February 2004. Documentation (constructed with Doc++) for MW base classes is here. Here is an example application.

Most of the real applications using MW that people have built so far are to solve mathematical Optimization problems. To see some of these real-world solvers, see our solvers page.

The diagram below indicates the relationships between HTCondor, PVM, and the MWDriver.

MW News

  • MW Version 0.2.2 released -- download source from menu on the left. New version has support for running on native MS-Windows (without cygwin), many bugfixes, and new "blackbox" example for running standalone executables as workers (1/9/2006)
  • MW Version 0.2.1 released -- download source from menu on the left. New version has minor bugfixes, users guide and cleaned up examples. (7/10/2005)
  • MW Version 0.2 released -- download source from menu on the left. New version supports 64 bit architecture, Windows (via cygwin), and new chirp mode with MW-Files, which allows MW-File to work on clipped version of condor. (5/31/2005)
  • MW is now distributed under an LGPL license. (10/04/2002)
  • Interactive Master-Worker (IMW) Prototype available. IMW, written by Jichuan Chang, is a lightweight interactive master-worker framework written in Java (except for a C++ HTCondor User Log Reader) which utilizes HTCondor's Java Universe. Here is a brief introduction, and a README. You can download and try it. (05/31/2002)
  • MW mailing list "mw@cs.wisc.edu" created!
    To join the list, please send a blank email message to mw-subscribe@cs.wisc.edu, or else visit this page to fill out a subscription. Please also use the list to send MW bug reports.
  • MW-Independent is back! MW-Independent implements the resource and communication (RMComm) layer using memory copy, and bundles the master/workers into a single process. It can be used to debug an MW application code before submitting your job to HTCondorPvm/MW-File/MW-Socket.
  • Pre-compiled MW libraries installed in the UW HTCondor Pool (cs.wisc.edu). Look under /unsup/condor/mw.

MW People

  • Jeff Linderoth: email, web page.
  • Wen-Han Goh: email.
  • Peter Keller email,web page.
  • Greg Thain: web page.
  • Steve Wright: email, web page.

MW Alumni

This material is based upon work supported by the National Science Foundation under Grant No. 0330538.