Condor World Map


This is an automatically generated series of maps that display some of the Condor pools in the world. Each square on the map represents one operating, autonomous Condor pool that has reported its presence within the last sixty days. This map is updated daily. The size of each pool is indicated by color; a single pool may contain as little as one or as many as thousands of individual machines. There are some subtleties to this map; please read on for more details.
Please use the following citation for this data:
Douglas Thain, Todd Tannenbaum, and Miron Livny, "How to Measure a Large Open Source Distributed System", Concurrency and Computation: Practice and Experience, volume 8, number 15, December 2006.
Totals By Country
World
Totals By U.S. State
(gif | eps | pdf)
color: (gif | eps | pdf)
b/w: (gif | eps | pdf)
(gif | eps | pdf)
Europe
United States
Asia
color: (gif | eps | pdf)
b/w: (gif | eps | pdf)
color: (gif | eps | pdf)
b/w: (gif | eps | pdf)
color: (gif | eps | pdf)
b/w: (gif | eps | pdf)

These maps were created by Douglas Thain and are currently maintained by the Condor Team.

About This Map

What does this map show?
This map displays all of the Condor pools worldwide that are willing to make their presence known to us at the University of Wisconsin. Within the United States, each pool is plotted to the nearest city. Outside the United States, each pool is plotted at the capital of the host country. A grid structure is used to visually distinguish multiple pools that would otherwise fall in the same spot.

What is a Condor pool?
A Condor pool is a collection of computers that pool their resources in order to make all of their combined power available to any of the participants. A Condor pool can range from just one machine to many thousands of machines.

What is the Condor Project?
The Condor Project is a research program led by Miron Livny at the University of Wisconsin. Since 1985, the Condor Project has researched and developed new ways for ordinary people to discover, share, and exploit large amounts of computing power.

How is this data collected?
Many Condor pools in the world send periodic messages to to a server at the University of Wisconsin, giving a short summary of the pool name, internet address, number of hosts, and a few other details. Some of these messages are delivered every two weeks by email, while others are delivered every five minutes by UDP. Some pools report in both manners, some by one only, and some not at all. We periodically collect these messages and transform the internet addresses into geographic coordinates to produce this map.

Does this map show all Condor pools in the world?
No, it certainly does not. This is primarily because we do not receive periodic messages from all installed Condor pools. The most common reason for this is that Condor pool managers can freely choose not to send us messages, perhaps due to concerns of security and privacy. Another reason might be that they are behind a firewall or private network, so the address that we see cannot be resolved to a physical location. Other reasons include incorrect or incomplete WHOIS entries, and so on... We don't know exactly how many operating Condor pools there are in the world. We do know of the ones on this map, and we also know that the Condor software is downloaded from our web site about a thousand times a month.

How do you map internet addresses to geographic coordinates?
There are a variety of techniques to map IP addresses, none of them completely reliable. If you need to do this in a high volume way for an important task, there are many commercial services that can do it with varying levels of accuracy. Here's how we do it on the cheap:

  1. Call DNS to convert an IP address to a domain name.
  2. Use a static table to determine the whois server for the top-level domain.
  3. Recursively call WHOIS on the domain, and scrape the output for text that looks like an address:
    1. If a response contains a (city, state, zip) pattern, assume it is a U.S. address and map the pattern to coordinates via a static table provided by the postal service.
    2. If a response contains a country name, assume it is an international address, and map it to the coordinates to the capital of that country.
  4. If all else fails, infer the country from the top-level domain name.

Is the plotting accurate? Why do some points fall in the ocean?
Condor pools tend to clump together. It we plotted them accurately, you wouldn't be able to see much. So, we line up the squares in a grid so that they can be seen and counted. Due to the grid structure, some squares are significantly out of position, and some, such as Miami, fall in the ocean. Our map data has several significant omissions. For example, Ireland appears to be missing.

Why do the Condor pools clump together?
There are several possible reasons for a clump. One is that pools really are physically clumped together. This is the result of social effects. If one person or project establishes a Condor pool, there is encouragement and help for others nearby to set up Condor pools. Of course, we offer lots of encouragement to do this in Madison, Wisconsin, so there is a clump there. Similar things happen at other institutions. Another reason is that many addresses in the WHOIS system give the location of the administrator of a large organization, or perhaps that organization's internet service provider, not the physical location of the machine itself. Thus, we see a big clump in the San Francisco bay area, where the network operations of many California organizations are centered. Outside of the United States, we are only able to resolve location to the correct country, and then simply plot the squares around the capital.

I want my Condor pool to be on this map!
Great, thanks for helping out. First, make sure that your central manager has a complete and valid domain name. Then, set the following configuration variables:

CONDOR_DEVELOPERS = condor-admin@cs.wisc.edu
CONDOR_DEVELOPERS_COLLECTOR = condor.cs.wisc.edu

I DO NOT want my Condor pool to be on this map!
That's fine, too. Set the following configuration variables:

CONDOR_DEVELOPERS = NONE
CONDOR_DEVELOPERS_COLLECTOR = NONE

Condor Home Page