The raw computation power of today's PC clusters running the Linux operating system rivals that of commercial supercomputers at a fraction of their purchase price. As a part of that high price tag, some tools and system support come with the supercomputer package. However, a multitude of vendors will slap together a number of commodity computer boxes and call it a cluster while providing nothing more than a power cord for each box. Thus, the lack of good administration and application support tools represents a hidden operation cost that is often overlooked when purchasing a cluster. This hidden cost greatly adds to the cluster's total cost of ownership. The work presented in this paper addresses the area of cluster administration and application support by presenting the Cluster Command & Control (C3) tool suite developed at Oak Ridge National Laboratory for use in administrating and using its high performance cluster computing resources.
We first cover the history of the Cluster Command and Control (C3) tool suite from initial v1.0 release to present state of v2.7 and include brief directions for v3.0 as well as the planned v4.0. We then present the requisite C3 tools environment followed by C3 command specifications with command syntax and some simple command line usage examples. A brief coverage is then give to some implementation issues between a serial and parallel implementation. We conclude the discussion of C3 v2.x with advanced usage techniques by showing how the C3 tools may be embedded into other commands and applications to create end-to-end cluster solutions. We close our discussion by providing information on C3 v3.0 that will be released later in 2001, as well as plans for a v4.0 high-scalability release.