This section is still being developed as the usage on my cluster
evolves, but so far we tend to write our own sets of message passing
routines to communicate between processes on different machines.
Many applications, particularly in the computational genomics
areas, are massively and trivially parallelisable, meaning that
perfect distribution can be achieved by spreading tasks equally across
the machines (for example, when analysing a whole genome using a
technique that operates on a single gene/proteom, each processor can
work on one gene at a time independent of all the other
processors).
So far we have not found the need to use a professional queueing
system, but obviously that is highly dependent on the type of
applications you wish to run.
For the single most important program we run (our ab initio
protein folding simulation program), using the Pentium 3 1 GHz
processor machine as a reference frame, the Athlon 1.2 GHz processor
machine is about 16% faster on average, the Pentium 4 1.7 GHz machine
is about 25-32% faster on average, and the Athlon 1.5 GHz processor is
about 80% faster on average (yes, the Athlon 1.5 GHz is faster than
the Xeon 1.7 GHz since the Xeon executes only six instructions per
clock (IPC) whereas the Athlon executes nine IPC (you do the math!)).
These machines are incredibly stable both in terms of hardware and
software once they have been debugged (usually some in a new batch of
machines have hardware problems), running constantly under very heavy
loads. One example is given below. Reboots have generally occurred
when a circuit breaker is tripped.
