[TriLUG] Building a beowulf (or other) cluster

[Joel J. Adamson]

Justis Peters <jtrilug at indythinker.com> writes:

> On 03/28/2011 11:07 AM, Joseph Mack NA3T wrote:
>> On Mon, 28 Mar 2011, Ron Kelley wrote:
>>
>>> I would like to install some sort of distributed process management
>>> tool so we can enable N-nodes to run the computations
>>> simultaneously.
>>
>> You only use a beowulf if the job cannot fit inside a single
>> machine/node. This usually means that the job needs more memory than
>> a single node holds. If this is your situation, you then recode the
>> app to use the nodes in parallel. This usually means using mpi or
>> omp.
>>
>> If each job can be run in a single node, then you need a job farm
>> (lots of machines with a job dispatcher).
> Ron,
>
> I agree with Joe's take on your issue. You said, "Our processing
> happens in batch jobs and can easily be run on multiple servers at the
> same time." That sounds like an "embarrassingly parallel workload"
> (http://en.wikipedia.org/wiki/Embarrassingly_parallel), which is good
> news.

Good news, yes: embarrassingly parallel doesn't mean you should be
embarrassed.  I found for my own purposes that

make -j

did the trick.  In other words, parallel /jobs/ was what I needed,
rather than parallel code (as with OpenMP).

Joel

-- 
Joel J. Adamson
Servedio Lab
University of North Carolina at Chapel Hill

FSF Member #8164
http://www.unc.edu/~adamsonj
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 229 bytes
Desc: not available
URL: <http://www.trilug.org/pipermail/trilug/attachments/20110328/9da4b19a/attachment.pgp>