It seems these days more and more research environments are taking advantage of using a cluster of computers for running parallel programs on them. So what is a cluster? In the computing world there are two types of clustering. One type is for high redundancy and fail-over for usually one high profile computer. In this scenario, let’s say it’s a web server, there would be multiple computers all synchronized together into a cluster so that if any one of the computers died another one would take it’s place. This is what I thought clustering was when I was in the private sector.  When I got into academics is when I realized the other cluster use. The second type of cluster is where you just put lots of computers together and use them all at the same time (in parallel) to compute some type of large data set or algorithm. It’s what I focus on mostly these days and the topic of this discussion.

The main idea here is let’s say for example, you run a program that takes one month to run on one processor. This could be your own personal desktop even. Many times, the program computes some calculation over and over again while changing a variable each iteration. A good example is “over time”. So the same calculation, but time changes in it.

Let’s say that if we wrote the program so that it could send one calculation to one cpu and another calculation (with time incremented) to another cpu we theoretically could cut our time in half to solve the same problem. So running it on 1 cpu takes a month, 2 cpu’s takes 15 days, 4 cpu’s takes a week, and so on. You can see that for certain problems you can vastly increase the turn around time to analyze results.

Now, mind you, this is just the basic idea and there are a lot of issues related to this and in many cases there is a limit to how much performance you can get. Meaning that at some point the time to calculate the problem slows down when you have too many resources. The overhead to manage the communication between hosts takes over and essentially takes longer to process and collect all the output data.

If this is a little technical don’t worry… the main idea is that there are many benefits for using lots of computing resources to solve problems.