I am building a Supercomputer made out of 4x 16 core machines running at 2.93 Ghz, stock. I was wondering if the head node for the cluster is required to have any performance characteristics? I mean to say, should it be a good computer? Will it matter if I throw on a cheap pentium 4 or will I be better off with a duo core or ever duo core with hyperthreading etc.... The idea is that the head-node relays info from the workstation, and therefore isn't too important (spec-wise), is that right?

Also, when connecting the computers together to a "router" could I simply use a cheapo router that I might have left over from an old internet provider? Is there a difference in the hardware? Will the bandwidth of only 100 mbps off the boards really be constrained by the router, too much, realistically?Ricky ( the master cluster node ) should be equal to the Lucys in spec. Its not a requirement, but for best performance its best to be balanced between Ricky and all the Lucys.

You can get by with 100mbps network, but I'd go with a Gigabit 1000mbps network. You can get by with a low cost switch or router acting as a switch etc.

Curious as to what you will be crunching for data and what OS of choice you will be using for your cluster?This thread has my attention. May I ask questions?
Have you made any working model yet? Some kind of down scaled working Super computer micro version?
Is there some project others have done that serves as a guide?

Thanks for letting me ask. It really depends on what sort of cluster you are running and how it is designed. While the head node is generally uses to launch jobs, it often COORDINATES a lot of the rest of the things that go on in the cluster.

In Hadoop for EXAMPLE the head node coordinates resources in the cluster as well as manages the distributed file system, it needs to be a pretty powerful node to do this, in some cases, more powerful than the individual data processing nodes.I have built up 4 separate server computers which will be running microsoft server 2008 or 2012 with its hpc pack (b/c I do not LINUX as they say). They work off of 4 supermicro X8DTT-F boards with 2 x5570 chips on each board giving 8 cores with hyper-threading i.e. like having 16 cores (I like thinking of it that way). The chips run at 2.93 Ghz, but theoretically i could overclock but not good for a Xeon so forgetting about it right away. It's my first time building up a cluster so I'm not sure of how it all comes together. My workstation itself is a majestic quad core that is good for smallish simulations (4000 real- time plotted entities) but not enough for the incredibly large matrix feeds I will need in the future, so instead of investing $500 in computer time, I decided to build my own cluster.

The hardware part was kinda difficult to get down but that's all figured out now. Stacked up on layers of headers the boards layout to be the size of a small or medium atx case (probably could just stick it in one). I haven't had any previous projects or attempts with the system. In my mind, as I have understood what you have been saying, I would have thought that the cluster head would do all that you have already said...but do I really need a 16 thread machine to run all those machines? Is it not really overkill since the machine won't be computing at all? Reason for me asking is that I read some other sites where it pointed only to the clock speed of the head node and the fact that it will "just" be the via point to allow users to access the cluster.
And if the cluster spreads to be 8 machines or more, what can i do? Should I invest in some cheap network cards (1 PCI lane - the board only has one) and maybe a cheap ssd for the head node, since it might be writing something down, mid-processing? (the cluster nodes have simple 7200 rpm hdd's)

I will be running matlab with the hpc programming pack for my first codes and such. I will need to display lots of particulate interactions with this system so if you are familiar with the hpc systemology of matlab (which i think takes care of node MANAGEMENT (as I read on their website and talked to SOMEBODY on the phone about)) if you could give me some advice before I go out and buy either something that will not work well or is too expensive, I'd be grateful. Maybe tmi but I do my best.
You would really need to look into what the head node actually does with what you are doing and go from there. Why not try a cluster with 3 of your Xeon machines then use one of them as a head node and then test using an older machine as your head node instead and see how that affects performance. It could also be worth considering running the head node on one of the processing nodes to see how that works, either on the same OS or in a virtual machine. I'm not aware of Matlab having any sort of complex filesystem so you could well be okay with a lower spec head node.

You should definitely be using a gigabit network for this but you probably don't need any separate network cards, just use the onboard ones and a reasonable Gigabit switch.Quote from: camerongray on October 29, 2015, 09:10:06 AM

You should definitely be using a gigabit network for this but you probably don't need any separate network cards, just use the onboard ones and a reasonable Gigabit switch.