C/C++ Framework for distributed computing in a dynamic cluster -
I am looking for a framework to use in a C ++ distributed number crunching application.
Setup looks like this:
A master node that breaks the problem domain into small independent tasks. Work has been built in different capacities of worker nodes (such as CPU type / GPU-enabled). Work nodes are dynamically added to the calculation grid, because they are available, it can also be that a worker node dies, without saying goodbye.
To complete this setup I am searching for a fast C / C + + framework.
In short, my main requirements are:
- Worker / Work
- Dynamically add / remove nodes
- Target Network: 1G - 10G Ethernet (corporate network, good performance is not required on the internet)
- Optional: Encrypted and certified communication
What you definitely want to do with you MPI-2 added dynamic process management features and I Most of the deficit that is currently widely used to be Lagukrnon made the offer in.
One advantage of using C ++ + MPI is that combining scientific and technical computing, though my impression is that this niche dynamic process management is not used very much as the use of MPI The cause of computational science is on the biggest supercomputers dealing with increasing problems, so one can guess that it is a quick reference for your purposes. Will be ample.
One of the disadvantages of using C ++ + MPI is that the MPI was not designed to tolerate the failure of processes during execution. There is a debate on whether the dynamic process management facilities allow you to program your fault tolerance. But there can be no debate that it can be difficult.
You will get your first 3 needs out-of-the-box. For encrypted and authenticated communication, you have to do most of your own, MPI bus passes around messages almost It seems that for most MPI users, personal interconnects (often separate from corporate or enterprise networks themselves), parallel applications on cluster or supercomputer with encryption and authentication Running is a matter of great concern.
Comments
Post a Comment