Distributed memory computer

J. Nieplocha, R.J. Harrison and R.J. Littlefield, Global arrays A portable "shared-memory" programming model for distributed memory computers, in Supercomputing 94 (Washington D.C., 1994). 

Message passing, collective communication, and similar alternatives for programming software libraries for large-scale applications on distributed-memory computer systems. 

T. R. Furlam and H. F. King,/. Comput. Chem., submitted for publication. Implementation of a Parallel Direct SCF Algorithm on a Distributed Memory Computer. 

S. J. Plimpton, in The Fifth Distributed Memory Computing Conference, Proceedings, Charleston, SC, April 8-12, 1990, D. W. Walker and Q. F. Stoute, Eds., IEEE Computer Society Press, Los Alamitos, CA, 1990, pp. 478-483. Molecular Dynamics Simulations of Short-Range Forces on 1024-Node Hypercubes. 

Marquez, A. M., J. Oviedo, J. F. Sanz, and M. Dupuis. Parallel computation of second derivatives of RHF energy on distributed memory computers. /. Comp. Chem. 18 159-168,1997. 

Petitet, A., R. C. Whaley, J. Dongarra, and A. Cleary. HPL—a portable implementation of the high-performance Linpack benchmark for distributed-memory computers, http //www.netlib.org/benchmark/hpl/. 

Each of the individual nodes discussed in the previous section can be a MIMD parallel computer. Larger MIMD machines can be constructed by connecting many MIMD nodes via a high-performance network. While each node can have shared memory, memory is typically not shared between the nodes, at least not at the hardware level. Such machines are referred to as distributed memory computers or clusters, and in this section we consider such parallel computers in more detail. 

Petitet, A., Whaley, R.C., Dongarra, and Cleary A. (2008) HPL - a Portable Implementation of the High-Performance Unpack Benchmark for Distributed-Memory Computers, http //mvw.netlib.org/benchmark/hpl (last accessed 3 November 2010). 

S. Kocak and H.U. Akay. (2001) Parallel Schur Cointlement Method for Large-Scale Systems on Distributed Memory Computers. AppUed Mathematical Modelling 25. 

Parallel programming for shared memory computers is relatively straightforward, as all processors can access any location in the common memory. Processor access to the global memory is coordinated by a switch. The complexity of this switch increases rapidly with the number of processors, resulting in longer memory access times, and shared memory machines therefore usually consist of a small number of processors, typically less than 100. In distributed memory computers, there is, in principle, no such bottleneck, as communication networks are constructed in such a way that communication between one pair of processors is typically independent of communication between another pair. 

Figure 1 Common network topologies for distributed memory computers (a) bus (b) ring (c) two-dimensional mesh (d) torus (e) hypercube (f) full interconnection. Squares represent processors, and lines represent communication paths...

An important consideration in the parallelization of quantum chemistry algorithms for distributed memory computers is the data distribution. The simplest approach is to replicate all the data on all the nodes. Considering, for example, a parallel direct HF computation, this means that each node must store the Fock matrix, the density matrix, the eigenvectors and a variety of other matrices depending on the implementation. Thus, the storage requirement on each node becomes 0 n ), where n is the number of basis functions, and for the large basis sets that can be handled in a reasonable amount of time on a massively parallel computer, this storage requirement may become prohibitive. 

---