Memory bottleneck

Over the last decade, chip designers have struggled with such limits and tradeoffs, best exemplified by the twin problems of memory bottleneck and power management. Although CPUs are a thousand times faster today than they were... 

From Eq. 6.10 it follows that the dimension n must grow at the same rate as p to maintain a constant efficiency as the number of processes increases. If n increases at the same rate as p, however, the memory requirement per process n /p + 2n) will increase with the number of processes. Thus, a fc-fold increase in p, with a concomitant increase in n to keep the efficiency constant, will lead to a fc-fold increase in the memory required per process, creating a potential memory bottleneck. Measured performance data for a parallel matrix-vector multiplication algorithm using a row-distributed matrix are presented in section 5.3.2. 

Large data structures should be distributed to avoid memory bottlenecks... 

Parallel Fock maf rix compufaf ion using replicated density and Fock matrices is easy to implement and achieves high parallel efficiency. Using replicated matrices, however, may create a memory bottleneck the Fock and density matrices are both, nominally, of size O(n ), and keeping a copy of the entire density and Fock matrix for every process may not be possible when the number of basis functions is very large. This memory bottleneck can be avoided by distributing the matrices, and we will explore this approach in the next section. 

To avoid a potential memory bottleneck in parallel Hartree-Fock computations for very large systems, the density and Fock matrices must be distributed, and in this section we will investigate a distributed data parallel... 

From our experiences with D C, we have found that it is best to use separate cutoffs for the matrices and the electrostatics. It may be that the LSCF approach would benefit from employing a small cutoff for P, H, and F, and a large cutoff for the simplified Coulombic expressions. To eliminate any memory bottleneck, the two-electron integrals could be recalculated as needed. Conversely, our D C scheme could be made more efficient by adopting Stewart s simplified NDDO formulas at large distances, rather than the approach we currently use, which recalculates all required two-electron integrals using the standard NDDO formulas. 

Definition of Critical and Rate-Limiting Bottlenecks" The hypothesis of local equilibrium within the reservoirs means that the set of transitions from reservoir to reservoir can be described as a Markov process without memory, with the transition probabilities given by eq. 4. Assuming the canonical ensemble and microscopic reversibility, the rate constant Wji, for transitions from reservoir i to reservoir j can be written... 

Considering long-range MD, a variety of approximate methods have been developed to overcome the bottleneck that characterizes the forces treatment these include particle mesh algorithms, hierarchical methods, and fast multipole methods. One of the most promising developments is the cell multipole method, which scales linearly with N, requires only modest memory, and is well suited to highly parallel and vector computers. 

We assume that the total physical memory, required by the problem and algorithm, will fit into the total available memory. For situations - often called out-of-core problems-when this is not true, it is the sustainable bandwidth to secondary media, like disk and tape, which becomes the bottleneck. A very interesting paper on what can be done using an algorithm that has both modest I/O bandwidth requirements and a substantial latency tolerance can found in [48]. 

If we think of the items in the above list as being ordered in terms of decreasing approximate importance. The first three items on the list are concerned with the physical memory of the machine and the processing power of each node comes only in fourth place and the network and secondary media follow thereafter. This ordering puts the focus on the primary bottleneck in (parallel) computers used for scientific computing with large datasets. This is especially true if the node CPUs are microprocessors, but also to a large extent if they are vector processors. 

A newer measure of an algorithm s theoretical performance is its Mop-Cost which is defined exactly as the Flop-cost except that Memory Operations (Mops) are counted instead of Floating-Point Operations (Flops). A Mop is a load from, or a store to, fast memory. There are sound theoretical reasons why Mops should be a better indicator of practical performance than Flops, especially on recent computers employing vector or RISC architectures, and this has been discussed in detail by Frisch et al. [62] to cut a long story short, the Mops measure is useful because, on modern computers and in contrast to older ones, memory traffic generally presents a tighter bottleneck than floating-point arithmetic. 

The small number of coefficients needed can be pre-tabulated and held in memory, and we retain the computational simplicity of the Cartesian formulation along with the vital transformation properties of the spherical Gaus-sians. The coefficients [A, B p, P , i, j,k , i, j, kf ,s,t, ] are simple to construct, and the accumulation of sums like (208) can mostly be done in integer aritb-metic. The extensive cancellation which occurs for higher angular momentum spinors can therefore be done exactly without rounding error. The computational bottlenecks encountered in our preliminary work with the complex recurrence relations for direct constraction of Eg[A,B ,p,P ,n,l,m ,r, 1, m s,t,u] given by [107] are completely eliminated. The calculation of these coefficients and the spinor coefficients of the next section now constitutes a trivial part of the computational load. 

---