OpenMP

To address the shortcoming of the simple MPI model, Medvedev and coworkers [40] developed a hybrid OpenMP/MPI method that takes advantage of both distributed and shared memory features of these clusters of multiprocessor nodes. The features of this model are ... 

To use OpenMP on each node to implement the parallel work on each node. 

Hybrid OpenMP/MPI parallelization is used to compute Hv. The multithreaded OpenMP library is used for parallelization within each node, while MPI is used for communication between nodes. The subroutine fipsi that computes Hv contains two arrays of the same size the results of the portion of the calculation local to a node are stored in hps local, while the results of the portion of the calculation that requires communication are stored in hpsbujf. On each node, a dedicated (master) OpenMP thread performs all the communication, while other threads perform local parts of the work. In general, one uses as many threads as there are processors on a given node. The following scheme presents an overview of the execution of subroutine, hpsi ... 

The shared memory OpenMP library is used for parallelization within each node. The evaluation of the action of potential energy, rotational kinetic energy, and T2 kinetic energy are local to each node. These local calculations are performed with the help of a task farm. Each thread dynamically obtains a triple (/2, il, ir) of radial indices and performs evaluation of first the kinetic energy and then the potential energy contribution to hps local( , i2, il, ir) for all rotational indices. 

Figure 8. Schematic of distribution of the wave packet over the nodes and the communication pattern for the hybrid OpenMP/MPI method. The wave packet is distributed over four nodes, each of which contains four processors. All of the communication is handled by the shaded processors the other processors are simultaneously performing computations. Note that there is only nearest-neighbor communication between nodes.

Figure 9. Typical speed-up curve for the hybrid OpenMP/Ml approach. The symbols represent achieved speed-up solid line shows ideal speed-up. The deviation from ideal speed-up occurs when computation time becomes comparable to computation time on the nodes.

Component Resolution Time step (s) Species MPI/openMP Run time 24 h... 

Another multi-threaded implementation of the matrix-vector multiplication, using OpenMP, is shown in Figure 4.6. This implementation is much... 

A routine to compute the matrix vector product c = Ab using OpenMP. 

A routine to compute the matrix vector product c = Ab using a hybrid multi-threading and message-passing technique. The algorithm shown combines the MPI and OpenMP parallelizations illustrated in Figures 4.4 and 4.6. Since all MPI calls are from the main thread, MPI need only support the "funneled" level of multi-threading. 

OpenMP, and the OpenMP standard includes complete details for programming with OpenMP. 

OpenMP Application Program Interface. Version 2.5, May 2005. http //www. openmp.org. 

Quinn, M. J. Parallel Programming in C with MPI and OpenMP. New York McGraw-Hill, 2003. 

We will use C code fragments to illustrate OpenMP the equivalent C++ code fragments would be similar to those for C, but the use of OpenMP in Fortran looks somewhat different, although the concepts are the same. The C and C++ code annotations in OpenMP are specified via the pragma mechanism, which provides a way to specify language extensions in a portable way. 

OpenMP functions. These functions are declared in the file... 

Pragmas take the form of a C preprocessor statement/ and compilers that do not recognize a pragma will ignore them. OpenMP pragmas begin with the keyword omp, which is followed by the desired directive ... 

OpenMP provides functions to query the OpenMP environment, control the behavior of OpenMP, acquire and release mutexes, and obtain timing information. A few of these functions are summarized in Table C.l. The omp.h... 

OpenMP also provides a few environment variables one of these is the OMP NUM THREADS variable, and a Bash shell statement of the form... 

The OpenMP directives shown in Table C.2 can be divided into those that specify parallel regions, work-sharing constructs, and synchronization constructs. The primary OpenMP directive is parallel, and the block of code that follows this statement is a parallel region that will be executed by multiple threads, namely a team of threads created by the thread encountering... 

Selected OpenMP directives and their effect. C/C++ directives are shown, and... 

---