Multi-threading

The relationship between processes, threads, and processors within a node. Each process has its own memory space shared among one or more threads. The threads are mapped to processors by the operating system. 

C language malloc function or the C++ new operator) and file descriptors, but each thread also stores some data unique to that particular thread for example, the address of the instruction currently being executed, the processor registers, and a stack storing temporary data generated by function calls are all specific to a given thread. It is also possible for threads to have private memory and for processes to share memory however, for our purposes, the simplified picture presented above will suffice. 

The threads we have discussed so far are known as kernel threads. These threads are provided by the operating system, which can schedule different threads to run simultaneously on different processors. Another type of thread is a user thread (or user-space thread), which is scheduled by the program itself. There are two ways to perform this scheduling an operating system timer can 

---

Multi-Threaded Computing A process model that uses hardware/software mechanisms to switch between multiple execution threads. 

Nielsen IMB, Janssen CL. Multi-threading a new dimension to massively parallel scientific computation. Comp Phys Commun 2000 128 238-244. 

Pseudocode iUustrating a race condition in a multi-threaded program. Two threads are created, each reading and modifying a datum at the same memory location, datu m. 

Another multi-threaded implementation of the matrix-vector multiplication, using OpenMP, is shown in Figure 4.6. This implementation is much... 

The four levels of multi-threading support defined by the MPI standard... 

A routine to compute the matrix vector product c = Ab using a hybrid multi-threading and message-passing technique. The algorithm shown combines the MPI and OpenMP parallelizations illustrated in Figures 4.4 and 4.6. Since all MPI calls are from the main thread, MPI need only support the "funneled" level of multi-threading. 

Appendix B briefly discusses programming with Pthreads, and the text by Lewis and Berg is a good source for more information on multi-threaded programming with Pthreads. Appendix C gives a brief introduction to... 

The use of MPI in a multi-threaded environment is briefly discussed in section 4.4. 

During the multi-threaded parallel execution inside of an OpenMP worksharing region, it is sometimes necessary to explicitly coordinate the activity of the threads. We have already seen an example of implicit synchronization with the reduction clause of the for directive, which causes each thread to ensure that updates to the reduction variables occur in a thread-safe manner. 

---