Parallel programming paradigms

Parallel programming paradigms and environments differ in many aspects. These variations arise from at least four factors ... 

The Linda model supports many different parallel programming paradigms. At its simplest, the distributed-data environment offers an improvement over raw message passing by promoting an uncoupled programming style. This is because the synchronization of processes resulting from access to distributed data is minimal. However, arbitrarily complex shared and distributed data structures may be created. 

As far as sound synthesis is concerned, parallel supercomputing is commonly associated with the possibility of breaking the speed limits imposed by SISD machines. However much faster sequential computers become, parallel computing will still offer interesting programming paradigms for the development of new synthesis techniques. From a composer s perspective, it is this latter aspect of parallel computing that excites interest. 

Two main approaches are used to build parallel supercomputers SIMD (for Single Instruction Multiple Data) and MIMD (for Multiple Instructions Multiple Data). Whilst the former employs many processors simultaneously to execute the same program on different data, the latter employs several processors to execute different instructions on different data. Both approaches underpin different programming paradigms and have their own strengths and weaknesses. 

At present, we are somewhere in-between these points the barrier to utilizing GPU hardware for general purpose computation has been reduced by the introduction of general purpose GPU programming models such as NVIDIA s Compute Unified Device Architecture (CUDA) [15] and AMD s Stream [20]. However, algorithmic paradigm shifts are often required in existing codes to maximize such performance offered by the massively parallel GPU hardware. 

---