Using a Row-Distributed Matrix

Data residing on process P in a parallel matrix-vector multiplication Ab = c, where A is a row-distributed n xn matrix, and b and c are replicated vectors of length n. The process count is p, and Pi holds the rows of A numbered/n/p through (i +l)n/p—1 and computes the corresponding elements of c. A final all-to-all broadcast puts the entire c vector on all processes. 

From Eq. 6.10 it follows that the dimension n must grow at the same rate as p to maintain a constant efficiency as the number of processes increases. If n increases at the same rate as p, however, the memory requirement per process n /p + 2n) will increase with the number of processes. Thus, a fc-fold increase in p, with a concomitant increase in n to keep the efficiency constant, will lead to a fc-fold increase in the memory required per process, creating a potential memory bottleneck. Measured performance data for a parallel matrix-vector multiplication algorithm using a row-distributed matrix are presented in section 5.3.2. 

---