1. Header Files
#include<stdio.h>
#include<mpi.h>
-
stdio.h→ used for input/output operations likeprintf(). -
mpi.h→ required to use MPI functions for parallel programming.
MPI is a library used in parallel computing systems and clusters.
2. Matrix Size Definitions
#define NUM_ROWS_A 3
#define NUM_COLUMNS_A 3
#define NUM_ROWS_B 3
#define NUM_COLUMNS_B 3
These define the dimensions of matrices.
Matrix A → 3 × 3
Matrix B → 3 × 3
Since:
So result matrix C will also be 3 × 3.
3. MPI Message Tags
#define MASTER_TO_SLAVE_TAG 1
#define SLAVE_TO_MASTER_TAG 4
MPI uses tags to identify messages.
| Tag | Meaning |
|---|---|
| 1 | Message from master to slave |
| 4 | Message from slave to master |
This prevents confusion between different messages.
4. Function Declarations
void makeAB();
void printArray();
These functions:
makeAB()
-
Initializes matrices A and B.
printArray()
-
Prints matrices A, B and result C.
5. Global Variables
int rank;
int size;
rank
Process ID.
Example with 4 processes:
| Process | Rank |
|---|---|
| Master | 0 |
| Slave 1 | 1 |
| Slave 2 | 2 |
| Slave 3 | 3 |
size
Total number of processes running.
Example:
mpirun -np 4 program
Then:
size = 4
6. Matrix Declaration
double mat_a[3][3];
double mat_b[3][3];
double mat_result[3][3];
These are:
| Matrix | Meaning |
|---|---|
| mat_a | Input matrix A |
| mat_b | Input matrix B |
| mat_result | Result matrix C |
7. MPI Initialization
MPI_Init(&argc, &argv);
This starts MPI environment.
Without this function MPI programs cannot run.
8. Get Process Rank
MPI_Comm_rank(MPI_COMM_WORLD, &rank);
Each process receives a unique rank number.
Example output:
Process 0
Process 1
Process 2
Process 3
9. Get Number of Processes
MPI_Comm_size(MPI_COMM_WORLD, &size);
This gives total processes.
Example:
size = 4
10. Master Process (Rank 0)
if (rank == 0)
Process 0 acts as the master.
The master:
-
Creates matrices
-
Divides work
-
Sends data to slaves
-
Collects results
11. Create Matrices
makeAB();
Function fills matrices.
Example matrix A:
0 1 2
1 2 3
2 3 4
Matrix B:
0 0 0
0 1 2
0 2 4
12. Start Time Measurement
start_time = MPI_Wtime();
MPI provides a timer to measure parallel execution time.
13. Work Distribution
for (i = 1; i < size; i++)
Master sends work to each slave process.
Calculate Portion
portion = (NUM_ROWS_A / (size - 1));
Example:
rows of A = 3
slaves = 3
portion = 1 row per slave
Each slave processes one row of matrix A.
14. Determine Row Range
low_bound = (i - 1) * portion;
upper_bound = low_bound + portion;
Example:
| Slave | Rows |
|---|---|
| 1 | row 0 |
| 2 | row 1 |
| 3 | row 2 |
15. Send Data to Slaves
Send row range
MPI_Isend(&low_bound,1,MPI_INT,i,...)
MPI_Isend(&upper_bound,1,MPI_INT,i,...)
Send matrix data
MPI_Isend(&mat_a[low_bound][0], ...)
Here the master sends only the required rows.
16. Broadcast Matrix B
MPI_Bcast(&mat_b,...)
This sends matrix B to all processes.
Reason:
Every slave needs matrix B to perform multiplication.
Broadcast is faster than sending individually.
17. Slave Process Work
if(rank > 0)
All slaves execute this block.
Step 1: Receive Work
MPI_Recv(&low_bound...)
MPI_Recv(&upper_bound...)
MPI_Recv(&mat_a...)
Each slave receives:
-
Row range
-
Portion of matrix A
18. Matrix Multiplication
for(i=low_bound;i<upper_bound;i++)
for(j=0;j<NUM_COLUMNS_B;j++)
for(k=0;k<NUM_ROWS_B;k++)
This performs:
Example calculation:
C[0][0] = A[0][0]*B[0][0] +
A[0][1]*B[1][0] +
A[0][2]*B[2][0]
19. Send Results Back
After computation slaves send results to master.
MPI_Isend(&low_bound...)
MPI_Isend(&upper_bound...)
MPI_Isend(&mat_result...)
Master now knows:
-
Which rows are computed
-
Their result values
20. Master Collects Results
for(i=1;i<size;i++)
Master receives data from each slave.
MPI_Recv(&low_bound...)
MPI_Recv(&upper_bound...)
MPI_Recv(&mat_result...)
The result matrix is reconstructed.
21. Stop Time
end_time = MPI_Wtime();
Execution time:
Running Time = end_time - start_time
This shows parallel performance.
22. Print Matrices
printArray();
This prints:
1️⃣ Matrix A
2️⃣ Matrix B
3️⃣ Result matrix C
Example output:
Matrix A
0 1 2
1 2 3
2 3 4
Matrix B
0 0 0
0 1 2
0 2 4
Result Matrix
0 5 10
0 8 16
0 11 22
23. Finalize MPI
MPI_Finalize();
This closes MPI environment.
Every MPI program must end with this function.
Overall Program Flow
Start MPI
│
Master creates matrices
│
Master divides rows
│
Master sends rows to slaves
│
Broadcast matrix B
│
Slaves perform multiplication
│
Slaves send results
│
Master gathers results
│
Print result
│
End MPI
Key Concepts Used
| Concept | Meaning |
|---|---|
| MPI_Init | Start MPI |
| MPI_Comm_rank | Get process ID |
| MPI_Comm_size | Get number of processes |
| MPI_Isend | Non-blocking send |
| MPI_Recv | Receive message |
| MPI_Bcast | Broadcast data |
| MPI_Wtime | Measure execution time |
Why MPI is Used
Advantages:
✔ Faster computation
✔ Parallel processing
✔ Efficient for large matrices
✔ Used in supercomputers and clusters
0 Comments