A6 - MIMD

image: T Latham, lecture slides

• Multiple Instructions Multiple Data (MIMD)

image: T Latham, lecture slides

• there are two main architectures:

  • Shared Memory Parallel (SMP) system: all processors are connected to a shared memory
  • distributed memory system/cluster: each processor has its own memory, and any communication is via a network

• programming for SMP machines involves a single program issuing multiple instruction streams, called threads, to operate on different parts of the shared memory

• OpenMP is the appropriate scientific programming model

• programming for distributed memory machines involves multiple copies of the same program running on each node of the cluster, called tasks

• each program issues one instruction stream to operate on data in the local memory

• data is transferred between tasks via message passing

• the appropriate scientific programming model is MPI

• modern super computers are clusters of SMP nodes

• in some cases, it is appropriate to use a hybrid MP/MPI programming model