BLOG

简单地来理解 MPI，它是一个定义了多个原语的消息传递接口，这一接口主要被用于多进程间的通信。它的竞品包括 RPC，Distributed Shared Memory 等。关于它们的比较可以参考论文 Message Passing, Remote Procedure Calls and Distributed Shared Memory as Communication Paradigms for Distributed Systems。

MPI 的详细文档可以参考 MPI Forum，这里有 MPI 各个版本的文档（目前发布到了 3.1）。在当代的 MPI 中，接口已经有相当多。不过对我们而言最重要的只有三个部分，也对应着 MPI 文档 中的第三和第五章节：端到端通信，数据类型，和集合通信（Collective Communication）。

端到端通信部分主要实现了从一个进程到另一个进程的通信，核心功能由两个原语提供：

int MPI_Send(const void *buf, int count, MPI_Datatype datatype, int dest,
    int tag, MPI_Comm comm)

int MPI_Recv(void *buf, int count, MPI_Datatype datatype,
    int source, int tag, MPI_Comm comm, MPI_Status *status)

Message Passing Interface (MPI) is a standardized and portable message-passing standard designed to function on parallel computing architectures. The MPI standard defines the syntax and semantics of library routines that are useful to a wide range of users writing portable message-passing programs in C, C++, and Fortran. There are several open-source MPI implementations, which fostered the development of a parallel software industry, and encouraged development of portable and scalable large-scale parallel applications.

消息传递接口 （MPI） 是一种标准化的可移植消息传递标准，旨在在并行计算架构上运行。MPI 标准定义了库例程的语法和语义，这些例程对于使用 C、C++ 和 Fortran 编写可移植消息传递程序的广大用户非常有用。有几个开源的MPI实现，它们促进了并行软件行业的发展，并鼓励了可移植和可扩展的大规模并行应用程序的开发。

The message passing interface effort began in the summer of 1991 when a small group of researchers started discussions at a mountain retreat in Austria. Out of that discussion came a Workshop on Standards for Message Passing in a Distributed Memory Environment, held on April 29–30, 1992 in Williamsburg, Virginia. Attendees at Williamsburg discussed the basic features essential to a standard message-passing interface and established a working group to continue the standardization process. Jack Dongarra, Tony Hey, and David W. Walker put forward a preliminary draft proposal, "MPI1", in November 1992. In November 1992 a meeting of the MPI working group took place in Minneapolis and decided to place the standardization process on a more formal footing. The MPI working group met every 6 weeks throughout the first 9 months of 1993. The draft MPI standard was presented at the Supercomputing '93 conference in November 1993. After a period of public comments, which resulted in some changes in MPI, version 1.0 of MPI was released in June 1994. These meetings and the email discussion together constituted the MPI Forum, membership of which has been open to all members of the high-performance-computing community.

消息传递接口工作始于1991年夏天，当时一小群研究人员在奥地利的一个山区度假胜地开始讨论。讨论结束后，于 1992 年 4 月 29 日至 30 日在弗吉尼亚州威廉斯堡举行了分布式内存环境中消息传递标准研讨会。威廉斯堡的与会者讨论了标准消息传递接口的基本功能，并成立了一个工作组来继续标准化过程。Jack Dongarra，Tony Hey和David W. Walker于1992年11月提出了“MPI1”的初步提案草案。1992年11月，MPI工作组在明尼阿波利斯举行了一次会议，决定将标准化过程置于更正式的基础上。MPI工作组在1993年前9个月每6周举行一次会议。MPI 标准草案于 1993 年 11 月在超级计算 '93 会议上提出。经过一段时间的公众意见，MPI发生了一些变化，MPI的1.0版本于1994年6月发布。这些会议和电子邮件讨论共同构成了MPI论坛，其成员已向高性能计算社区的所有成员开放。

The MPI effort involved about 80 people from 40 organizations, mainly in the United States and Europe. Most of the major vendors of concurrent computers were involved in the MPI effort, collaborating with researchers from universities, government laboratories, and industry.

MPI的工作涉及来自40个组织的约80人，主要在美国和欧洲。大多数并发计算机的主要供应商都参与了MPI的工作，与来自大学，政府实验室和工业界的研究人员合作。

MPI provides parallel hardware vendors with a clearly defined base set of routines that can be efficiently implemented. As a result, hardware vendors can build upon this collection of standard low-level routines to create higher-level routines for the distributed-memory communication environment supplied with their parallel machines. MPI provides a simple-to-use portable interface for the basic user, yet one powerful enough to allow programmers to use the high-performance message passing operations available on advanced machines.

MPI 为并行硬件供应商提供了一组明确定义的基本例程，这些例程可以有效实现。因此，硬件供应商可以基于此标准低级例程集合进行构建，为并行计算机提供的分布式内存通信环境创建更高级别例程。MPI 为基本用户提供了一个简单易用的可移植界面，但功能强大到足以让程序员使用高级机器上可用的高性能消息传递操作。

In an effort to create a universal standard for message passing, researchers did not base it off of a single system but it incorporated the most useful features of several systems, including those designed by IBM, Intel, nCUBE, PVM, Express, P4 and PARMACS. The message-passing paradigm is attractive because of wide portability and can be used in communication for distributed-memory and shared-memory multiprocessors, networks of workstations, and a combination of these elements. The paradigm can apply in multiple settings, independent of network speed or memory architecture.

为了创建消息传递的通用标准，研究人员并没有将其建立在单个系统的基础上，而是结合了多个系统最有用的功能，包括由IBM，Intel，nCUBE，PVM，Express，P4和PARMACS设计的功能。消息传递范例具有吸引力，因为它具有广泛的可移植性，可用于分布式内存和共享内存多处理器、工作站网络以及这些元素的组合的通信。该范例可以应用于多种设置，与网络速度或内存架构无关。

Support for MPI meetings came in part from DARPA and from the U.S. National Science Foundation (NSF) under grant ASC-9310330, NSF Science and Technology Center Cooperative agreement number CCR-8809615, and from the European Commission through Esprit Project P6643. The University of Tennessee also made financial contributions to the MPI Forum.

对MPI会议的支持部分来自DARPA和美国国家科学基金会（NSF）的ASC-9310330赠款，NSF科学和技术中心合作协议编号CCR-8809615，以及欧盟委员会通过Esprit项目P6643。田纳西大学也为MPI论坛提供了财政捐助。

MPI is a communication protocol for programming parallel computers. Both point-to-point and collective communication are supported. MPI "is a message-passing application programmer interface, together with protocol and semantic specifications for how its features must behave in any implementation." MPI's goals are high performance, scalability, and portability. MPI remains the dominant model used in high-performance computing today.

MPI 是一种用于对并行计算机进行编程的通信协议。支持点对点和集体通信。MPI“是一个消息传递应用程序编程器接口，以及其功能在任何实现中必须如何表现的协议和语义规范。MPI 的目标是高性能、可扩展性和可移植性。MPI 仍然是当今高性能计算中使用的主导模型。

MPI is not sanctioned by any major standards body; nevertheless, it has become a de facto standard for communication among processes that model a parallel program running on a distributed memory system. Actual distributed memory supercomputers such as computer clusters often run such programs.

MPI不受任何主要标准机构的批准;尽管如此，它已成为模拟在分布式内存系统上运行的并行程序的进程之间通信的事实标准。实际的分布式内存超级计算机（如计算机集群）经常运行此类程序。

The principal MPI-1 model has no shared memory concept, and MPI-2 has only a limited distributed shared memory concept. Nonetheless, MPI programs are regularly run on shared memory computers, and both MPICH and Open MPI can use shared memory for message transfer if it is available. Designing programs around the MPI model (contrary to explicit shared memory models) has advantages when running on NUMA architectures since MPI encourages memory locality. Explicit shared memory programming was introduced in MPI-3.

主 MPI-1 模型没有共享内存概念，MPI-2 只有有限的分布式共享内存概念。尽管如此，MPI 程序定期在共享内存计算机上运行，MPICH 和 Open MPI 都可以使用共享内存进行消息传输（如果可用）。在 NUMA 体系结构上运行时，围绕 MPI 模型设计程序（与显式共享内存模型相反）具有优势，因为 MPI 鼓励内存局部性。显式共享内存编程在 MPI-3 中引入。

Although MPI belongs in layers 5 and higher of the OSI Reference Model, implementations may cover most layers, with sockets and Transmission Control Protocol (TCP) used in the transport layer.

尽管 MPI 属于 OSI 参考模型的第 5 层及更高层，但实现可能涵盖大多数层，在传输层中使用套接字和传输控制协议 （TCP）。

Most MPI implementations consist of a specific set of routines directly callable from C, C++, Fortran (i.e., an API) and any language able to interface with such libraries, including C#, Java or Python. The advantages of MPI over older message passing libraries are portability (because MPI has been implemented for almost every distributed memory architecture) and speed (because each implementation is in principle optimized for the hardware on which it runs).

大多数 MPI 实现由一组特定的例程组成，这些例程可以直接从 C、C++、Fortran（即 API）和任何能够与此类库交互的语言（包括 C#、Java 或 Python）调用。与旧的消息传递库相比，MPI 的优势在于可移植性（因为 MPI 已针对几乎所有分布式内存体系结构实现）和速度（因为每个实现原则上都针对其运行的硬件进行了优化）。

MPI uses Language Independent Specifications (LIS) for calls and language bindings. The first MPI standard specified ANSI C and Fortran-77 bindings together with the LIS. The draft was presented at Supercomputing 1994 (November 1994) and finalized soon thereafter. About 128 functions constitute the MPI-1.3 standard which was released as the final end of the MPI-1 series in 2008.

MPI 使用语言独立规范 （LIS） 进行调用和语言绑定。第一个 MPI 标准指定了 ANSI C 和 Fortran-77 与 LIS 的绑定。该草案在1994年超级计算大会（1994年11月）上提出，此后不久就定稿了。大约128个功能构成了MPI-1.3标准，该标准于2008年作为MPI-1系列的最终版本发布。

At present, the standard has several versions: version 1.3 (commonly abbreviated MPI-1), which emphasizes message passing and has a static runtime environment, MPI-2.2 (MPI-2), which includes new features such as parallel I/O, dynamic process management and remote memory operations, and MPI-3.1 (MPI-3), which includes extensions to the collective operations with non-blocking versions and extensions to the one-sided operations. MPI-2's LIS specifies over 500 functions and provides language bindings for ISO C, ISO C++, and Fortran 90. Object interoperability was also added to allow easier mixed-language message passing programming. A side-effect of standardizing MPI-2, completed in 1996, was clarifying the MPI-1 standard, creating the MPI-1.2.

目前，该标准有几个版本：版本1.3（通常缩写为MPI-1），强调消息传递并具有静态运行时环境，MPI-2.2（MPI-2），包括并行I / O，动态进程管理和远程内存操作等新功能，以及MPI-3.1（MPI-3），其中包括对具有非阻塞版本的集体操作的扩展以及对单侧操作的扩展。MPI-2 的 LIS 指定了 500 多个函数，并为 ISO C、ISO C++ 和 Fortran 90 提供了语言绑定。还添加了对象互操作性，以允许更轻松地进行混合语言消息传递编程。1996年完成的MPI-2标准化的一个副作用是澄清了MPI-1标准，创建了MPI-1.2。

MPI-2 is mostly a superset of MPI-1, although some functions have been deprecated. MPI-1.3 programs still work under MPI implementations compliant with the MPI-2 standard.

MPI-2 主要是 MPI-1 的超集，尽管某些函数已被弃用。MPI-1.3 程序仍然可以在符合 MPI-2 标准的 MPI 实现下工作。

MPI-3 includes new Fortran 2008 bindings, while it removes deprecated C++ bindings as well as many deprecated routines and MPI objects.

MPI-3 包括新的 Fortran 2008 绑定，同时删除了已弃用的C++绑定以及许多已弃用的例程和 MPI 对象。

MPI is often compared with Parallel Virtual Machine (PVM), which is a popular distributed environment and message passing system developed in 1989, and which was one of the systems that motivated the need for standard parallel message passing. Threaded shared memory programming models (such as Pthreads and OpenMP) and message passing programming (MPI/PVM) can be considered complementary and have been used together on occasion in, for example, servers with multiple large shared-memory nodes.

MPI经常与并行虚拟机（PVM）进行比较，后者是1989年开发的流行的分布式环境和消息传递系统，也是激发标准并行消息传递需求的系统之一。线程共享内存编程模型（如 Pthreads 和 OpenMP）和消息传递编程 （MPI/PVM） 可以被认为是互补的，并且有时在具有多个大型共享内存节点的服务器中一起使用。

The MPI interface is meant to provide essential virtual topology, synchronization, and communication functionality between a set of processes (that have been mapped to nodes/servers/computer instances) in a language-independent way, with language-specific syntax (bindings), plus a few language-specific features. MPI programs always work with processes, but programmers commonly refer to the processes as processors. Typically, for maximum performance, each CPU (or core in a multi-core machine) will be assigned just a single process. This assignment happens at runtime through the agent that starts the MPI program, normally called mpirun or mpiexec.

MPI 接口旨在以与语言无关的方式在一组进程（已映射到节点/服务器/计算机实例）之间提供基本的虚拟拓扑、同步和通信功能，具有特定于语言的语法（绑定）以及一些特定于语言的功能。MPI 程序始终与进程一起工作，但程序员通常将进程称为处理器。通常，为了获得最佳性能，每个 CPU（或多核计算机中的内核）将只分配一个进程。此分配在运行时通过启动 MPI 程序的代理（通常称为 mpirun 或 mpiexec）进行。

MPI library functions include, but are not limited to, point-to-point rendezvous-type send/receive operations, choosing between a Cartesian or graph-like logical process topology, exchanging data between process pairs (send/receive operations), combining partial results of computations (gather and reduce operations), synchronizing nodes (barrier operation) as well as obtaining network-related information such as the number of processes in the computing session, current processor identity that a process is mapped to, neighboring processes accessible in a logical topology, and so on. Point-to-point operations come in synchronous, asynchronous, buffered, and ready forms, to allow both relatively stronger and weaker semantics for the synchronization aspects of a rendezvous-send. Many outstanding[clarification needed] operations are possible in asynchronous mode, in most implementations.

MPI 库功能包括但不限于点对点会合型发送/接收操作，在笛卡尔或类似图形的逻辑过程拓扑之间进行选择，在进程对之间交换数据（发送/接收操作），组合部分计算结果（收集和减少操作），同步节点（屏障操作）以及获取与网络相关的信息，例如计算会话中的进程数， 进程映射到的当前处理器标识、逻辑拓扑中可访问的相邻进程等。点对点操作以同步、异步、缓冲和就绪形式出现，以便为会合发送的同步方面提供相对较强和较弱的语义。在大多数实现中，在异步模式下可以执行许多出色的 [clarification needed] 操作。

MPI-1 and MPI-2 both enable implementations that overlap communication and computation, but practice and theory differ. MPI also specifies thread safe interfaces, which have cohesion and coupling strategies that help avoid hidden state within the interface. It is relatively easy to write multithreaded point-to-point MPI code, and some implementations support such code. Multithreaded collective communication is best accomplished with multiple copies of Communicators, as described below.

MPI-1 和 MPI-2 都支持与通信和计算重叠的实现，但实践和理论不同。MPI 还指定线程安全接口，这些接口具有内聚和耦合策略，可帮助避免接口中的隐藏状态。编写多线程点对点 MPI 代码相对容易，并且某些实现支持此类代码。多线程集体通信最好通过多个通信器副本来完成，如下所述。

MPI provides several features. The following concepts provide context for all of those abilities and help the programmer to decide what functionality to use in their application programs. Four of MPI's eight basic concepts are unique to MPI-2.

MPI 提供了多种功能。以下概念为所有这些功能提供了上下文，并帮助程序员决定在其应用程序中使用哪些功能。MPI的八个基本概念中有四个是MPI-2独有的。

Communicator objects connect groups of processes in the MPI session. Each communicator gives each contained process an independent identifier and arranges its contained processes in an ordered topology. MPI also has explicit groups, but these are mainly good for organizing and reorganizing groups of processes before another communicator is made. MPI understands single group intracommunicator operations, and bilateral intercommunicator communication. In MPI-1, single group operations are most prevalent. Bilateral operations mostly appear in MPI-2 where they include collective communication and dynamic in-process management.

通信器对象连接 MPI 会话中的进程组。每个通信器为每个包含的进程提供一个独立的标识符，并在有序拓扑中排列其包含的进程。MPI 也有显式组，但这些组主要用于在创建另一个通信器之前组织和重组进程组。MPI 了解单组内部通信器操作和双边通信器间通信。在 MPI-1 中，单组操作最为普遍。双边行动主要出现在MPI-2中，其中包括集体沟通和动态进程内管理。

Communicators can be partitioned using several MPI commands. These commands include MPI_COMM_SPLIT, where each process joins one of several colored sub-communicators by declaring itself to have that color.

可以使用多个 MPI 命令对通信器进行分区。这些命令包括 MPI_COMM_SPLIT ，其中每个进程通过声明自己具有该颜色来加入多个彩色子通信器之一。

A number of important MPI functions involve communication between two specific processes. A popular example is MPI_Send, which allows one specified process to send a message to a second specified process. Point-to-point operations, as these are called, are particularly useful in patterned or irregular communication, for example, a data-parallel architecture in which each processor routinely swaps regions of data with specific other processors between calculation steps, or a master–slave architecture in which the master sends new task data to a slave whenever the prior task is completed.

许多重要的 MPI 函数涉及两个特定进程之间的通信。一个流行的例子是 MPI_Send ，它允许一个指定的进程向另一个指定的进程发送消息。点对点操作在模式化或不规则通信中特别有用，例如，数据并行架构，其中每个处理器在计算步骤之间定期与特定的其他处理器交换数据区域，或者主从架构，其中主从架构在前一个任务完成时向从站发送新的任务数据。

MPI-1 specifies mechanisms for both blocking and non-blocking point-to-point communication mechanisms, as well as the so-called 'ready-send' mechanism whereby a send request can be made only when the matching receive request has already been made.

MPI-1 指定了阻塞和非阻塞点对点通信机制的机制，以及所谓的“就绪发送”机制，该机制仅在已经发出匹配的接收请求时才能发出发送请求。

Collective functions involve communication among all processes in a process group (which can mean the entire process pool or a program-defined subset). A typical function is the MPI_Bcast call (short for "broadcast"). This function takes data from one node and sends it to all processes in the process group. A reverse operation is the MPI_Reduce call, which takes data from all processes in a group, performs an operation (such as summing), and stores the results on one node. MPI_Reduce is often useful at the start or end of a large distributed calculation, where each processor operates on a part of the data and then combines it into a result.

集合函数涉及流程组（可以表示整个流程池或程序定义的子集）中所有流程之间的通信。一个典型的函数是 MPI_Bcast 调用（“广播”的缩写）。此函数从一个节点获取数据，并将其发送到进程组中的所有进程。反向操作是一种 MPI_Reduce 调用，它从组中的所有进程获取数据，执行操作（例如求和），并将结果存储在一个节点上。 MPI_Reduce 通常在大型分布式计算的开始或结束时很有用，其中每个处理器对数据的一部分进行操作，然后将其组合成一个结果。

Other operations perform more sophisticated tasks, such as MPI_Alltoall which rearranges n items of data such that the nth node gets the nth item of data from each.

其他操作执行更复杂的任务，例如 MPI_Alltoall 重新排列 n 项数据，以便第 n 个节点从每个数据项获取第 n 项数据。

Many MPI functions require that you specify the type of data which is sent between processes. This is because MPI aims to support heterogeneous environments where types might be represented differently on the different nodes (for example they might be running different CPU architectures that have different endianness), in which case MPI implementations can perform data conversion. Since the C language does not allow a type itself to be passed as a parameter, MPI predefines the constants MPI_INT, MPI_CHAR, MPI_DOUBLE to correspond with int, char, double, etc.

许多 MPI 函数要求您指定在进程之间发送的数据类型。这是因为 MPI 旨在支持异构环境，其中类型可能在不同的节点上以不同的方式表示（例如，它们可能运行具有不同字节序的不同 CPU 体系结构），在这种情况下，MPI 实现可以执行数据转换。由于 C 语言不允许类型本身作为参数传递，因此 MPI 预定义了常量 、 以 MPI_DOUBLE 对应于 int MPI_INT 、 MPI_CHAR char 、 double 等。

Here is an example in C that passes arrays of ints from all processes to one. The one receiving process is called the "root" process, and it can be any designated process but normally it will be process 0. All the processes ask to send their arrays to the root with MPI_Gather, which is equivalent to having each process (including the root itself) call MPI_Send and the root make the corresponding number of ordered MPI_Recv calls to assemble all of these arrays into a larger one:

下面是 C 中的一个示例，它将 s 数组 int 从所有进程传递到一个进程。一个接收进程称为“根”进程，它可以是任何指定的进程，但通常它将是进程 0。所有进程都要求使用 将其 MPI_Gather 数组发送到根，这相当于让每个进程（包括根本身）调用，根进行相应数量的有序 MPI_Recv 调用 MPI_Send ，以将所有这些数组组装成一个更大的数组：

int send_array[100];
int root = 0; /* or whatever */
int num_procs, *recv_array;
MPI_Comm_size(comm, &num_procs);
recv_array = malloc(num_procs * sizeof(send_array));
MPI_Gather(send_array, sizeof(send_array) / sizeof(*send_array), MPI_INT,
           recv_array, sizeof(send_array) / sizeof(*send_array), MPI_INT,
           root, comm);

However, you may instead wish to send data as one block as opposed to 100 ints. To do this define a "contiguous block" derived data type:

但是，您可能希望将数据作为一个块而不是 100 int 秒发送。为此，请定义一个“连续块”派生数据类型：

MPI_Datatype newtype;
MPI_Type_contiguous(100, MPI_INT, &newtype);
MPI_Type_commit(&newtype);
MPI_Gather(array, 1, newtype, receive_array, 1, newtype, root, comm);

For passing a class or a data structure, MPI_Type_create_struct creates an MPI derived data type from MPI_predefined data types, as follows:

对于传递类或数据结构， MPI_Type_create_struct 从数据类型创建 MPI_predefined MPI 派生数据类型，如下所示：

int MPI_Type_create_struct(int count,
                           int *blocklen,
                           MPI_Aint *disp,
                           MPI_Datatype *type,
                           MPI_Datatype *newtype)

where: 哪里：

• count is a number of blocks, and specifies the length (in elements) of the arrays blocklen, disp, and type.

  count 是块的数量，并指定数组 blocklen 、 disp 和 type 的长度（以元素为单位）。

• blocklen contains numbers of elements in each block,

  blocklen 包含每个块中的元素数量，

• disp contains byte displacements of each block,

  disp 包含每个块的字节位移，

• type contains types of element in each block.

  type 包含每个块中的元素类型。

• newtype (an output) contains the new derived type created by this function

  newtype （输出）包含此函数创建的新派生类型

The disp (displacements) array is needed for data structure alignment, since the compiler may pad the variables in a class or data structure. The safest way to find the distance between different fields is by obtaining their addresses in memory. This is done with MPI_Get_address, which is normally the same as C's & operator but that might not be true when dealing with memory segmentation.

disp （位移）数组是数据结构对齐所必需的，因为编译器可以在类或数据结构中填充变量。查找不同字段之间距离的最安全方法是在内存中获取它们的地址。这是用 完成 MPI_Get_address 的，它通常与 C 的 & 运算符相同，但在处理内存分割时可能并非如此。

Passing a data structure as one block is significantly faster than passing one item at a time, especially if the operation is to be repeated. This is because fixed-size blocks do not require serialization during transfer.

将数据结构作为一个块传递比一次传递一个项目要快得多，尤其是在要重复操作时。这是因为固定大小的块在传输过程中不需要序列化。

Given the following data structures:

给定以下数据结构：

struct A {
    int f;
    short p;
};

struct B {
    struct A a;
    int pp, vp;
};

Here's the C code for building an MPI-derived data type:

下面是用于构建 MPI 派生数据类型的 C 代码：

static const int blocklen[] = {1, 1, 1, 1};
static const MPI_Aint disp[] = {
    offsetof(struct B, a) + offsetof(struct A, f),
    offsetof(struct B, a) + offsetof(struct A, p),
    offsetof(struct B, pp),
    offsetof(struct B, vp)
};
static MPI_Datatype type[] = {MPI_INT, MPI_SHORT, MPI_INT, MPI_INT};
MPI_Datatype newtype;
MPI_Type_create_struct(sizeof(type) / sizeof(*type), blocklen, disp, type, &newtype);
MPI_Type_commit(&newtype);

MPI-2 defines three one-sided communications operations, MPI_Put, MPI_Get, and MPI_Accumulate, being a write to remote memory, a read from remote memory, and a reduction operation on the same memory across a number of tasks, respectively. Also defined are three different methods to synchronize this communication (global, pairwise, and remote locks) as the specification does not guarantee that these operations have taken place until a synchronization point.

MPI-2 定义了三个单侧通信操作，分别 MPI_Put MPI_Get MPI_Accumulate 是写入远程内存、从远程内存读取和跨多个任务对同一内存的缩减操作。还定义了三种不同的方法来同步此通信（全局、成对和远程锁定），因为规范不保证这些操作在同步点之前已经发生。

These types of call can often be useful for algorithms in which synchronization would be inconvenient (e.g. distributed matrix multiplication), or where it is desirable for tasks to be able to balance their load while other processors are operating on data.

这些类型的调用通常对于同步不方便的算法（例如分布式矩阵乘法）很有用，或者当其他处理器对数据进行操作时，任务需要能够平衡其负载。

The key aspect is "the ability of an MPI process to participate in the creation of new MPI processes or to establish communication with MPI processes that have been started separately." The MPI-2 specification describes three main interfaces by which MPI processes can dynamically establish communications, MPI_Comm_spawn, MPI_Comm_accept/MPI_Comm_connect and MPI_Comm_join. The MPI_Comm_spawn interface allows an MPI process to spawn a number of instances of the named MPI process. The newly spawned set of MPI processes form a new MPI_COMM_WORLD intracommunicator but can communicate with the parent and the intercommunicator the function returns. MPI_Comm_spawn_multiple is an alternate interface that allows the different instances spawned to be different binaries with different arguments.

关键方面是“MPI 进程参与创建新 MPI 进程或与单独启动的 MPI 进程建立通信的能力”。MPI-2 规范描述了三个主要接口，MPI 进程可以通过这些接口动态建立通信、 MPI_Comm_spawn MPI_Comm_accept / MPI_Comm_connect 和 MPI_Comm_join 。该 MPI_Comm_spawn 接口允许 MPI 进程生成命名 MPI 进程的多个实例。新生成的一组 MPI 进程形成了一个新的 MPI_COMM_WORLD 内部通信器，但可以与函数返回的父通信器和内部通信器进行通信。 MPI_Comm_spawn_multiple 是一个备用接口，它允许生成的不同实例是具有不同参数的不同二进制文件。

The parallel I/O feature is sometimes called MPI-IO, and refers to a set of functions designed to abstract I/O management on distributed systems to MPI, and allow files to be easily accessed in a patterned way using the existing derived datatype functionality.

并行 I/O 功能有时称为 MPI-IO，指的是一组函数，旨在将分布式系统上的 I/O 管理抽象为 MPI，并允许使用现有的派生数据类型功能以模式化方式轻松访问文件。

The little research that has been done on this feature indicates that it may not be trivial to get high performance gains by using MPI-IO. For example, an implementation of sparse matrix-vector multiplications using the MPI I/O library shows a general behavior of minor performance gain, but these results are inconclusive. It was not until the idea of collective I/O implemented into MPI-IO that MPI-IO started to reach widespread adoption. Collective I/O substantially boosts applications' I/O bandwidth by having processes collectively transform the small and noncontiguous I/O operations into large and contiguous ones, thereby reducing the locking and disk seek overhead. Due to its vast performance benefits, MPI-IO also became the underlying I/O layer for many state-of-the-art I/O libraries, such as HDF5 and Parallel NetCDF. Its popularity also triggered research on collective I/O optimizations, such as layout-aware I/O and cross-file aggregation.

对此功能所做的少量研究表明，通过使用 MPI-IO 获得高性能提升可能并非易事。例如，使用 MPI I/O 库实现稀疏矩阵向量乘法时，显示了性能略有提升的一般行为，但这些结果尚无定论。直到在MPI-IO中实现集体I/O的想法，MPI-IO才开始被广泛采用。集合 I/O 通过让进程将小型和不连续的 I/O 操作共同转换为大型和连续的 I/O 操作，从而减少锁定和磁盘寻道开销，从而显著提高应用程序的 I/O 带宽。由于其巨大的性能优势，MPI-IO也成为许多最先进的I / O库（如HDF5和并行NetCDF）的底层I / O层。它的流行也引发了对集体 I/O 优化的研究，例如布局感知 I/O 和跨文件聚合。

• The initial implementation of the MPI 1.x standard was MPICH, from Argonne National Laboratory (ANL) and Mississippi State University. IBM also was an early implementor, and most early 90s supercomputer companies either commercialized MPICH, or built their own implementation. LAM/MPI from Ohio Supercomputer Center was another early open implementation. ANL has continued developing MPICH for over a decade, and now offers MPICH-3.2, implementing the MPI-3.1 standard.

  MPI 1.x标准的最初实施是MPICH，来自阿贡国家实验室（ANL）和密西西比州立大学。IBM也是早期的实现者，大多数90年代早期的超级计算机公司要么将MPICH商业化，要么建立自己的实现。俄亥俄州超级计算机中心的LAM/MPI是另一个早期的开放实现。ANL十多年来一直在开发MPICH，现在提供MPICH-3.2，实施MPI-3.1标准。

• Open MPI (not to be confused with OpenMP) was formed by the merging FT-MPI, LA-MPI, LAM/MPI, and PACX-MPI, and is found in many TOP-500 supercomputers.

  Open MPI（不要与OpenMP混淆）是由FT-MPI，LA-MPI，LAM/MPI和PACX-MPI合并而成的，在许多TOP-500超级计算机中都可以找到。

Many other efforts are derivatives of MPICH, LAM, and other works, including, but not limited to, commercial implementations from HPE, Intel, Microsoft, and NEC.

许多其他努力是MPICH，LAM和其他作品的衍生品，包括但不限于HPE，Intel，Microsoft和NEC的商业实施。

While the specifications mandate a C and Fortran interface, the language used to implement MPI is not constrained to match the language or languages it seeks to support at runtime. Most implementations combine C, C++ and assembly language, and target C, C++, and Fortran programmers. Bindings are available for many other languages, including Perl, Python, R, Ruby, Java, and CL (see #Language bindings).

虽然规范要求使用 C 和 Fortran 接口，但用于实现 MPI 的语言不受限制，以匹配它在运行时寻求支持的语言。大多数实现结合了 C、C++ 和汇编语言，以及面向 C、C++ 和 Fortran 程序员。绑定可用于许多其他语言，包括Perl，Python，R，Ruby，Java和CL（请参阅 #Language 绑定）。

The ABI of MPI implementations are roughly split between MPICH and Open MPI derivatives, so that a library from one family works as a drop-in replacement of one from the same family, but direct replacement across families is impossible. The French CEA maintains a wrapper interface to facilitate such switches.

MPI 实现的 ABI 大致分为 MPICH 和 Open MPI 衍生产品，因此来自一个系列的库可以作为来自同一系列的库的直接替换，但跨系列直接替换是不可能的。法国CEA维护了一个包装器接口，以方便此类切换。

MPI hardware research focuses on implementing MPI directly in hardware, for example via processor-in-memory, building MPI operations into the microcircuitry of the RAM chips in each node. By implication, this approach is independent of language, operating system, and CPU, but cannot be readily updated or removed.

MPI 硬件研究侧重于直接在硬件中实现 MPI，例如通过内存中的处理器，将 MPI 操作构建到每个节点中 RAM 芯片的微电路中。这意味着，此方法独立于语言、操作系统和 CPU，但不能轻易更新或删除。

Another approach has been to add hardware acceleration to one or more parts of the operation, including hardware processing of MPI queues and using RDMA to directly transfer data between memory and the network interface controller without CPU or OS kernel intervention.

另一种方法是将硬件加速添加到操作的一个或多个部分，包括 MPI 队列的硬件处理和使用 RDMA 直接在内存和网络接口控制器之间传输数据，而无需 CPU 或操作系统内核干预。

mpicc (and similarly mpic++, mpif90, etc.) is a program that wraps over an existing compiler to set the necessary command-line flags when compiling code that uses MPI. Typically, it adds a few flags that enable the code to be the compiled and linked against the MPI library.

mpicc（以及类似的 mpic++、mpif90 等）是一个程序，它包装在现有编译器上，以便在编译使用 MPI 的代码时设置必要的命令行标志。通常，它会添加一些标志，使代码能够编译并链接到 MPI 库。

Bindings are libraries that extend MPI support to other languages by wrapping an existing MPI implementation such as MPICH or Open MPI.

绑定是通过包装现有的 MPI 实现（如 MPICH 或 Open MPI）将 MPI 支持扩展到其他语言的库。

The two managed Common Language Infrastructure .NET implementations are Pure Mpi.NET and MPI.NET, a research effort at Indiana University licensed under a BSD-style license. It is compatible with Mono, and can make full use of underlying low-latency MPI network fabrics.

两个托管的公共语言基础结构 .NET 实现是 Pure Mpi.NET 和 MPI.NET，这是印第安纳大学的一项研究工作，在 BSD 样式的许可证下获得许可。它与Mono兼容，可以充分利用底层的低延迟MPI网络结构。

Although Java does not have an official MPI binding, several groups attempt to bridge the two, with different degrees of success and compatibility. One of the first attempts was Bryan Carpenter's mpiJava, essentially a set of Java Native Interface (JNI) wrappers to a local C MPI library, resulting in a hybrid implementation with limited portability, which also has to be compiled against the specific MPI library being used.

尽管Java没有正式的MPI绑定，但有几个小组试图将两者联系起来，并取得了不同程度的成功和兼容性。最早的尝试之一是Bryan Carpenter的mpiJava，本质上是一组Java原生接口（JNI）包装器到本地C MPI库，导致混合实现具有有限的可移植性，也必须针对正在使用的特定MPI库进行编译。

However, this original project also defined the mpiJava API (a de facto MPI API for Java that closely followed the equivalent C++ bindings) which other subsequent Java MPI projects adopted. One less-used API is MPJ API, which was designed to be more object-oriented and closer to Sun Microsystems' coding conventions. Beyond the API, Java MPI libraries can be either dependent on a local MPI library, or implement the message passing functions in Java, while some like P2P-MPI also provide peer-to-peer functionality and allow mixed-platform operation.

然而，这个最初的项目也定义了mpiJava API（事实上的JavaMPI API，紧随等效的C++绑定），其他后续的Java MPI项目也采用了mpiJava API。一个较少使用的API是MPJ API，它被设计为更加面向对象，更接近Sun Microsystems的编码约定。除了 API 之外，Java MPI 库可以依赖于本地 MPI 库，也可以在 Java 中实现消息传递函数，而像 P2P-MPI 这样的库也提供点对点功能并允许混合平台操作。

Some of the most challenging parts of Java/MPI arise from Java characteristics such as the lack of explicit pointers and the linear memory address space for its objects, which make transferring multidimensional arrays and complex objects inefficient. Workarounds usually involve transferring one line at a time and/or performing explicit de-serialization and casting at both the sending and receiving ends, simulating C or Fortran-like arrays by the use of a one-dimensional array, and pointers to primitive types by the use of single-element arrays, thus resulting in programming styles quite far from Java conventions.

Java/MPI 的一些最具挑战性的部分来自 Java 特性，例如缺少显式指针和对象的线性内存地址空间，这使得传输多维数组和复杂对象效率低下。解决方法通常包括一次传输一行和/或在发送端和接收端执行显式反序列化和强制转换，使用一维数组模拟 C 或类似 Fortran 的数组，以及使用单元素数组指向基元类型的指针，从而导致编程风格与 Java 约定相去甚远。

Another Java message passing system is MPJ Express. Recent versions can be executed in cluster and multicore configurations. In the cluster configuration, it can execute parallel Java applications on clusters and clouds. Here Java sockets or specialized I/O interconnects like Myrinet can support messaging between MPJ Express processes. It can also utilize native C implementation of MPI using its native device. In the multicore configuration, a parallel Java application is executed on multicore processors. In this mode, MPJ Express processes are represented by Java threads.

另一个Java消息传递系统是MPJ Express。最新版本可以在群集和多核配置中执行。在集群配置中，它可以在集群和云上执行并行的 Java 应用程序。在这里，Java套接字或像Myrinet这样的专用I / O互连可以支持MPJ Express进程之间的消息传递。它还可以使用其本机设备利用 MPI 的本机 C 实现。在多核配置中，并行 Java 应用程序在多核处理器上执行。在这种模式下，MPJ Express 进程由 Java 线程表示。

There is a Julia language wrapper for MPI.

有一个用于 MPI 的 Julia 语言包装器。

There are a few academic implementations of MPI using MATLAB. MATLAB has its own parallel extension library implemented using MPI and PVM.

有一些使用 MATLAB 的 MPI 学术实现。MATLAB 有自己的并行扩展库，使用 MPI 和 PVM 实现。

The OCamlMPI Module implements a large subset of MPI functions and is in active use in scientific computing. An 11,000-line OCaml program was "MPI-ified" using the module, with an additional 500 lines of code and slight restructuring and ran with excellent results on up to 170 nodes in a supercomputer.OCamlMPI 模块实现了 MPI 函数的大量子集，并在科学计算中得到积极应用。使用该模块对11,000行OCaml程序进行了“MPI化”，增加了500行代码和轻微的重组，并在超级计算机的多达170个节点上运行，并取得了出色的结果。

PARI/GP can be built to use MPI as its multi-thread engine, allowing to run parallel PARI and GP programs on MPI clusters unmodified.

PARI/GP 可以构建为 MPI 作为其多线程引擎，允许在未经修改的 MPI 集群上运行并行 PARI 和 GP 程序。

MPI implementations in Python include: pyMPI, mpi4py, pypar, MYMPI, and the MPI submodule in ScientificPython. pyMPI is notable because it is a variant python interpreter, while pypar, MYMPI, and ScientificPython's module are import modules. They make it the coder's job to decide where the call to MPI_Init belongs.

Python 中的 MPI 实现包括：pyMPI、mpi4py、pypar、MYMPI 和 ScientificPython 中的 MPI 子模块。pyMPI值得注意，因为它是一个变体python解释器，而pypar，MYMPI和ScientificPython的模块是导入模块。他们让编码员的工作是决定调用 MPI_Init 属于哪里。

In 2006 the Boost C++ Libraries acquired Boost:MPI which included the MPI Python Bindings. This is of particular help for mixing C++ and Python. As of October 2016 Boost:MPI's Python bindings still have unfixed packaging bugs in CentOS.

2006年，Boost C++ Libraries收购了Boost：MPI，其中包括MPI Python Bindings。这对于混合C++和Python特别有帮助。截至 2016 年 10 月，Boost：MPI 的 Python 绑定在 CentOS 中仍有未修复的打包错误。

R bindings of MPI include Rmpi and pbdMPI, where Rmpi focuses on manager-workers parallelism while pbdMPI focuses on SPMD parallelism. Both implementations fully support Open MPI or MPICH2.

MPI 的 R 绑定包括 Rmpi 和 pbdMPI，其中 Rmpi 专注于管理器-worker 并行性，而 pbdMPI 专注于 SPMD 并行性。这两种实现都完全支持Open MPI或MPICH2。

Here is a "Hello, World!" program in MPI written in C. In this example, we send a "hello" message to each processor, manipulate it trivially, return the results to the main process, and print the messages.

这是一个用 C 语言编写的 MPI 中的“Hello， World！”程序。在此示例中，我们向每个处理器发送“hello”消息，简单操作它，将结果返回到主进程，然后打印消息。

/*
  "Hello World" MPI Test Program
*/
#include <assert.h>
#include <stdio.h>
#include <string.h>
#include <mpi.h>

int main(int argc, char **argv)
{
    char buf[256];
    int my_rank, num_procs;

    /* Initialize the infrastructure necessary for communication */
    MPI_Init(&argc, &argv);

    /* Identify this process */
    MPI_Comm_rank(MPI_COMM_WORLD, &my_rank);

    /* Find out how many total processes are active */
    MPI_Comm_size(MPI_COMM_WORLD, &num_procs);

    /* Until this point, all programs have been doing exactly the same.
       Here, we check the rank to distinguish the roles of the programs */
    if (my_rank == 0) {
        int other_rank;
        printf("We have %i processes.\n", num_procs);

        /* Send messages to all other processes */
        for (other_rank = 1; other_rank < num_procs; other_rank++)
        {
            sprintf(buf, "Hello %i!", other_rank);
            MPI_Send(buf, 256, MPI_CHAR, other_rank,
                     0, MPI_COMM_WORLD);
        }

        /* Receive messages from all other processes */
        for (other_rank = 1; other_rank < num_procs; other_rank++)
        {
            MPI_Recv(buf, 256, MPI_CHAR, other_rank,
                     0, MPI_COMM_WORLD, MPI_STATUS_IGNORE);
            printf("%s\n", buf);
        }

    } else {

        /* Receive message from process #0 */
        MPI_Recv(buf, 256, MPI_CHAR, 0,
                 0, MPI_COMM_WORLD, MPI_STATUS_IGNORE);
        assert(memcmp(buf, "Hello ", 6) == 0);

        /* Send message to process #0 */
        sprintf(buf, "Process %i reporting for duty.", my_rank);
        MPI_Send(buf, 256, MPI_CHAR, 0,
                 0, MPI_COMM_WORLD);

    }

    /* Tear down the communication infrastructure */
    MPI_Finalize();
    return 0;
}

When run with 4 processes, it should produce the following output:

当使用 4 个进程运行时，它应生成以下输出：

$ mpicc example.c && mpiexec -n 4 ./a.out
We have 4 processes.
Process 1 reporting for duty.
Process 2 reporting for duty.
Process 3 reporting for duty.

Here, mpiexec is a command used to execute the example program with 4 processes, each of which is an independent instance of the program at run time and assigned ranks (i.e. numeric IDs) 0, 1, 2, and 3. The name mpiexec is recommended by the MPI standard, although some implementations provide a similar command under the name mpirun. The MPI_COMM_WORLD is the communicator that consists of all the processes.

这里，是一个命令，用于执行具有 4 个进程的示例程序，每个进程在运行时都是程序的独立实例， mpiexec 并分配了秩（即数字 ID）0、1、2 和 3。MPI 标准建议使用该名称，尽管某些实现在名称 mpiexec mpirun 下提供了类似的命令。这是 MPI_COMM_WORLD 由所有进程组成的通信器。

A single program, multiple data (SPMD) programming model is thereby facilitated, but not required; many MPI implementations allow multiple, different, executables to be started in the same MPI job. Each process has its own rank, the total number of processes in the world, and the ability to communicate between them either with point-to-point (send/receive) communication, or by collective communication among the group. It is enough for MPI to provide an SPMD-style program with MPI_COMM_WORLD, its own rank, and the size of the world to allow algorithms to decide what to do. In more realistic situations, I/O is more carefully managed than in this example. MPI does not stipulate how standard I/O (stdin, stdout, stderr) should work on a given system. It generally works as expected on the rank-0 process, and some implementations also capture and funnel the output from other processes.

因此，单个程序，多个数据（SPMD）编程模型得到了促进，但不是必需的;许多 MPI 实现允许在同一 MPI 作业中启动多个不同的可执行文件。每个进程都有自己的等级、世界上的进程总数，以及通过点对点（发送/接收）通信或通过组之间的集体通信在它们之间进行通信的能力。对于 MPI 来说，提供一个 SPMD 风格的程序 MPI_COMM_WORLD 就足够了，它有自己的等级和世界的大小，允许算法决定做什么。在更现实的情况下，I/O 的管理比此示例更仔细。MPI 没有规定标准 I/O（stdin、stdout、stderr）应该如何在给定系统上工作。它通常在 rank-0 进程上按预期工作，并且某些实现还会捕获和汇集来自其他进程的输出。

MPI uses the notion of process rather than processor. Program copies are mapped to processors by the MPI runtime. In that sense, the parallel machine can map to one physical processor, or to N processors, where N is the number of available processors, or even something in between. For maximum parallel speedup, more physical processors are used. This example adjusts its behavior to the size of the world N, so it also seeks to scale to the runtime configuration without compilation for each size variation, although runtime decisions might vary depending on that absolute amount of concurrency available.

MPI 使用进程而不是处理器的概念。程序副本由 MPI 运行时映射到处理器。从这个意义上说，并行机可以映射到一个物理处理器，也可以映射到N个处理器，其中N是可用处理器的数量，甚至介于两者之间。为了获得最大的并行加速，使用了更多的物理处理器。此示例将其行为调整为全局 N 的大小，因此它还寻求扩展到运行时配置，而无需针对每个大小变化进行编译，尽管运行时决策可能会因可用的绝对并发量而异。

Adoption of MPI-1.2 has been universal, particularly in cluster computing, but acceptance of MPI-2.1 has been more limited. Issues include:

MPI-1.2 的采用是普遍的，特别是在集群计算中，但对 MPI-2.1 的接受度更为有限。问题包括：

1. MPI-2 implementations include I/O and dynamic process management, and the size of the middleware is substantially larger. Most sites that use batch scheduling systems cannot support dynamic process management. MPI-2's parallel I/O is well accepted.

   [citation needed]

   MPI-2 实现包括 I/O 和动态进程管理，中间件的大小要大得多。大多数使用批处理计划系统的网站都不支持动态流程管理。MPI-2 的并行 I/O 被广泛接受。 [citation needed]

2. Many MPI-1.2 programs were developed before MPI-2. Portability concerns initially slowed adoption, although wider support has lessened this.

   许多MPI-1.2程序是在MPI-2之前开发的。可移植性问题最初减缓了采用速度，尽管更广泛的支持已经减少了这种情况。

3. Many MPI-1.2 applications use only a subset of that standard (16-25 functions) with no real need for MPI-2 functionality.

   许多 MPI-1.2 应用程序仅使用该标准的子集（16-25 个函数），而不需要 MPI-2 功能。

Some aspects of the MPI's future appear solid; others less so. The MPI Forum reconvened in 2007 to clarify some MPI-2 issues and explore developments for a possible MPI-3, which resulted in versions MPI-3.0 (September 2012) and MPI-3.1 (June 2015).

MPI未来的某些方面似乎很稳固;其他人则不然。MPI论坛于2007年重新召开，以澄清MPI-2的一些问题并探讨可能的MPI-3的发展，从而产生了MPI-3.0（2012年9月）和MPI-3.1（2015年6月）的版本。

Architectures are changing, with greater internal concurrency (multi-core), better fine-grained concurrency control (threading, affinity), and more levels of memory hierarchy. Multithreaded programs can take advantage of these developments more easily than single-threaded applications. This has already yielded separate, complementary standards for symmetric multiprocessing, namely OpenMP. MPI-2 defines how standard-conforming implementations should deal with multithreaded issues, but does not require that implementations be multithreaded, or even thread-safe. MPI-3 adds the ability to use shared-memory parallelism within a node. Implementations of MPI such as Adaptive MPI, Hybrid MPI, Fine-Grained MPI, MPC and others offer extensions to the MPI standard that address different challenges in MPI.

体系结构正在发生变化，具有更大的内部并发性（多核）、更好的细粒度并发控制（线程、亲和力）和更多级别的内存层次结构。多线程程序可以比单线程应用程序更容易地利用这些开发。这已经为对称多处理产生了单独的互补标准，即OpenMP。MPI-2 定义了符合标准的实现应如何处理多线程问题，但不要求实现是多线程的，甚至不要求线程安全的实现。MPI-3 增加了在节点内使用共享内存并行性的功能。MPI 的实现，如自适应 MPI、混合 MPI、细粒度 MPI、MPC 等，提供了对 MPI 标准的扩展，解决了 MPI 中的不同挑战。

Astrophysicist Jonathan Dursi wrote an opinion piece calling MPI obsolescent, pointing to newer technologies like the Chapel language, Unified Parallel C, Hadoop, Spark and Flink. At the same time, nearly all of the projects in the Exascale Computing Project build explicitly on MPI; MPI has been shown to scale to the largest machines as of the early 2020s and is widely considered to stay relevant for a long time to come.

天体物理学家乔纳森·杜尔西（Jonathan Dursi）写了一篇评论文章，称MPI已经过时，指出了Chapel语言，统一并行C，Hadoop，Spark和Flink等新技术。同时，百万兆次级计算项目中的几乎所有项目都明确地建立在 MPI 上;截至 2020 年代初，MPI 已被证明可以扩展到最大的机器，并且被广泛认为在未来很长一段时间内保持相关性。