分布式文件系统(如 HDFS)上的 OpenMPI

OpenMPI on distributed file system like HDFS

像 HDFS(Hadoop 分布式文件系统)这样的分布式文件系统是否支持 OpenMPI?

Hadoop 不是使用 MPI 实现的,因为

  1. MPI 是一个消息传递接口,而 MapReduce 是 Google 最初为 运行 大数据应用程序开发的工具。
  2. Hadoop/Mapreduce 中可用的主要功能之一是容错。但它在 MPI 中不受支持。

参考
Why isn't Hadoop implemented using MPI?
MPI and MapReduce

让我们回答您的问题,

一篇文章说,

The prospect of running MPI jobs under YARN is an enticing proposition. As mentioned, the vast stores of data now living in HDFS can be accessed by non-MapReduce applications like those written in MPI. (Please note, however, that the intimate interface of MapReduce to HDFS is part of the MapReduce framework and is not automatic in other frameworks and must be managed by the user’s application.)

running Open MPI under YARN 的前景已由 Open MPI 团队的 Ralph H. Castain 进行了调查。

您可以参考下面link了解更多信息,

The New Hadoop