从 MPICH 切换到 OpenMPI
Switch from MPICH to OpenMPI
我的 Ubuntu 20.04.
中有 mpich 和 openmpi
$ dpkg -l | grep mpi | grep lib
...
ii libmpich-dev:amd64 3.3.2-2build1 amd64 Development files for MPICH
ii libmpich12:amd64 3.3.2-2build1 amd64 Shared libraries for MPICH
...
ii libopenmpi-dev:amd64 4.0.3-0ubuntu1 amd64 high performance message passing library -- header files
ii libopenmpi3:amd64 4.0.3-0ubuntu1 amd64 high performance message passing library -- shared library
...
ii openmpi-bin 4.0.3-0ubuntu1 amd64 high performance message passing library -- binaries
ii openmpi-common 4.0.3-0ubuntu1 all high performance message passing library -- common files
..
$ dpkg -l | grep mpich
...
ii mpich 3.3.2-2build1 amd64 Implementation of the MPI Message Passing Interface standard
默认的（可能是后来安装的）好像是mpich。
怎么改成openmpi？
我想确保需要更改的所有内容都是真实的。
到目前为止，我正在考虑 headers、可执行文件、库。
我不知道哪些是所有目录，链接等，必须改变。
比如建议cmake -DMPI_CC_COMPILER=/.../mpicc。
评论中提到它有效。但是：
我不确定它是否真的修复了所有问题headers，等等
我需要一个方法：
2.1。适用于系统中的所有用户
2.2。不需要那些宏
2.3。也适用于 cmake
以外的其他编译方法
至于2.3，我现在尝试配置petsc
$ ./configure --with-cc=mpicc --with-fc=mpif90 -with-cxx=mpicxx --with- make-np=10 --with-shared-libraries --download-f2cblaslapack --download-mumps --download-scalapack --with-debugging=0 COPTFLAGS="-O -O3 -march=native -mtune=native" FOPTF LAGS="-O -O3 -march=native -mtune=native" CXXOPTFLAGS="-O -O3 -march=native -mtune=native"
我得到了
Your libraries are from MPICH but it appears your mpiexec is from OpenMPI
可以用 update-alternatives 解决这个问题吗？
我发现了这个，这让我觉得它可以，但在我的系统中它没有正确配置：
$ type mpiexec
mpiexec is hashed (/usr/bin/mpiexec)
$ ll /usr/bin/mpiexec
lrwxrwxrwx 1 root root 25 Jan 21 11:11 /usr/bin/mpiexec -> /etc/alternatives/mpiexec
$ ll /etc/alternatives/mpiexec
lrwxrwxrwx 1 root root 24 Jan 21 11:11 /etc/alternatives/mpiexec -> /usr/bin/mpiexec.openmpi
$ ll /usr/bin/mpiexec.openmpi
lrwxrwxrwx 1 root root 7 Apr 15 2020 /usr/bin/mpiexec.openmpi -> orterun
$ type mpirun
mpirun is /usr/bin/mpirun
$ ll /usr/bin/mpirun
lrwxrwxrwx 1 root root 24 Jan 21 11:11 /usr/bin/mpirun -> /etc/alternatives/mpirun
$ ll /etc/alternatives/mpirun
lrwxrwxrwx 1 root root 23 Jan 21 11:11 /etc/alternatives/mpirun -> /usr/bin/mpirun.openmpi
$ ll /usr/bin/mpirun.openmpi
lrwxrwxrwx 1 root root 7 Apr 15 2020 /usr/bin/mpirun.openmpi -> orterun
$ type mpicc
mpicc is hashed (/usr/bin/mpicc)
$ ll /usr/bin/mpicc
lrwxrwxrwx 1 root root 21 Feb 25 18:54 /usr/bin/mpicc -> /etc/alternatives/mpi
$ ll /etc/alternatives/mpi
lrwxrwxrwx 1 root root 20 Feb 25 18:54 /etc/alternatives/mpi -> /usr/bin/mpicc.mpich
相关
- https://unix.stackexchange.com/questions/413099/flip-between-openmpi-and-mpich-as-default-using-linux-terminal
- 这个？ https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=896189
- 这个？ https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=912437
- https://unix.stackexchange.com/questions/81992/better-way-to-add-alternative-using-update-alternatives
- https://askubuntu.com/questions/964600/how-to-add-slave-to-existing-update-alternatives-link-group
似乎所有 alternatives，除了一个（link 组 mpi），都已经为 openmpi
$ update-alternatives --get-selections | grep mpi
h5pcc auto /usr/bin/h5pcc.openmpi
mpi auto /usr/bin/mpicc.mpich
mpi-x86_64-linux-gnu auto /usr/lib/x86_64-linux-gnu/openmpi/include
mpirun auto /usr/bin/mpirun.openmpi
正确设置 link 组 mpi（并避免容易出错的个人 link 操作）
sudo apt-get install --reinstall openmpi-bin
（拥有 mpicc.openmpi 的包）。
这显然解决了一切。
到目前为止一切正常。
“历史”注释：
我发现（奇怪的是）mpicc.openmpi 不在 update-alternatives 中，而不是其他 3 个 link 组
$ update-alternatives --list mpirun
/usr/bin/mpirun.mpich
/usr/bin/mpirun.openmpi
$ update-alternatives --list h5pcc
/usr/bin/h5pcc.mpich
/usr/bin/h5pcc.openmpi
$ update-alternatives --list mpi-x86_64-linux-gnu
/usr/include/x86_64-linux-gnu/mpich
/usr/lib/x86_64-linux-gnu/openmpi/include
$ update-alternatives --list mpi
/usr/bin/mpicc.mpich
即使它安装在我的系统中
$ ll /usr/bin/mpicc*
lrwxrwxrwx 1 root root 21 Feb 25 18:54 /usr/bin/mpicc -> /etc/alternatives/mpi
-rwxr-xr-x 1 root root 11K Mar 22 2020 /usr/bin/mpicc.mpich
lrwxrwxrwx 1 root root 12 Apr 15 2020 /usr/bin/mpicc.openmpi -> opal_wrapper
为什么不放在第一位呢？我还是不知道。
我决定使用 reinstall，因为手动处理 link 组可能会很麻烦
$ update-alternatives --query mpi
Name: mpi
Link: /usr/bin/mpicc
Slaves:
mpiCC /usr/bin/mpiCC
mpiCC.1.gz /usr/share/man/man1/mpiCC.1.gz
mpic++ /usr/bin/mpic++
mpic++.1.gz /usr/share/man/man1/mpic++.1.gz
mpicc.1.gz /usr/share/man/man1/mpicc.1.gz
mpicxx /usr/bin/mpicxx
mpicxx.1.gz /usr/share/man/man1/mpicxx.1.gz
mpif77 /usr/bin/mpif77
mpif77.1.gz /usr/share/man/man1/mpif77.1.gz
mpif90 /usr/bin/mpif90
mpif90.1.gz /usr/share/man/man1/mpif90.1.gz
mpifort /usr/bin/mpifort
mpifort.1.gz /usr/share/man/man1/mpifort.1.gz
Status: auto
Best: /usr/bin/mpicc.mpich
Value: /usr/bin/mpicc.mpich
Alternative: /usr/bin/mpicc.mpich
Priority: 40
Slaves:
mpiCC /usr/bin/mpicxx.mpich
mpiCC.1.gz /usr/share/man/man1/mpicxx.mpich.1.gz
mpic++ /usr/bin/mpicxx.mpich
mpic++.1.gz /usr/share/man/man1/mpicxx.mpich.1.gz
mpicc.1.gz /usr/share/man/man1/mpicc.mpich.1.gz
mpicxx /usr/bin/mpicxx.mpich
mpicxx.1.gz /usr/share/man/man1/mpicxx.mpich.1.gz
mpif77 /usr/bin/mpifort.mpich
mpif77.1.gz /usr/share/man/man1/mpif77.mpich.1.gz
mpif90 /usr/bin/mpifort.mpich
mpif90.1.gz /usr/share/man/man1/mpif90.mpich.1.gz
mpifort /usr/bin/mpifort.mpich
mpifort.1.gz /usr/share/man/man1/mpifort.mpich.1.gz
我的 Ubuntu 20.04.
中有 mpich 和 openmpi$ dpkg -l | grep mpi | grep lib
...
ii libmpich-dev:amd64 3.3.2-2build1 amd64 Development files for MPICH
ii libmpich12:amd64 3.3.2-2build1 amd64 Shared libraries for MPICH
...
ii libopenmpi-dev:amd64 4.0.3-0ubuntu1 amd64 high performance message passing library -- header files
ii libopenmpi3:amd64 4.0.3-0ubuntu1 amd64 high performance message passing library -- shared library
...
ii openmpi-bin 4.0.3-0ubuntu1 amd64 high performance message passing library -- binaries
ii openmpi-common 4.0.3-0ubuntu1 all high performance message passing library -- common files
..
$ dpkg -l | grep mpich
...
ii mpich 3.3.2-2build1 amd64 Implementation of the MPI Message Passing Interface standard
默认的（可能是后来安装的）好像是mpich。
怎么改成openmpi？
我想确保需要更改的所有内容都是真实的。 到目前为止，我正在考虑 headers、可执行文件、库。 我不知道哪些是所有目录，链接等，必须改变。
比如cmake -DMPI_CC_COMPILER=/.../mpicc。
评论中提到它有效。但是：
我不确定它是否真的修复了所有问题headers，等等
我需要一个方法：
2.1。适用于系统中的所有用户
2.2。不需要那些宏
2.3。也适用于
以外的其他编译方法cmake
至于2.3，我现在尝试配置petsc
$ ./configure --with-cc=mpicc --with-fc=mpif90 -with-cxx=mpicxx --with- make-np=10 --with-shared-libraries --download-f2cblaslapack --download-mumps --download-scalapack --with-debugging=0 COPTFLAGS="-O -O3 -march=native -mtune=native" FOPTF LAGS="-O -O3 -march=native -mtune=native" CXXOPTFLAGS="-O -O3 -march=native -mtune=native"
我得到了
Your libraries are from MPICH but it appears your mpiexec is from OpenMPI
可以用 update-alternatives 解决这个问题吗？
我发现了这个，这让我觉得它可以，但在我的系统中它没有正确配置：
$ type mpiexec
mpiexec is hashed (/usr/bin/mpiexec)
$ ll /usr/bin/mpiexec
lrwxrwxrwx 1 root root 25 Jan 21 11:11 /usr/bin/mpiexec -> /etc/alternatives/mpiexec
$ ll /etc/alternatives/mpiexec
lrwxrwxrwx 1 root root 24 Jan 21 11:11 /etc/alternatives/mpiexec -> /usr/bin/mpiexec.openmpi
$ ll /usr/bin/mpiexec.openmpi
lrwxrwxrwx 1 root root 7 Apr 15 2020 /usr/bin/mpiexec.openmpi -> orterun
$ type mpirun
mpirun is /usr/bin/mpirun
$ ll /usr/bin/mpirun
lrwxrwxrwx 1 root root 24 Jan 21 11:11 /usr/bin/mpirun -> /etc/alternatives/mpirun
$ ll /etc/alternatives/mpirun
lrwxrwxrwx 1 root root 23 Jan 21 11:11 /etc/alternatives/mpirun -> /usr/bin/mpirun.openmpi
$ ll /usr/bin/mpirun.openmpi
lrwxrwxrwx 1 root root 7 Apr 15 2020 /usr/bin/mpirun.openmpi -> orterun
$ type mpicc
mpicc is hashed (/usr/bin/mpicc)
$ ll /usr/bin/mpicc
lrwxrwxrwx 1 root root 21 Feb 25 18:54 /usr/bin/mpicc -> /etc/alternatives/mpi
$ ll /etc/alternatives/mpi
lrwxrwxrwx 1 root root 20 Feb 25 18:54 /etc/alternatives/mpi -> /usr/bin/mpicc.mpich
相关
- https://unix.stackexchange.com/questions/413099/flip-between-openmpi-and-mpich-as-default-using-linux-terminal
- 这个？ https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=896189
- 这个？ https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=912437
- https://unix.stackexchange.com/questions/81992/better-way-to-add-alternative-using-update-alternatives
- https://askubuntu.com/questions/964600/how-to-add-slave-to-existing-update-alternatives-link-group
似乎所有 alternatives，除了一个（link 组 mpi），都已经为 openmpi
$ update-alternatives --get-selections | grep mpi
h5pcc auto /usr/bin/h5pcc.openmpi
mpi auto /usr/bin/mpicc.mpich
mpi-x86_64-linux-gnu auto /usr/lib/x86_64-linux-gnu/openmpi/include
mpirun auto /usr/bin/mpirun.openmpi
正确设置 link 组 mpi（并避免容易出错的个人 link 操作）
sudo apt-get install --reinstall openmpi-bin
（拥有 mpicc.openmpi 的包）。
这显然解决了一切。
到目前为止一切正常。
“历史”注释：
我发现（奇怪的是）mpicc.openmpi 不在 update-alternatives 中，而不是其他 3 个 link 组
$ update-alternatives --list mpirun
/usr/bin/mpirun.mpich
/usr/bin/mpirun.openmpi
$ update-alternatives --list h5pcc
/usr/bin/h5pcc.mpich
/usr/bin/h5pcc.openmpi
$ update-alternatives --list mpi-x86_64-linux-gnu
/usr/include/x86_64-linux-gnu/mpich
/usr/lib/x86_64-linux-gnu/openmpi/include
$ update-alternatives --list mpi
/usr/bin/mpicc.mpich
即使它安装在我的系统中
$ ll /usr/bin/mpicc*
lrwxrwxrwx 1 root root 21 Feb 25 18:54 /usr/bin/mpicc -> /etc/alternatives/mpi
-rwxr-xr-x 1 root root 11K Mar 22 2020 /usr/bin/mpicc.mpich
lrwxrwxrwx 1 root root 12 Apr 15 2020 /usr/bin/mpicc.openmpi -> opal_wrapper
为什么不放在第一位呢？我还是不知道。
我决定使用 reinstall，因为手动处理 link 组可能会很麻烦
$ update-alternatives --query mpi
Name: mpi
Link: /usr/bin/mpicc
Slaves:
mpiCC /usr/bin/mpiCC
mpiCC.1.gz /usr/share/man/man1/mpiCC.1.gz
mpic++ /usr/bin/mpic++
mpic++.1.gz /usr/share/man/man1/mpic++.1.gz
mpicc.1.gz /usr/share/man/man1/mpicc.1.gz
mpicxx /usr/bin/mpicxx
mpicxx.1.gz /usr/share/man/man1/mpicxx.1.gz
mpif77 /usr/bin/mpif77
mpif77.1.gz /usr/share/man/man1/mpif77.1.gz
mpif90 /usr/bin/mpif90
mpif90.1.gz /usr/share/man/man1/mpif90.1.gz
mpifort /usr/bin/mpifort
mpifort.1.gz /usr/share/man/man1/mpifort.1.gz
Status: auto
Best: /usr/bin/mpicc.mpich
Value: /usr/bin/mpicc.mpich
Alternative: /usr/bin/mpicc.mpich
Priority: 40
Slaves:
mpiCC /usr/bin/mpicxx.mpich
mpiCC.1.gz /usr/share/man/man1/mpicxx.mpich.1.gz
mpic++ /usr/bin/mpicxx.mpich
mpic++.1.gz /usr/share/man/man1/mpicxx.mpich.1.gz
mpicc.1.gz /usr/share/man/man1/mpicc.mpich.1.gz
mpicxx /usr/bin/mpicxx.mpich
mpicxx.1.gz /usr/share/man/man1/mpicxx.mpich.1.gz
mpif77 /usr/bin/mpifort.mpich
mpif77.1.gz /usr/share/man/man1/mpif77.mpich.1.gz
mpif90 /usr/bin/mpifort.mpich
mpif90.1.gz /usr/share/man/man1/mpif90.mpich.1.gz
mpifort /usr/bin/mpifort.mpich
mpifort.1.gz /usr/share/man/man1/mpifort.mpich.1.gz