gcc/gfortran 11 在默认位置找不到 headers/libraries
gcc/gfortran 11 cannot find headers/libraries in default locations
使用自制软件将 python 3.8.6 升级到 3.9.10 后，我的 Cython 扩展不再没有明确添加 /usr/local/include（对于我的 Intel MacBook）或 /opt/homebrew/include 到 include_dirs 我的分机。
我的setup.py.in:
import os, sys
from numpy.distutils.core import setup, Extension
from Cython.Build import cythonize
link_arguments = []
extra_include_dirs = []
if (sys.platform == 'darwin'):
link_arguments.append("-Wl,-rpath")
link_arguments.append("-Wl,@loader_path/")
if os.path.exists('/opt/homebrew/'):
extra_include_dirs.append("/opt/homebrew/include/")
else:
extra_include_dirs.append("/usr/local/include/")
else:
link_arguments.append("-Wl,-rpath=${CMAKE_SOURCE_DIR}/lib/")
pynwp_extension = Extension(
name="pynwp",
sources=["${CMAKE_CURRENT_SOURCE_DIR}/lambert.f90", "${CMAKE_CURRENT_SOURCE_DIR}/pynwp.f90", "${CMAKE_CURRENT_SOURCE_DIR}/readAtmosphereGen.f90", "${CMAKE_CURRENT_SOURCE_DIR}/ptogrot.f", "${CMAKE_CURRENT_SOURCE_DIR}/bilin1.f", "${CMAKE_CURRENT_SOURCE_DIR}/fl2pres_f.f","${CMAKE_CURRENT_SOURCE_DIR}/message.c","${CMAKE_CURRENT_SOURCE_DIR}/gridWindDirCorrection.F"],
libraries=["HirlamUtils_fPIC", "eccodes_f90", "jasper"],
library_dirs=["${PROJECT_BINARY_DIR}", "${CMAKE_SOURCE_DIR}/build${CMAKE_BUILD_TYPE}/src/libHirlamUtils/", "/opt/homebrew/lib/"],
extra_link_args = link_arguments,
include_dirs=["${CMAKE_SOURCE_DIR}/include", "/usr/lib64/gfortran/modules/",
"${CMAKE_SOURCE_DIR}/build${CMAKE_BUILD_TYPE}"] + extra_include_dirs,
extra_f90_compile_args=["-DLINUX", "-DIS_LITTLE_ENDIAN", "-DUSEWALLTIME", "-DHAS_BLAS", "-DHAS_LAPACK", "-DGRIB32", "-DTIMING", "-DPREC32", "-fno-whole-file", "-g", "-fbounds-check"]
#compiler_directives={'language_level' : "3"}
)
setup(name="pynwp",
author="me",
author_email="me!me.com",
version="1.0.1",
description="Python wrapper for pynwp",
package_dir={"": "${CMAKE_CURRENT_SOURCE_DIR}"},
url="http://emaddc.eu",
license="MIT License",
ext_modules=[pynwp_extension]
)
在上面的文件中，我对自制库目录的位置（临时）进行了硬编码，并根据找到的 /opt/homebrew 为包含目录添加了一些功能。如果我从文件中删除它，编译将失败，因为找不到 eccodes.mod，当我 运行 命令由 python/CMake 手动生成时查看输出：
buildDebug git:(master) ✗ /opt/homebrew/bin/gfortran -Wall -g -fno-second-underscore -fPIC -O3 -funroll-loops -I<project_dir>//include -I/usr/lib64/gfortran/modules/ -I<project_dir>//buildDebug -Ibuild/src.macosx-12-arm64-3.9/build/src.macosx-12-arm64-3.9 -I/opt/homebrew/lib/python3.9/site-packages/numpy/core/include -Ibuild/src.macosx-12-arm64-3.9/numpy/distutils/include -I/opt/homebrew/opt/python@3.9/Frameworks/Python.framework/Versions/3.9/include/python3.9 -c -c <project_dir>//src/pynwp/readAtmosphereGen.f90 -o build/temp.macosx-12-arm64-3.9<project_dir>//src/pynwp/readAtmosphereGen.o -DLINUX -DIS_LITTLE_ENDIAN -DUSEWALLTIME -DHAS_BLAS -DHAS_LAPACK -DGRIB32 -DTIMING -DPREC32 -fno-whole-file -g -fbounds-check
f951: Warning: Nonexistent include directory '/usr/lib64/gfortran/modules/' [-Wmissing-include-dirs]
f951: Warning: Nonexistent include directory 'build/src.macosx-12-arm64-3.9/build/src.macosx-12-arm64-3.9' [-Wmissing-include-dirs]
f951: Warning: Nonexistent include directory 'build/src.macosx-12-arm64-3.9/numpy/distutils/include' [-Wmissing-include-dirs]
<project_dir>//src/pynwp/readAtmosphereGen.f90:3:7:
3 | use eccodes
| 1
Fatal Error: Cannot open module file 'eccodes.mod' for reading at (1): No such file or directory
compilation terminated.
此扩展是使用 CMake 构建的更大项目的一部分。 运行手动执行gfortran命令时，出现同样的错误提示找不到eccodes.mod。但是，该文件位于 gfortran/gcc 搜索路径的默认位置：
locate eccodes.mod
/opt/homebrew/Cellar/eccodes/2.24.2/include/eccodes.mod
/opt/homebrew/include/eccodes.mod
以及 gfortran 的搜索路径：
gfortran -E -Wp,-v -
#include <...> search starts here:
/opt/homebrew/include
/opt/homebrew/Cellar/gcc/11.2.0_3/bin/../lib/gcc/11/gcc/aarch64-apple-darwin21/11/include
/opt/homebrew/Cellar/gcc/11.2.0_3/bin/../lib/gcc/11/gcc/aarch64-apple-darwin21/11/include-fixed
/Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk/usr/include
/Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk/System/Library/Frameworks
End of search list.
另一个项目使用gcc/ld也有类似的问题。我需要明确地将 LINK_DIRECTORIES(/opt/homebrew/lib) 添加到 CMakeList.txt 以便 gcc 找到 eccodes 库。没有它，我得到：
gcc-11: warning: this compiler does not support X86 (arch flags ignored)
ld: library not found for -leccodes
collect2: error: ld returned 1 exit status
make[2]: *** [src/smoothModeS-v51/smoothModeS-v51.x] Error 1
make[1]: *** [src/smoothModeS-v51/CMakeFiles/smoothModeS-v51.x.dir/all] Error 2
make: *** [all] Error 2
将路径添加到 CPATH 和 LIBRARY_PATH 没有效果。
此方法有效但似乎难以维护。 gfortran/gcc 不再在默认路径中搜索但库和 headers/modules?
我错过了什么以及发生了什么变化
编辑
刚刚发现使用与 python 扩展名相似的代码并且还使用 eccodes 的独立可执行文件有类似的问题。如果我不在 CMakeLists.txt 中包含 INCLUDE_DIRECTORIES(/opt/homebrew/include) ，我会得到：
cd <project_dir>/buildDebug/src/collocEHS && /opt/homebrew/bin/gfortran -I<project_dir>/buildDebug -I<project_dir>/include -I<project_dir>/src/readASTERIX2 -I<project_dir>/src/geomag70_linux -I<project_dir>/src/libDTG -fallow-argument-mismatch -ffpe-trap=invalid,zero,overflow -DPREC32 -DLINUX -DIS_LITTLE_ENDIAN -DUSEWALLTIME -DHAS_BLAS -DHAS_LAPACK -DGRIB32 -DTIMING -ffixed-line-length-none -g -fcheck=all -Wall -fcheck=bounds -O0 -g -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk -c <project_dir>/src/pynwp/readAtmosphereGen.f90 -o CMakeFiles/collocEHSv2.dir/__/pynwp/readAtmosphereGen.f90.o
<project_dir>/src/pynwp/readAtmosphereGen.f90:3:7:
3 | use eccodes
| 1
Fatal Error: Cannot open module file 'eccodes.mod' for reading at (1): No such file or directory
compilation terminated.
make[2]: *** [src/collocEHS/CMakeFiles/collocEHSv2.dir/__/pynwp/readAtmosphereGen.f90.o] Error 1
make[1]: *** [src/collocEHS/CMakeFiles/collocEHSv2.dir/all] Error 2
make: *** [all] Error 2
所以我猜问题与 python 无关，但更多与 gcc/gfortran（gcc 版本 11.2.0（Homebrew GCC 11.2.0_3）有关。
编辑 2
笔记本电脑重新启动解决了 python 扩展构建和 setup.py 需要额外目录的问题。对于使用 CMake 的正常构建，我仍然需要额外的 INCLUDE_DIRECTORIES 和 LINK_DIRECTORIES 命令 gcc/gfortran 来查找 brew 在 /opt/homebrew 中安装的库（或 /usr/local for intel MacBook）。
根据自制软件开发者的说法，这是期望的行为：/opt/homebrew 和 /usr/local 是要手动添加到例如 CMake 项目中的“特殊”目录。这在我关于 home-brew 的 GitHub 的错误报告中进行了解释，请参阅 https://github.com/Homebrew/homebrew-core/issues/95561。
我无法通过文档证实这一点。
使用自制软件将 python 3.8.6 升级到 3.9.10 后，我的 Cython 扩展不再没有明确添加 /usr/local/include（对于我的 Intel MacBook）或 /opt/homebrew/include 到 include_dirs 我的分机。
我的setup.py.in:
import os, sys
from numpy.distutils.core import setup, Extension
from Cython.Build import cythonize
link_arguments = []
extra_include_dirs = []
if (sys.platform == 'darwin'):
link_arguments.append("-Wl,-rpath")
link_arguments.append("-Wl,@loader_path/")
if os.path.exists('/opt/homebrew/'):
extra_include_dirs.append("/opt/homebrew/include/")
else:
extra_include_dirs.append("/usr/local/include/")
else:
link_arguments.append("-Wl,-rpath=${CMAKE_SOURCE_DIR}/lib/")
pynwp_extension = Extension(
name="pynwp",
sources=["${CMAKE_CURRENT_SOURCE_DIR}/lambert.f90", "${CMAKE_CURRENT_SOURCE_DIR}/pynwp.f90", "${CMAKE_CURRENT_SOURCE_DIR}/readAtmosphereGen.f90", "${CMAKE_CURRENT_SOURCE_DIR}/ptogrot.f", "${CMAKE_CURRENT_SOURCE_DIR}/bilin1.f", "${CMAKE_CURRENT_SOURCE_DIR}/fl2pres_f.f","${CMAKE_CURRENT_SOURCE_DIR}/message.c","${CMAKE_CURRENT_SOURCE_DIR}/gridWindDirCorrection.F"],
libraries=["HirlamUtils_fPIC", "eccodes_f90", "jasper"],
library_dirs=["${PROJECT_BINARY_DIR}", "${CMAKE_SOURCE_DIR}/build${CMAKE_BUILD_TYPE}/src/libHirlamUtils/", "/opt/homebrew/lib/"],
extra_link_args = link_arguments,
include_dirs=["${CMAKE_SOURCE_DIR}/include", "/usr/lib64/gfortran/modules/",
"${CMAKE_SOURCE_DIR}/build${CMAKE_BUILD_TYPE}"] + extra_include_dirs,
extra_f90_compile_args=["-DLINUX", "-DIS_LITTLE_ENDIAN", "-DUSEWALLTIME", "-DHAS_BLAS", "-DHAS_LAPACK", "-DGRIB32", "-DTIMING", "-DPREC32", "-fno-whole-file", "-g", "-fbounds-check"]
#compiler_directives={'language_level' : "3"}
)
setup(name="pynwp",
author="me",
author_email="me!me.com",
version="1.0.1",
description="Python wrapper for pynwp",
package_dir={"": "${CMAKE_CURRENT_SOURCE_DIR}"},
url="http://emaddc.eu",
license="MIT License",
ext_modules=[pynwp_extension]
)
在上面的文件中，我对自制库目录的位置（临时）进行了硬编码，并根据找到的 /opt/homebrew 为包含目录添加了一些功能。如果我从文件中删除它，编译将失败，因为找不到 eccodes.mod，当我 运行 命令由 python/CMake 手动生成时查看输出：
buildDebug git:(master) ✗ /opt/homebrew/bin/gfortran -Wall -g -fno-second-underscore -fPIC -O3 -funroll-loops -I<project_dir>//include -I/usr/lib64/gfortran/modules/ -I<project_dir>//buildDebug -Ibuild/src.macosx-12-arm64-3.9/build/src.macosx-12-arm64-3.9 -I/opt/homebrew/lib/python3.9/site-packages/numpy/core/include -Ibuild/src.macosx-12-arm64-3.9/numpy/distutils/include -I/opt/homebrew/opt/python@3.9/Frameworks/Python.framework/Versions/3.9/include/python3.9 -c -c <project_dir>//src/pynwp/readAtmosphereGen.f90 -o build/temp.macosx-12-arm64-3.9<project_dir>//src/pynwp/readAtmosphereGen.o -DLINUX -DIS_LITTLE_ENDIAN -DUSEWALLTIME -DHAS_BLAS -DHAS_LAPACK -DGRIB32 -DTIMING -DPREC32 -fno-whole-file -g -fbounds-check
f951: Warning: Nonexistent include directory '/usr/lib64/gfortran/modules/' [-Wmissing-include-dirs]
f951: Warning: Nonexistent include directory 'build/src.macosx-12-arm64-3.9/build/src.macosx-12-arm64-3.9' [-Wmissing-include-dirs]
f951: Warning: Nonexistent include directory 'build/src.macosx-12-arm64-3.9/numpy/distutils/include' [-Wmissing-include-dirs]
<project_dir>//src/pynwp/readAtmosphereGen.f90:3:7:
3 | use eccodes
| 1
Fatal Error: Cannot open module file 'eccodes.mod' for reading at (1): No such file or directory
compilation terminated.
此扩展是使用 CMake 构建的更大项目的一部分。 运行手动执行gfortran命令时，出现同样的错误提示找不到eccodes.mod。但是，该文件位于 gfortran/gcc 搜索路径的默认位置：
locate eccodes.mod
/opt/homebrew/Cellar/eccodes/2.24.2/include/eccodes.mod
/opt/homebrew/include/eccodes.mod
以及 gfortran 的搜索路径：
gfortran -E -Wp,-v -
#include <...> search starts here:
/opt/homebrew/include
/opt/homebrew/Cellar/gcc/11.2.0_3/bin/../lib/gcc/11/gcc/aarch64-apple-darwin21/11/include
/opt/homebrew/Cellar/gcc/11.2.0_3/bin/../lib/gcc/11/gcc/aarch64-apple-darwin21/11/include-fixed
/Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk/usr/include
/Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk/System/Library/Frameworks
End of search list.
另一个项目使用gcc/ld也有类似的问题。我需要明确地将 LINK_DIRECTORIES(/opt/homebrew/lib) 添加到 CMakeList.txt 以便 gcc 找到 eccodes 库。没有它，我得到：
gcc-11: warning: this compiler does not support X86 (arch flags ignored)
ld: library not found for -leccodes
collect2: error: ld returned 1 exit status
make[2]: *** [src/smoothModeS-v51/smoothModeS-v51.x] Error 1
make[1]: *** [src/smoothModeS-v51/CMakeFiles/smoothModeS-v51.x.dir/all] Error 2
make: *** [all] Error 2
将路径添加到 CPATH 和 LIBRARY_PATH 没有效果。
此方法有效但似乎难以维护。 gfortran/gcc 不再在默认路径中搜索但库和 headers/modules?
我错过了什么以及发生了什么变化编辑
刚刚发现使用与 python 扩展名相似的代码并且还使用 eccodes 的独立可执行文件有类似的问题。如果我不在 CMakeLists.txt 中包含 INCLUDE_DIRECTORIES(/opt/homebrew/include) ，我会得到：
cd <project_dir>/buildDebug/src/collocEHS && /opt/homebrew/bin/gfortran -I<project_dir>/buildDebug -I<project_dir>/include -I<project_dir>/src/readASTERIX2 -I<project_dir>/src/geomag70_linux -I<project_dir>/src/libDTG -fallow-argument-mismatch -ffpe-trap=invalid,zero,overflow -DPREC32 -DLINUX -DIS_LITTLE_ENDIAN -DUSEWALLTIME -DHAS_BLAS -DHAS_LAPACK -DGRIB32 -DTIMING -ffixed-line-length-none -g -fcheck=all -Wall -fcheck=bounds -O0 -g -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk -c <project_dir>/src/pynwp/readAtmosphereGen.f90 -o CMakeFiles/collocEHSv2.dir/__/pynwp/readAtmosphereGen.f90.o
<project_dir>/src/pynwp/readAtmosphereGen.f90:3:7:
3 | use eccodes
| 1
Fatal Error: Cannot open module file 'eccodes.mod' for reading at (1): No such file or directory
compilation terminated.
make[2]: *** [src/collocEHS/CMakeFiles/collocEHSv2.dir/__/pynwp/readAtmosphereGen.f90.o] Error 1
make[1]: *** [src/collocEHS/CMakeFiles/collocEHSv2.dir/all] Error 2
make: *** [all] Error 2
所以我猜问题与 python 无关，但更多与 gcc/gfortran（gcc 版本 11.2.0（Homebrew GCC 11.2.0_3）有关。
编辑 2
笔记本电脑重新启动解决了 python 扩展构建和 setup.py 需要额外目录的问题。对于使用 CMake 的正常构建，我仍然需要额外的 INCLUDE_DIRECTORIES 和 LINK_DIRECTORIES 命令 gcc/gfortran 来查找 brew 在 /opt/homebrew 中安装的库（或 /usr/local for intel MacBook）。
根据自制软件开发者的说法，这是期望的行为：/opt/homebrew 和 /usr/local 是要手动添加到例如 CMake 项目中的“特殊”目录。这在我关于 home-brew 的 GitHub 的错误报告中进行了解释，请参阅 https://github.com/Homebrew/homebrew-core/issues/95561。
我无法通过文档证实这一点。