在 Windows 上使用 MKL 编译 NumPy - DLL 加载失败
Compile NumPy with MKL on Windows - DLL load failed
我一直在尝试使用 MSVC 编译器和 Intel MKL 在 Windows 10 上从源代码编译 NumPy。
我是 运行 Windows 10.0.18363，Microsoft Visual Studio 2019 (16.8.4) 和 Intel MKL 2017.8.275。
我使用最新的 Python 和 NumPy 以最少的设置重现了该问题。
下载最新的 Python (3.9.1) 和最新的 NumPy (1.20.1) 源。
打开 VS 命令提示符，解压缩 Python 源，使用 PCbuild\build.bat
构建
运行 mklvars.bat intel64 设置正确的环境变量。
将英特尔编译器（ifort 需要）添加到 PATH：
set PATH=C:\Program Files (x86)\IntelSWTools\compilers_and_libraries\windows\bin\intel64;%PATH%
创建虚拟环境，从 Python 构建中复制一些文件并激活虚拟环境：
copy Python\PCbuild\amd64\python39.dll venv\Scripts
copy Python\PC\pyconfig.h venv\Include
从源代码构建 NumPy 并安装：pip install . -v
尝试导入 NumPy：python -c "import numpy"
错误信息
Traceback (most recent call last):
File "C:\cygwin\home\user\numpy_clean_env\venv\lib\site-packages\numpy\core\__init__.py", line 22, in <module>
from . import multiarray
File "C:\cygwin\home\user\numpy_clean_env\venv\lib\site-packages\numpy\core\multiarray.py", line 12, in <module>
from . import overrides
File "C:\cygwin\home\user\numpy_clean_env\venv\lib\site-packages\numpy\core\overrides.py", line 7, in <module>
from numpy.core._multiarray_umath import (
ImportError: DLL load failed while importing _multiarray_umath: The specified module could not be found.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\cygwin\home\user\numpy_clean_env\venv\lib\site-packages\numpy\__init__.py", line 145, in <module>
from . import core
File "C:\cygwin\home\user\numpy_clean_env\venv\lib\site-packages\numpy\core\__init__.py", line 48, in <module>
raise ImportError(msg)
ImportError:
IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.
We have compiled some common reasons and troubleshooting tips at:
https://numpy.org/devdocs/user/troubleshooting-importerror.html
Please note and check the following:
* The Python version is: Python3.9 from "C:\cygwin\home\user\numpy_clean_env\venv\Scripts\python.exe"
* The NumPy version is: "1.20.1"
and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.
Original error was: DLL load failed while importing _multiarray_umath: The specified module could not be found.
其他信息
MKL 库在编译期间被拾取，因为它 returns:
FOUND:
libraries = ['mkl_rt']
library_dirs = ['C:\Program Files (x86)\IntelSWTools\compilers_and_libraries\windows\mkl\lib\intel64']
define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]
include_dirs = ['C:\Program Files (x86)\IntelSWTools\compilers_and_libraries\windows\mkl', 'C:\Program Files (x86)\IntelSWTools\compilers_and_libraries\windows\mkl\include', 'C:\Program Files (x86)\IntelSWTools\compilers_and_libraries\windows\mkl\lib']
我尝试使用 Dependencies（较新版本的 Dependency Walker）分析 DLL 解析，但似乎 MKL DLL 加载正常。有一些 DLL 显示为未正确加载，但据我了解，这是由 Windows API 设置的检查软件限制引起的，而不是由此 DLL 的实际问题引起的，所以我认为系统设置正确。
以下是在分析 _multiarray_umath.pyd:
时在依赖项中导致 NOT_FOUND 的唯一 DLL 的列表
EMCLIENT.dll (NOT_FOUND) :
HvsiFileTrust.dll (NOT_FOUND) :
UpdateAPI.dll (NOT_FOUND) :
api-ms-win-core-comm-l1-1-0.dll (NOT_FOUND) :
api-ms-win-core-psapi-ansi-l1-1-0.dll (NOT_FOUND) :
api-ms-win-core-psapi-l1-1-0.dll (NOT_FOUND) :
api-ms-win-core-stringansi-l1-1-0.dll (NOT_FOUND) :
api-ms-win-core-versionansi-l1-1-0.dll (NOT_FOUND) :
api-ms-win-core-versionansi-l1-1-1.dll (NOT_FOUND) :
api-ms-win-core-winrt-string-l1-1-0.dll (NOT_FOUND) :
api-ms-win-core-wow64-l1-1-0.dll (NOT_FOUND) :
api-ms-win-core-wow64-l1-1-1.dll (NOT_FOUND) :
api-ms-win-core-xstate-l2-1-0.dll (NOT_FOUND) :
api-ms-win-core-xstate-l2-1-1.dll (NOT_FOUND) :
api-ms-win-coreui-secruntime-l1-1-0.dll (NOT_FOUND) :
ext-ms-mf-pal-l2-1-0.dll (NOT_FOUND) :
ext-ms-onecore-appmodel-emclient-l1-1-0.dll (NOT_FOUND) :
ext-ms-onecore-defaultdiscovery-l1-1-0.dll (NOT_FOUND) :
ext-ms-onecore-orientation-l1-1-0.dll (NOT_FOUND) :
ext-ms-onecore-shellchromeapi-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-appmodel-deployment-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-appmodel-usercontext-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-appmodel-viewscalefactor-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-audiocore-pal-l1-2-0.dll (NOT_FOUND) :
ext-ms-win-com-suspendresiliency-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-core-winrt-remote-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-dwmapi-ext-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-dxcore-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-networking-wlanstorage-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-ntuser-window-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-ntuser-window-l1-1-1.dll (NOT_FOUND) :
ext-ms-win-ntuser-window-l1-1-2.dll (NOT_FOUND) :
ext-ms-win-ntuser-window-l1-1-3.dll (NOT_FOUND) :
ext-ms-win-ntuser-window-l1-1-4.dll (NOT_FOUND) :
ext-ms-win-ras-tapi32-l1-1-1.dll (NOT_FOUND) :
ext-ms-win-ro-typeresolution-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-rtcore-minuser-input-l1-1-1.dll (NOT_FOUND) :
ext-ms-win-rtcore-minuser-private-ext-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-security-capauthz-ext-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-security-chambers-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-shell-knownfolderext-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-shell-shell32-l1-2-0.dll (NOT_FOUND) :
ext-ms-win-shell-shlwapi-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-shell-tabbedtitlebar-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-shell32-shellcom-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-shell32-shellfolders-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-ui-viewmanagement-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-uiacore-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-wer-xbox-l1-1-0.dll (NOT_FOUND) :
ext-ms-win32-subsystem-query-l1-1-0.dll (NOT_FOUND) :
ext-ms-windowscore-deviceinfo-l1-1-0.dll (NOT_FOUND) :
我也尝试按照 this guide 并且我可以确认 os.path.exists('_multiarray_umath.pyd') returns True，但是 ctypes.WinDLL('_multiarray_umath.pyd') 无法加载 DLL（或一个它的依赖项）。
工作示例
步骤与上述相同，但未执行 mklvars.bat，因此未链接至 Intel MKL。
过去，此设置用于 Python 3.6、VS2015 和类似版本的 Intel MKL。
我可能遗漏了什么，欢迎任何建议。
已按照 Numpy 邮件列表 (permalink) 上的建议解决。
Python >= 3.8 will no longer use PATH for resolving dependencies of
extension modules. Use os.add_dll_directory(mkl_bin_path)
https://docs.python.org/3/library/os.html#os.add_dll_directory in all
your scripts before importing numpy or add the call to a
_distributor_init.py file in the numpy package directory.
我一直在尝试使用 MSVC 编译器和 Intel MKL 在 Windows 10 上从源代码编译 NumPy。 我是 运行 Windows 10.0.18363，Microsoft Visual Studio 2019 (16.8.4) 和 Intel MKL 2017.8.275。
我使用最新的 Python 和 NumPy 以最少的设置重现了该问题。
下载最新的 Python (3.9.1) 和最新的 NumPy (1.20.1) 源。
打开 VS 命令提示符，解压缩 Python 源，使用
构建PCbuild\build.bat运行
mklvars.bat intel64设置正确的环境变量。将英特尔编译器（ifort 需要）添加到 PATH：
set PATH=C:\Program Files (x86)\IntelSWTools\compilers_and_libraries\windows\bin\intel64;%PATH%创建虚拟环境，从 Python 构建中复制一些文件并激活虚拟环境：
copy Python\PCbuild\amd64\python39.dll venv\Scripts copy Python\PC\pyconfig.h venv\Include从源代码构建 NumPy 并安装：
pip install . -v尝试导入 NumPy：
python -c "import numpy"
错误信息
Traceback (most recent call last):
File "C:\cygwin\home\user\numpy_clean_env\venv\lib\site-packages\numpy\core\__init__.py", line 22, in <module>
from . import multiarray
File "C:\cygwin\home\user\numpy_clean_env\venv\lib\site-packages\numpy\core\multiarray.py", line 12, in <module>
from . import overrides
File "C:\cygwin\home\user\numpy_clean_env\venv\lib\site-packages\numpy\core\overrides.py", line 7, in <module>
from numpy.core._multiarray_umath import (
ImportError: DLL load failed while importing _multiarray_umath: The specified module could not be found.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\cygwin\home\user\numpy_clean_env\venv\lib\site-packages\numpy\__init__.py", line 145, in <module>
from . import core
File "C:\cygwin\home\user\numpy_clean_env\venv\lib\site-packages\numpy\core\__init__.py", line 48, in <module>
raise ImportError(msg)
ImportError:
IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.
We have compiled some common reasons and troubleshooting tips at:
https://numpy.org/devdocs/user/troubleshooting-importerror.html
Please note and check the following:
* The Python version is: Python3.9 from "C:\cygwin\home\user\numpy_clean_env\venv\Scripts\python.exe"
* The NumPy version is: "1.20.1"
and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.
Original error was: DLL load failed while importing _multiarray_umath: The specified module could not be found.
其他信息
MKL 库在编译期间被拾取，因为它 returns:
FOUND:
libraries = ['mkl_rt']
library_dirs = ['C:\Program Files (x86)\IntelSWTools\compilers_and_libraries\windows\mkl\lib\intel64']
define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]
include_dirs = ['C:\Program Files (x86)\IntelSWTools\compilers_and_libraries\windows\mkl', 'C:\Program Files (x86)\IntelSWTools\compilers_and_libraries\windows\mkl\include', 'C:\Program Files (x86)\IntelSWTools\compilers_and_libraries\windows\mkl\lib']
我尝试使用 Dependencies（较新版本的 Dependency Walker）分析 DLL 解析，但似乎 MKL DLL 加载正常。有一些 DLL 显示为未正确加载，但据我了解，这是由 Windows API 设置的检查软件限制引起的，而不是由此 DLL 的实际问题引起的，所以我认为系统设置正确。
以下是在分析 _multiarray_umath.pyd:
NOT_FOUND 的唯一 DLL 的列表
EMCLIENT.dll (NOT_FOUND) :
HvsiFileTrust.dll (NOT_FOUND) :
UpdateAPI.dll (NOT_FOUND) :
api-ms-win-core-comm-l1-1-0.dll (NOT_FOUND) :
api-ms-win-core-psapi-ansi-l1-1-0.dll (NOT_FOUND) :
api-ms-win-core-psapi-l1-1-0.dll (NOT_FOUND) :
api-ms-win-core-stringansi-l1-1-0.dll (NOT_FOUND) :
api-ms-win-core-versionansi-l1-1-0.dll (NOT_FOUND) :
api-ms-win-core-versionansi-l1-1-1.dll (NOT_FOUND) :
api-ms-win-core-winrt-string-l1-1-0.dll (NOT_FOUND) :
api-ms-win-core-wow64-l1-1-0.dll (NOT_FOUND) :
api-ms-win-core-wow64-l1-1-1.dll (NOT_FOUND) :
api-ms-win-core-xstate-l2-1-0.dll (NOT_FOUND) :
api-ms-win-core-xstate-l2-1-1.dll (NOT_FOUND) :
api-ms-win-coreui-secruntime-l1-1-0.dll (NOT_FOUND) :
ext-ms-mf-pal-l2-1-0.dll (NOT_FOUND) :
ext-ms-onecore-appmodel-emclient-l1-1-0.dll (NOT_FOUND) :
ext-ms-onecore-defaultdiscovery-l1-1-0.dll (NOT_FOUND) :
ext-ms-onecore-orientation-l1-1-0.dll (NOT_FOUND) :
ext-ms-onecore-shellchromeapi-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-appmodel-deployment-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-appmodel-usercontext-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-appmodel-viewscalefactor-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-audiocore-pal-l1-2-0.dll (NOT_FOUND) :
ext-ms-win-com-suspendresiliency-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-core-winrt-remote-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-dwmapi-ext-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-dxcore-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-networking-wlanstorage-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-ntuser-window-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-ntuser-window-l1-1-1.dll (NOT_FOUND) :
ext-ms-win-ntuser-window-l1-1-2.dll (NOT_FOUND) :
ext-ms-win-ntuser-window-l1-1-3.dll (NOT_FOUND) :
ext-ms-win-ntuser-window-l1-1-4.dll (NOT_FOUND) :
ext-ms-win-ras-tapi32-l1-1-1.dll (NOT_FOUND) :
ext-ms-win-ro-typeresolution-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-rtcore-minuser-input-l1-1-1.dll (NOT_FOUND) :
ext-ms-win-rtcore-minuser-private-ext-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-security-capauthz-ext-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-security-chambers-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-shell-knownfolderext-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-shell-shell32-l1-2-0.dll (NOT_FOUND) :
ext-ms-win-shell-shlwapi-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-shell-tabbedtitlebar-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-shell32-shellcom-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-shell32-shellfolders-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-ui-viewmanagement-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-uiacore-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-wer-xbox-l1-1-0.dll (NOT_FOUND) :
ext-ms-win32-subsystem-query-l1-1-0.dll (NOT_FOUND) :
ext-ms-windowscore-deviceinfo-l1-1-0.dll (NOT_FOUND) :
我也尝试按照 this guide 并且我可以确认 os.path.exists('_multiarray_umath.pyd') returns True，但是 ctypes.WinDLL('_multiarray_umath.pyd') 无法加载 DLL（或一个它的依赖项）。
工作示例
步骤与上述相同，但未执行 mklvars.bat，因此未链接至 Intel MKL。
过去，此设置用于 Python 3.6、VS2015 和类似版本的 Intel MKL。 我可能遗漏了什么，欢迎任何建议。
已按照 Numpy 邮件列表 (permalink) 上的建议解决。
Python >= 3.8 will no longer use PATH for resolving dependencies of extension modules. Use os.add_dll_directory(mkl_bin_path) https://docs.python.org/3/library/os.html#os.add_dll_directory in all your scripts before importing numpy or add the call to a _distributor_init.py file in the numpy package directory.