setuptools package_dir 复制到错误的文件夹
setuptools package_dir copies into wrong folder
目前我正在尝试用 setuptools、cmake 和相应的本机库打包一个 python 包。构建步骤如预期的那样 运行ning。但是,当我尝试安装该软件包时,出现了一些意想不到的位置,如下所示。
所以我有以下项目结构:
project
│ setup.py
│
└───mypackage
│ │ __init__.py
│ │ some more files...
│ │
│ └───subpackage
│ │ __init__.py
│ │ some more files...
│
└───src
│ example1.cpp
│ example2.cpp
│ some more files...
一切都编译得很好,但如果我 运行 python setup.py install
,它将安装以下结构:
留意 所有已编译的 dll 行
mypackage-<version>-py3.9-win-amd64.egg
│
└───mypackage
│ │ __init__.py
│ │ some more files...
│ │
│ └───subpackage
│ │ __init__.py
│ │ some more files...
└── all compiled dlls
我期望的是:
mypackage-<version>-py3.9-win-amd64.egg
│
└───mypackage
│ __init__.py
│ some more files...
└── **all compiled dlls**
└───subpackage
│ __init__.py
│ some more files...
我的 setup.py 看起来像这样:
[code to build]
setup(
name="mypackage",
version="1.0.0",
packages=['mypackage','mypackage.subpackage'],
author=...,
author_email=...,
description=...,
license=...,
keywords=...,
url=...,
tests_require=[
...
],
package_data={
'mypackage': ['lib/*.*', 'lib/*/*/*', 'share/*/*'],
},
test_suite=...,
ext_modules=[CMakeExtension("mypackage")],
cmdclass={"build_ext": CMakeBuild},
zip_safe=False
)
有没有办法告诉设置工具将 package_data 文件复制到 mypackage 文件夹?
我认为 'mypackage': ['lib/*.*', 'lib/*/*/*', 'share/*/*'],
会那样做。还是我必须编写自己的逻辑才能完成?
好的。没关系。我刚刚弄明白了:
我指定了 https://github.com/pybind/cmake_example/blob/ce0ea77878522a85da0be13cf9e425626d05827e/setup.py#L46 DCMAKE_LIBRARY_OUTPUT_DIRECTORY
。所以所有的库都会在这个目录下。 'mypackage': ['lib/*.*', 'lib/*/*/*', 'share/*/*']
这一行完全没有效果。
所以将 mypackage
添加到 https://github.com/pybind/cmake_example/blob/ce0ea77878522a85da0be13cf9e425626d05827e/setup.py#L30 (ext_dir
) 将解决我的问题
添加来自以下链接的代码,以防它们在某些时候不再可访问。
# -*- coding: utf-8 -*-
import os
import re
import subprocess
import sys
from setuptools import setup, Extension
from setuptools.command.build_ext import build_ext
# Convert distutils Windows platform specifiers to CMake -A arguments
PLAT_TO_CMAKE = {
"win32": "Win32",
"win-amd64": "x64",
"win-arm32": "ARM",
"win-arm64": "ARM64",
}
# A CMakeExtension needs a sourcedir instead of a file list.
# The name must be the _single_ output extension from the CMake build.
# If you need multiple extensions, see scikit-build.
class CMakeExtension(Extension):
def __init__(self, name, sourcedir=""):
Extension.__init__(self, name, sources=[])
self.sourcedir = os.path.abspath(sourcedir)
class CMakeBuild(build_ext):
def build_extension(self, ext):
extdir = os.path.abspath(os.path.dirname(self.get_ext_fullpath(ext.name)))
# required for auto-detection of auxiliary "native" libs
if not extdir.endswith(os.path.sep):
extdir += os.path.sep
cfg = "Debug" if self.debug else "Release"
# CMake lets you override the generator - we need to check this.
# Can be set with Conda-Build, for example.
cmake_generator = os.environ.get("CMAKE_GENERATOR", "")
# Set Python_EXECUTABLE instead if you use PYBIND11_FINDPYTHON
# EXAMPLE_VERSION_INFO shows you how to pass a value into the C++ code
# from Python.
cmake_args = [
"-DCMAKE_LIBRARY_OUTPUT_DIRECTORY={}".format(extdir),
"-DPYTHON_EXECUTABLE={}".format(sys.executable),
"-DEXAMPLE_VERSION_INFO={}".format(self.distribution.get_version()),
"-DCMAKE_BUILD_TYPE={}".format(cfg), # not used on MSVC, but no harm
]
build_args = []
if self.compiler.compiler_type != "msvc":
# Using Ninja-build since it a) is available as a wheel and b)
# multithreads automatically. MSVC would require all variables be
# exported for Ninja to pick it up, which is a little tricky to do.
# Users can override the generator with CMAKE_GENERATOR in CMake
# 3.15+.
if not cmake_generator:
try:
import ninja # noqa: F401
cmake_args += ["-GNinja"]
except ImportError:
pass
else:
# Single config generators are handled "normally"
single_config = any(x in cmake_generator for x in {"NMake", "Ninja"})
# CMake allows an arch-in-generator style for backward compatibility
contains_arch = any(x in cmake_generator for x in {"ARM", "Win64"})
# Specify the arch if using MSVC generator, but only if it doesn't
# contain a backward-compatibility arch spec already in the
# generator name.
if not single_config and not contains_arch:
cmake_args += ["-A", PLAT_TO_CMAKE[self.plat_name]]
# Multi-config generators have a different way to specify configs
if not single_config:
cmake_args += [
"-DCMAKE_LIBRARY_OUTPUT_DIRECTORY_{}={}".format(cfg.upper(), extdir)
]
build_args += ["--config", cfg]
if sys.platform.startswith("darwin"):
# Cross-compile support for macOS - respect ARCHFLAGS if set
archs = re.findall(r"-arch (\S+)", os.environ.get("ARCHFLAGS", ""))
if archs:
cmake_args += ["-DCMAKE_OSX_ARCHITECTURES={}".format(";".join(archs))]
# Set CMAKE_BUILD_PARALLEL_LEVEL to control the parallel build level
# across all generators.
if "CMAKE_BUILD_PARALLEL_LEVEL" not in os.environ:
# self.parallel is a Python 3 only way to set parallel jobs by hand
# using -j in the build_ext call, not supported by pip or PyPA-build.
if hasattr(self, "parallel") and self.parallel:
# CMake 3.12+ only.
build_args += ["-j{}".format(self.parallel)]
if not os.path.exists(self.build_temp):
os.makedirs(self.build_temp)
subprocess.check_call(
["cmake", ext.sourcedir] + cmake_args, cwd=self.build_temp
)
subprocess.check_call(
["cmake", "--build", "."] + build_args, cwd=self.build_temp
)
# The information here can also be placed in setup.cfg - better separation of
# logic and declaration, and simpler if you include description/version in a file.
setup(
name="cmake_example",
version="0.0.1",
author="Dean Moldovan",
author_email="dean0x7d@gmail.com",
description="A test project using pybind11 and CMake",
long_description="",
ext_modules=[CMakeExtension("cmake_example")],
cmdclass={"build_ext": CMakeBuild},
zip_safe=False,
extras_require={"test": ["pytest"]},
)
目前我正在尝试用 setuptools、cmake 和相应的本机库打包一个 python 包。构建步骤如预期的那样 运行ning。但是,当我尝试安装该软件包时,出现了一些意想不到的位置,如下所示。 所以我有以下项目结构:
project
│ setup.py
│
└───mypackage
│ │ __init__.py
│ │ some more files...
│ │
│ └───subpackage
│ │ __init__.py
│ │ some more files...
│
└───src
│ example1.cpp
│ example2.cpp
│ some more files...
一切都编译得很好,但如果我 运行 python setup.py install
,它将安装以下结构:
留意 所有已编译的 dll 行
mypackage-<version>-py3.9-win-amd64.egg
│
└───mypackage
│ │ __init__.py
│ │ some more files...
│ │
│ └───subpackage
│ │ __init__.py
│ │ some more files...
└── all compiled dlls
我期望的是:
mypackage-<version>-py3.9-win-amd64.egg
│
└───mypackage
│ __init__.py
│ some more files...
└── **all compiled dlls**
└───subpackage
│ __init__.py
│ some more files...
我的 setup.py 看起来像这样:
[code to build]
setup(
name="mypackage",
version="1.0.0",
packages=['mypackage','mypackage.subpackage'],
author=...,
author_email=...,
description=...,
license=...,
keywords=...,
url=...,
tests_require=[
...
],
package_data={
'mypackage': ['lib/*.*', 'lib/*/*/*', 'share/*/*'],
},
test_suite=...,
ext_modules=[CMakeExtension("mypackage")],
cmdclass={"build_ext": CMakeBuild},
zip_safe=False
)
有没有办法告诉设置工具将 package_data 文件复制到 mypackage 文件夹?
我认为 'mypackage': ['lib/*.*', 'lib/*/*/*', 'share/*/*'],
会那样做。还是我必须编写自己的逻辑才能完成?
好的。没关系。我刚刚弄明白了:
我指定了 https://github.com/pybind/cmake_example/blob/ce0ea77878522a85da0be13cf9e425626d05827e/setup.py#L46 DCMAKE_LIBRARY_OUTPUT_DIRECTORY
。所以所有的库都会在这个目录下。 'mypackage': ['lib/*.*', 'lib/*/*/*', 'share/*/*']
这一行完全没有效果。
所以将 mypackage
添加到 https://github.com/pybind/cmake_example/blob/ce0ea77878522a85da0be13cf9e425626d05827e/setup.py#L30 (ext_dir
) 将解决我的问题
添加来自以下链接的代码,以防它们在某些时候不再可访问。
# -*- coding: utf-8 -*-
import os
import re
import subprocess
import sys
from setuptools import setup, Extension
from setuptools.command.build_ext import build_ext
# Convert distutils Windows platform specifiers to CMake -A arguments
PLAT_TO_CMAKE = {
"win32": "Win32",
"win-amd64": "x64",
"win-arm32": "ARM",
"win-arm64": "ARM64",
}
# A CMakeExtension needs a sourcedir instead of a file list.
# The name must be the _single_ output extension from the CMake build.
# If you need multiple extensions, see scikit-build.
class CMakeExtension(Extension):
def __init__(self, name, sourcedir=""):
Extension.__init__(self, name, sources=[])
self.sourcedir = os.path.abspath(sourcedir)
class CMakeBuild(build_ext):
def build_extension(self, ext):
extdir = os.path.abspath(os.path.dirname(self.get_ext_fullpath(ext.name)))
# required for auto-detection of auxiliary "native" libs
if not extdir.endswith(os.path.sep):
extdir += os.path.sep
cfg = "Debug" if self.debug else "Release"
# CMake lets you override the generator - we need to check this.
# Can be set with Conda-Build, for example.
cmake_generator = os.environ.get("CMAKE_GENERATOR", "")
# Set Python_EXECUTABLE instead if you use PYBIND11_FINDPYTHON
# EXAMPLE_VERSION_INFO shows you how to pass a value into the C++ code
# from Python.
cmake_args = [
"-DCMAKE_LIBRARY_OUTPUT_DIRECTORY={}".format(extdir),
"-DPYTHON_EXECUTABLE={}".format(sys.executable),
"-DEXAMPLE_VERSION_INFO={}".format(self.distribution.get_version()),
"-DCMAKE_BUILD_TYPE={}".format(cfg), # not used on MSVC, but no harm
]
build_args = []
if self.compiler.compiler_type != "msvc":
# Using Ninja-build since it a) is available as a wheel and b)
# multithreads automatically. MSVC would require all variables be
# exported for Ninja to pick it up, which is a little tricky to do.
# Users can override the generator with CMAKE_GENERATOR in CMake
# 3.15+.
if not cmake_generator:
try:
import ninja # noqa: F401
cmake_args += ["-GNinja"]
except ImportError:
pass
else:
# Single config generators are handled "normally"
single_config = any(x in cmake_generator for x in {"NMake", "Ninja"})
# CMake allows an arch-in-generator style for backward compatibility
contains_arch = any(x in cmake_generator for x in {"ARM", "Win64"})
# Specify the arch if using MSVC generator, but only if it doesn't
# contain a backward-compatibility arch spec already in the
# generator name.
if not single_config and not contains_arch:
cmake_args += ["-A", PLAT_TO_CMAKE[self.plat_name]]
# Multi-config generators have a different way to specify configs
if not single_config:
cmake_args += [
"-DCMAKE_LIBRARY_OUTPUT_DIRECTORY_{}={}".format(cfg.upper(), extdir)
]
build_args += ["--config", cfg]
if sys.platform.startswith("darwin"):
# Cross-compile support for macOS - respect ARCHFLAGS if set
archs = re.findall(r"-arch (\S+)", os.environ.get("ARCHFLAGS", ""))
if archs:
cmake_args += ["-DCMAKE_OSX_ARCHITECTURES={}".format(";".join(archs))]
# Set CMAKE_BUILD_PARALLEL_LEVEL to control the parallel build level
# across all generators.
if "CMAKE_BUILD_PARALLEL_LEVEL" not in os.environ:
# self.parallel is a Python 3 only way to set parallel jobs by hand
# using -j in the build_ext call, not supported by pip or PyPA-build.
if hasattr(self, "parallel") and self.parallel:
# CMake 3.12+ only.
build_args += ["-j{}".format(self.parallel)]
if not os.path.exists(self.build_temp):
os.makedirs(self.build_temp)
subprocess.check_call(
["cmake", ext.sourcedir] + cmake_args, cwd=self.build_temp
)
subprocess.check_call(
["cmake", "--build", "."] + build_args, cwd=self.build_temp
)
# The information here can also be placed in setup.cfg - better separation of
# logic and declaration, and simpler if you include description/version in a file.
setup(
name="cmake_example",
version="0.0.1",
author="Dean Moldovan",
author_email="dean0x7d@gmail.com",
description="A test project using pybind11 and CMake",
long_description="",
ext_modules=[CMakeExtension("cmake_example")],
cmdclass={"build_ext": CMakeBuild},
zip_safe=False,
extras_require={"test": ["pytest"]},
)