auditwheel 修复未按预期工作

auditwheel repair not working as expected

我正在尝试为 mysqlclient
造轮子 基础镜像 Dockerfile

FROM amazonlinux:2018.03
# Need to set "ulimit -n" to a small value to stop yum from hanging:
# https://bugzilla.redhat.com/show_bug.cgi?id=1715254#c1
RUN ulimit -n 1024 && \
    yum -y update && \
    yum -y install \
    gcc \
    openssl-devel \
    zip \
    zlib-devel \
    libffi-devel && \
    yum -y clean all && \
    rm -rf /var/cache/yum

# Install Python, pip and boto
# boto3 is available to lambda processes by default
# Runtimes - https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtimes.html
COPY ./.cache /cache
WORKDIR /cache
RUN mkdir Python && \
    tar xzf Python.tgz -C Python --strip-components 1 && \
    cd Python && \
    ./configure --enable-optimizations && \
    make altinstall && \
    ln -s /usr/local/bin/python3.7 /usr/local/bin/python3 && \
    cd .. && \
    python3 get-pip.py  && \
    python3 -m pip install --upgrade pip && \
    python3 -V && \
    pip -V  && \
    python3 -m pip install botocore==1.13.34 boto3==1.10.34 && \
    rm -rf /cache

COPY ./scripts/packager.sh /scripts/packager.sh
WORKDIR /.build

ENV PIP_WHEEL_DIR=/.wheelhouse
ENV PIP_FIND_LINKS=/.wheelhouse
ENV PIP_DESTINATION_DIRECTORY=/.wheelhouse
# Make it possible to build numpy:
# https://github.com/numpy/numpy/issues/14147
ENV CFLAGS="-std=c99"
ENTRYPOINT ["/bin/bash", "-ex"]
CMD ["/scripts/packager.sh"]

构建镜像 Dockerfile

ARG BASE_IMAGE
FROM ${BASE_IMAGE}
RUN yum install mysql-devel unzip wget bzip2 gcc-c++ -y
RUN pip install auditwheel
RUN cd /tmp && \
    wget http://nixos.org/releases/patchelf/patchelf-0.10/patchelf-0.10.tar.bz2 && \
    tar xf patchelf-0.10.tar.bz2 && \
    cd patchelf-0.10 && \
    ./configure && \
    make install

构建脚本:

cp -r /src/* /.build
pip download -r requirements.txt
pip wheel -r requirements.txt
auditwheel repair --plat manylinux2014_x86_64 -w /.wheelhouse /.wheelhouse/mysqlclient-1.4.6-cp37-cp37m-linux_x86_64.whl
auditwheel show /.wheelhouse/mysqlclient-1.4.6-cp37-cp37m-manylinux2014_x86_64.whl
pip install --no-index -r requirements.txt -t .
zip -r /.package/lambda.zip *

构建日志:

builder_1  | + pip download -r requirements.txt
builder_1  | Looking in links: /.wheelhouse
builder_1  | Processing /.wheelhouse/mysqlclient-1.4.6-cp37-cp37m-manylinux2014_x86_64.whl
builder_1  |   File was already downloaded /.wheelhouse/mysqlclient-1.4.6-cp37-cp37m-manylinux2014_x86_64.whl
builder_1  | Successfully downloaded mysqlclient
builder_1  | + pip wheel -r requirements.txt
builder_1  | Looking in links: /.wheelhouse
builder_1  | Processing /.wheelhouse/mysqlclient-1.4.6-cp37-cp37m-manylinux2014_x86_64.whl
builder_1  |   File was already downloaded /.wheelhouse/mysqlclient-1.4.6-cp37-cp37m-manylinux2014_x86_64.whl
builder_1  | Skipping mysqlclient, due to already being wheel.
builder_1  | + auditwheel repair --plat manylinux2014_x86_64 -w /.wheelhouse /.wheelhouse/mysqlclient-1.4.6-cp37-cp37m-linux_x86_64.whl
builder_1  | INFO:auditwheel.main_repair:Repairing mysqlclient-1.4.6-cp37-cp37m-linux_x86_64.whl
builder_1  | INFO:auditwheel.wheeltools:Previous filename tags: linux_x86_64
builder_1  | INFO:auditwheel.wheeltools:New filename tags: manylinux2014_x86_64
builder_1  | INFO:auditwheel.wheeltools:Previous WHEEL info tags: cp37-cp37m-linux_x86_64
builder_1  | INFO:auditwheel.wheeltools:New WHEEL info tags: cp37-cp37m-manylinux2014_x86_64
builder_1  | INFO:auditwheel.main_repair:
builder_1  | Fixed-up wheel written to /.wheelhouse/mysqlclient-1.4.6-cp37-cp37m-manylinux2014_x86_64.whl
builder_1  | + auditwheel show /.wheelhouse/mysqlclient-1.4.6-cp37-cp37m-manylinux2014_x86_64.whl
builder_1  | 
builder_1  | mysqlclient-1.4.6-cp37-cp37m-manylinux2014_x86_64.whl is consistent
builder_1  | with the following platform tag: "linux_x86_64".
builder_1  | 
builder_1  | The wheel references external versioned symbols in these system-
builder_1  | provided shared libraries: libc.so.6 with versions {'GLIBC_2.3',
builder_1  | 'GLIBC_2.17', 'GLIBC_2.14', 'GLIBC_2.16', 'GLIBC_2.4', 'GLIBC_2.7',
builder_1  | 'GLIBC_2.3.4', 'GLIBC_2.3.2', 'GLIBC_2.2.5', 'GLIBC_2.8'},
builder_1  | libpthread.so.0 with versions {'GLIBC_2.2.5'},
builder_1  | libk5crypto-83914fc5.so.3.1 with versions {'k5crypto_3_MIT'},
builder_1  | libkrb5-5171cf0b.so.3.3 with versions {'krb5_3_MIT'},
builder_1  | libcrypto-1763ce4d.so.1.0.2k with versions {'libcrypto.so.10',
builder_1  | 'OPENSSL_1.0.1_EC'}, libgcc_s.so.1 with versions {'GCC_4.2.0',
builder_1  | 'GCC_3.3', 'GCC_3.0'}, libm.so.6 with versions {'GLIBC_2.2.5'},
builder_1  | libdl.so.2 with versions {'GLIBC_2.2.5'}, libstdc++.so.6 with versions
builder_1  | {'CXXABI_1.3', 'GLIBCXX_3.4'}, libcrypto.so.10 with versions
builder_1  | {'libcrypto.so.10', 'OPENSSL_1.0.1_EC'}, libssl.so.10 with versions
builder_1  | {'libssl.so.10'}, libkrb5support-1e00ebb8.so.0.1 with versions
builder_1  | {'krb5support_0_MIT'}, libkeyutils-caf8c69c.so.1.5 with versions
builder_1  | {'KEYUTILS_1.0', 'KEYUTILS_0.3', 'KEYUTILS_1.5'}, libresolv.so.2 with
builder_1  | versions {'GLIBC_2.2.5'}, libc-2-8d824254.17.so with versions
builder_1  | {'GLIBC_2.2.5'}, libmysqlclient-f67de14d.so.1020.3.15 with versions
builder_1  | {'libmysqlclient_1020.0'}, libkeyutils.so.1 with versions
builder_1  | {'KEYUTILS_1.0', 'KEYUTILS_0.3', 'KEYUTILS_1.5'}, libkrb5support.so.0
builder_1  | with versions {'krb5support_0_MIT'}, libk5crypto.so.3 with versions
builder_1  | {'k5crypto_3_MIT'}, libkrb5.so.3 with versions {'krb5_3_MIT'}
builder_1  | 
builder_1  | This constrains the platform tag to "manylinux2014_x86_64". In order
builder_1  | to achieve a more compatible tag, you would need to recompile a new
builder_1  | wheel from source on a system with earlier versions of these
builder_1  | libraries, such as a recent manylinux image.
builder_1  | + pip install --no-index -r requirements.txt -t .
builder_1  | Looking in links: /.wheelhouse
builder_1  | Processing /.wheelhouse/mysqlclient-1.4.6-cp37-cp37m-manylinux2014_x86_64.whl
builder_1  | Installing collected packages: mysqlclient
builder_1  | Successfully installed mysqlclient-1.4.6
builder_1  | + zip -r /.package/lambda.zip MySQLdb lambda.py mysqlclient-1.4.6.dist-info mysqlclient.libs requirements.txt
builder_1  |   adding: MySQLdb/ (stored 0%)
builder_1  |   adding: MySQLdb/_exceptions.py (deflated 63%)
builder_1  |   adding: MySQLdb/converters.py (deflated 62%)
builder_1  |   adding: MySQLdb/constants/ (stored 0%)
builder_1  |   adding: MySQLdb/constants/ER.py (deflated 65%)
builder_1  |   adding: MySQLdb/constants/CR.py (deflated 57%)
builder_1  |   adding: MySQLdb/constants/FIELD_TYPE.py (deflated 40%)
builder_1  |   adding: MySQLdb/constants/FLAG.py (deflated 26%)
builder_1  |   adding: MySQLdb/constants/__init__.py (deflated 2%)
builder_1  |   adding: MySQLdb/constants/CLIENT.py (deflated 32%)
builder_1  |   adding: MySQLdb/constants/__pycache__/ (stored 0%)
builder_1  |   adding: MySQLdb/constants/__pycache__/FIELD_TYPE.cpython-37.pyc (deflated 32%)
builder_1  |   adding: MySQLdb/constants/__pycache__/__init__.cpython-37.pyc (deflated 10%)
builder_1  |   adding: MySQLdb/constants/__pycache__/CLIENT.cpython-37.pyc (deflated 31%)
builder_1  |   adding: MySQLdb/constants/__pycache__/CR.cpython-37.pyc (deflated 42%)
builder_1  |   adding: MySQLdb/constants/__pycache__/FLAG.cpython-37.pyc (deflated 25%)
builder_1  |   adding: MySQLdb/constants/__pycache__/ER.cpython-37.pyc (deflated 58%)
builder_1  |   adding: MySQLdb/cursors.py (deflated 71%)
builder_1  |   adding: MySQLdb/compat.py (deflated 46%)
builder_1  |   adding: MySQLdb/_mysql.cpython-37m-x86_64-linux-gnu.so (deflated 73%)
builder_1  |   adding: MySQLdb/connections.py (deflated 68%)
builder_1  |   adding: MySQLdb/__init__.py (deflated 62%)
builder_1  |   adding: MySQLdb/times.py (deflated 70%)
builder_1  |   adding: MySQLdb/release.py (deflated 7%)
builder_1  |   adding: MySQLdb/__pycache__/ (stored 0%)
builder_1  |   adding: MySQLdb/__pycache__/times.cpython-37.pyc (deflated 52%)
builder_1  |   adding: MySQLdb/__pycache__/converters.cpython-37.pyc (deflated 48%)
builder_1  |   adding: MySQLdb/__pycache__/cursors.cpython-37.pyc (deflated 58%)
builder_1  |   adding: MySQLdb/__pycache__/connections.cpython-37.pyc (deflated 54%)
builder_1  |   adding: MySQLdb/__pycache__/__init__.cpython-37.pyc (deflated 45%)
builder_1  |   adding: MySQLdb/__pycache__/_exceptions.cpython-37.pyc (deflated 62%)
builder_1  |   adding: MySQLdb/__pycache__/release.cpython-37.pyc (deflated 13%)
builder_1  |   adding: MySQLdb/__pycache__/compat.cpython-37.pyc (deflated 18%)
builder_1  |   adding: lambda.py (deflated 29%)
builder_1  |   adding: mysqlclient-1.4.6.dist-info/ (stored 0%)
builder_1  |   adding: mysqlclient-1.4.6.dist-info/INSTALLER (stored 0%)
builder_1  |   adding: mysqlclient-1.4.6.dist-info/top_level.txt (stored 0%)
builder_1  |   adding: mysqlclient-1.4.6.dist-info/METADATA (deflated 59%)
builder_1  |   adding: mysqlclient-1.4.6.dist-info/RECORD (deflated 51%)
builder_1  |   adding: mysqlclient-1.4.6.dist-info/LICENSE (deflated 62%)
builder_1  |   adding: mysqlclient-1.4.6.dist-info/WHEEL (deflated 3%)
builder_1  |   adding: mysqlclient.libs/ (stored 0%)
builder_1  |   adding: mysqlclient.libs/libgcc_s-7-20170915-8c34ab4d.so.1 (deflated 56%)
builder_1  |   adding: mysqlclient.libs/libcom_err-7d8345a5.so.2.1 (deflated 68%)
builder_1  |   adding: mysqlclient.libs/libssl-54f048ca.so.1.0.2k (deflated 70%)
builder_1  |   adding: mysqlclient.libs/libresolv-2-f42a639d.17.so (deflated 58%)
builder_1  |   adding: mysqlclient.libs/libkeyutils-caf8c69c.so.1.5 (deflated 69%)
builder_1  |   adding: mysqlclient.libs/libmysqlclient-f67de14d.so.1020.3.15 (deflated 70%)
builder_1  |   adding: mysqlclient.libs/libdl-2-d78ceb63.17.so (deflated 71%)
builder_1  |   adding: mysqlclient.libs/librt-2-dffdb84f.17.so (deflated 64%)
builder_1  |   adding: mysqlclient.libs/libk5crypto-83914fc5.so.3.1 (deflated 67%)
builder_1  |   adding: mysqlclient.libs/libgssapi_krb5-594507f6.so.2.2 (deflated 68%)
builder_1  |   adding: mysqlclient.libs/libcrypto-1763ce4d.so.1.0.2k (deflated 60%)
builder_1  |   adding: mysqlclient.libs/libz-5de0a349.so.1.2.8 (deflated 49%)
builder_1  |   adding: mysqlclient.libs/libstdc++-d81bf616.so.6.0.24 (deflated 74%)
builder_1  |   adding: mysqlclient.libs/libm-2-e75c71fd.17.so (deflated 36%)
builder_1  |   adding: mysqlclient.libs/libkrb5-5171cf0b.so.3.3 (deflated 67%)
builder_1  |   adding: mysqlclient.libs/libselinux-e812e8aa.so.1 (deflated 57%)
builder_1  |   adding: mysqlclient.libs/libc-2-8d824254.17.so (deflated 61%)
builder_1  |   adding: mysqlclient.libs/libpthread-2-764a16ea.17.so (deflated 66%)
builder_1  |   adding: mysqlclient.libs/libkrb5support-1e00ebb8.so.0.1 (deflated 71%)
builder_1  |   adding: requirements.txt (stored 0%)
builder_1  | + chmod -R 777 /.build /.package /.wheelhouse

最后我的代码失败了 Runtime.ImportModuleError: Unable to import module 'lambda': /var/task/MySQLdb/../mysqlclient.libs/libcrypto-1763ce4d.so.1.0.2k: version 'libcrypto.so.10' not found (required by /var/task/MySQLdb/../mysqlclient.libs/libssl-54f048ca.so.1.0.2k

我的问题:为什么 libcrypto.so.10 没有复制到方向盘上,如何修复?

P.S。问题已经 exists

由于bug in the patcheif we need to use this repo with patched for auditwheel patcheif version. Full example can be found here