"ERROR: Could not build wheels for pyarrow which use PEP 517 and cannot be installed directly" on armv7 architecture with Linux Debian Buster

Question

我使用 python 包 numpy、scipy、pandas 和 [=19] 为 armv7 架构构建了一个 Docker 映像=] 使用来自 piwheels 的包。基本图像是 Python:3.7-buster.

如果我是运行带有此图像的容器，容器总是重新启动并给我错误日志“ValueError：此方法需要安装 pyarrow”：

Traceback (most recent call last):
  File "main_prog.py", line 3, in <module>
    upload_data()
  File "/usr/src/app/bigquery.py", line 39, in upload_data
    job = client.load_table_from_dataframe(dataframe, table_id, job_config=job_config)  # Make an API request.
  File "/usr/local/lib/python3.7/site-packages/google/cloud/bigquery/client.py", line 2574, in load_table_from_dataframe
    raise ValueError("This method requires pyarrow to be installed")
ValueError: This method requires pyarrow to be installed

所以我尝试直接在我的 Docker 文件中安装 pyarrow：

RUN pip3 install pyarrow

在镜像构建过程中出现错误“错误：无法为使用 PEP 517 且无法直接安装的 pyarrow 构建轮子”：

> [10/11] RUN pip3 install pyarrow:
#14 164.9   copying pyarrow/tests/parquet/test_parquet_writer.py -> build/lib.linux-armv7l-3.7/pyarrow/tests/parquet
#14 164.9   running build_ext
#14 164.9   creating /tmp/pip-install-jiim0m92/pyarrow_07d2ad5142d7405fa1b4bb2fe83e0428/build/temp.linux-armv7l-3.7
#14 164.9   -- Running cmake for pyarrow
#14 164.9   cmake -DPYTHON_EXECUTABLE=/usr/local/bin/python -DPython3_EXECUTABLE=/usr/local/bin/python  -DPYARROW_BUILD_CUDA=off -DPYARROW_BUILD_FLIGHT=off -DPYARROW_BUILD_GANDIVA=off -DPYARROW_BUILD_DATASET=off -DPYARROW_BUILD_ORC=off -DPYARROW_BUILD_PARQUET=off -DPYARROW_BUILD_PLASMA=off -DPYARROW_BUILD_S3=off -DPYARROW_BUILD_HDFS=off -DPYARROW_USE_TENSORFLOW=off -DPYARROW_BUNDLE_ARROW_CPP=off -DPYARROW_BUNDLE_BOOST=off -DPYARROW_GENERATE_COVERAGE=off -DPYARROW_BOOST_USE_SHARED=on -DPYARROW_PARQUET_USE_SHARED=on -DCMAKE_BUILD_TYPE=release /tmp/pip-install-jiim0m92/pyarrow_07d2ad5142d7405fa1b4bb2fe83e0428
#14 164.9   error: command 'cmake' failed with exit status 1
#14 164.9   ----------------------------------------
#14 164.9   ERROR: Failed building wheel for pyarrow
#14 164.9 Failed to build pyarrow
#14 164.9 ERROR: Could not build wheels for pyarrow which use PEP 517 and cannot be installed directly

然后喜欢它的推荐here我试过了：

RUN pip3 install pandas-gbq==0.14.0

和

RUN pip install --upgrade 'google-cloud-bigquery[bqstorage,pandas]'

但没有任何效果，每次我都遇到与上述相同的错误。我在 piwheels nor on PyPi.

上都找不到 armv7 的 pyarrow 轮子

有人知道答案吗？感谢您的帮助！

Answer 1

我通过使用带有 Node-RED

的单独容器映像解决了这个问题

FROM nodered/node-red:latest

RUN npm install node-red-contrib-google-cloud

我可以在其中使用 google-云包。这个容器现在处理上传到 google-cloud 的任务。要将 node-red 与 docker 一起使用，我访问了 this site and this 是我安装的 google-cloud-package。

"ERROR: Could not build wheels for pyarrow which use PEP 517 and cannot be installed directly" on armv7 architecture with Linux Debian Buster

"ERROR: Could not build wheels for pyarrow which use PEP 517 and cannot be installed directly" on armv7 architecture with Linux Debian Buster

python-3.x

google-bigquery

dockerfile

pyarrow

armv7