Apache Beam and Dataflow Fatal Python error: XXX block stack underflow

Question

按照 https://beam.apache.org/documentation/runtime/environments/ 中的说明，当我使用提到的空 Dockerfile 构建自定义容器时，我收到“致命 Python 错误：XXX 块堆栈下溢”错误我的 stackdriver 日志，我不知道为什么。有任何想法吗？提前致谢！

Docker 文件：

FROM apache/beam_python3.8_sdk:2.33.0

启动管道的命令：

python3 -m apache_beam.examples.wordcount \
  --input gs://dataflow-samples/shakespeare/kinglear.txt \
  --output "gs://valid_container/andy/counts" \
  --runner DataflowRunner \
  --project variant-transforms \
  --region us-central1 \
  --temp_location "gs://valid_container/andy//tmp/" \
  --experiment=use_runner_v2 \
  --subnetwork regions/us-central1/subnetworks/default \
  --sdk_container_image=gcr.io/my/andy-custom \

日志：

Info
2021-11-11 10:51:38.421 EST
" Building wheel for apache-beam (setup.py): still running... "
Info
2021-11-11 10:51:43.508 EST
" Building wheel for apache-beam (setup.py): finished with status 'done' "
Info
2021-11-11 10:51:43.550 EST
" Created wheel for apache-beam: filename=apache_beam-2.33.0-cp38-cp38-linux_x86_64.whl size=11599177 sha256=930936df83b0d709dc3dfa5a9dc822bef81ee90a24fe4548aca0cdca23c07fdf "
Info
2021-11-11 10:51:43.550 EST
" Stored in directory: /tmp/pip-ephem-wheel-cache-d6jhjgl1/wheels/a0/3e/f1/ebd365270e306e57577d9b1c6f6a4788937cc5ed5fc9f9a4c5 "
Info
2021-11-11 10:51:43.563 EST
"Successfully built apache-beam "
Info
2021-11-11 10:51:44.938 EST
"Installing collected packages: apache-beam "
Info
2021-11-11 10:51:44.939 EST
" Attempting uninstall: apache-beam "
Info
2021-11-11 10:51:44.941 EST
" Found existing installation: apache-beam 2.33.0 "
Info
2021-11-11 10:51:45.212 EST
" Uninstalling apache-beam-2.33.0: "
Info
2021-11-11 10:51:48.022 EST
" Successfully uninstalled apache-beam-2.33.0 "
Info
2021-11-11 10:51:50.360 EST
"Successfully installed apache-beam-2.33.0 "
Info
2021-11-11 10:51:50.360 EST
"WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv "
Info
2021-11-11 10:51:50.749 EST
"2021/11/11 15:51:50 Executing: python -m apache_beam.runners.worker.sdk_worker_main "
Info
2021-11-11 10:52:21.204 EST
"Fatal Python error: XXX block stack underflow "
Info
2021-11-11 10:52:21.205 EST
"Python runtime state: initialized "
Info
2021-11-11 10:52:21.205 EST
" "
Info
2021-11-11 10:52:21.205 EST
"Thread 0x00007fc2bdff3700 (most recent call first): "
Info
2021-11-11 10:52:21.206 EST
" File "/usr/local/lib/python3.8/threading.py", line 870 in run "
Info
2021-11-11 10:52:21.207 EST
" File "/usr/local/lib/python3.8/threading.py", line 932 in _bootstrap_inner "
Info
2021-11-11 10:52:21.207 EST
" File "/usr/local/lib/python3.8/threading.py", line 890 in _bootstrap "
Info
2021-11-11 10:52:21.207 EST
" "

Answer 1

我的猜测是 Python 版本不匹配。 python3 --version 给出了什么？是不是和容器一样3.8？

Apache Beam and Dataflow Fatal Python error: XXX block stack underflow

Apache Beam and Dataflow Fatal Python error: XXX block stack underflow

google-cloud-dataflow

apache-beam