工作人员无法使用 rasterio 反序列化
Workers fails to deserialize with rasterio
在 Google 云上部署官方 Dask Helm 图表后，我用一些额外的 conda 包更新了环境，特别是 xarray 和 rasterio。
如果我尝试 运行 我的代码，我将从工作日志中返回此错误并且程序停止。
Traceback (most recent call last): File
"/opt/conda/lib/python3.7/site-packages/tornado/ioloop.py", line 743,
in _run_callback ret = callback() File
"/opt/conda/lib/python3.7/site-packages/tornado/ioloop.py", line 767,
in _discard_future_result future.result() File
"/opt/conda/lib/python3.7/site-packages/tornado/gen.py", line 742, in
run yielded = self.gen.throw(*exc_info) # type: ignore File
"/opt/conda/lib/python3.7/site-packages/distributed/worker.py", line
661, in handle_scheduler self.ensure_computing]) File
"/opt/conda/lib/python3.7/site-packages/tornado/gen.py", line 735, in
run value = future.result() File
"/opt/conda/lib/python3.7/site-packages/tornado/gen.py", line 742, in
run yielded = self.gen.throw(*exc_info) # type: ignore File
"/opt/conda/lib/python3.7/site-packages/distributed/core.py", line
386, in handle_stream msgs = yield comm.read() File
"/opt/conda/lib/python3.7/site-packages/tornado/gen.py", line 735, in
run value = future.result() File
"/opt/conda/lib/python3.7/site-packages/tornado/gen.py", line 742, in
run yielded = self.gen.throw(*exc_info) # type: ignore File
"/opt/conda/lib/python3.7/site-packages/distributed/comm/tcp.py", line
206, in read deserializers=deserializers) File
"/opt/conda/lib/python3.7/site-packages/tornado/gen.py", line 735, in
run value = future.result() File
"/opt/conda/lib/python3.7/site-packages/tornado/gen.py", line 209, in
wrapper yielded = next(result) File
"/opt/conda/lib/python3.7/site-packages/distributed/comm/utils.py",
line 82, in from_frames res = _from_frames() File
"/opt/conda/lib/python3.7/site-packages/distributed/comm/utils.py",
line 68, in _from_frames deserializers=deserializers) File
"/opt/conda/lib/python3.7/site-packages/distributed/protocol/core.py",
line 132, in loads value = _deserialize(head, fs,
deserializers=deserializers) File
"/opt/conda/lib/python3.7/site-packages/distributed/protocol/serialize.py",
line 184, in deserialize return loads(header, frames) File
"/opt/conda/lib/python3.7/site-packages/distributed/protocol/serialize.py",
line 57, in pickle_loads return pickle.loads(b''.join(frames)) File
"/opt/conda/lib/python3.7/site-packages/distributed/protocol/pickle.py",
line 59, in loads return pickle.loads(x) File
"/opt/conda/lib/python3.7/site-packages/rasterio/init.py", line
22, in from rasterio._base import gdal_version ImportError:
libzstd.so.1: cannot open shared object file: No such file or
directory
根据我的理解，问题似乎是库 libzstdl 丢失或损坏，对吗？我无法尝试重新安装它，因为我没有管理员权限。舵车基于官方 dask/docker 版本
谁能帮我看看在哪个渠道上报这个问题比较好？
您的所有客户和员工的版本似乎都不相同。请注意，您正在使用的 EXTRA_CONDA_PACKAGES= 环境变量需要在客户端和工作人员规范中同时使用，而不仅仅是一个。
您也可以尝试 client.get_versions(check=True) 验证一些对 Dask 更重要的包是否已同步。
已解决在 helm 文件中添加：
env:
- name: EXTRA_APT_PACKAGES
value : libzstd1
在 Google 云上部署官方 Dask Helm 图表后，我用一些额外的 conda 包更新了环境，特别是 xarray 和 rasterio。 如果我尝试 运行 我的代码，我将从工作日志中返回此错误并且程序停止。
Traceback (most recent call last): File "/opt/conda/lib/python3.7/site-packages/tornado/ioloop.py", line 743, in _run_callback ret = callback() File "/opt/conda/lib/python3.7/site-packages/tornado/ioloop.py", line 767, in _discard_future_result future.result() File "/opt/conda/lib/python3.7/site-packages/tornado/gen.py", line 742, in run yielded = self.gen.throw(*exc_info) # type: ignore File "/opt/conda/lib/python3.7/site-packages/distributed/worker.py", line 661, in handle_scheduler self.ensure_computing]) File "/opt/conda/lib/python3.7/site-packages/tornado/gen.py", line 735, in run value = future.result() File "/opt/conda/lib/python3.7/site-packages/tornado/gen.py", line 742, in run yielded = self.gen.throw(*exc_info) # type: ignore File "/opt/conda/lib/python3.7/site-packages/distributed/core.py", line 386, in handle_stream msgs = yield comm.read() File "/opt/conda/lib/python3.7/site-packages/tornado/gen.py", line 735, in run value = future.result() File "/opt/conda/lib/python3.7/site-packages/tornado/gen.py", line 742, in run yielded = self.gen.throw(*exc_info) # type: ignore File "/opt/conda/lib/python3.7/site-packages/distributed/comm/tcp.py", line 206, in read deserializers=deserializers) File "/opt/conda/lib/python3.7/site-packages/tornado/gen.py", line 735, in run value = future.result() File "/opt/conda/lib/python3.7/site-packages/tornado/gen.py", line 209, in wrapper yielded = next(result) File "/opt/conda/lib/python3.7/site-packages/distributed/comm/utils.py", line 82, in from_frames res = _from_frames() File "/opt/conda/lib/python3.7/site-packages/distributed/comm/utils.py", line 68, in _from_frames deserializers=deserializers) File "/opt/conda/lib/python3.7/site-packages/distributed/protocol/core.py", line 132, in loads value = _deserialize(head, fs, deserializers=deserializers) File "/opt/conda/lib/python3.7/site-packages/distributed/protocol/serialize.py", line 184, in deserialize return loads(header, frames) File "/opt/conda/lib/python3.7/site-packages/distributed/protocol/serialize.py", line 57, in pickle_loads return pickle.loads(b''.join(frames)) File "/opt/conda/lib/python3.7/site-packages/distributed/protocol/pickle.py", line 59, in loads return pickle.loads(x) File "/opt/conda/lib/python3.7/site-packages/rasterio/init.py", line 22, in from rasterio._base import gdal_version ImportError: libzstd.so.1: cannot open shared object file: No such file or directory
根据我的理解，问题似乎是库 libzstdl 丢失或损坏，对吗？我无法尝试重新安装它，因为我没有管理员权限。舵车基于官方 dask/docker 版本 谁能帮我看看在哪个渠道上报这个问题比较好？
您的所有客户和员工的版本似乎都不相同。请注意，您正在使用的 EXTRA_CONDA_PACKAGES= 环境变量需要在客户端和工作人员规范中同时使用，而不仅仅是一个。
您也可以尝试 client.get_versions(check=True) 验证一些对 Dask 更重要的包是否已同步。
已解决在 helm 文件中添加：
env:
- name: EXTRA_APT_PACKAGES
value : libzstd1