dask-kubernetes：使用大写用户名创建 pod 时出现问题

Question

我正在 GKE 上学习 dask-kubernetes。

我偶然发现了一个异步错误（错误：异步：从未检索到任务异常）。

有关问题，请参阅以下步骤。

但是，对于使用远程 Kubernetes 集群部署 dask-kubernetes 的额外指导表示赞赏（注意我使用了 helm with good experience here but want to try the native approach as I can't scale the helm approach）。

创建集群：

$ gcloud container clusters create --machine-type n1-standard-2 --num-nodes 2 --zone us-central1-b --cluster-version latest k8scluster

创建一个新的 python 环境：

$ conda create -n daskk8s python=3.8
$ conda activate daskk8s

安装包：

$ conda install -c conda-forge dask-kubernetes

通过将其放入我的 .bash_profile:

来验证项目

$ export GOOGLE_APPLICATION_CREDENTIALS="/Users/Ray/Downloads/daskk8s-f92d8191517a.json"

开始记事本：

$ jupyter notebook

运行命令和使用示例 here

from dask_kubernetes import KubeCluster, make_pod_spec

pod_spec = make_pod_spec(image='daskdev/dask:latest',
                         memory_limit='4G', memory_request='4G',
                         cpu_limit=1, cpu_request=1,
                         env={'EXTRA_PIP_PACKAGES': 'fastparquet git+https://github.com/dask/distributed'})

cluster = KubeCluster(pod_spec)

cluster.scale(1)
ERROR:asyncio:Task exception was never retrieved
future: <Task finished name='Task-8' coro=<_wrap_awaitable() done, defined at /opt/anaconda3/envs/daskk8s/lib/python3.8/asyncio/tasks.py:677> exception=ApiException()>
Traceback (most recent call last):
  File "/opt/anaconda3/envs/daskk8s/lib/python3.8/asyncio/tasks.py", line 684, in _wrap_awaitable
    return (yield from awaitable.__await__())
  File "/opt/anaconda3/envs/daskk8s/lib/python3.8/site-packages/distributed/deploy/spec.py", line 50, in _
    await self.start()
  File "/opt/anaconda3/envs/daskk8s/lib/python3.8/site-packages/dask_kubernetes/core.py", line 78, in start
    raise e
  File "/opt/anaconda3/envs/daskk8s/lib/python3.8/site-packages/dask_kubernetes/core.py", line 68, in start
    self._pod = await self.core_api.create_namespaced_pod(
  File "/opt/anaconda3/envs/daskk8s/lib/python3.8/site-packages/kubernetes_asyncio/client/api_client.py", line 162, in __call_api
    response_data = await self.request(
  File "/opt/anaconda3/envs/daskk8s/lib/python3.8/site-packages/kubernetes_asyncio/client/rest.py", line 224, in POST
    return (await self.request("POST", url,
  File "/opt/anaconda3/envs/daskk8s/lib/python3.8/site-packages/kubernetes_asyncio/client/rest.py", line 181, in request
    raise ApiException(http_resp=r)
kubernetes_asyncio.client.rest.ApiException: (422)
Reason: Unprocessable Entity
HTTP response headers: <CIMultiDictProxy('Audit-Id': '89b614f5-2e9c-46bd-8d7d-b834e1a18f1a', 'Content-Type': 'application/json', 'Date': 'Fri, 13 Mar 2020 01:21:14 GMT', 'Content-Length': '1607')>
HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"Pod \"dask-Ray-ece943dc-8lj4mj\" is invalid: [metadata.generateName: Invalid value: \"dask-Ray-ece943dc-8\": a DNS-1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*'), metadata.name: Invalid value: \"dask-Ray-ece943dc-8lj4mj\": a DNS-1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*')]","reason":"Invalid","details":{"name":"dask-Ray-ece943dc-8lj4mj","kind":"Pod","causes":[{"reason":"FieldValueInvalid","message":"Invalid value: \"dask-Ray-ece943dc-8\": a DNS-1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*')","field":"metadata.generateName"},{"reason":"FieldValueInvalid","message":"Invalid value: \"dask-Ray-ece943dc-8lj4mj\": a DNS-1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*')","field":"metadata.name"}]},"code":422}

Answer 1

将小写名称指定为 KubeCluster:

KubeCluster(pod_spec, name='ray')

dask-kubernetes：使用大写用户名创建 pod 时出现问题

dask-kubernetes: Issue creating pod with uppercase username

google-kubernetes-engine

dask

dask-kubernetes