通过 GKE 气流中的 KubernetesPodOperator 安装卷问题

Question

我的 DAG 卡在 运行 状态，并在堆栈驱动程序中显示一条日志消息 "had an event of type Pending"。我的 DAG 在没有卷安装的情况下工作正常，但看起来卷安装有问题。你能请人帮忙吗？

我正在尝试将“/data/storage”路径挂载到容器中的“/storage”位置。

from airflow import DAG
from airflow.contrib.operators.kubernetes_pod_operator import KubernetesPodOperator
from airflow.contrib.kubernetes.volume import Volume
from airflow.contrib.kubernetes.volume_mount import VolumeMount
from datetime import timedelta
default_args = {
                    'owner': '',
                    'start_date': '2020-03-31 03:40:34',
                    'depend_on_past': False,
                    'email_on_failure': False,
                    'email_on_retry': False,
                    'retries': 0,
                    'retry_delay': timedelta(minutes=5)
                    } 
volume_mount = VolumeMount('test-volume',
                            mount_path='/data/storage',
                            sub_path='storage',
                            read_only=True)
volume_config= {
    'persistentVolumeClaim':
      {
        'claimName': 'test-volume'
      }
    }
volume = Volume(name='test-volume', configs=volume_config)
dag = DAG('dag-name', default_args=default_args, schedule_interval=None, catchup=False, concurrency=1, max_active_runs=1)
task1 = KubernetesPodOperator(
                                    # The ID specified for the task.
                                    task_id='task1',
                                    # Name of task you want to run, used to generate Pod ID.
                                    name='task1',
                                    namespace='default',
                                    env_vars={'var1': 'value1'},
                                    image='someimage',
                                    volumes=[volume],
                                    volume_mounts=[volume_mount],
                                    arguments=['arg.json'],
                                    startup_timeout_seconds=3600,
                                    image_pull_policy='Always',
                                    dag=dag)

Answer 1

既然你已经定义了mount_path='/data/storage'，我认为没有必要添加sub_path='storage'。我们可以在 here as well as the example provided in here.

中提供的文档字符串中看到它

Answer 2

你看过这个? first you need to create a persistent volume and then update your DAG with a volume and a volume mount. Here is the full example