如果计划间隔小于 v1.10.2 的 Airflow.cfg 中的 min_file_process_interval 值,则未按计划拾取 Dags

Dags not being picked up as per its schedule if the schedule interval is less than min_file_process_interval value in Airflow.cfg of v1.10.2

在 airflow.cfg 文件中,我将 min_file_process_interval 值配置为 120 秒。 dag 的计划间隔为每分钟后 运行。 但它仅在每 120 秒后被安排一次(根据 min_file_process_interval 值)。这是预期的吗?

我将 min_file_process_interval 更改为 200 秒,然后在 200 秒后开始提取 dag 计划。 澄清一下,如果情况相反,即 dag 计划间隔是 2 分钟,min_file_process_interval 是 1 分钟,那么 dag 将 运行 按照它的计划运行。 下面是我的爸爸:

from airflow import DAG
from airflow.operators.bash_operator import BashOperator
from datetime import datetime, timedelta

default_args = {
    'owner': 'airflow',
    'depends_on_past': False,
    'start_date': datetime(2019, 5, 14),
    'retries': 0,
}

dag = DAG('print', catchup=False, default_args=default_args, schedule_interval='*/1 * * * *')

t1 = BashOperator(
    task_id='print_date',
    bash_command='echo `date` , ',
    dag=dag)

每气流 documents:

In cases where there are only a small number of DAG definition files, the loop could potentially process the DAG definition files many times a minute. To control the rate of DAG file processing, the min_file_process_interval can be set to a higher value. This parameter ensures that a DAG definition file is not processed more often than once every min_file_process_interval seconds.

这意味着气流将每 min_file_process_interval 秒处理一次。因此,您应该将 DAG 调度间隔设置为 min_file_process_interval 的倍数。