仅限工作日的气流追赶

Question

运行 2018 年 12 月的每一天的历史导入都需要一个赶上时间表达式 '0 0 12 * * MON-FRI' 的 Dag。为什么当 dag 启动并且 catchup=True 时调度程序运行周末？ catchup 参数是否遵守调度间隔？

Answer 1

你的表达 doesn't work. But 0 0 * 12 MON-FRI or 0 0 * 12 1-5 会。

Airflow 使用 croniter，您可以在家玩：

$ cal 12 2018
   December 2018
Su Mo Tu We Th Fr Sa
                   1
 2  3  4  5  6  7  8
 9 10 11 12 13 14 15
16 17 18 19 20 21 22
23 24 25 26 27 28 29
30 31

$ python -c '
from croniter import croniter as cr
from datetime import datetime as dt
c=cr("0 0 * 12 MON-FRI", dt(2018,12,1))
for i in range(1,31):
  print(f"{i:>2}: ", c.get_next(dt))'
 1:  2018-12-03 00:00:00
 2:  2018-12-04 00:00:00
 3:  2018-12-05 00:00:00
 4:  2018-12-06 00:00:00
 5:  2018-12-07 00:00:00
 6:  2018-12-10 00:00:00
 7:  2018-12-11 00:00:00
 8:  2018-12-12 00:00:00
 9:  2018-12-13 00:00:00
...
21:  2018-12-31 00:00:00
22:  2019-12-02 00:00:00
23:  2019-12-03 00:00:00
24:  2019-12-04 00:00:00
25:  2019-12-05 00:00:00
26:  2019-12-06 00:00:00
27:  2019-12-09 00:00:00
28:  2019-12-10 00:00:00
29:  2019-12-11 00:00:00
30:  2019-12-12 00:00:00

它不应该 "run weekends" 但您可能会发现 execution_date（由 start_date 和 schedule_interval 确定）不是 DAG 的日期是运行。例如。为上面的#1 安排的 dag_run 将在 #2 过去时开始运行ning，等等。此外，默认情况下这些将是 UTC，因此运行 #5 将开始于#6 UTC，在纽约是：2018-12-09 19:00:00-05:00

参见：

python -c '
from croniter import croniter as cr; from datetime import datetime as dt
from pendulum import datetime as pdt, timezone as ptz
c=cr("0 0 * 12 MON-FRI", pdt(2018,12,1))
for i in range(1,31):
 print(f"{i:>2}: ", ptz("America/New_York").convert(c.get_next(dt)))'
 1:  2018-12-02 19:00:00-05:00
 2:  2018-12-03 19:00:00-05:00
 3:  2018-12-04 19:00:00-05:00
 4:  2018-12-05 19:00:00-05:00
 5:  2018-12-06 19:00:00-05:00
 6:  2018-12-09 19:00:00-05:00
 7:  2018-12-10 19:00:00-05:00
 8:  2018-12-11 19:00:00-05:00
 9:  2018-12-12 19:00:00-05:00
10:  2018-12-13 19:00:00-05:00
11:  2018-12-16 19:00:00-05:00
12:  2018-12-17 19:00:00-05:00
13:  2018-12-18 19:00:00-05:00
14:  2018-12-19 19:00:00-05:00
15:  2018-12-20 19:00:00-05:00
16:  2018-12-23 19:00:00-05:00
17:  2018-12-24 19:00:00-05:00
18:  2018-12-25 19:00:00-05:00
19:  2018-12-26 19:00:00-05:00
20:  2018-12-27 19:00:00-05:00
21:  2018-12-30 19:00:00-05:00
22:  2019-12-01 19:00:00-05:00
23:  2019-12-02 19:00:00-05:00
24:  2019-12-03 19:00:00-05:00
25:  2019-12-04 19:00:00-05:00
26:  2019-12-05 19:00:00-05:00
27:  2019-12-08 19:00:00-05:00
28:  2019-12-09 19:00:00-05:00
29:  2019-12-10 19:00:00-05:00
30:  2019-12-11 19:00:00-05:00

仅限工作日的气流追赶

Airflow catchup weekdays only

cron

directed-acyclic-graphs

airflow