systemd 的气流：`airflow.pid` 与 `airflow-monitor.pid`

Question

我的 systemd 单元文件正在运行（如下）。

但是 airflow-monitor.pid 文件暂时变为只读，这有时会阻止气流启动。如果发生这种情况，我们的解决方法是删除 airflow-monitor.pid。这与 airflow.pid 不同。

看起来 airflow.pid 是 gunicorn，airflow-monitor.pid 是一个 python 进程作为 airflow 网络服务器。

系统单元文件：

[Unit]
Description=Airflow webserver daemon
After=network.target postgresql.service mysql.service redis.service rabbitmq-server.service
Wants=postgresql.service mysql.service redis.service rabbitmq-server.service

[Service]
# by default we just set $AIRFLOW_HOME to its default dir: $HOME/airflow , so lets skip this for now
EnvironmentFile=/home/airflow/airflow/airflow.systemd.environment

#WorkingDirectory=/home/airflow/airflow-venv
#Environment=PATH="/home/airflow/airflow-venv/bin:$PATH"
PIDFile=/home/airflow/airflow/airflow.pid
User=airflow
Group=airflow
Type=simple
# this was originally the file webserver.pid but did not run
#ExecStart=/bin/bash -c 'source /home/airflow/airflow-venv/bin/activate ; /home/airflow/airflow-venv/bin/airflow webserver -p 8080 --pid /home/airflow/airflow/airflow.pid --daemon'
#ExecStart=/home/airflow/airflow-venv/bin/airflow webserver -p 8080 --pid /home/airflow/airflow/airflow.pid --daemon
ExecStart=/usr/local/bin/airflow webserver -p 8080 --pid /home/airflow/airflow/airflow.pid --daemon
Restart=on-failure
RestartSec=5s
PrivateTmp=true

[Install]
WantedBy=multi-user.target

这里是 pid 文件的输出：

airflow@airflow:~$ cat airflow/airflow.pid 
8397 
airflow@airflow:~$ cat     airflow/airflow-monitor.pid 
8377 
airflow@airflow:~$ ps faux | grep 8377 
airflow 26004 0.0 0.0 14224 976 pts/0 S+ 18:05 0:00 | \_ grep --color=auto 8377 airflow 8377 0.4 1.0 399676 83804 ? Ss Aug23 6:14 /usr/bin/python /usr/local/bin/airflow webserver -p 8080 --pid /home/airflow/airflow/airflow.pid --daemon 
airflow@airflow:~$ ps faux | grep 8397 
airflow 26028 0.0 0.0 14224 940 pts/0 R+ 18:05 0:00 | \_ grep --color=auto 8397 airflow 8397 0.0 0.6 186652 55496 ? S Aug23 0:32 gunicorn: master [airflow-webserver]

Answer 1

不太清楚为什么 airflow-monitor.pid 变成只读的，但是你可以通过不运行带有 --daemon 的网络服务器完全避免这个 pid 文件。我认为 systemd 没有必要。

相关代码块：https://github.com/apache/incubator-airflow/blob/master/airflow/bin/cli.py#L754-L765

systemd 的气流：`airflow.pid` 与 `airflow-monitor.pid`

Airflow with systemd: `airflow.pid` vs `airflow-monitor.pid`

python

systemd

airflow

apache-airflow