重新启动运行 supervisord 程序的 docker 会保留 pid 文件并在重新启动时导致错误

Question

我有一个 docker 通过 supervisord 运行 django celery worker，程序设置非常简单

[program:celery_priority]
command=python manage.py celery worker -E -Q priority --concurrency=2 --loglevel=ERROR
directory=/var/lib/app
stdout_events_enabled = true
stderr_events_enabled = true
stopwaitsecs = 600

[program:celery_medium]
command=python manage.py celery worker -E -Q medium --concurrency=2 --loglevel=ERROR
directory=/var/lib/app
stdout_events_enabled = true
stderr_events_enabled = true
stopwaitsecs = 600

[program:celerycam]
command=python manage.py celerycam
directory=/var/lib/app
stdout_events_enabled = true
stderr_events_enabled = true
stopwaitsecs = 600

我们的部署周期使用 fig 来管理 dockers，下面是我们的 fig.yml 文件对于 worker

的样子

worker:
  build: .docker/worker
  command: normal
  volumes_from:
    - appdata
  hostname: workerprod
  domainname: project.internal
  links:
    - redis
    - rabbit
    - appdata
    - mail

我们面临的问题是，当我们尝试使用 fig restart worker 时，supervisord 程序失败，因为它发现 pid 中存在冲突并出现以下错误

[130.211.XX.XX] out: worker_1     | celery_medium stderr | [2015-02-13 13:40:54,271: WARNING/MainProcess] ERROR: Pidfile (/tmp/med_celery.pid) already exists.
[130.211.XX.XX] out: worker_1     | Seems we're already running? (pid: 17)
[130.211.XX.XX] out: worker_1     | celery_priority stderr | [2015-02-13 13:40:54,272: WARNING/MainProcess] ERROR: Pidfile (/tmp/priority_celery.pid) already exists.
[130.211.XX.XX] out: worker_1     | Seems we're already running? (pid: 16)
[130.211.XX.XX] out: worker_1     | 2015-02-13 18:40:54,359 INFO exited: celery_medium (exit status 0; expected)
[130.211.XX.XX] out: worker_1     | 2015-02-13 18:40:54,359 INFO exited: celery_priority (exit status 0; expected)
[130.211.XX.XX] out: worker_1     | 2015-02-13 18:40:55,360 INFO success: celerycam entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)

然而，当我们使用 fig -d up worker 时它起作用了，因为显然 up fig 试图重新创建容器而不是使用现有容器。但这会导致所有链接服务也重新创建，从而丢失 RabbitMQ 数据和 Redis 缓存。

有没有办法使用简单的 fig restart worker 重新启动 docker 并确保在重新启动时清除 pid？请指教

Answer 1

创建一个 ENTRYPOINT 脚本，在运行启动您的 CMD 之前清理任何状态数据。例如

FROM someotherimage
COPY entrypoint.sh /entrypoint.sh
ENTRYPOINT ["/entrypoint.sh"]

在 entrypoint.sh 中：

#!/bin/sh
rm -f /tmp/*.pid
exec "$@"

ENTRYPOINT 脚本将在每次容器启动时运行，并确保在运行执行容器命令之前清除 /tmp 中的任何 pid 文件.

重新启动运行 supervisord 程序的 docker 会保留 pid 文件并在重新启动时导致错误

Restarting a docker that runs supervisord programs keeps pid files and causes error at restart

celery

supervisord

docker

fig