Why Celery worker gives "OSError: Socket closed"

Question

我使用 rabbitMQ 的 celery worker 在工作几分钟后不断给我一个套接字错误——见下文。我想知道问题的主要原因是什么？我认为它可能是防火墙。但是，禁用防火墙并没有解决问题。我正在 Windows 10 机器上工作。

C:\Users\user_\Desktop\Aida>celery -A tasks worker -l info -P eventlet

 -------------- celery@user-RazerBlade v4.2.0 (windowlicker)
---- **** -----
--- * ***  * -- Windows-10-10.0.17134-SP0 2018-07-13 12:27:03
-- * - **** ---
- ** ---------- [config]
- ** ---------- .> app:         tasks:0x4b95370
- ** ---------- .> transport:   amqp://guest:**@localhost:5672//
- ** ---------- .> results:     redis://
- *** --- * --- .> concurrency: 4 (eventlet)
-- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
--- ***** -----
 -------------- [queues]
                .> queueA           exchange=(direct) key=tasks.task_1
                .> queueB           exchange=(direct) key=tasks.task_2
                .> queueC           exchange=(direct) key=tasks.task_3
                .> queueD           exchange=(direct) key=tasks.task_4

[tasks]
  . tasks.task_1
  . tasks.task_2
  . tasks.task_3
  . tasks.task_4

[2018-07-13 12:27:03,119: INFO/MainProcess] Connected to amqp://guest:**@127.0.0.1:5672//
[2018-07-13 12:27:03,133: INFO/MainProcess] mingle: searching for neighbors
[2018-07-13 12:27:04,166: INFO/MainProcess] mingle: all alone
[2018-07-13 12:27:04,212: INFO/MainProcess] pidbox: Connected to amqp://guest:**@127.0.0.1:5672//.
[2018-07-13 12:27:04,217: INFO/MainProcess] celery@user-RazerBlade ready.
[2018-07-13 12:30:04,270: WARNING/MainProcess] Traceback (most recent call last):
[2018-07-13 12:30:04,271: WARNING/MainProcess] File "c:\users\user_\appdata\local\programs\python\python37-32\lib\site-packages\eventlet\hubs\selects.py", line 55, in wait
    listeners.get(fileno, noop).cb(fileno)
[2018-07-13 12:30:04,276: WARNING/MainProcess] File "c:\users\user_\appdata\local\programs\python\python37-32\lib\site-packages\celery\worker\pidbox.py", line 120, in loop
    connection.drain_events(timeout=1.0)
[2018-07-13 12:30:04,277: WARNING/MainProcess] File "c:\users\user_\appdata\local\programs\python\python37-32\lib\site-packages\kombu\connection.py", line 301, in drain_events
    return self.transport.drain_events(self.connection, **kwargs)
[2018-07-13 12:30:04,278: WARNING/MainProcess] File "c:\users\user_\appdata\local\programs\python\python37-32\lib\site-packages\kombu\transport\pyamqp.py", line 103, in drain_events
    return connection.drain_events(**kwargs)
[2018-07-13 12:30:04,279: WARNING/MainProcess] File "c:\users\user_\appdata\local\programs\python\python37-32\lib\site-packages\amqp\connection.py", line 491, in drain_events
    while not self.blocking_read(timeout):
[2018-07-13 12:30:04,281: WARNING/MainProcess] File "c:\users\user_\appdata\local\programs\python\python37-32\lib\site-packages\amqp\connection.py", line 496, in blocking_read
    frame = self.transport.read_frame()
[2018-07-13 12:30:04,285: WARNING/MainProcess] File "c:\users\user_\appdata\local\programs\python\python37-32\lib\site-packages\amqp\transport.py", line 243, in read_frame
    frame_header = read(7, True)
[2018-07-13 12:30:04,286: WARNING/MainProcess] File "c:\users\user_\appdata\local\programs\python\python37-32\lib\site-packages\amqp\transport.py", line 426, in _read
    raise IOError('Socket closed')
[2018-07-13 12:30:04,287: WARNING/MainProcess] OSError: Socket closed
[2018-07-13 12:30:04,288: WARNING/MainProcess] Removing descriptor: 756

Answer 1

我不是 windows 用户，但我会尝试给你一些提示。

(1) - 检查您的 rabbitmq 配置是否与您的连接字符串匹配（特别是在 linux 中，我可以将 rabbitmq 配置为仅侦听特定接口，这样它就不会接受连接，比如环回接口） ;我要检查的配置选项是 NODE_IP_ADDRESS

(2) - 假设你 100% 确定你的 rabbitmq 是运行我会检查 NODENAME 是否与你的实际系统配置匹配（即检查主机名的正确性）

(3) - 最明显但仍然值得检查的是你的 rabbitmq 用户是否正确（我看到你有 guest 用户，你自己创建了这个用户吗？你是否赋予它特定权限虚拟主机？）

(4) - 如果您的连接字符串正确，这会导致另一个明显的问题。我的连接字符串看起来像这样 BROKER_URL = 'amqp://my_user:very_secret_password@localhost:5672/my_vhost_name（在您的日志中，您似乎没有将连接字符串配置为使用虚拟主机）

Answer 2

我在运行芹菜工人使用 --without-heartbeat 选项时也遇到了这个错误。删除 --without-heartbeat 选项使它消失了。

我认为要正确地使 --without-heartbeat 工作，我们需要更改代理的心跳设置以阻止它断开不发送心跳的消费者。不过还没试过。

Why Celery worker gives "OSError: Socket closed"

Why Celery worker gives "OSError: Socket closed"

worker

rabbitmq

celery

broker

python-3.x