使用消息队列的线程同步不按顺序
thread synchronization using message queues is not in order
以下代码来自Raymond's Pycon keynotes on cuncurrency,非常棒的介绍。有点长,先从问题说起:
- 如果 消息 一起添加到 队列 ,并且应该在线程被认为完成之前打印它们,他们怎么会印刷混乱?在线程初始化之间添加 100 毫秒的延迟解决了这个问题,并且输出符合预期。
-
import Queue, time
counter_queue = Queue.Queue()
counter = 0
def counter_manager():
# I have EXCLUSIVE rights to update the counter variable
global counter
while True:
increment = counter_queue.get()
counter += 1
print_queue.put(['The count is {}\n'.format(counter),
'----------\n'])
counter_queue.task_done()
t = threading.Thread(target=counter_manager)
t.daemon = True
t.start()
del t
print_queue = Queue.Queue()
def print_manager():
# I have EXCLUSIVE rights to call the print keyword
while True:
job = print_queue.get()
for line in job:
print line
print_queue.task_done()
t = threading.Thread(target=print_manager)
t.daemon = True
t.start()
del t
def worker_threads():
counter_queue.put(1)
print_queue.put(['Starting up.. with message queues'])
worker_threads = []
for i in range(10):
t = threading.Thread(target=worker)
worker_threads.append(t)
t.start()
# time.sleep(0.1)
for t in worker_threads:
t.join()
counter_queue.join()
print_queue.put(['Finishing up'])
print_queue.join()
它使用counter_manager
和print_queue
作为守护进程从工作线程接收消息,并顺序执行它们。据我了解,这应该保留执行顺序。但是,我得到以下输出:
Starting up.. with message queues
The couns is 1The couns is 2
--------------------
The couns is 3
----------The couns is 4
The couns is 5
The couns is 6
----------
----------
----------
The couns is 7
----------The couns is 8
The couns is 9----------
----------The couns is 10
----------
Finishing up
计数器正确递增,但打印消息仍然混乱。
如果我取消注释睡眠语句,线程初始化会延迟 100 毫秒并且输出是正确的
Starting up.. with message queues
The couns is 1
----------
The couns is 2
----------
The couns is 3
----------
The couns is 4
----------
The couns is 5
----------
The couns is 6
----------
The couns is 7
----------
The couns is 8
----------
The couns is 9
----------
The couns is 10
----------
Finishing up
使用队列,打印应该是有序的。这不对吗?
工人代码
def worker():
global counter
counter += 1
print 'The couns is {}'.format(counter) # read the var (race cond.)
print '----------'
我无法重现这个问题。为了达到 运行,我添加了一个 threading
导入并将 worker_threads
函数重命名为 worker
。
输出有 "couns" 但您的代码显示 "count"。您能否验证您提供的输出和代码是否一致?
您的输出似乎只在行级别交错。
我想知道以前版本的代码是否在 counter_manager
中有两个不同的 .put
(不是原子的)后来合并成一个多行的 .put
?
以下代码来自Raymond's Pycon keynotes on cuncurrency,非常棒的介绍。有点长,先从问题说起:
- 如果 消息 一起添加到 队列 ,并且应该在线程被认为完成之前打印它们,他们怎么会印刷混乱?在线程初始化之间添加 100 毫秒的延迟解决了这个问题,并且输出符合预期。
-
import Queue, time
counter_queue = Queue.Queue()
counter = 0
def counter_manager():
# I have EXCLUSIVE rights to update the counter variable
global counter
while True:
increment = counter_queue.get()
counter += 1
print_queue.put(['The count is {}\n'.format(counter),
'----------\n'])
counter_queue.task_done()
t = threading.Thread(target=counter_manager)
t.daemon = True
t.start()
del t
print_queue = Queue.Queue()
def print_manager():
# I have EXCLUSIVE rights to call the print keyword
while True:
job = print_queue.get()
for line in job:
print line
print_queue.task_done()
t = threading.Thread(target=print_manager)
t.daemon = True
t.start()
del t
def worker_threads():
counter_queue.put(1)
print_queue.put(['Starting up.. with message queues'])
worker_threads = []
for i in range(10):
t = threading.Thread(target=worker)
worker_threads.append(t)
t.start()
# time.sleep(0.1)
for t in worker_threads:
t.join()
counter_queue.join()
print_queue.put(['Finishing up'])
print_queue.join()
它使用counter_manager
和print_queue
作为守护进程从工作线程接收消息,并顺序执行它们。据我了解,这应该保留执行顺序。但是,我得到以下输出:
Starting up.. with message queues
The couns is 1The couns is 2
--------------------
The couns is 3
----------The couns is 4
The couns is 5
The couns is 6
----------
----------
----------
The couns is 7
----------The couns is 8
The couns is 9----------
----------The couns is 10
----------
Finishing up
计数器正确递增,但打印消息仍然混乱。
如果我取消注释睡眠语句,线程初始化会延迟 100 毫秒并且输出是正确的
Starting up.. with message queues
The couns is 1
----------
The couns is 2
----------
The couns is 3
----------
The couns is 4
----------
The couns is 5
----------
The couns is 6
----------
The couns is 7
----------
The couns is 8
----------
The couns is 9
----------
The couns is 10
----------
Finishing up
使用队列,打印应该是有序的。这不对吗?
工人代码
def worker():
global counter
counter += 1
print 'The couns is {}'.format(counter) # read the var (race cond.)
print '----------'
我无法重现这个问题。为了达到 运行,我添加了一个 threading
导入并将 worker_threads
函数重命名为 worker
。
输出有 "couns" 但您的代码显示 "count"。您能否验证您提供的输出和代码是否一致?
您的输出似乎只在行级别交错。
我想知道以前版本的代码是否在 counter_manager
中有两个不同的 .put
(不是原子的)后来合并成一个多行的 .put
?