无法将项目从一个队列排列到另一个队列

Question

我处理两个 python 队列。
我的问题的简短描述：
客户通过 waiting queue(q1)，然后他们（客户）得到服务。
waiting queue 的大小不能大于 N（在我的程序中为 10）。
如果 waiting queue 已满，客户端将转到 outside queue(q2，大小 20)。如果外部队列已满，客户将被拒绝并且不被服务。
每个离开等待队列的客户端都允许队列外的另一个客户端加入等待队列。

使用队列应该是线程安全的。

下面我大致实现了我想要的。但我遇到了问题 - 在执行 serve 函数期间将客户端从外部队列 (q1) 排队到等待队列 (q2)。我想我丢失或忘记了一些重要的东西。我认为此语句 q1.put(client) 会永久阻塞，但不知道为什么。

import time
import threading
from random import randrange
from Queue import Queue, Full as FullQueue


class Client(object):
    def __repr__(self):
        return '<{0}: {1}>'.format(self.__class__.__name__, id(self))


def serve(q1, q2):
    while True:
        if not q2.empty():
            client = q2.get()
            print '%s leaved outside queue' % client
            q1.put(client)
            print '%s is in the waiting queue' % client
            q2.task_done()

        client = q1.get()
        print '%s leaved waiting queue for serving' % client
        time.sleep(2)  # Do something with client
        q1.task_done()


def main():
    waiting_queue = Queue(10)
    outside_queue = Queue(20)

    for _ in range(2):
        worker = threading.Thread(target=serve, args=(waiting_queue, outside_queue))
        worker.setDaemon(True)
        worker.start()

    delays = [randrange(1, 5) for _ in range(100)]

    # Every d seconds 10 clients enter to the waiting queue
    for d in delays:
        time.sleep(d)
        for _ in range(10):
            client = Client()
            try:
                waiting_queue.put_nowait(client)
            except FullQueue:
                print 'Waiting queue is full. Please line up in outside queue.'
                try:
                    outside_queue.put_nowait(client)
                except FullQueue:
                    print 'Outside queue is full. Please go out.'

    waiting_queue.join()
    outside_queue.join()
    print 'Done'

Answer 1

我终于找到了解决办法。我检查文档更仔细 If full() returns True it doesn’t guarantee that a subsequent call to get() will not blockhttps://docs.python.org/2/library/queue.html#Queue.Queue.full

这就是为什么 q1.full() 在一些线程中不可靠的原因。我在将项目插入队列并检查队列已满之前添加了互斥锁：

class Client(object):
    def __init__(self, ident):
        self.ident = ident

    def __repr__(self):
        return '<{0}: {1}>'.format(self.__class__.__name__, self.ident)


def serve(q1, q2, mutex):
    while True:
        client = q1.get()
        print '%s leaved waiting queue for serving' % client
        time.sleep(2)  # Do something with client
        q1.task_done()

        with mutex:
            if not q2.empty() and not q1.full():
                client = q2.get()
                print '%s leaved outside queue' % client
                q1.put(client)
                print '%s is in the waiting queue' % client
                q2.task_done()


def main():
    waiting_queue = Queue(10)
    outside_queue = Queue(20)

    lock = threading.RLock()

    for _ in range(2):
        worker = threading.Thread(target=serve, args=(waiting_queue, outside_queue, lock))
        worker.setDaemon(True)
        worker.start()

    # Every 1-5 seconds 10 clients enter to the waiting room
    i = 1  # Used for unique <int> client's id
    while True:
        delay = randrange(1, 5)
        time.sleep(delay)
        for _ in range(10):
            client = Client(i)
            try:
                lock.acquire()
                if not waiting_queue.full():
                    waiting_queue.put(client)
                else:
                    outside_queue.put_nowait(client)
            except FullQueue:
                # print 'Outside queue is full. Please go out.'
                pass
            finally:
                lock.release()

            i += 1

    waiting_queue.join()
    outside_queue.join()
    print 'Done'

现在效果很好。

无法将项目从一个队列排列到另一个队列

Cannot to line up item from one queue to another

python

queue

multithreading

blockingqueue

python-multithreading