锁在多处理中如何区分？

Question

假设您有两个使用 manager.list() 创建的列表，以及两个使用 manager.Lock() 创建的锁。您如何将每个锁分配给每个列表？我做的像

lock1 = manager.Lock()
lock2 = manager.Lock()
list1 = manager.list()
list2 = manager.list()

当我想从列表中write/read

lock1.acquire()
list1.pop(0)
lock1.release()

lock2.acquire()
list2.pop(0)
lock2.released()

今天我意识到没有任何东西可以将 lock1 与 list1 相关联。我是不是误解了这些功能？

Answer 1

TL;DR 是的，这可能是 XY problem!

如果您创建一个 multiprocessing.Manager() 并使用其方法创建容器基元（.list 和 .dict），它们将已经同步，您无需处理你自己的同步原语

from multiprocessing import Manager, Process, freeze_support

def my_function(d, lst):
    lst.append([x**2 for x in d.values()])

def main():
    with Manager() as manager:  # context-managed SyncManager
        normal_dict = {'a': 1, 'b': 2}
        managed_synchronized_dict = manager.dict(normal_dict)
        managed_synchronized_list = manager.list()  # used to store results
        p = Process(
            target=my_function,
            args=(managed_synchronized_dict, managed_synchronized_list)
        )
        p.start()
        p.join()
        print(managed_synchronized_list)

if __name__ == '__main__':
    freeze_support()
    main()

% python3 ./test_so_66603485.py
[[1, 4]]

multiprocessing.Array,也是同步的

注意：proxy objects 不能直接与其 Python collection 等价物进行比较

Note: The proxy types in multiprocessing do nothing to support comparisons by value. So, for instance, we have:
>>> manager.list([1,2,3]) == [1,2,3]
False
One should just use a copy of the referent instead when making comparisons.

一些混淆可能来自 Synchronization Primitives 上的多处理文档部分，这意味着应该使用管理器来创建同步原语，而实际上管理器已经可以为您进行同步

Synchronization primitives

Generally synchronization primitives are not as necessary in a multiprocess program as they are in a multithreaded program. See the documentation for threading module.

Note that one can also create synchronization primitives by using a manager object – see Managers.

如果简单地使用multiprocessing.Manager()、per the docs，它

Returns a started SyncManager object which can be used for sharing objects between processes. The returned manager object corresponds to a spawned child process and has methods which will create shared objects and return corresponding proxies.

来自SyncManager section

Its methods create and return Proxy Objects for a number of commonly used data types to be synchronized across processes. This notably includes shared lists and dictionaries.

这意味着您可能已经拥有了大部分想要的东西

manager object 具有构建托管类型的方法
通过Proxy Objects

最后，总结一下关于 Lock objects

实例的评论中的线程

没有固有的方法可以判断某个命名锁是用于除 meta-information 之外的任何特定内容，例如它的名称、评论、文档..相反，它们可以自由地用于无论您有什么同步需求
一些有用的 class/container 可以用来管理锁和任何它应该同步的东西——一个普通的 multiprocessing.Manager (SyncManager) 的 .list 和.dict 这样做，并且存在各种其他有用的构造，例如 Pipe and Queue
一个锁可用于同步任意数量的操作，但拥有更多锁可能很有价值trade-off，因为它们可能会不必要地阻止对资源的访问
a variety of synchronization primitives also exist 用途不同

value = my_queue.get()  # already synchronized

if not my_lock1.acquire(timeout=5):  # False if cannot acquire
    raise CustomException("failed to acquire my_lock1 after 5 seconds")
try:
    with my_lock2():    # blocks until acquired
        some_shared_mutable = some_container_of_mutables[-1]
        some_shared_mutable = some_object_with_mutables.get()
        if foo(value, some_shared_mutable):
            action1(value, some_shared_mutable)
            action2(value, some_other_shared_mutable)
finally:
    lock2.release()

锁在多处理中如何区分？

How are locks differenciated in multiprocessing?

python

mutex

locking

ipc

multiprocessing

Synchronization primitives