Python

Question

我在尝试让线程 (threading.Thread) subclass 在进程 (multiprocessing.Process) subclass 中工作时遇到问题。

这是演示该问题的最简单的工作示例。我制作了一个“SubProcess”（multiprocessing.Process 的实例），它将包含一个子“WorkerThread”（threading.Thread 的实例）。执行终止于 subProcess.start()

import multiprocessing
import threading

class SubProcess(multiprocessing.Process):
    def __init__(self):
        multiprocessing.Process.__init__(self, daemon=True)

        #Instantiate this one worker
        self.workerThread = WorkerThread()

    def run(self):
        #Start the worker
        self.workerThread.start()

class WorkerThread(threading.Thread):
    def __init__(self):
        threading.Thread.__init__(self, daemon=True)

    def run(self):
        #Real work code goes here
        pass


if __name__ == '__main__':
    #Program starts here
    
    #Instantiate the SubProcess class - works fine
    subProcess = SubProcess()

    #Start the subProcess - Execution stops here - see Traceback below
    subProcess.start()

    subProcess.join()

这是输出回溯：

Traceback (most recent call last):
  File "[***]\simplestExampleError.py", line 31, in <module>
    subProcess.start()
  File "C:\Python39\lib\multiprocessing\process.py", line 121, in start
    self._popen = self._Popen(self)
  File "C:\Python39\lib\multiprocessing\context.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "C:\Python39\lib\multiprocessing\context.py", line 327, in _Popen
    return Popen(process_obj)
  File "C:\Python39\lib\multiprocessing\popen_spawn_win32.py", line 93, in __init__
    reduction.dump(process_obj, to_child)
  File "C:\Python39\lib\multiprocessing\reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
TypeError: cannot pickle '_thread.lock' object

[***]>Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Python39\lib\multiprocessing\spawn.py", line 107, in spawn_main
    new_handle = reduction.duplicate(pipe_handle,
  File "C:\Python39\lib\multiprocessing\reduction.py", line 79, in duplicate
    return _winapi.DuplicateHandle(
PermissionError: [WinError 5] Access is denied

我已经查看了类似的问题，但关键区别在于它们将“WorkerThread”定义为指向一个带有预定义参数的函数（不是 [= 的子class 32=]).
示例：在子流程中，他们定义了 workerThread = Thread(target=print_all_the_things, args=("a", self.num))

我需要能够在运行时更改 WorkerThread 的 class 变量。
示例：在子流程中，我会做类似 self.workerThread.input = "INPUT" 的事情，而两者都是运行

如有任何帮助，我们将不胜感激！

Answer 1

从您发布的代码中我看不出有任何理由（当然，您只发布了非常少的代码）为什么工作线程的实例化需要在 SubProcess.__init__ 方法中完成。我会简单地在 run 方法中初始化 workerThread 属性：

class SubProcess(multiprocessing.Process):
    def __init__(self):
        multiprocessing.Process.__init__(self, daemon=True)

    def run(self):
        #Instantiate this one worker
        self.workerThread = WorkerThread()
        #Start the worker
        self.workerThread.start()

我不认为从 Process 和 Thread classes 派生 classes 是一个致命的缺陷，但它不是很灵活。如果使用 SubProcess class 的代码想要为进程分配 name 属性怎么办？在您当前的实现中，如果您不重新设计 __init__ 方法，它就无法实现。以下实现更简洁，使 classes 可在非多处理、非多线程场景中重用（为此我可能会选择 run 以外的其他方法名称，以更好地描述由方法):

import multiprocessing
import threading

class SubProcess:
    def __init__(self):
        # set whatever attributes are required (that can be pickled)
        ...

    def run(self): # or a more descriptive method name
        #Instantiate this one worker
        worker = Worker()
        self.workerThread = threading.Thread(target=worker.run, daemon=True)
        #Start the worker
        self.workerThread.start()

class Worker:
    def __init__(self):
        # set whatever attributes are required
        ...

    def run(self): # or a more descriptive method name
        #Real work code goes here
        pass


if __name__ == '__main__':
    subprocess = SubProcess()
    # In this context, the Process does not need to be a daemon process
    # since we are waiting for its complettion:
    p = mulitprocessing.Process(target=subprocess.run, name='SubProcess')
    p.start()
    p.join()

Python - 在 Process 子类中创建 Thread 子类的问题 - 线程和多处理

Python - Issue with creating a Thread subclass within a Process subclass - threading & multiprocessing

multithreading

multiprocessing