将多处理模块与 BaseHTTPRequestHandler 一起使用
Using multiprocessing module with BaseHTTPRequestHandler
我正在尝试使用 python 多处理模块 运行 另一个使用 http.server.BaseHTTPRequestHandler 模块的线程中的服务器。我被卡住了,运行正在陷入“_thread.lock”问题。
我不想使用线程模块,因为我宁愿使用真正的多线程和多处理模块。
如果有人知道我做错了什么,或者可以指出一个好的库供我使用,那就太棒了。
import multiprocessing
from http.server import ThreadingHTTPServer, BaseHTTPRequestHandler
if __name__ == '__main__':
httpd = ThreadingHTTPServer(('localhost', 4433), BaseHTTPRequestHandler)
manager = multiprocessing.Manager()
manager.http_server = httpd
running_server = multiprocessing.Process(target=manager.http_server.serve_forever)
running_server.start()
堆栈跟踪:
File "/Users/redacted/python/test2/test1.py", line 10, in <module>
running_server.start()
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/process.py", line 121, in start
self._popen = self._Popen(self)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/context.py", line 224, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/context.py", line 284, in _Popen
return Popen(process_obj)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 32, in __init__
super().__init__(process_obj)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/popen_fork.py", line 19, in __init__
self._launch(process_obj)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 47, in _launch
reduction.dump(process_obj, fp)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
TypeError: cannot pickle '_thread.lock' object
Python 在使用多进程模块时使用 pickle 将对象传递给另一个进程。在您的情况下,httpserver 中使用的线程锁不可腌制。所以它报告错误。
您可以做的是完全像这样在另一个进程中启动 http 服务器:
import multiprocessing
from http.server import ThreadingHTTPServer, BaseHTTPRequestHandler
def startServer():
httpd = ThreadingHTTPServer(('localhost', 4433), BaseHTTPRequestHandler)
httpd.serve_forever()
if __name__ == '__main__':
manager = multiprocessing.Manager()
running_server = multiprocessing.Process(target=startServer)
running_server.start()
此外,您可能想尝试 4433 以外的其他端口。我无法连接到我的 windows 机器上的这个端口。但是如果我使用 8000 一切正常。
我正在尝试使用 python 多处理模块 运行 另一个使用 http.server.BaseHTTPRequestHandler 模块的线程中的服务器。我被卡住了,运行正在陷入“_thread.lock”问题。
我不想使用线程模块,因为我宁愿使用真正的多线程和多处理模块。
如果有人知道我做错了什么,或者可以指出一个好的库供我使用,那就太棒了。
import multiprocessing
from http.server import ThreadingHTTPServer, BaseHTTPRequestHandler
if __name__ == '__main__':
httpd = ThreadingHTTPServer(('localhost', 4433), BaseHTTPRequestHandler)
manager = multiprocessing.Manager()
manager.http_server = httpd
running_server = multiprocessing.Process(target=manager.http_server.serve_forever)
running_server.start()
堆栈跟踪:
File "/Users/redacted/python/test2/test1.py", line 10, in <module>
running_server.start()
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/process.py", line 121, in start
self._popen = self._Popen(self)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/context.py", line 224, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/context.py", line 284, in _Popen
return Popen(process_obj)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 32, in __init__
super().__init__(process_obj)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/popen_fork.py", line 19, in __init__
self._launch(process_obj)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 47, in _launch
reduction.dump(process_obj, fp)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
TypeError: cannot pickle '_thread.lock' object
Python 在使用多进程模块时使用 pickle 将对象传递给另一个进程。在您的情况下,httpserver 中使用的线程锁不可腌制。所以它报告错误。 您可以做的是完全像这样在另一个进程中启动 http 服务器:
import multiprocessing
from http.server import ThreadingHTTPServer, BaseHTTPRequestHandler
def startServer():
httpd = ThreadingHTTPServer(('localhost', 4433), BaseHTTPRequestHandler)
httpd.serve_forever()
if __name__ == '__main__':
manager = multiprocessing.Manager()
running_server = multiprocessing.Process(target=startServer)
running_server.start()
此外,您可能想尝试 4433 以外的其他端口。我无法连接到我的 windows 机器上的这个端口。但是如果我使用 8000 一切正常。