pyzmq 上下文有时初始化缓慢
pyzmq context sometimes slow to initialize
我最近开始使用 ZeroMQ，并且观察到 有时 ZeroMQ 上下文的初始化可能需要多达 40 秒。通常它比那快得多。
例如：
import zmq
# the next line will sometimes take 40 seconds
# or so but is usually near instantaneous
with zmq.Context.instance() as context:
# connect / bind to socket, do stuff
pass
这种行为似乎与我是否使用上下文管理器无关，例如以下行为与上述行为相同：
import zmq
# the next line will sometimes take 40 seconds
# or so but is usually near instantaneous
context = zmq.Context.instance()
# connect / bind to socket, do stuff
context.term()
这是运行宁在 Python 3.6.8 在 Ubuntu 18.04.2
我无法确定减速的任何模式。
这种行为的可能原因是什么？
更新
根据@user3666197 的建议，我花了一些时间并设法让减速在第二个 运行 上显现出来。之后事情又回到了"normal"。
测试脚本的第二个 运行 开始减速：
= zmq.Context()] took + 704 [us] to execute
1.~ 704 us
2.~ 131 766 816 us
3.~ 145 us
4.~ 137 us
当时在 htop 上观察内存并没有显示内存已满，但交换已经处于活动状态。
更新 2
使用 @user3666197 建议脚本的 p运行ed-down 版本：
import sys
import zmq
clock = zmq.Stopwatch()
clock.start()
foo = zmq.Context()
bar = clock.stop()
print("{0:>2s}: {1:>9d}".format(sys.argv[1], bar))
foo.term()
并循环调用，例如：
for x in $(seq 1 20);
do python3 test.zmq.py $x;
done
我可以在原始服务器上重复这个问题但还没有在另一个服务器上遇到它，这表明环境问题（或者与特定发行版 and/or python 版本有关，因为 "good" 服务器较旧）。
这对我来说已经足够了，但我稍后会回来并收集足够的信息来向维护者提交一些东西。
如果你从未使用过 ZeroMQ，
你可能会喜欢先看看
，然后再深入了解更多细节
Q : " What are the possible causes of this ( only "sometimes" present ) behaviour? "
I haven't been able to establish any pattern to the slowdowns.
a ) 上面提到的那 40 秒，
可能是有意义的，但是使用 RAM 进行 swap-I/O 操作。我无法想象这样的时间块有任何其他原因（在我使用 ZeroMQ 的最后 10 年里，从来没有注意到 ZeroMQ 有接近 3/4 分钟的延迟，从来没有）
b ) 现实检查
总是在提出任何索赔之前衡量各自的实际执行时间：
让我们开始在可重复和记录的测试用例上变得系统和公平，好吗？
像“有时”这样的类别是不值得的，数据是。作为开始，我们可以使用以下模板化方法来区分任何实际延迟的根本原因：
M = "[{0:_>40s}] took +{1:_>9d} [us] to execute"
N = 0
from zmq import Stopwatch; aClk = Stopwatch()
#------------------------------
pass; aClk.start()
import zmq
pass; _ = aClk.stop(); print( M.format( "IMPORT", _ ) )
#------------------------------
pass; aClk.start()
aCTX = zmq.Context()
pass; _ = aClk.stop(); print( M.format( "= zmq.Context()", _ ) )
#------------------------------
pass; aClk.start()
aCTX.term()
pass; _ = aClk.stop(); print( M.format( ".term()", _ ) )
#------------------------------
pass; aClk.start()
context = zmq.Context.instance()
pass; _ = aClk.stop(); print( M.format( "= zmq.Context.instance()", _ ) )
#------------------------------
pass; aClk.start()
context.term()
pass; _ = aClk.stop(); print( M.format( ".term()", _ ) )
#------------------------------
pass; aClk.start()
with zmq.Context.instance() as aCtxInCtxMGR:
N +=1
pass; _ = aClk.stop(); print( M.format( "with … as aCtxInCtxMGR:", _ ) )
#------------------------------
pass; aClk.start()
aCTX = zmq.Context( 0 )
pass; _ = aClk.stop(); print( M.format( "= zmq.Context( 0 )", _ ) )
#------------------------------
pass; aClk.start()
aCTX.term()
pass; _ = aClk.stop(); print( M.format( ".term()", _ ) )
#------------------------------
pass; aClk.start()
aCTX = zmq.Context( 10 )
pass; _ = aClk.stop(); print( M.format( "= zmq.Context( 10 )", _ ) )
#------------------------------
pass; aClk.start()
aCTX.term()
pass; _ = aClk.stop(); print( M.format( ".term()", _ ) )
#------------------------------
pass; aClk.start()
aCTX = zmq.Context( 20 )
pass; _ = aClk.stop(); print( M.format( "= zmq.Context( 20 )", _ ) )
#------------------------------
pass; aClk.start()
aCTX.term()
pass; _ = aClk.stop(); print( M.format( ".term()", _ ) )
#------------------------------
pass; aClk.start()
aCTX = zmq.Context( 50 )
pass; _ = aClk.stop(); print( M.format( "= zmq.Context( 50 )", _ ) )
#------------------------------
pass; aClk.start()
aCTX.term()
pass; _ = aClk.stop(); print( M.format( ".term()", _ ) )
#------------------------------
pass; aClk.start()
aCTX = zmq.Context( 99 )
pass; _ = aClk.stop(); print( M.format( "= zmq.Context( 99 )", _ ) )
#------------------------------
pass; aClk.start()
aCTX.term()
pass; _ = aClk.stop(); print( M.format( ".term()", _ ) )
#------------------------------
等等
广告似是而非
您的实际里程可能会有所不同
（并且将会）
[__________________________________IMPORT] took +_____1198 [us] to execute
[_________________________= zmq.Context()] took +_____1523 [us] to execute
[________________= zmq.Context.instance()] took +_____1348 [us] to execute
[_________________with … as aCtxInCtxMGR:] took +_____7909 [us] to execute
[_____________________= zmq.Context( 0 )] took +______966 [us] to execute
[_____________________= zmq.Context( 10 )] took +______944 [us] to execute
[_____________________= zmq.Context( 20 )] took +______936 [us] to execute
[_____________________= zmq.Context( 50 )] took +______962 [us] to execute
[_____________________= zmq.Context( 99 )] took +_____1015 [us] to execute
[__________________________________IMPORT] took +______879 [us] to execute
[_________________________= zmq.Context()] took +_____1076 [us] to execute
[_________________________________.term()] took +______881 [us] to execute
[________________= zmq.Context.instance()] took +_____1105 [us] to execute
[_________________________________.term()] took +______896 [us] to execute
[_________________with … as aCtxInCtxMGR:] took +_____1474 [us] to execute
[_____________________= zmq.Context( 0 )] took +_____1163 [us] to execute
[_________________________________.term()] took +______982 [us] to execute
[_____________________= zmq.Context( 10 )] took +_____2646 [us] to execute
[_________________________________.term()] took +______873 [us] to execute
[_____________________= zmq.Context( 20 )] took +_____2689 [us] to execute
[_________________________________.term()] took +______981 [us] to execute
[_____________________= zmq.Context( 50 )] took +_____2772 [us] to execute
[_________________________________.term()] took +______910 [us] to execute
[_____________________= zmq.Context( 99 )] took +_____1086 [us] to execute
[_________________________________.term()] took +_____1010 [us] to execute
我最近开始使用 ZeroMQ，并且观察到 有时 ZeroMQ 上下文的初始化可能需要多达 40 秒。通常它比那快得多。
例如：
import zmq
# the next line will sometimes take 40 seconds
# or so but is usually near instantaneous
with zmq.Context.instance() as context:
# connect / bind to socket, do stuff
pass
这种行为似乎与我是否使用上下文管理器无关，例如以下行为与上述行为相同：
import zmq
# the next line will sometimes take 40 seconds
# or so but is usually near instantaneous
context = zmq.Context.instance()
# connect / bind to socket, do stuff
context.term()
这是运行宁在 Python 3.6.8 在 Ubuntu 18.04.2
我无法确定减速的任何模式。
这种行为的可能原因是什么？
更新
根据@user3666197 的建议，我花了一些时间并设法让减速在第二个 运行 上显现出来。之后事情又回到了"normal"。
测试脚本的第二个 运行 开始减速：
= zmq.Context()] took + 704 [us] to execute
1.~ 704 us
2.~ 131 766 816 us
3.~ 145 us
4.~ 137 us
当时在 htop 上观察内存并没有显示内存已满，但交换已经处于活动状态。
更新 2
使用 @user3666197 建议脚本的 p运行ed-down 版本：
import sys
import zmq
clock = zmq.Stopwatch()
clock.start()
foo = zmq.Context()
bar = clock.stop()
print("{0:>2s}: {1:>9d}".format(sys.argv[1], bar))
foo.term()
并循环调用，例如：
for x in $(seq 1 20);
do python3 test.zmq.py $x;
done
我可以在原始服务器上重复这个问题但还没有在另一个服务器上遇到它，这表明环境问题（或者与特定发行版 and/or python 版本有关，因为 "good" 服务器较旧）。
这对我来说已经足够了，但我稍后会回来并收集足够的信息来向维护者提交一些东西。
如果你从未使用过 ZeroMQ，
你可能会喜欢先看看
，然后再深入了解更多细节
Q : " What are the possible causes of this ( only "sometimes" present ) behaviour? "
I haven't been able to establish any pattern to the slowdowns.
a ) 上面提到的那 40 秒，
可能是有意义的，但是使用 RAM 进行 swap-I/O 操作。我无法想象这样的时间块有任何其他原因（在我使用 ZeroMQ 的最后 10 年里，从来没有注意到 ZeroMQ 有接近 3/4 分钟的延迟，从来没有）
b ) 现实检查
总是在提出任何索赔之前衡量各自的实际执行时间：
让我们开始在可重复和记录的测试用例上变得系统和公平，好吗？
像“有时”这样的类别是不值得的，数据是。作为开始，我们可以使用以下模板化方法来区分任何实际延迟的根本原因：
M = "[{0:_>40s}] took +{1:_>9d} [us] to execute"
N = 0
from zmq import Stopwatch; aClk = Stopwatch()
#------------------------------
pass; aClk.start()
import zmq
pass; _ = aClk.stop(); print( M.format( "IMPORT", _ ) )
#------------------------------
pass; aClk.start()
aCTX = zmq.Context()
pass; _ = aClk.stop(); print( M.format( "= zmq.Context()", _ ) )
#------------------------------
pass; aClk.start()
aCTX.term()
pass; _ = aClk.stop(); print( M.format( ".term()", _ ) )
#------------------------------
pass; aClk.start()
context = zmq.Context.instance()
pass; _ = aClk.stop(); print( M.format( "= zmq.Context.instance()", _ ) )
#------------------------------
pass; aClk.start()
context.term()
pass; _ = aClk.stop(); print( M.format( ".term()", _ ) )
#------------------------------
pass; aClk.start()
with zmq.Context.instance() as aCtxInCtxMGR:
N +=1
pass; _ = aClk.stop(); print( M.format( "with … as aCtxInCtxMGR:", _ ) )
#------------------------------
pass; aClk.start()
aCTX = zmq.Context( 0 )
pass; _ = aClk.stop(); print( M.format( "= zmq.Context( 0 )", _ ) )
#------------------------------
pass; aClk.start()
aCTX.term()
pass; _ = aClk.stop(); print( M.format( ".term()", _ ) )
#------------------------------
pass; aClk.start()
aCTX = zmq.Context( 10 )
pass; _ = aClk.stop(); print( M.format( "= zmq.Context( 10 )", _ ) )
#------------------------------
pass; aClk.start()
aCTX.term()
pass; _ = aClk.stop(); print( M.format( ".term()", _ ) )
#------------------------------
pass; aClk.start()
aCTX = zmq.Context( 20 )
pass; _ = aClk.stop(); print( M.format( "= zmq.Context( 20 )", _ ) )
#------------------------------
pass; aClk.start()
aCTX.term()
pass; _ = aClk.stop(); print( M.format( ".term()", _ ) )
#------------------------------
pass; aClk.start()
aCTX = zmq.Context( 50 )
pass; _ = aClk.stop(); print( M.format( "= zmq.Context( 50 )", _ ) )
#------------------------------
pass; aClk.start()
aCTX.term()
pass; _ = aClk.stop(); print( M.format( ".term()", _ ) )
#------------------------------
pass; aClk.start()
aCTX = zmq.Context( 99 )
pass; _ = aClk.stop(); print( M.format( "= zmq.Context( 99 )", _ ) )
#------------------------------
pass; aClk.start()
aCTX.term()
pass; _ = aClk.stop(); print( M.format( ".term()", _ ) )
#------------------------------
等等
广告似是而非
您的实际里程可能会有所不同
（并且将会）
[__________________________________IMPORT] took +_____1198 [us] to execute
[_________________________= zmq.Context()] took +_____1523 [us] to execute
[________________= zmq.Context.instance()] took +_____1348 [us] to execute
[_________________with … as aCtxInCtxMGR:] took +_____7909 [us] to execute
[_____________________= zmq.Context( 0 )] took +______966 [us] to execute
[_____________________= zmq.Context( 10 )] took +______944 [us] to execute
[_____________________= zmq.Context( 20 )] took +______936 [us] to execute
[_____________________= zmq.Context( 50 )] took +______962 [us] to execute
[_____________________= zmq.Context( 99 )] took +_____1015 [us] to execute
[__________________________________IMPORT] took +______879 [us] to execute
[_________________________= zmq.Context()] took +_____1076 [us] to execute
[_________________________________.term()] took +______881 [us] to execute
[________________= zmq.Context.instance()] took +_____1105 [us] to execute
[_________________________________.term()] took +______896 [us] to execute
[_________________with … as aCtxInCtxMGR:] took +_____1474 [us] to execute
[_____________________= zmq.Context( 0 )] took +_____1163 [us] to execute
[_________________________________.term()] took +______982 [us] to execute
[_____________________= zmq.Context( 10 )] took +_____2646 [us] to execute
[_________________________________.term()] took +______873 [us] to execute
[_____________________= zmq.Context( 20 )] took +_____2689 [us] to execute
[_________________________________.term()] took +______981 [us] to execute
[_____________________= zmq.Context( 50 )] took +_____2772 [us] to execute
[_________________________________.term()] took +______910 [us] to execute
[_____________________= zmq.Context( 99 )] took +_____1086 [us] to execute
[_________________________________.term()] took +_____1010 [us] to execute