pyzmq 上下文有时初始化速度很慢

问题描述 投票:0回答:1

我最近开始使用 ZeroMQ,并观察到有时ZeroMQ 上下文的初始化可能需要长达 40 秒的时间。通常比这快得多。

例如:

import zmq

# the next line will sometimes take 40 seconds
# or so but is usually near instantaneous
with zmq.Context.instance() as context:
    # connect / bind to socket, do stuff
    pass

此行为似乎与我是否使用上下文管理器无关,例如以下行为与上面相同:

import zmq

# the next line will sometimes take 40 seconds
# or so but is usually near instantaneous
context = zmq.Context.instance()

# connect / bind to socket, do stuff

context.term()

它在 Ubuntu 18.04.2 上的 Python 3.6.8 中运行

我无法建立任何放缓的模式。

造成这种行为的可能原因是什么?

更新

根据@user3666197的建议,我花了一些时间并设法在第二次运行时体现出速度减慢。此后一切恢复“正常”。

第二次运行测试脚本时速度开始变慢:

 = zmq.Context()] took +      704 [us] to execute
 1.~                          704  us
 2.~                  131 766 816  us
 3.~                          145  us
 4.~                          137  us

当时在

htop
上查看内存根本没有显示内存已满,但交换已经处于活动状态。

更新2

使用@user3666197建议脚本的精简版本:

import sys

import zmq
clock = zmq.Stopwatch()

clock.start()
foo = zmq.Context()
bar = clock.stop()
print("{0:>2s}: {1:>9d}".format(sys.argv[1], bar))
foo.term()

并循环调用它,例如:

for x in $(seq 1 20);
do python3 test.zmq.py $x;
done

我可以在原始服务器上重复这个问题但尚未在另一台服务器上遇到它,这表明存在环境问题(或者与特定发行版版本和/或Python版本作为“好”服务器有关)年纪大了)。

目前这对我来说已经足够了,但我稍后会回来并收集足够的信息以向维护者提交一些内容。

python zeromq pyzmq
1个回答
2
投票

导致这种行为(仅“有时”存在)的可能原因是什么? “
我无法建立任何放缓模式。

a)上面提到的
这 40 秒可能是有意义的,但需要 RAM 来交换 I/O 操作。我无法想象这种时间块的任何其他原因(在我使用 ZeroMQ 的最后 10 年里,我从未注意到 ZeroMQ 中有任何接近 3/4 分钟的延迟,从来没有)

b)现实检查
在提出任何索赔之前,始终测量各自的实际执行时间:
让我们开始对可重复和记录的测试用例进行系统和公平,好吗?
像“有时”这样的类别不值得,数据值得。首先,我们可以使用以下模板方法来区分任何实际延迟的根本原因:

M = "[{0:_>40s}] took +{1:_>9d} [us] to execute"
N = 0
from zmq import Stopwatch; aClk = Stopwatch()
#------------------------------
pass;                      aClk.start()
import zmq
pass;                  _ = aClk.stop(); print( M.format( "IMPORT", _ ) )
#------------------------------
pass;                      aClk.start()
aCTX = zmq.Context()
pass;                  _ = aClk.stop(); print( M.format( "= zmq.Context()", _ ) )
#------------------------------
pass;                      aClk.start()
aCTX.term()
pass;                  _ = aClk.stop(); print( M.format( ".term()", _ ) )
#------------------------------
pass;                      aClk.start()
context = zmq.Context.instance()
pass;                  _ = aClk.stop(); print( M.format( "= zmq.Context.instance()", _ ) )
#------------------------------
pass;                      aClk.start()
context.term()
pass;                  _ = aClk.stop(); print( M.format( ".term()", _ ) )
#------------------------------
pass;                      aClk.start()
with zmq.Context.instance() as aCtxInCtxMGR:
     N +=1
pass;                  _ = aClk.stop(); print( M.format( "with … as aCtxInCtxMGR:", _ ) )
#------------------------------
pass;                      aClk.start()
aCTX = zmq.Context(  0 )
pass;                  _ = aClk.stop(); print( M.format( "= zmq.Context(  0 )", _ ) )
#------------------------------
pass;                      aClk.start()
aCTX.term()
pass;                  _ = aClk.stop(); print( M.format( ".term()", _ ) )
#------------------------------
pass;                      aClk.start()
aCTX = zmq.Context( 10 )
pass;                  _ = aClk.stop(); print( M.format( "= zmq.Context( 10 )", _ ) )
#------------------------------
pass;                      aClk.start()
aCTX.term()
pass;                  _ = aClk.stop(); print( M.format( ".term()", _ ) )
#------------------------------
pass;                      aClk.start()
aCTX = zmq.Context( 20 )
pass;                  _ = aClk.stop(); print( M.format( "= zmq.Context( 20 )", _ ) )
#------------------------------
pass;                      aClk.start()
aCTX.term()
pass;                  _ = aClk.stop(); print( M.format( ".term()", _ ) )
#------------------------------
pass;                      aClk.start()
aCTX = zmq.Context( 50 )
pass;                  _ = aClk.stop(); print( M.format( "= zmq.Context( 50 )", _ ) )
#------------------------------
pass;                      aClk.start()
aCTX.term()
pass;                  _ = aClk.stop(); print( M.format( ".term()", _ ) )
#------------------------------
pass;                      aClk.start()
aCTX = zmq.Context( 99 )
pass;                  _ = aClk.stop(); print( M.format( "= zmq.Context( 99 )", _ ) )
#------------------------------
pass;                      aClk.start()
aCTX.term()
pass;                  _ = aClk.stop(); print( M.format( ".term()", _ ) )
#------------------------------

等等
广告貌似合理


您的实际里程可能会有所不同(并且将会)

[__________________________________IMPORT] took +_____1198 [us] to execute
[_________________________= zmq.Context()] took +_____1523 [us] to execute
[________________= zmq.Context.instance()] took +_____1348 [us] to execute
[_________________with … as aCtxInCtxMGR:] took +_____7909 [us] to execute
[_____________________= zmq.Context(  0 )] took +______966 [us] to execute
[_____________________= zmq.Context( 10 )] took +______944 [us] to execute
[_____________________= zmq.Context( 20 )] took +______936 [us] to execute
[_____________________= zmq.Context( 50 )] took +______962 [us] to execute
[_____________________= zmq.Context( 99 )] took +_____1015 [us] to execute

[__________________________________IMPORT] took +______879 [us] to execute
[_________________________= zmq.Context()] took +_____1076 [us] to execute
[_________________________________.term()] took +______881 [us] to execute
[________________= zmq.Context.instance()] took +_____1105 [us] to execute
[_________________________________.term()] took +______896 [us] to execute
[_________________with … as aCtxInCtxMGR:] took +_____1474 [us] to execute
[_____________________= zmq.Context(  0 )] took +_____1163 [us] to execute
[_________________________________.term()] took +______982 [us] to execute
[_____________________= zmq.Context( 10 )] took +_____2646 [us] to execute
[_________________________________.term()] took +______873 [us] to execute
[_____________________= zmq.Context( 20 )] took +_____2689 [us] to execute
[_________________________________.term()] took +______981 [us] to execute
[_____________________= zmq.Context( 50 )] took +_____2772 [us] to execute
[_________________________________.term()] took +______910 [us] to execute
[_____________________= zmq.Context( 99 )] took +_____1086 [us] to execute
[_________________________________.term()] took +_____1010 [us] to execute
© www.soinside.com 2019 - 2024. All rights reserved.