我对多处理星图的理解是否正确?
Is my understanding of multiprocessing's starmap correct?
我写了一个小程序来检查一堆哈希,我想知道这是否是 starmap 的正确应用,是否应该进行其他处理。我目前的理解是starmap应该在worker之间分工,但是好像并没有做到。
from multiprocessing import Pool
from hashlib import sha256
import datetime
start = datetime.datetime.now()
def hash(text):
return sha256(text.encode("utf-8")).hexdigest()
string = "Baguette "
def f(r, z, string):
for i in range(r, z):
j = hash(string + str(i))
if j.startswith("000000"):
print(i, j[0:6])
if __name__ == '__main__':
with Pool(8) as p:
p.starmap(f, [[0, 10000000, string]])
p.starmap(f, [[20000000, 30000000, string]])
p.starmap(f, [[30000000, 40000000, string]])
p.starmap(f, [[40000000, 50000000, string]])
print(datetime.datetime.now()-start)
Python apparently has 9 processes open, but only one is using the CPU (11.9%)
from multiprocessing import Pool
from hashlib import sha256
import datetime
start = datetime.datetime.now()
def hash(text):
return sha256(text.encode("utf-8")).hexdigest()
string = "Baguette "
def f(r, z, string):
for i in range(r, z):
j = hash(string + str(i))
if j.startswith("000000"):
print(i, j[0:6])
if __name__ == '__main__':
with Pool(8) as p:
p.starmap(f, [[0, 10000000, string], [20000000, 30000000, string], [30000000, 40000000, string], [40000000, 50000000, string]])
print(datetime.datetime.now()-start)
谢谢@flakes 的好回答:)
flakes 的评论:
我写了一个小程序来检查一堆哈希,我想知道这是否是 starmap 的正确应用,是否应该进行其他处理。我目前的理解是starmap应该在worker之间分工,但是好像并没有做到。
from multiprocessing import Pool
from hashlib import sha256
import datetime
start = datetime.datetime.now()
def hash(text):
return sha256(text.encode("utf-8")).hexdigest()
string = "Baguette "
def f(r, z, string):
for i in range(r, z):
j = hash(string + str(i))
if j.startswith("000000"):
print(i, j[0:6])
if __name__ == '__main__':
with Pool(8) as p:
p.starmap(f, [[0, 10000000, string]])
p.starmap(f, [[20000000, 30000000, string]])
p.starmap(f, [[30000000, 40000000, string]])
p.starmap(f, [[40000000, 50000000, string]])
print(datetime.datetime.now()-start)
Python apparently has 9 processes open, but only one is using the CPU (11.9%)
from multiprocessing import Pool
from hashlib import sha256
import datetime
start = datetime.datetime.now()
def hash(text):
return sha256(text.encode("utf-8")).hexdigest()
string = "Baguette "
def f(r, z, string):
for i in range(r, z):
j = hash(string + str(i))
if j.startswith("000000"):
print(i, j[0:6])
if __name__ == '__main__':
with Pool(8) as p:
p.starmap(f, [[0, 10000000, string], [20000000, 30000000, string], [30000000, 40000000, string], [40000000, 50000000, string]])
print(datetime.datetime.now()-start)
谢谢@flakes 的好回答:)
flakes 的评论: