从线性数据处理到同时处理

问题描述 投票:2回答:1

我正在尝试了解多处理。假设我在列表中存储了大量的0到1之间的值。

a = [rand() for _ in range(1e9)]

并且如果值大于0.5,我想做点什么。

for i in a:
    if i > 0.5:
        do_something()

因此将线性评估每个值。但是我可以同时评估几个值吗?

is_above = (i for i in a if i > 0.5)

import multiprocessing as mp
pool = mp.Pool()
task = pool.imap(do_something, is_above, chunksize=1e9/4) # 1e9 elements divided by 4 cores?
pool.close()
pool.join()

这段代码会按预期工作吗?

编辑:这是一个具体的示例,带有质数:

import multiprocessing as mp

def is_prime(num):
    if num > 2:
        for i in range(2, num // 2):
            if (num % i) == 0:
                break
        else:
            print(num)


start = 1_000_000_000
stop = 1_000_000_500
a = (i for i in range(start, stop))

if __name__ == '__main__':
    p = mp.Pool()
    t = p.imap(is_prime, a, chunksize=int((stop - start) / 6)) #6 cores
    p.close()
    p.join()

python multithreading iterator multiprocessing generator
1个回答
© www.soinside.com 2019 - 2024. All rights reserved.