Numpy种子有时不适用于dask功能

Question

在运行分布式分布式函数调用时，播种有时会失败。希望将种子值传递给大多数时间都有效的一组MC模拟试验。但不总是。该问题归结为以下示例：

from dask.distributed import Client
import numpy as np

def get_rand4seed(seedx):
    np.random.seed(seedx)
    rand1 = np.random.rand(1)[0]
    return seedx, rand1

seedrange = 100
seed_ids = np.arange(0,seedrange).tolist()

client = Client()
a = client.map(get_rand4seed, seed_ids)
results = client.gather(a)
client.close()

for result in results:
    # take seed packed in result and calculate correct 1st random number
    np.random.seed(result[0])
    correct_result = np.random.rand(1)[0]

    # comparing with 1st random number calculated in parallelized func
    comparison = 'seed=%s, dask=%s, correct=%s' % (result[0], result[1], correct_result)
    if result[1] != correct_result:
        print('DIFF: %s' % comparison)
    else:
        pass
        #print(comparison)

通常情况下，有5％到10％的情况是不正确的，并且在第10个左右的项目之后，错误可能性似乎更大。而且，有时所有100项都是正确的。结果示例：

DIFF: seed=10, dask=0.6503742417395917, correct=0.771320643266746
DIFF: seed=18, dask=0.5054533737348429, correct=0.6503742417395917
DIFF: seed=26, dask=0.038561680881409655, correct=0.30793495262497084
DIFF: seed=34, dask=0.780100460524675, correct=0.038561680881409655
DIFF: seed=69, dask=0.6063543377764754, correct=0.29624916167243354
DIFF: seed=77, dask=0.29624916167243354, correct=0.9191090317991818
DIFF: seed=85, dask=0.6575115686178157, correct=0.620373814553256
DIFF: seed=93, dask=0.3072410093435699, correct=0.6063543377764754

Answer 1

Python 3.6.9，dask 2.9.0

我无法运行您的代码...抱怨不使用if __name__ == '__main__':，然后它给了我

NameError: name 'results' is not defined
distributed.nanny - WARNING - Restarting worker

因此，我改为查看dask.Bag并重写了您的代码，如下所示：

import dask.bag as db
import numpy as np


def get_rand4seed(seedx):
    np.random.seed(seedx)
    rand1 = np.random.rand(1)[0]
    return seedx, rand1


seedrange = 100
b = db.from_sequence(np.arange(seedrange), npartitions=4)
results = b.map(get_rand4seed).compute()
for result in results:
    np.random.seed(result[0])
    correct_result = np.random.rand(1)[0]
    comparison = 'seed=%s, dask=%s, correct=%s' % (
        result[0], result[1], correct_result)
    if result[1] != correct_result:
        print('DIFF: %s' % comparison)
    else:
        pass

代码执行完美，不打印任何内容，我想这意味着一切正常。