我可以通过使用dask.distributed.LocalCluster
的空参数构造函数隐式创建dask.distributed.Client
。但是,当我尝试使用关键字参数显式定义集群时,它将失败并引发异常:TypeError: cannot pickle '_thread.RLock' object
。
下面是一个简单的复制器:
import dask.bag
from dask.distributed import Client, LocalCluster
def main():
with Client():
dask.bag.from_sequence(range(5)).map(print).compute() # prints 0 through 4
cluster = LocalCluster()
with Client(cluster=cluster): # raises TypeError: cannot pickle '_thread.RLock' object
dask.bag.from_sequence(range(5)).map(print).compute()
if __name__ == "__main__":
main()
问题是您为Client
构造函数提供了错误的关键字参数。您提供的cluster
关键字arg被解释为应该传递给工作程序的关键字参数。由于无法腌制LocalCluster
,因此会生成您看到的异常。
[Client(cluster=cluster)
应替换为Client(address=cluster)
或仅替换为Client(cluster)
。
然后是一个完整的工作示例:
import dask.bag
from dask.distributed import Client, LocalCluster
def main():
cluster = LocalCluster()
with Client(address=cluster):
dask.bag.from_sequence(range(5)).map(print).compute()
if __name__ == "__main__":
main()