用Dask执行Qiskit时出现问题。工作器不启动

Question

我正在尝试使用Dask执行Qiskit。当我提交一个使用Qiskit执行量子电路模拟的函数时，它不能正常工作，此外，worker开始产生回调错误。所以我决定开始在worker上预加载Qiskit，但它没有工作。工人们优雅地停止，没有任何错误。然而，导入另一个python包作为numpy，它们就能正常工作。知道为什么工人不能加载Qiskit吗？

这是一个问题的痕迹。

[user@c6601 ~]$ conda --version
conda 4.7.12
[user@c6601 ~]$ conda activate qiskit
(qiskit) [user@c6601 ~]$ 
(qiskit) [user@c6601 ~]$ python --version
Python 3.7.7
(qiskit) [user@c6601 ~]$ python -c "import qiskit; print(qiskit.__qiskit_version__)"
{'qiskit-terra': '0.14.1', 'qiskit-aer': '0.5.1', 'qiskit-ignis': '0.3.0', 'qiskit-ibmq-provider': '0.7.1', 'qiskit-aqua': None, 'qiskit': '0.19.2'}
(qiskit) [user@c6601 ~]$ dask-worker --version
dask-worker, version 2.17.0
(qiskit) [user@c6601 ~]$ dask-scheduler --scheduler-file /tmp/sched.json&
[1] 16228
(qiskit) [user@c6601 ~]$ distributed.scheduler - INFO - -----------------------------------------------
distributed.http.proxy - INFO - To route to workers diagnostics web server please install jupyter-server-proxy: python -m pip install jupyter-server-proxy
distributed.scheduler - INFO - -----------------------------------------------
distributed.scheduler - INFO - Clear task state
distributed.scheduler - INFO -   Scheduler at:    tcp://10.120.66.1:8786
distributed.scheduler - INFO -   dashboard at:                     :8787
(qiskit) [user@c6601 ~]$ dask-worker --scheduler-file /tmp/sched.json --preload "import qiskit;print(qiskit.__qiskit_version__)"
distributed.utils - INFO - Reload module tmp2c9jac8m from .py file
{'qiskit-terra': '0.14.1', 'qiskit-aer': '0.5.1', 'qiskit-ignis': '0.3.0', 'qiskit-ibmq-provider': '0.7.1', 'qiskit-aqua': '0.7.1', 'qiskit': '0.19.2'}
{'qiskit-terra': '0.14.1', 'qiskit-aer': '0.5.1', 'qiskit-ignis': '0.3.0', 'qiskit-ibmq-provider': '0.7.1', 'qiskit-aqua': '0.7.1', 'qiskit': '0.19.2'}
distributed.preloading - INFO - Import preload module: /scratch/4070613/tmp2c9jac8m.py
distributed.dask_worker - INFO - End worker

但是numpy的加载没有问题

(qiskit) [user@c6601 ~]$ dask-worker --scheduler-file /tmp/sched.json --preload "import numpy;print(numpy.__version__)"
distributed.utils - INFO - Reload module tmpm2y2lp42 from .py file
1.18.1
1.18.1
distributed.preloading - INFO - Import preload module: /scratch/4070613/tmpm2y2lp42.py
distributed.nanny - INFO -         Start Nanny at: 'tcp://10.120.66.1:46577'
distributed.utils - INFO - Reload module tmpzhdz9u4h from .py file
1.18.1
1.18.1
distributed.preloading - INFO - Import preload module: /scratch/4070613/tmpzhdz9u4h.py
distributed.worker - INFO -       Start worker at:    tcp://10.120.66.1:34459
distributed.worker - INFO -          Listening to:    tcp://10.120.66.1:34459
distributed.worker - INFO -          dashboard at:          10.120.66.1:45970
distributed.worker - INFO - Waiting to connect to:     tcp://10.120.66.1:8786
distributed.worker - INFO - -------------------------------------------------
distributed.worker - INFO -               Threads:                          4
distributed.worker - INFO -                Memory:                   21.47 GB
distributed.worker - INFO -       Local Directory: /mnt/netapp2/Home_FT2/home/cesga/user/dask-worker-space/worker-dleqkfmk
distributed.worker - INFO - -------------------------------------------------
distributed.scheduler - INFO - Register worker <Worker 'tcp://10.120.66.1:34459', name: tcp://10.120.66.1:34459, memory: 0, processing: 0>
distributed.scheduler - INFO - Starting worker compute stream, tcp://10.120.66.1:34459
distributed.core - INFO - Starting established connection
distributed.worker - INFO -         Registered to:     tcp://10.120.66.1:8786
distributed.worker - INFO - -------------------------------------------------
distributed.core - INFO - Starting established connection

Answer 1

Dask做了很多一些库没有设计好的事情。 常见的原因包括以下几点。

1. 从许多不同的线程运行函数。 一些CC++库有全局状态，如果管理不当，会在没有警告的情况下终止程序。
2. 序列化。 Dask有时需要移动Python对象。 许多库不知道如何将它们的对象变成字节并返回，所以会失败。 不过这些通常错误比较大。