无法在jupyter上dd.read_sql，内核崩溃

Question

我来这里是因为我不明白我的问题。 我创建了一个 dockerfile + compose，它创建了 1 个 dask 调度程序和 2 个工作人员：

docker-compose.yaml：

version: '3.8'

services:
  dask-scheduler:
    build:
      context: .
      dockerfile: dask.Dockerfile
    command: ["dask", "scheduler", "--host", "0.0.0.0"]
    ports:
      - "50101:8786"
      - "50100:8787"
    networks:
      - default

  dask-worker:
    build:
      context: .
      dockerfile: dask.Dockerfile
    command: ["dask", "worker", "dask-scheduler:8786", "--memory-limit", "4G"]
    deploy:
      mode: replicated
      replicas: 2
    networks:
      - default

dask.Dockerfile

FROM python:3.11.0-bullseye

RUN apt update -y && \
    apt upgrade -y

RUN apt-get install -y \
    rustc \
    libpq-dev

RUN pip install --upgrade pip

RUN pip install setuptools_rust

RUN pip install \
    dask[complete] \
    bokeh \
    lz4

EXPOSE 8786
EXPOSE 8787

当我从笔记本连接到客户端时，没有问题。我什至可以运行测试：

client.submit(np.random.random, 2903192, pure=False).key

但是当我尝试 read_sql 时，内核崩溃了。

在调度程序上，我只得到这个：

dask-scheduler-1  | 2024-01-22 10:11:09,823 - distributed.scheduler - INFO - Receive client connection: Client-9063b1d4-b90e-11ee-9f28-a652689ec955
dask-scheduler-1  | 2024-01-22 10:11:09,824 - distributed.core - INFO - Starting established connection to tcp://192.168.65.1:56693
dask-scheduler-1  | 2024-01-22 10:11:12,921 - distributed.core - INFO - Connection to tcp://192.168.65.1:56693 has been closed.
dask-scheduler-1  | 2024-01-22 10:11:12,921 - distributed.scheduler - INFO - Remove client Client-9063b1d4-b90e-11ee-9f28-a652689ec955
dask-scheduler-1  | 2024-01-22 10:11:12,922 - distributed.scheduler - INFO - Close client connection: Client-9063b1d4-b90e-11ee-9f28-a652689ec955

没有任何东西发送给任何工人。

这是read_sql代码：

df = dd.read_sql_table(
    table_name="table",
    index_col='stock_qty',
    con="postgresql+psycopg2://username:password@IP:PORT/RAW"
)

你知道可能是什么问题吗？

Answer 1

我自己解决了这个问题。

我创建了一个新的anaconda环境，问题自己解决了。