arm64 - 如何安装 sudachiPy - 日语 SpaCy 需要

问题描述 投票：0回答：1

我应该在序言中说，我确实按照 

SpaCy
文档安装了 
SpaCy
库和感兴趣的模型。

pip install -U pip setuptools wheel
pip install -U 'spacy[apple]'
python -m spacy download zh_core_web_sm
python -m spacy download en_core_web_sm
python -m spacy download fr_core_news_sm
python -m spacy download de_core_news_sm
python -m spacy download ja_core_news_sm
python -m spacy download es_core_news_sm

目前一直停留在在 Docker 环境中使用 

ja_core_news_sm
的 Python 镜像在 Docker 中安装 
python:3.11-slim
。我正在我的 
SpaCy
Docker 镜像上串联安装一些其他 
arm64
管道模块，这就是冲突的根源。其他管道模型是：

我收到以下错误：

    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 71.7/71.7 MB 4.4 MB/s eta 0:00:00
 Building wheels for collected packages: sudachipy
   Building wheel for sudachipy (pyproject.toml): started
   Building wheel for sudachipy (pyproject.toml): finished with status 'error'
   error: subprocess-exited-with-error
   
   × Building wheel for sudachipy (pyproject.toml) did not run successfully.
   │ exit code: 1
   ╰─> [39 lines of output]
       running bdist_wheel
       running build
       running build_py
       creating build
       creating build/lib.linux-aarch64-cpython-311
       creating build/lib.linux-aarch64-cpython-311/sudachipy
       copying py_src/sudachipy/config.py -> build/lib.linux-aarch64-cpython-311/sudachipy
       copying py_src/sudachipy/__init__.py -> build/lib.linux-aarch64-cpython-311/sudachipy
       copying py_src/sudachipy/command_line.py -> build/lib.linux-aarch64-cpython-311/sudachipy
       copying py_src/sudachipy/errors.py -> build/lib.linux-aarch64-cpython-311/sudachipy
       creating build/lib.linux-aarch64-cpython-311/sudachipy/dictionary
       copying py_src/sudachipy/dictionary/__init__.py -> build/lib.linux-aarch64-cpython-311/sudachipy/dictionary
       creating build/lib.linux-aarch64-cpython-311/sudachipy/tokenizer
       copying py_src/sudachipy/tokenizer/__init__.py -> build/lib.linux-aarch64-cpython-311/sudachipy/tokenizer
       creating build/lib.linux-aarch64-cpython-311/sudachipy/morphemelist
       copying py_src/sudachipy/morphemelist/__init__.py -> build/lib.linux-aarch64-cpython-311/sudachipy/morphemelist
       creating build/lib.linux-aarch64-cpython-311/sudachipy/morpheme
       copying py_src/sudachipy/morpheme/__init__.py -> build/lib.linux-aarch64-cpython-311/sudachipy/morpheme
       copying py_src/sudachipy/sudachipy.pyi -> build/lib.linux-aarch64-cpython-311/sudachipy
       creating build/lib.linux-aarch64-cpython-311/sudachipy/resources
       copying py_src/sudachipy/resources/sudachi.json -> build/lib.linux-aarch64-cpython-311/sudachipy/resources
       copying py_src/sudachipy/resources/rewrite.def -> build/lib.linux-aarch64-cpython-311/sudachipy/resources
       copying py_src/sudachipy/resources/unk.def -> build/lib.linux-aarch64-cpython-311/sudachipy/resources
       copying py_src/sudachipy/resources/char.def -> build/lib.linux-aarch64-cpython-311/sudachipy/resources
       warning: build_py: byte-compiling is disabled, skipping.
       
       running build_ext
       running build_rust
       error: can't find Rust compiler
       
       If you are using an outdated pip version, it is possible a prebuilt wheel is available for this package but pip is not able to install from it. Installing from the wheel would avoid the need for a Rust compiler.
       
       To update pip, run:
       
           pip install --upgrade pip
       
       and then retry package installation.
       
       If you did intend to build this package from source, try installing a Rust compiler from your system package manager and ensure it is on the PATH during installation. Alternatively, rustup (available at https://rustup.rs) is the recommended way to download and update the Rust compiler toolchain.
       [end of output]
   
   note: This error originates from a subprocess, and is likely not a problem with pip.
   ERROR: Failed building wheel for sudachipy
 Failed to build sudachipy
 ERROR: Could not build wheels for sudachipy, which is required to install pyproject.toml-based projects

这是我的 Dockerfile：

# Use the official Python image, with Python 3.11
FROM python:3.11-slim

# Set environment variables to reduce Python bytecode generation and buffering
ENV PYTHONUNBUFFERED=1 \
    PYTHONDONTWRITEBYTECODE=1

# Set working directory
WORKDIR /app

# Install essential dependencies including Python development headers and GCC
RUN apt-get update && \
    apt-get install -y --no-install-recommends \
    libpq-dev \
    gcc \
    ffmpeg \
    python3-dev \
    libc-dev \
    && apt-get clean && \
    rm -rf /var/lib/apt/lists/*

# Update pip and install Python packages
COPY ./docker-requirements.txt /app/
RUN pip install --upgrade pip && \
    pip install --no-cache-dir -r docker-requirements.txt

# Copy application code to container
COPY . /app

# Expose the port the app runs on
EXPOSE 5000

# Make the entrypoint script executable
RUN chmod +x /app/shell_scripts/entrypoint.sh /app/shell_scripts/wait-for-it.sh /app/shell_scripts/docker-ngrok-tunnel.sh

# Define entrypoint
ENTRYPOINT ["/app/shell_scripts/entrypoint.sh"]

进一步阅读，这是一个存档模块。

这是来自 PyPi 的注释，它解释了这个问题，因为我的 Mac 具有 M2 芯片：

Binary wheels
We provide binary builds for macOS (10.14+), Windows and Linux only for x86_64 architecture. x86 32-bit architecture is not supported and is not tested. MacOS source builds seem to work on ARM-based (Aarch64) Macs, but this architecture also is not tested and require installing Rust toolchain and Cargo.

我似乎无法安装所有型号

python docker spacy rust-cargo arm64
1个回答
0
投票

感谢 @aab 在 Rust 编译器的方向上的推动。

银弹除了安装 Rust 编译器之外还升级了我的 

es
SpaCy
管道，因为 
sudachiPy
依赖于 
Rust
编译器。

# Use the official Python image, with Python 3.11
FROM python:3.11-slim

# Set environment variables to reduce Python bytecode generation and buffering
ENV PYTHONUNBUFFERED=1 \
    PYTHONDONTWRITEBYTECODE=1

# Set working directory
WORKDIR /app

# Install essential dependencies including Python development headers and GCC
RUN apt-get update && \
    apt-get install -y --no-install-recommends \
    python3-dev \
    build-essential \
    git \
    libpq-dev \
    gcc \
    ffmpeg \
    libc-dev \
    curl \
    && apt-get clean && \
    rm -rf /var/lib/apt/lists/*

# Install Rust
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
ENV PATH="/root/.cargo/bin:${PATH}"

# Update pip and install Python packages
COPY ./docker-requirements.txt /app/
RUN pip install --upgrade pip && \
    pip install --no-cache-dir -r docker-requirements.txt

# Install Cython, SpaCy and language models
RUN pip install -U pip setuptools wheel && \
    pip install -U spacy && \
    pip install --upgrade 'sudachipy>=0.6.8' && \
    python -m spacy download zh_core_web_sm && \
    python -m spacy download en_core_web_sm && \
    python -m spacy download fr_core_news_md && \
    python -m spacy download de_core_news_sm && \
    python -m spacy download es_core_news_md && \
    python -m spacy download ja_core_news_sm 

# Copy application code to container
COPY . /app

# Expose the port the app runs on
EXPOSE 5000

# Make the entrypoint script executable
RUN chmod +x /app/shell_scripts/entrypoint.sh /app/shell_scripts/wait-for-it.sh /app/shell_scripts/docker-ngrok-tunnel.sh

# Define entrypoint
ENTRYPOINT ["/app/shell_scripts/entrypoint.sh"]
最新问题
© www.soinside.com 2019 - 2024. All rights reserved.