在 python 脚本中导入一个 python 包（这是一个 docker 容器）

Question

我正在

python script

中运行

conda enviroment

（我必须在 HPC 上执行此操作）。我需要在 python 脚本中导入的一个包（称为 scrublet）无法使用

pip or conda

安装，因此我决定使用它的 docker 容器。 你知道我该怎么做并将其导入我的 python 脚本吗？ 当我尝试安装软件包时出现此错误：

  ERROR: Command errored out with exit status 1:
   command: /mnt/miniconda/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-qvxac2gl/annoy_7749eb3100d143f695dfd00778678327/setup.py'"'"'; __file__='"'"'/tmp/pip-install-qvxac2gl/annoy_7749eb3100d143f695dfd00778678327/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-yeevfzda
       cwd: /tmp/pip-install-qvxac2gl/annoy_7749eb3100d143f695dfd00778678327/
  Complete output (16 lines):
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-3.7
  creating build/lib.linux-x86_64-3.7/annoy
  copying annoy/__init__.py -> build/lib.linux-x86_64-3.7/annoy
  copying annoy/__init__.pyi -> build/lib.linux-x86_64-3.7/annoy
  copying annoy/py.typed -> build/lib.linux-x86_64-3.7/annoy
  running build_ext
  building 'annoy.annoylib' extension
  creating build/temp.linux-x86_64-3.7
  creating build/temp.linux-x86_64-3.7/src
  gcc -pthread -B /mnt/miniconda/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/mnt/miniconda/include/python3.7m -c src/annoymodule.cc -o build/temp.linux-x86_64-3.7/src/annoymodule.o -D_CRT_SECURE_NO_WARNINGS -fpermissive -march=native -O3 -ffast-math -fno-associative-math -DANNOYLIB_MULTITHREADED_BUILD -std=c++14
  gcc: error: unrecognized command line option ‘-std=c++14’
  error: command 'gcc' failed with exit status 1
  ----------------------------------------
  ERROR: Failed building wheel for annoy
  Running setup.py clean for annoy
Successfully built scrublet
Failed to build annoy
Installing collected packages: annoy, scrublet
    Running setup.py install for annoy ... error
    ERROR: Command errored out with exit status 1:
     command: /mnt/miniconda/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-qvxac2gl/annoy_7749eb3100d143f695dfd00778678327/setup.py'"'"'; __file__='"'"'/tmp/pip-install-qvxac2gl/annoy_7749eb3100d143f695dfd00778678327/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-3asrawrx/install-record.txt --single-version-externally-managed --user --prefix= --compile --install-headers /home/moumbeib/.local/include/python3.7m/annoy
         cwd: /tmp/pip-install-qvxac2gl/annoy_7749eb3100d143f695dfd00778678327/
    Complete output (16 lines):
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-3.7
    creating build/lib.linux-x86_64-3.7/annoy
    copying annoy/__init__.py -> build/lib.linux-x86_64-3.7/annoy
    copying annoy/__init__.pyi -> build/lib.linux-x86_64-3.7/annoy
    copying annoy/py.typed -> build/lib.linux-x86_64-3.7/annoy
    running build_ext
    building 'annoy.annoylib' extension
    creating build/temp.linux-x86_64-3.7
    creating build/temp.linux-x86_64-3.7/src
    gcc -pthread -B /mnt/miniconda/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/mnt/miniconda/include/python3.7m -c src/annoymodule.cc -o build/temp.linux-x86_64-3.7/src/annoymodule.o -D_CRT_SECURE_NO_WARNINGS -fpermissive -march=native -O3 -ffast-math -fno-associative-math -DANNOYLIB_MULTITHREADED_BUILD -std=c++14
    gcc: error: unrecognized command line option ‘-std=c++14’
    error: command 'gcc' failed with exit status 1
    ----------------------------------------
ERROR: Command errored out with exit status 1: /mnt/miniconda/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-qvxac2gl/annoy_7749eb3100d143f695dfd00778678327/setup.py'"'"'; __file__='"'"'/tmp/pip-install-qvxac2gl/annoy_7749eb3100d143f695dfd00778678327/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-3asrawrx/install-record.txt --single-version-externally-managed --user --prefix= --compile --install-headers /home/moumbeib/.local/include/python3.7m/annoy Check the logs for full command output.

Answer 1

如果您的 Python 代码仅打包在 Docker 镜像中，则无法将其导入其他 Python 环境。

Docker 的一个重要特性是文件系统隔离。容器无法访问其他容器或主机中的文件，主机也无法访问容器的文件系统。像 Python 解释器这样的东西通过读取它从磁盘导入的模块的源代码来工作，如果文件只在图像或容器中，主机 Python 解释器就看不到它们。

请注意，这并不意味着无法提取文件（请参阅Docker：将文件从 Docker 容器复制到主机），只是您必须做很多工作才能使 Python

import

语句真正起作用.

这使得 Docker 不是此用例的理想打包设置。 Python 的原生打包工具相当强大，如果您可以将相关文件嵌入到 Docker 镜像中，那么您应该考虑创建一个原生 Python 包。 Python Packaging User Guide 讨论了标准工具，并介绍了创建可以使用

pip

进行安装的包的过程。