为了解决其他人失败的 Conda 解决方案,我将使用 subdir 覆盖重新运行他们的解决方案,例如,
# mimic linux-64
CONDA_SUBDIR=linux-64 mamba create -n foo python=3.11
最近,我遇到了更多的情况,用户具有其他系统级特征,例如 CUDA 版本或 GLIBC 版本,这似乎是问题所在。举一个具体的例子,如何模仿用户报告的 CUDA v11.8 和 GLIBC v2.27 在解决
tensorflow-gpu
中遇到问题这个问题?
天真地尝试:
CONDA_SUBDIR=linux-64 mamba create -n foo python=3.11 tensorflow-gpu
导致无法解决的情况,具体列出原因:
__glibc >=2.17
,系统上缺少该功能;”__cuda
,系统上缺少该功能;”但是,我怀疑这仅仅是我的系统上的问题,而不是让我深入了解本机系统上的人会遇到的情况。
如何在没有它们的系统上模仿这些特征?
这些系统级特征称为 虚拟包,并在《Conda 用户指南》的任务 > 管理虚拟包下进行了描述。检查
conda info
将显示 virtual package:
部分,其中列出了当前检测到的系统特征。例如,
$ conda info
# [abridged output]
virtual packages : __archspec=1=skylake
__osx=11.7.10=0
__unix=0=0
文档描述可以使用CONDA_OVERRIDE_[name]
环境变量覆盖虚拟包。例如,我们可以通过以下方式模仿特定的 CUDA 版本:
$ CONDA_OVERRIDE_CUDA=11.8 conda info
# [abridged output]
virtual packages : __archspec=1=skylake
__cuda=11.8=0
__osx=11.7.10=0
__unix=0=0
解决方案中的覆盖
$ CONDA_SUBDIR=linux-64 \
CONDA_OVERRIDE_CUDA=11.8 \
CONDA_OVERRIDE_GLIBC=2.27 \
mamba create -n foo python=3.11 tensorflow-gpu
Looking for: ['python=3.11', 'tensorflow-gpu']
warning libmamba linux version not found, defaulting to '0'
conda-forge/linux-64 Using cache
conda-forge/noarch Using cache
bioconda/linux-64 Using cache
bioconda/noarch Using cache
pkgs/main/linux-64 No change
pkgs/r/noarch No change
pkgs/main/noarch No change
pkgs/r/linux-64 No change
Transaction
Prefix: /Users/user/miniforge3/envs/foo
Updating specs:
- python=3.11
- tensorflow-gpu
Package Version Build Channel Size
────────────────────────────────────────────────────────────────────────────────────────────
Install:
────────────────────────────────────────────────────────────────────────────────────────────
+ python_abi 3.11 4_cp311 conda-forge 6kB
+ _libgcc_mutex 0.1 conda_forge conda-forge 3kB
+ libstdcxx-ng 13.2.0 h7e041cc_3 conda-forge 4MB
+ ld_impl_linux-64 2.40 h41732ed_0 conda-forge 705kB
+ ca-certificates 2023.11.17 hbcca054_0 conda-forge 154kB
+ libgomp 13.2.0 h807b86a_3 conda-forge 422kB
+ _openmp_mutex 4.5 2_gnu conda-forge 24kB
+ libgcc-ng 13.2.0 h807b86a_3 conda-forge 774kB
+ libaec 1.1.2 h59595ed_1 conda-forge 35kB
+ libev 4.33 hd590300_2 conda-forge 113kB
+ libgfortran5 13.2.0 ha4646dd_3 conda-forge 1MB
+ giflib 5.2.1 h0b41bf4_3 conda-forge 77kB
+ c-ares 1.24.0 hd590300_0 conda-forge 156kB
+ ncurses 6.4 h59595ed_2 conda-forge 884kB
+ xz 5.2.6 h166bdaf_0 conda-forge 418kB
+ libzlib 1.2.13 hd590300_5 conda-forge 62kB
+ libffi 3.4.2 h7f98852_5 conda-forge 58kB
+ bzip2 1.0.8 hd590300_5 conda-forge 254kB
+ snappy 1.1.10 h9fff704_0 conda-forge 39kB
+ openssl 3.2.0 hd590300_1 conda-forge 3MB
+ libjpeg-turbo 3.0.0 hd590300_1 conda-forge 619kB
+ flatbuffers 23.5.26 h59595ed_1 conda-forge 2MB
+ cudatoolkit 11.8.0 h4ba93d1_12 conda-forge 716MB
+ keyutils 1.6.1 h166bdaf_0 conda-forge 118kB
+ re2 2023.03.02 h8c504da_0 conda-forge 201kB
+ libabseil 20230125.3 cxx17_h59595ed_0 conda-forge 1MB
+ icu 73.2 h59595ed_0 conda-forge 12MB
+ libuuid 2.38.1 h0b41bf4_0 conda-forge 34kB
+ libnsl 2.0.1 hd590300_0 conda-forge 33kB
+ libexpat 2.5.0 hcb278e6_1 conda-forge 78kB
+ libgfortran-ng 13.2.0 h69a702a_3 conda-forge 24kB
+ libedit 3.1.20191231 he28a2e2_2 conda-forge 124kB
+ readline 8.2 h8228510_1 conda-forge 281kB
+ tk 8.6.13 noxft_h4845f30_101 conda-forge 3MB
+ zstd 1.5.5 hfc55251_0 conda-forge 545kB
+ zlib 1.2.13 hd590300_5 conda-forge 93kB
+ libprotobuf 3.21.12 hfc55251_2 conda-forge 2MB
+ libpng 1.6.39 h753d276_0 conda-forge 283kB
+ libsqlite 3.44.2 h2797004_0 conda-forge 846kB
+ libnghttp2 1.58.0 h47da74e_1 conda-forge 632kB
+ libssh2 1.11.0 h0841786_0 conda-forge 271kB
+ libopenblas 0.3.25 pthreads_h413a1c8_0 conda-forge 6MB
+ krb5 1.21.2 h659d440_0 conda-forge 1MB
+ libgrpc 1.54.3 hb20ce57_0 conda-forge 6MB
+ libblas 3.9.0 20_linux64_openblas conda-forge 14kB
+ libcurl 8.5.0 hca28451_0 conda-forge 389kB
+ libcblas 3.9.0 20_linux64_openblas conda-forge 14kB
+ liblapack 3.9.0 20_linux64_openblas conda-forge 14kB
+ hdf5 1.14.3 nompi_h4f84152_100 conda-forge 4MB
+ cuda-version 11.8 h70ddcb2_2 conda-forge 21kB
+ tzdata 2023c h71feb2d_0 conda-forge Cached
+ nccl 2.19.4.1 h6103f9b_0 conda-forge 118MB
+ cudnn 8.8.0.121 hcdd5f01_4 conda-forge 479MB
+ python 3.11.7 hab00c5b_0_cpython conda-forge 31MB
+ wheel 0.42.0 pyhd8ed1ab_0 conda-forge Cached
+ setuptools 68.2.2 pyhd8ed1ab_0 conda-forge Cached
+ pip 23.3.2 pyhd8ed1ab_0 conda-forge 1MB
+ cached_property 1.5.2 pyha770c72_1 conda-forge Cached
+ pysocks 1.7.1 pyha2e5f31_6 conda-forge Cached
+ attrs 23.1.0 pyh71513ae_1 conda-forge Cached
+ pycparser 2.21 pyhd8ed1ab_0 conda-forge Cached
+ packaging 23.2 pyhd8ed1ab_0 conda-forge Cached
+ blinker 1.7.0 pyhd8ed1ab_0 conda-forge 18kB
+ pyjwt 2.8.0 pyhd8ed1ab_0 conda-forge 25kB
+ pyasn1 0.5.1 pyhd8ed1ab_0 conda-forge 64kB
+ cachetools 5.3.2 pyhd8ed1ab_0 conda-forge 15kB
+ click 8.1.7 unix_pyh707e725_0 conda-forge 84kB
+ zipp 3.17.0 pyhd8ed1ab_0 conda-forge 19kB
+ charset-normalizer 3.3.2 pyhd8ed1ab_0 conda-forge Cached
+ idna 3.6 pyhd8ed1ab_0 conda-forge Cached
+ certifi 2023.11.17 pyhd8ed1ab_0 conda-forge Cached
+ gast 0.5.4 pyhd8ed1ab_0 conda-forge 24kB
+ absl-py 2.0.0 pyhd8ed1ab_0 conda-forge 105kB
+ typing_extensions 4.9.0 pyha770c72_0 conda-forge 36kB
+ termcolor 2.3.0 pyhd8ed1ab_0 conda-forge 12kB
+ six 1.16.0 pyh6c4a22f_0 conda-forge Cached
+ python-flatbuffers 23.5.26 pyhd8ed1ab_0 conda-forge 34kB
+ keras 2.15.0 pyhd8ed1ab_0 conda-forge 900kB
+ cached-property 1.5.2 hd8ed1ab_1 conda-forge 4kB
+ pyasn1-modules 0.3.0 pyhd8ed1ab_0 conda-forge 96kB
+ rsa 4.9 pyhd8ed1ab_0 conda-forge Cached
+ importlib-metadata 7.0.0 pyha770c72_0 conda-forge 26kB
+ pyu2f 0.1.5 pyhd8ed1ab_0 conda-forge Cached
+ google-pasta 0.2.0 pyh8c360ce_0 conda-forge 43kB
+ astunparse 1.6.3 pyhd8ed1ab_0 conda-forge 16kB
+ markdown 3.5.1 pyhd8ed1ab_0 conda-forge 77kB
+ brotli-python 1.1.0 py311hb755f60_1 conda-forge 351kB
+ markupsafe 2.1.3 py311h459d7ec_1 conda-forge 27kB
+ multidict 6.0.4 py311h459d7ec_1 conda-forge 61kB
+ frozenlist 1.4.1 py311h459d7ec_0 conda-forge 61kB
+ tensorboard-data-server 0.7.0 py311h63ff55d_1 conda-forge 5MB
+ numpy 1.26.2 py311h64a7726_0 conda-forge 8MB
+ protobuf 4.21.12 py311hcafe171_0 conda-forge 389kB
+ wrapt 1.14.1 py311hd4cff14_1 conda-forge 61kB
+ grpcio 1.54.3 py311hcafe171_0 conda-forge 800kB
+ cffi 1.16.0 py311hb3a22ac_0 conda-forge 300kB
+ yarl 1.9.3 py311h459d7ec_0 conda-forge 122kB
+ h5py 3.10.0 nompi_py311hebc2b07_101 conda-forge 1MB
+ ml_dtypes 0.2.0 py311h320fe9a_2 conda-forge 703kB
+ cryptography 41.0.7 py311hcb13ee4_1 conda-forge 2MB
+ urllib3 2.1.0 pyhd8ed1ab_0 conda-forge Cached
+ werkzeug 3.0.1 pyhd8ed1ab_0 conda-forge 242kB
+ aiosignal 1.3.1 pyhd8ed1ab_0 conda-forge Cached
+ opt_einsum 3.3.0 pyhc1e730c_2 conda-forge 58kB
+ oauthlib 3.2.2 pyhd8ed1ab_0 conda-forge 92kB
+ pyopenssl 23.3.0 pyhd8ed1ab_0 conda-forge 127kB
+ requests 2.31.0 pyhd8ed1ab_0 conda-forge Cached
+ requests-oauthlib 1.3.1 pyhd8ed1ab_0 conda-forge 22kB
+ aiohttp 3.9.1 py311h459d7ec_0 conda-forge 798kB
+ google-auth 2.25.2 pyhca7485f_0 conda-forge 103kB
+ google-auth-oauthlib 1.2.0 pyhd8ed1ab_0 conda-forge 26kB
+ tensorboard 2.15.1 pyhd8ed1ab_0 conda-forge 5MB
+ tensorflow-base 2.15.0 cuda118py311h0476658_0 conda-forge 145MB
+ tensorflow-estimator 2.15.0 cuda118py311heba7942_0 conda-forge 715kB
+ tensorflow 2.15.0 cuda118py311hfabe020_0 conda-forge 39kB
+ tensorflow-gpu 2.15.0 cuda118py311h0240f8b_0 conda-forge 38kB
Summary:
Install: 116 packages
Total download: 2GB
────────────────────────────────────────────────────────────────────────────────────────────