将安装从 Google colab 复制到标准环境

问题描述 投票:0回答:1

我已经按照here的建议在Google Colab中尝试使用“pip freeze”。但安装失败。我尝试在每次失败后注释掉失败的行,以下是其中的一些:

1.

Collecting en-core-web-sm@ https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.6.0/en_core_web_sm-3.6.0-py3-none-any.whl#sha256=83276fc78a70045627144786b52e1f2728ad5e29e5e43916ec37ea9c26a11212 (from -r requirements.txt (line 81))
  Downloading https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.6.0/en_core_web_sm-3.6.0-py3-none-any.whl (12.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 12.8/12.8 MB 3.9 MB/s eta 0:00:00
Processing c:\colabtools\dist\google-colab-1.0.0.tar.gz (from -r requirements.txt (line 133))
ERROR: Could not install packages due to an OSError: [Errno 2] No such file or directory: '\\colabtools\\dist\\google-colab-1.0.0.tar.gz'
Collecting en-core-web-sm@ https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.6.0/en_core_web_sm-3.6.0-py3-none-any.whl#sha256=83276fc78a70045627144786b52e1f2728ad5e29e5e43916ec37ea9c26a11212 (from -r requirements.txt (line 81))
  Downloading https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.6.0/en_core_web_sm-3.6.0-py3-none-any.whl (12.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 12.8/12.8 MB 4.1 MB/s eta 0:00:00
ERROR: jaxlib-0.4.14+cuda11.cudnn86-cp310-cp310-manylinux2014_x86_64.whl is not a supported wheel on this platform.
Collecting en-core-web-sm@ https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.6.0/en_core_web_sm-3.6.0-py3-none-any.whl#sha256=83276fc78a70045627144786b52e1f2728ad5e29e5e43916ec37ea9c26a11212 (from -r requirements.txt (line 81))
  Using cached https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.6.0/en_core_web_sm-3.6.0-py3-none-any.whl (12.8 MB)
Collecting kobert@ git+https://[email protected]/SKTBrain/KoBERT.git@47a69af87928fc24e20f571fe10c3cc9dd9af9a3 (from -r requirements.txt (line 198))
  Cloning https://****@github.com/SKTBrain/KoBERT.git (to revision 47a69af87928fc24e20f571fe10c3cc9dd9af9a3) to c:\users\asus\appdata\local\temp\pip-install-4vl8uu39\kobert_b3930649fc264e328975d602c51cea3f
  Running command git clone --filter=blob:none --quiet 'https://****@github.com/SKTBrain/KoBERT.git' 'C:\Users\ASUS\AppData\Local\Temp\pip-install-4vl8uu39\kobert_b3930649fc264e328975d602c51cea3f'
  Running command git rev-parse -q --verify 'sha^47a69af87928fc24e20f571fe10c3cc9dd9af9a3'
  Running command git fetch -q 'https://****@github.com/SKTBrain/KoBERT.git' 47a69af87928fc24e20f571fe10c3cc9dd9af9a3
  Resolved https://****@github.com/SKTBrain/KoBERT.git to commit 47a69af87928fc24e20f571fe10c3cc9dd9af9a3
  Preparing metadata (setup.py) ... done
Collecting kobert-tokenizer@ git+https://github.com/SKTBrain/KoBERT.git@47a69af87928fc24e20f571fe10c3cc9dd9af9a3#subdirectory=kobert_hf (from -r requirements.txt (line 199))
  Cloning https://github.com/SKTBrain/KoBERT.git (to revision 47a69af87928fc24e20f571fe10c3cc9dd9af9a3) to c:\users\asus\appdata\local\temp\pip-install-4vl8uu39\kobert-tokenizer_32e5f9c89d88429a8f5bf8eec559742e
  Running command git clone --filter=blob:none --quiet https://github.com/SKTBrain/KoBERT.git 'C:\Users\ASUS\AppData\Local\Temp\pip-install-4vl8uu39\kobert-tokenizer_32e5f9c89d88429a8f5bf8eec559742e'
  Running command git rev-parse -q --verify 'sha^47a69af87928fc24e20f571fe10c3cc9dd9af9a3'
  Running command git fetch -q https://github.com/SKTBrain/KoBERT.git 47a69af87928fc24e20f571fe10c3cc9dd9af9a3
  Resolved https://github.com/SKTBrain/KoBERT.git to commit 47a69af87928fc24e20f571fe10c3cc9dd9af9a3
  Preparing metadata (setup.py) ... done
Collecting pafy@ git+https://github.com/Cupcakus/pafy@45f0deb067bf7c420cdf83a0529fd5274c12de18 (from -r requirements.txt (line 266))
  Cloning https://github.com/Cupcakus/pafy (to revision 45f0deb067bf7c420cdf83a0529fd5274c12de18) to c:\users\asus\appdata\local\temp\pip-install-4vl8uu39\pafy_7afda1f9b3284cbd929dd3f9d6b7ea81
  Running command git clone --filter=blob:none --quiet https://github.com/Cupcakus/pafy 'C:\Users\ASUS\AppData\Local\Temp\pip-install-4vl8uu39\pafy_7afda1f9b3284cbd929dd3f9d6b7ea81'
  Running command git rev-parse -q --verify 'sha^45f0deb067bf7c420cdf83a0529fd5274c12de18'
  Running command git fetch -q https://github.com/Cupcakus/pafy 45f0deb067bf7c420cdf83a0529fd5274c12de18
  Resolved https://github.com/Cupcakus/pafy to commit 45f0deb067bf7c420cdf83a0529fd5274c12de18
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [17 lines of output]
      Traceback (most recent call last):
        File "C:\Users\ASUS\AppData\Local\Temp\pip-install-4vl8uu39\pafy_7afda1f9b3284cbd929dd3f9d6b7ea81\pafy\pafy.py", line 48, in <module>
          import youtube_dl
      ModuleNotFoundError: No module named 'youtube_dl'

      During handling of the above exception, another exception occurred:

      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "C:\Users\ASUS\AppData\Local\Temp\pip-install-4vl8uu39\pafy_7afda1f9b3284cbd929dd3f9d6b7ea81\setup.py", line 13, in <module>
          from pafy import __version__
        File "C:\Users\ASUS\AppData\Local\Temp\pip-install-4vl8uu39\pafy_7afda1f9b3284cbd929dd3f9d6b7ea81\pafy\__init__.py", line 7, in <module>
          from .pafy import new
        File "C:\Users\ASUS\AppData\Local\Temp\pip-install-4vl8uu39\pafy_7afda1f9b3284cbd929dd3f9d6b7ea81\pafy\pafy.py", line 51, in <module>
          raise ImportError(
      ImportError: pafy: youtube-dl not found; you can use the internal backend by setting the environmental variable PAFY_BACKEND to "internal". It is not enabled by default because it is not as well maintained as the youtube-dl backend.
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

就在那时我放弃了这个过程。 我猜前两个问题是因为在 google colab 之外,并且分别没有安装 cuda。 无论如何,有更好的方法吗?也许有一种方法可以将 Colab 中的环境直接下载为虚拟环境?

根据要求添加我的requirements.txt:

# This is output of "pip freeze", don't edit it by hand.
absl-py==1.4.0
aiohttp==3.8.5
aiosignal==1.3.1
alabaster==0.7.13
albumentations==1.3.1
altair==4.2.2
annotated-types==0.5.0
anyio==3.7.1
appdirs==1.4.4
argon2-cffi==21.3.0
argon2-cffi-bindings==21.2.0
array-record==0.4.1
arviz==0.15.1
astropy==5.3.1
astunparse==1.6.3
async-timeout==4.0.2
attrs==23.1.0
audioread==3.0.0
autograd==1.6.2
Babel==2.12.1
backcall==0.2.0
beautifulsoup4==4.11.2
bleach==6.0.0
blinker==1.4
blis==0.7.10
blosc2==2.0.0
bokeh==3.2.1
boto3==1.15.18
botocore==1.18.18
branca==0.6.0
build==0.10.0
CacheControl==0.13.1
cachetools==5.3.1
catalogue==2.0.9
certifi==2023.7.22
cffi==1.15.1
chardet==4.0.0
charset-normalizer==3.2.0
chex==0.1.7
click==8.1.6
click-plugins==1.1.1
cligj==0.7.2
cloudpickle==2.2.1
cmake==3.27.1
cmdstanpy==1.1.0
colorcet==3.0.1
coloredlogs==15.0.1
colorlover==0.3.0
community==1.0.0b1
confection==0.1.1
cons==0.4.6
contextlib2==21.6.0
contourpy==1.1.0
convertdate==2.4.0
cryptography==41.0.3
cufflinks==0.17.3
cvxopt==1.3.2
cvxpy==1.3.2
cycler==0.11.0
cymem==2.0.7
Cython==0.29.36
dask==2023.8.0
datascience==0.17.6
db-dtypes==1.1.1
dbus-python==1.2.18
debugpy==1.6.6
decorator==4.4.2
defusedxml==0.7.1
distributed==2023.8.0
distro==1.7.0
dlib==19.24.2
dm-tree==0.1.8
docutils==0.18.1
dopamine-rl==4.0.6
duckdb==0.8.1
earthengine-api==0.1.363
easydict==1.10
ecos==2.0.12
editdistance==0.6.2
en-core-web-sm @ https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.6.0/en_core_web_sm-3.6.0-py3-none-any.whl#sha256=83276fc78a70045627144786b52e1f2728ad5e29e5e43916ec37ea9c26a11212
entrypoints==0.4
ephem==4.1.4
et-xmlfile==1.1.0
etils==1.4.1
etuples==0.3.9
exceptiongroup==1.1.2
fastai==2.7.12
fastcore==1.5.29
fastdownload==0.0.7
fastjsonschema==2.18.0
fastprogress==1.0.3
fastrlock==0.8.1
filelock==3.12.2
Fiona==1.9.4.post1
firebase-admin==5.3.0
Flask==2.2.5
flatbuffers==23.5.26
flax==0.7.2
folium==0.14.0
fonttools==4.42.0
frozendict==2.3.8
frozenlist==1.4.0
fsspec==2023.6.0
future==0.18.3
gast==0.4.0
gcsfs==2023.6.0
GDAL==3.4.3
gdown==4.6.6
gensim==4.3.1
geographiclib==2.0
geopandas==0.13.2
geopy==2.3.0
gin-config==0.5.0
glob2==0.7
gluonnlp==0.8.0
google==2.0.3
google-api-core==2.11.1
google-api-python-client==2.84.0
google-auth==2.17.3
google-auth-httplib2==0.1.0
google-auth-oauthlib==1.0.0
google-cloud-bigquery==3.10.0
google-cloud-bigquery-connection==1.12.1
google-cloud-bigquery-storage==2.22.0
google-cloud-core==2.3.3
google-cloud-datastore==2.15.2
google-cloud-firestore==2.11.1
google-cloud-functions==1.13.2
google-cloud-language==2.9.1
google-cloud-storage==2.8.0
google-cloud-translate==3.11.3
# google-colab @ file:///colabtools/dist/google-colab-1.0.0.tar.gz#sha256=9343e2731377b2b311ba298483c3aa4af535df3acfa3b1387f621ac7a94cb86d
google-crc32c==1.5.0
google-pasta==0.2.0
google-resumable-media==2.5.0
googleapis-common-protos==1.60.0
googledrivedownloader==0.4
graphviz==0.8.4
greenlet==2.0.2
grpc-google-iam-v1==0.12.6
grpcio==1.56.2
grpcio-status==1.48.2
gspread==3.4.2
gspread-dataframe==3.3.1
gym==0.25.2
gym-notices==0.0.8
h5netcdf==1.2.0
h5py==3.9.0
holidays==0.30
holoviews==1.17.0
html5lib==1.1
httpimport==1.3.1
httplib2==0.22.0
huggingface-hub==0.16.4
humanfriendly==10.0
humanize==4.7.0
hyperopt==0.2.7
idna==3.4
imageio==2.31.1
imageio-ffmpeg==0.4.8
imagesize==1.4.1
imbalanced-learn==0.10.1
imgaug==0.4.0
importlib-metadata==6.8.0
importlib-resources==6.0.1
imutils==0.5.4
inflect==7.0.0
iniconfig==2.0.0
intel-openmp==2023.2.0
ipykernel==5.5.6
ipython==7.34.0
ipython-genutils==0.2.0
ipython-sql==0.5.0
ipywidgets==7.7.1
itsdangerous==2.1.2
jax==0.4.14
# jaxlib @ https://storage.googleapis.com/jax-releases/cuda11/jaxlib-0.4.14+cuda11.cudnn86-cp310-cp310-manylinux2014_x86_64.whl#sha256=09c439923a785df517e3f470158dd3e46de454a01eca80935eed4ba383b5e90a
jeepney==0.7.1
jieba==0.42.1
Jinja2==3.1.2
jmespath==0.10.0
joblib==1.3.2
JPype1==1.4.1
jsonpickle==3.0.1
jsonschema==4.19.0
jsonschema-specifications==2023.7.1
jupyter-client==6.1.12
jupyter-console==6.1.0
jupyter-server==1.24.0
jupyter_core==5.3.1
jupyterlab-pygments==0.2.2
jupyterlab-widgets==3.0.8
kaggle==1.5.16
keras==2.12.0
keyring==23.5.0
kiwisolver==1.4.4
kobert @ git+https://[email protected]/SKTBrain/KoBERT.git@47a69af87928fc24e20f571fe10c3cc9dd9af9a3
kobert-tokenizer @ git+https://github.com/SKTBrain/KoBERT.git@47a69af87928fc24e20f571fe10c3cc9dd9af9a3#subdirectory=kobert_hf
konlpy==0.6.0
langcodes==3.3.0
launchpadlib==1.10.16
lazr.restfulclient==0.14.4
lazr.uri==1.0.6
lazy_loader==0.3
libclang==16.0.6
librosa==0.10.0.post2
lightgbm==4.0.0
linkify-it-py==2.0.2
lit==16.0.6
llvmlite==0.39.1
locket==1.0.0
logical-unification==0.4.6
LunarCalendar==0.0.9
lxml==4.9.3
Markdown==3.4.4
markdown-it-py==3.0.0
MarkupSafe==2.1.3
matplotlib==3.7.1
matplotlib-inline==0.1.6
matplotlib-venn==0.11.9
mdit-py-plugins==0.4.0
mdurl==0.1.2
miniKanren==1.0.3
missingno==0.5.2
mistune==0.8.4
mizani==0.9.2
mkl==2023.2.0
ml-dtypes==0.2.0
mlxtend==0.22.0
more-itertools==10.1.0
moviepy==1.0.3
mpmath==1.3.0
msgpack==1.0.5
multidict==6.0.4
multipledispatch==1.0.0
multitasking==0.0.11
murmurhash==1.0.9
music21==9.1.0
mxnet==1.9.1
mxnet-cu101==1.9.1
natsort==8.4.0
nbclient==0.8.0
nbconvert==6.5.4
nbformat==5.9.2
nest-asyncio==1.5.7
networkx==3.1
nibabel==4.0.2
nltk==3.7
notebook==6.4.8
numba==0.56.4
numexpr==2.8.5
numpy==1.23.5
oauth2client==4.1.3
oauthlib==3.2.2
onnxruntime==1.15.1
opencv-contrib-python==4.8.0.76
opencv-python==4.8.0.76
opencv-python-headless==4.8.0.76
openpyxl==3.1.2
opt-einsum==3.3.0
optax==0.1.7
orbax-checkpoint==0.3.2
osqp==0.6.2.post8
packaging==23.1
pafy @ git+https://github.com/Cupcakus/pafy@45f0deb067bf7c420cdf83a0529fd5274c12de18
pandas==1.5.3
pandas-datareader==0.10.0
pandas-gbq==0.17.9
pandocfilters==1.5.0
panel==1.2.1
param==1.13.0
parso==0.8.3
partd==1.4.0
pathlib==1.0.1
pathy==0.10.2
patsy==0.5.3
pexpect==4.8.0
pickleshare==0.7.5
Pillow==9.4.0
pip-tools==6.13.0
platformdirs==3.10.0
plotly==5.15.0
plotnine==0.12.2
pluggy==1.2.0
polars==0.17.3
pooch==1.6.0
portpicker==1.5.2
prefetch-generator==1.0.3
preshed==3.0.8
prettytable==3.8.0
proglog==0.1.10
progressbar2==4.2.0
prometheus-client==0.17.1
promise==2.3
prompt-toolkit==3.0.39
prophet==1.1.4
proto-plus==1.22.3
protobuf==3.20.3
psutil==5.9.5
psycopg2==2.9.7
ptyprocess==0.7.0
py-cpuinfo==9.0.0
py-hanspell @ git+https://github.com/ssut/py-hanspell.git@fdc6ca50c19f1c85971437a072d89d4e5ce024b8
py4j==0.10.9.7
pyarrow==9.0.0
pyasn1==0.5.0
pyasn1-modules==0.3.0
pycocotools==2.0.6
pycparser==2.21
pyct==0.5.0
pydantic==2.1.1
pydantic_core==2.4.0
pydata-google-auth==1.8.2
pydot==1.4.2
pydot-ng==2.0.0
pydotplus==2.0.2
PyDrive==1.3.1
PyDrive2==1.6.3
pyerfa==2.0.0.3
pygame==2.5.0
Pygments==2.16.1
PyGObject==3.42.1
PyJWT==2.3.0
pykospacing @ git+https://github.com/haven-jeon/PyKoSpacing.git@5bb0b3e8f62c1936a430745297821a9ecd78beb1
pymc==5.7.2
PyMeeus==0.5.12
pymystem3==0.2.0
PyOpenGL==3.1.7
pyOpenSSL==23.2.0
pyparsing==3.1.1
pyproj==3.6.0
pyproject_hooks==1.0.0
PySocks==1.7.1
pytensor==2.14.2
pytest==7.4.0
python-apt==0.0.0
python-dateutil==2.8.2
python-louvain==0.16
python-slugify==8.0.1
python-utils==3.7.0
pytz==2023.3
pyviz-comms==2.3.2
PyWavelets==1.4.1
PyYAML==6.0.1
pyzmq==23.2.1
qdldl==0.1.7.post0
qudida==0.0.4
referencing==0.30.2
regex==2023.6.3
requests==2.31.0
requests-oauthlib==1.3.1
requirements-parser==0.5.0
rich==13.5.2
rpds-py==0.9.2
rpy2==3.4.2
rsa==4.9
s3transfer==0.3.7
safetensors==0.3.2
scikit-image==0.19.3
scikit-learn==1.2.2
scipy==1.10.1
scs==3.2.3
seaborn==0.12.2
SecretStorage==3.3.1
Send2Trash==1.8.2
sentencepiece==0.1.96
shapely==2.0.1
six==1.16.0
sklearn-pandas==2.2.0
smart-open==6.3.0
sniffio==1.3.0
snowballstemmer==2.2.0
sortedcontainers==2.4.0
soundfile==0.12.1
soupsieve==2.4.1
soxr==0.3.5
soynlp==0.0.493
spacy==3.6.1
spacy-legacy==3.0.12
spacy-loggers==1.0.4
Sphinx==5.0.2
sphinxcontrib-applehelp==1.0.5
sphinxcontrib-devhelp==1.0.3
sphinxcontrib-htmlhelp==2.0.2
sphinxcontrib-jsmath==1.0.1
sphinxcontrib-qthelp==1.0.4
sphinxcontrib-serializinghtml==1.1.6
SQLAlchemy==2.0.19
sqlparse==0.4.4
srsly==2.4.7
statsmodels==0.14.0
sympy==1.12
tables==3.8.0
tabulate==0.9.0
tbb==2021.10.0
tblib==2.0.0
tenacity==8.2.2
tensorboard==2.12.3
tensorboard-data-server==0.7.1
tensorflow==2.12.0
tensorflow-datasets==4.9.2
tensorflow-estimator==2.12.0
tensorflow-gcs-config==2.12.0
tensorflow-hub==0.14.0
tensorflow-io-gcs-filesystem==0.33.0
tensorflow-metadata==1.13.1
tensorflow-probability==0.20.1
tensorstore==0.1.41
termcolor==2.3.0
terminado==0.17.1
text-unidecode==1.3
textblob==0.17.1
tf-slim==1.1.0
thinc==8.1.11
threadpoolctl==3.2.0
tifffile==2023.7.18
tinycss2==1.2.1
tokenizers==0.13.3
toml==0.10.2
tomli==2.0.1
toolz==0.12.0
torch @ https://download.pytorch.org/whl/cu118/torch-2.0.1%2Bcu118-cp310-cp310-linux_x86_64.whl#sha256=a7a49d459bf4862f64f7bc1a68beccf8881c2fa9f3e0569608e16ba6f85ebf7b
torchaudio @ https://download.pytorch.org/whl/cu118/torchaudio-2.0.2%2Bcu118-cp310-cp310-linux_x86_64.whl#sha256=26692645ea061a005c57ec581a2d0425210ac6ba9f923edf11cc9b0ef3a111e9
torchdata==0.6.1
torchsummary==1.5.1
torchtext==0.15.2
torchvision @ https://download.pytorch.org/whl/cu118/torchvision-0.15.2%2Bcu118-cp310-cp310-linux_x86_64.whl#sha256=19ca4ab5d6179bbe53cff79df1a855ee6533c2861ddc7389f68349d8b9f8302a
tornado==6.3.1
tqdm==4.66.0
traitlets==5.7.1
transformers==4.31.0
triton==2.0.0
tweepy==4.13.0
typer==0.9.0
types-setuptools==68.0.0.3
typing_extensions==4.7.1
tzlocal==5.0.1
uc-micro-py==1.0.2
uritemplate==4.1.1
urllib3==1.25.11
vega-datasets==0.9.0
wadllib==1.3.6
wasabi==1.1.2
wcwidth==0.2.6
webcolors==1.13
webencodings==0.5.1
websocket-client==1.6.1
Werkzeug==2.3.6
widgetsnbextension==3.6.5
wordcloud==1.9.2
wrapt==1.14.1
xarray==2023.7.0
xarray-einstats==0.6.0
xgboost==1.7.6
xlrd==2.0.1
xyzservices==2023.7.0
yarl==1.9.2
yellowbrick==1.5
yfinance==0.2.27
youtube-dl==2021.12.17
youtube-transcript-api==0.6.1
zict==3.0.0
zipp==3.16.2
python installation pip google-colaboratory python-wheel
1个回答
0
投票

您的

requirements.txt
指定以下内容:

google-colab @ file:///colabtools/dist/google-colab-1.0.0.tar.gz#sha256=9343e2731377b2b311ba298483c3aa4af535df3acfa3b1387f621ac7a94cb86d

这指向本地计算机上的一个文件,而您显然没有该文件。

在 Google 工作环境之外使用它时,您应该将其替换为 pypi 中您需要的版本:

google-colab==1.0.0
© www.soinside.com 2019 - 2024. All rights reserved.