在 GitHub Actions 中缓存 python

问题描述 投票:0回答:1

我正在尝试在“自托管”GitHub Action 中缓存 Python 安装和依赖项,问题是恢复缓存的 pip 花费的时间太长并导致超时,我无法弄清楚原因。
我检查的内容:
-> 在 GitHub Actions 中缓存 python 依赖项问题
-> Github Actions 不重用缓存
-> Github actions 中缓存未正确加载

除了 GitHub 上的官方 Cache 存储库文档。以下是使用的配置

- name: Set up Python
  uses: actions/setup-python@v4
  id: python311
  with:
    python-version: '3.11.8'
    cache: 'pip'
    cache-dependency-path: 'modules/**/requirements*.txt'

- name: Cache Python dependencies
  uses: actions/[email protected]
  with:
    path: ~/.cache/pip
    key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements*.txt') }}
    restore-keys: |
      ${{ runner.os }}-pip-

- name: Cache Python interpreter
  uses: actions/[email protected]
  with:
    path: ${{ env.pythonLocation }} #/opt/hostedtoolcache/Python/3.11.8
    key: ${{ runner.os }}-python-${{ hashFiles('/opt/hostedtoolcache/Python/3.11.8') }}
    restore-keys: |
      ${{ runner.os }}-python-


- name: Install requirements
  run: |
    mkdir -p .output
    pip install -r modules/localfiles/requirements.test.txt

输出:

Version 3.11.8 was not found in the local cache
  Version 3.11.8 is available for downloading
  Download from "https://github.com/actions/python-versions/releases/download/3.11.8-7809691605/python-3.11.8-linux-20.04-x64.tar.gz"
  Extract downloaded archive
  /usr/bin/tar xz --warning=no-unknown-keyword -C /runner/_work/_temp/ee3bccf9-3f77-4c32-97e6-429f1d6adb00 -f /runner/_work/_temp/9f59a8fb-fb9d-4d39-b3df-ffb7b7288fac
  Execute installation script
  Check if Python hostedtoolcache folder exist...
  Creating Python hostedtoolcache folder...
  Create Python 3.11.8 folder
  Copy Python binaries to hostedtoolcache folder
  Create additional symlinks (Required for the UsePythonVersion Azure Pipelines task and the setup-python GitHub Action)
  Upgrading pip...
  Looking in links: /tmp/tmpr1ai5afl
  Requirement already satisfied: setuptools in /opt/hostedtoolcache/Python/3.11.8/x64/lib/python3.11/site-packages (65.5.0)
  Requirement already satisfied: pip in /opt/hostedtoolcache/Python/3.11.8/x64/lib/python3.11/site-packages (24.0)
  Collecting pip
  Downloading pip-24.0-py3-none-any.whl.metadata (3.6 kB)
  Downloading pip-24.0-py3-none-any.whl (2.1 MB)
  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.1/2.1 MB 299.7 kB/s eta 0:00:00
  Installing collected packages: pip
  Successfully installed pip-24.0
  Create complete file
  Successfully set up CPython (3.11.8)
/opt/hostedtoolcache/Python/3.11.8/x64/bin/pip cache dir
/home/runner/.cache/pip
Received 0 of 57628253 (0.0%), 0.0 MBs/sec
Received 25165824 of 57628253 (43.7%), 12.0 MBs/sec
Received 53433949 of 57628253 (92.7%), 17.0 MBs/sec
Received 53433949 of 57628253 (92.7%), 12.7 MBs/sec
Received 53433949 of 57628253 (92.7%), 10.2 MBs/sec
Received 53433949 of 57628253 (92.7%), 8.5 MBs/sec
Received 53433949 of 57628253 (92.7%), 7.3 MBs/sec
Received 53433949 of 57628253 (92.7%), 6.4 MBs/sec
Received 53433949 of 57628253 (92.7%), 5.7 MBs/sec
Received 53433949 of 57628253 (92.7%), 5.1 MBs/sec
Received 53433949 of 57628253 (92.7%), 4.6 MBs/sec
Received 53433949 of 57628253 (92.7%), 4.2 MBs/sec
Received 53433949 of 57628253 (92.7%), 3.9 MBs/sec
Received 53433949 of 57628253 (92.7%), 3.6 MBs/sec
Received 53433949 of 57628253 (92.7%), 3.4 MBs/sec
Received 53433949 of 57628253 (92.7%), 3.2 MBs/sec
Received 53433949 of 57628253 (92.7%), 3.0 MBs/sec
Received 53433949 of 57628253 (92.7%), 2.8 MBs/sec
Received 53433949 of 57628253 (92.7%), 2.7 MBs/sec
Received 53433949 of 57628253 (92.7%), 2.5 MBs/sec
Received 53433949 of 57628253 (92.7%), 2.4 MBs/sec
Received 53433949 of 57628253 (92.7%), 2.3 MBs/sec
Received 53433949 of 57628253 (92.7%), 2.2 MBs/sec
Received 53433949 of 57628253 (92.7%), 2.1 MBs/sec
Received 53433949 of 57628253 (92.7%), 2.0 MBs/sec
Received 53433949 of 57628253 (92.7%), 2.0 MBs/sec
Received 53433949 of 57628253 (92.7%), 1.9 MBs/sec
Received 53433949 of 57628253 (92.7%), 1.8 MBs/sec
Received 53433949 of 57628253 (92.7%), 1.8 MBs/sec
Received 53433949 of 57628253 (92.7%), 1.7 MBs/sec
Received 53433949 of 57628253 (92.7%), 1.6 MBs/sec
Received 53433949 of 57628253 (92.7%), 1.6 MBs/sec
Received 53433949 of 57628253 (92.7%), 1.5 MBs/sec
Received 53433949 of 57628253 (92.7%), 1.5 MBs/sec
Received 53433949 of 57628253 (92.7%), 1.5 MBs/sec
Received 53433949 of 57628253 (92.7%), 1.4 MBs/sec
Received 53433949 of 57628253 (92.7%), 1.4 MBs/sec
Received 53433949 of 57628253 (92.7%), 1.3 MBs/sec
Received 53433949 of 57628253 (92.7%), 1.3 MBs/sec
Received 53433949 of 57628253 (92.7%), 1.3 MBs/sec
Received 53433949 of 57628253 (92.7%), 1.2 MBs/sec
Received 53433949 of 57628253 (92.7%), 1.2 MBs/sec
Received 53433949 of 57628253 (92.7%), 1.2 MBs/sec
Received 53433949 of 57628253 (92.7%), 1.2 MBs/sec
Received 53433949 of 57628253 (92.7%), 1.1 MBs/sec
Received 53433949 of 57628253 (92.7%), 1.1 MBs/sec
Received 53433949 of 57628253 (92.7%), 1.1 MBs/sec
Received 53433949 of 57628253 (92.7%), 1.1 MBs/sec
Received 53433949 of 57628253 (92.7%), 1.0 MBs/sec
Received 53433949 of 57628253 (92.7%), 1.0 MBs/sec
Received 53433949 of 57628253 (92.7%), 1.0 MBs/sec
Received 53433949 of 57628253 (92.7%), 1.0 MBs/sec
Received 53433949 of 57628253 (92.7%), 1.0 MBs/sec
Received 53433949 of 57628253 (92.7%), 0.9 MBs/sec
Received 53433949 of 57628253 (92.7%), 0.9 MBs/sec
Received 53433949 of 57628253 (92.7%), 0.9 MBs/sec
Received 53433949 of 57628253 (92.7%), 0.9 MBs/sec
Received 53433949 of 57628253 (92.7%), 0.9 MBs/sec
Received 53433949 of 57628253 (92.7%), 0.9 MBs/sec
Received 53433949 of 57628253 (92.7%), 0.8 MBs/sec
Received 53433949 of 57628253 (92.7%), 0.8 MBs/sec
Received 53433949 of 57628253 (92.7%), 0.8 MBs/sec
Received 53433949 of 57628253 (92.7%), 0.8 MBs/sec
Received 53433949 of 57628253 (92.7%), 0.8 MBs/sec
Received 53433949 of 57628253 (92.7%), 0.8 MBs/sec
Received 53433949 of 57628253 (92.7%), 0.8 MBs/sec
Received 53433949 of 57628253 (92.7%), 0.8 MBs/sec
Received 53433949 of 57628253 (92.7%), 0.7 MBs/sec
Received 53433949 of 57628253 (92.7%), 0.7 MBs/sec
Received 53433949 of 57628253 (92.7%), 0.7 MBs/sec
Received 53433949 of 57628253 (92.7%), 0.7 MBs/sec
Received 53433949 of 57628253 (92.7%), 0.7 MBs/sec
Received 53433949 of 57628253 (92.7%), 0.7 MBs/sec
Received 53433949 of 57628253 (92.7%), 0.7 MBs/sec
Received 53433949 of 57628253 (92.7%), 0.7 MBs/sec
Received 53433949 of 57628253 (92.7%), 0.7 MBs/sec
Received 53433949 of 57628253 (92.7%), 0.7 MBs/sec
Received 53433949 of 57628253 (92.7%), 0.7 MBs/sec
Received 53433949 of 57628253 (92.7%), 0.6 MBs/sec
Received 53433949 of 57628253 (92.7%), 0.6 MBs/sec
Received 53433949 of 57628253 (92.7%), 0.6 MBs/sec
Received 53433949 of 57628253 (92.7%), 0.6 MBs/sec
Received 53433949 of 57628253 (92.7%), 0.6 MBs/sec
Received 53433949 of 57628253 (92.7%), 0.6 MBs/sec
Received 53433949 of 57628253 (92.7%), 0.6 MBs/sec
Received 53433949 of 57628253 (92.7%), 0.6 MBs/sec
Received 53433949 of 57628253 (92.7%), 0.6 MBs/sec
Received 53433949 of 57628253 (92.7%), 0.6 MBs/sec
Received 53433949 of 57628253 (92.7%), 0.6 MBs/sec
Received 53433949 of 57628253 (92.7%), 0.6 MBs/sec
Received 53433949 of 57628253 (92.7%), 0.6 MBs/sec
Received 53433949 of 57628253 (92.7%), 0.6 MBs/sec
Received 53433949 of 57628253 (92.7%), 0.5 MBs/sec
Received 53433949 of 57628253 (92.7%), 0.5 MBs/sec
Received 53433949 of 57628253 (92.7%), 0.5 MBs/sec
Received 53433949 of 57628253 (92.7%), 0.5 MBs/sec
Received 53433949 of 57628253 (92.7%), 0.5 MBs/sec
Received 53433949 of 57628253 (92.7%), 0.5 MBs/sec
Received 53433949 of 57628253 (92.7%), 0.5 MBs/sec
Received 53433949 of 57628253 (92.7%), 0.5 MBs/sec
Received 53433949 of 57628253 (92.7%), 0.5 MBs/sec
Warning: Failed to restore: The operation cannot be completed in timeout.
pip cache is not found
python caching pip github-actions cicd
1个回答
0
投票

我在 GitHub 中打开了一个 issue,并得到了这样的回复:

The performance of actions/setup-python action on your self-hosted runners can be influenced by a few key factors:

- Network bandwidth: Make sure that your network bandwidth is sufficient for the number of concurrent workflows.
- Disk space: Ensure your self-hosted runner has enough disk space to store the downloaded Python versions.
- Resource allocation: Check if other processes on your self-hosted runner machine are using significant system resources (CPU, memory, disk I/O).

Additionally, the setup-python action caches the initially installed Python version on your runner for future use. In subsequent runs with the identical Python version, the action retrieves the cached version, bypassing the installation process and speeding up execution as no further downloads are necessary.
Please review these aspects on your self-hosted runner. If the issue persists, kindly provide us with more detailed logs or a public repository to facilitate further investigation.

基于这个答案,我回顾了Linux的文档,这样做意识到我需要添加

RUNNER_TOOL_CACHE
,这将时间减少到近0秒

© www.soinside.com 2019 - 2024. All rights reserved.