运行时错误：CUDA 错误：没有可在 cuda 11.8 和 torch 2.0.0 的设备上执行的内核映像

Question

我想使用meta-llama/Llama-2-13b-chat-hf，但我遇到了这个错误：

RuntimeError: CUDA error: no kernel image is available for execution on the device
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.

For debugging consider passing CUDA_LAUNCH_BLOCKING=1.

Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

nvidia-smi 的输出是：

| NVIDIA-SMI 465.19.01    Driver Version: 465.19.01    CUDA Version: 11.3     |

NVCC：

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation

Built on Wed_Sep_21_10:33:58_PDT_2022

Cuda compilation tools, release 11.8, V11.8.89

Build cuda_11.8.r11.8/compiler.31833905_0

火炬版本：

torch==2.0.0+cu118
torchaudio==2.0.1+cu118
torchvision==0.15.1+cu118

变形金刚版本：

transformers==4.37.2

我有一些 2080ti 和 710，正在使用 Ubuntu 16。

我的输出中也得到了这个：

Found GPU9 NVIDIA GeForce GT 710 which is of cuda capability 3.5.
    PyTorch no longer supports this GPU because it is too old.
    The minimum cuda capability supported by this library is 3.7.

我从这里下载火炬版本。

我也尝试从源代码构建，但它给了我相同的输出。

bitsandbytes 的输出：

++++++++++++++++++++++++++ OTHER +++++++++++++++++++++++++++
COMPILED_WITH_CUDA = True
COMPUTE_CAPABILITIES_PER_GPU = ['7.5', '7.5', '7.5', '7.5', '7.5', '7.5', '7.5', '7.5', '7.5', '3.5']
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++ DEBUG INFO END ++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Running a quick check that:
    + library is importable
    + CUDA function is callable


WARNING: Please be sure to sanitize sensible info from any such env vars!

SUCCESS!
Installation was successful!

我运行这段代码来测试火炬

import torch
import sys
print('A', sys.version)
print('B', torch.__version__)
print('C', torch.cuda.is_available())
print('D', torch.backends.cudnn.enabled)
device = torch.device('cuda')
print('E', torch.cuda.get_device_properties(device))
print('F', torch.tensor([1.0, 2.0]).cuda())

它给了我这个输出：

A 3.11.7 (main, Dec 15 2023, 18:12:31) [GCC 11.2.0]
B 2.0.0+cu118
C True
D True
    UserWarning: 
    Found GPU9 NVIDIA GeForce GT 710 which is of cuda capability 3.5.
    PyTorch no longer supports this GPU because it is too old.
    The minimum cuda capability supported by this library is 3.7.
    
  warnings.warn(old_gpu_warn % (d, name, major, minor, min_arch // 10, min_arch % 10))
E _CudaDeviceProperties(name='NVIDIA GeForce RTX 2080 Ti', major=7, minor=5, total_memory=11019MB, multi_processor_count=68)
F tensor([1., 2.], device='cuda:0')

我应该怎么做才能解决这个问题？

Answer 1

如警告中所述：

    UserWarning: 
    Found GPU9 NVIDIA GeForce GT 710 which is of cuda capability 3.5.
    PyTorch no longer supports this GPU because it is too old.
    The minimum cuda capability supported by this library is 3.7.

主要问题是 torch 尝试在 GT710 上运行代码，由于不支持，程序崩溃了。为了解决这个问题，我添加了一个环境变量：

export CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7,8

所以 Torch 现在忽略了 GT710，它现在对我来说运行得很好。