当使用 CUDAExecutionProvider 运行 onnx 时，它会引发“FAIL：无法加载库 libonnxruntime_providers_cuda.so，并出现错误”

Question

2024-04-01 23:16:16.891282913 [E:onnxruntime:Default, provider_bridge_ort.cc:1480 TryGetProviderInfo_CUDA] /onnxruntime_src/onnxruntime/core/session/provider_bridge_ort.cc:1193 onnxruntime::Provider& onnxruntime::ProviderLibrary::Get() [ONNXRuntimeError] : 1 : FAIL : Failed to load library libonnxruntime_providers_cuda.so with error: libcublasLt.so.11: cannot open shared object file: No such file or directory

我使用以下cmd查找未找到所以

~ $ locate libonnxruntime_providers_cuda.so
/home/roroco/.pyenv/versions/3.9.1/lib/python3.9/site-packages/onnxruntime/capi/libonnxruntime_providers_cuda.so

我猜是因为env PATH问题，如何设置PATH让python onnx找到libonnxruntime_providers_cuda.so

我尝试找到so，so存在并手动将so目录添加到LD_LIBRARY_PATH，错误仍然存在

~/Dropbox/mix/real-esrgan/submodule/Real-ESRGAN $ locate libcublasLt.so
/home/roroco/.pyenv/versions/3.9.1/lib/python3.9/site-packages/nvidia/cublas/lib/libcublasLt.so.11
/home/roroco/.pyenv/versions/3.9.1/lib/python3.9/site-packages/nvidia/cublas/lib/libcublasLt.so.12
^[[A~/Dropbox/mix/real-esrgan/submodule/Real-ESRGAlocate libonnxruntime_providers_cuda.so
/home/roroco/.pyenv/versions/3.9.1/lib/python3.9/site-packages/onnxruntime/capi/libonnxruntime_providers_cuda.so
~/Dropbox/mix/real-esrgan/submodule/Real-ESRGAN $ PYTHON_PATH=. LD_LIBRARY_PATH="/home/roroco/.pyenv/versions/3.9.1/lib/python3.9/site-packages/onnxruntime/capi:/home/roroco/.pyenv/versions/3.9.1/lib/python3.9/site-packages/nvidia/cublas/lib" python onnx/run_onnx.py
2024-04-01 23:50:25.975940926 [E:onnxruntime:Default, provider_bridge_ort.cc:1480 TryGetProviderInfo_CUDA] /onnxruntime_src/onnxruntime/core/session/provider_bridge_ort.cc:1193 onnxruntime::Provider& onnxruntime::ProviderLibrary::Get() [ONNXRuntimeError] : 1 : FAIL : Failed to load library libonnxruntime_providers_cuda.so with error: libcudnn.so.8: cannot open shared object file: No such file or directory

2024-04-01 23:50:25.975963148 [W:onnxruntime:Default, onnxruntime_pybind_state.cc:747 CreateExecutionProviderInstance] Failed to create CUDAExecutionProvider. Please reference https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html#requirements to ensure all dependencies are met.
out /home/roroco/Dropbox/mix/real-esrgan/submodule/Real-ESRGAN/onnx/out.jpg

以下是我的完整 onnx 代码：

import os.path

import onnxruntime
import numpy as np
import cv2

# Load the ONNX model
d = os.path.abspath(f"{__file__}/..")
onnx_model_path = f'{d}/RealESRGAN_x4plus_anime_6B.onnx'
ort_session = onnxruntime.InferenceSession(onnx_model_path, providers=["CUDAExecutionProvider", 'AzureExecutionProvider',
                                                                       'CPUExecutionProvider'])  # in my old deivce with 18.04, cannot install libonnxruntime_providers_cuda, so cannot use cuda
# ort_session = onnxruntime.InferenceSession(onnx_model_path, providers=["CUDAExecutionProvider"])

# Load and preprocess input image
input_image_path = f"{d}/t.jpg"
input_image = cv2.imread(input_image_path)
input_image = cv2.cvtColor(input_image, cv2.COLOR_BGR2RGB)  # Convert BGR to RGB
input_image = input_image.astype(np.float32) / 255.0  # Normalize to [0, 1]
input_image = np.transpose(input_image, (2, 0, 1))  # Change the shape from HWC to CHW
input_image = np.expand_dims(input_image, axis=0)  # Add batch dimension

# Perform inference
ort_inputs = {ort_session.get_inputs()[0].name: input_image}
ort_outs = ort_session.run(None, ort_inputs)

# Post-process the output
output_image = ort_outs[0][0]  # Assuming only one output
output_image = np.transpose(output_image, (1, 2, 0))  # Change the shape from CHW to HWC
output_image = (output_image * 255).clip(0, 255).astype(np.uint8)  # Convert back to uint8 and clip values

# Display or save the output image
out = f"{d}/out.jpg"  # Adjust output path as needed
cv2.imwrite(out, cv2.cvtColor(output_image, cv2.COLOR_RGB2BGR))
print(f"out {out}")

Answer 1

我使用

locate

获取所有so路径并将so目录放入LD_LIBRARY_PATH中，它修复了。方法比较复杂，如果有更简单的方法请告诉我

PYTHON_PATH=. LD_LIBRARY_PATH="/home/roroco/.pyenv/versions/3.9.1/lib/python3.9/site-packages/onnxruntime/capi:/home/roroco/.pyenv/versions/3.9.1/lib/python3.9/site-packages/nvidia/cublas/lib:/home/roroco/.pyenv/versions/3.9.1/lib/python3.9/site-packages/nvidia/cudnn/lib:/home/roroco/.pyenv/versions/3.9.1/lib/python3.9/site-packages/nvidia/curand/lib:/home/roroco/.pyenv/versions/3.9.1/lib/python3.9/site-packages/nvidia/cufft/lib:/home/roroco/.pyenv/versions/3.9.1/lib/python3.9/site-packages/nvidia/cuda_runtime/lib" python onnx/run_onnx.py