系统和 conda 环境中的 HDF5 版本与 h5py 发生冲突

问题描述 投票:0回答:1

我正在尝试编译需要 HDF5 的 C++ 代码。我还使用 conda 环境来管理 Python 中后处理所需的包,以及启动调用基于 C++ 的可执行文件的参数组合运行。除了系统上安装的发行版之外,此 conda env 还包含 h5py 和其他 HDF5 发行版。

CMake 能够正确配置,并且可以在基础环境中构建。但是,当在激活 h5py 的 conda 环境中构建(这是调用参数组合运行的可执行文件所必需的)时,我最终出现以下错误:

[100%] Linking CXX executable simulation /usr/bin/ld: /usr/lib/x86_64-linux-gnu/hdf5/serial/libhdf5.so: undefined reference to `curl_global_init@CURL_OPENSSL_4' /usr/bin/ld: /usr/lib/x86_64-linux-gnu/hdf5/serial/libhdf5.so: undefined reference to `curl_easy_perform@CURL_OPENSSL_4' /usr/bin/ld: /usr/lib/x86_64-linux-gnu/hdf5/serial/libhdf5.so: undefined reference to `curl_slist_free_all@CURL_OPENSSL_4' /usr/bin/ld: /usr/lib/x86_64-linux-gnu/hdf5/serial/libhdf5.so: undefined reference to `curl_easy_setopt@CURL_OPENSSL_4' /usr/bin/ld: /usr/lib/x86_64-linux-gnu/hdf5/serial/libhdf5.so: undefined reference to `curl_easy_init@CURL_OPENSSL_4' /usr/bin/ld: /usr/lib/x86_64-linux-gnu/hdf5/serial/libhdf5.so: undefined reference to `curl_slist_append@CURL_OPENSSL_4' /usr/bin/ld: /usr/lib/x86_64-linux-gnu/hdf5/serial/libhdf5.so: undefined reference to `curl_easy_cleanup@CURL_OPENSSL_4' /usr/bin/ld: /usr/lib/x86_64-linux-gnu/hdf5/serial/libhdf5.so: undefined reference to `curl_global_cleanup@CURL_OPENSSL_4' collect2: error: ld returned 1 exit status gmake[3]: *** [src/CMakeFiles/simulation.dir/build.make:597: src/simulation] Error 1 gmake[2]: *** [CMakeFiles/Makefile2:2032: src/CMakeFiles/simulation.dir/all] Error 2 gmake[1]: *** [CMakeFiles/Makefile2:2039: src/CMakeFiles/simulation.dir/rule] Error 2 gmake: *** [Makefile:963: simulation] Error 2
在激活的conda env中使用

cmake -B./build/

进行配置可以工作,但发现错误的HDF5版本(系统版本,而不是h5py版本):

CMAKE_SOURCE_DIR: /home/path/to/my/env -- HDF5 CXX compiler wrapper is unable to compile a minimal HDF5 program. -- Found HDF5: /usr/lib/x86_64-linux-gnu/hdf5/serial/libhdf5_cpp.so;/usr/lib/x86_64-linux-gnu/hdf5/serial/libhdf5.so (found version "1.10.7") found components: CXX -- Found Boost: /usr/lib/x86_64-linux-gnu/cmake/Boost-1.74.0/BoostConfig.cmake (found version "1.74.0") -- HDF5 C compiler wrapper is unable to compile a minimal HDF5 program. -- Found HDF5: /usr/lib/x86_64-linux-gnu/hdf5/serial/libhdf5.so (found version "1.10.7") -- Found Boost: /usr/include (found version "1.74.0") found components: system serialization CMake Warning at build/_deps/highfive-src/CMakeLists.txt:117 (message): Unit tests have been DISABLED. -- Detected CPPZMQ Version - 4.10.0 -- cppzmq v4.10.0 -- Using the multi-header code from /home/path/to/my/env/build/_deps/json-src/include/ -- Configuring done -- Generating done -- Build files have been written to: /home/path/to/my/env/build
激活 conda env 时运行 

h5cc -showconfig

 显示 h5py 附带的 HDF5 版本 1.14.2 正在使用中:

SUMMARY OF THE HDF5 CONFIGURATION ================================= General Information: ------------------- HDF5 Version: 1.14.2 Configured on: Sun Aug 20 20:09:14 UTC 2023 Configured by: conda@8701a29b7fa1 Host system: x86_64-conda-linux-gnu Uname information: Linux 8701a29b7fa1 5.15.0-1041-azure #48-Ubuntu SMP Tue Jun 20 20:34:08 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux Byte sex: little-endian Installation point: /home/path/to/my/env Compiling Options: ------------------ Build Mode: production Debugging Symbols: no Asserts: no Profiling: no Optimization Level: high Linking Options: ---------------- Libraries: shared Statically Linked Executables: LDFLAGS: -Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags -Wl,--gc-sections -Wl,--allow-shlib-undefined -Wl,-rpath,/home/path/to/my/envlib -Wl,-rpath-link,/home/path/to/my/envlib -L/home/path/to/my/envlib H5_LDFLAGS: AM_LDFLAGS: -L/home/path/to/my/envlib -L/home/path/to/my/envlib Extra libraries: -lcrypto -lcurl -lpthread -lsz -lz -ldl -lm Archiver: /home/conda/feedstock_root/build_artifacts/hdf5_1692561980689/_build_env/bin/x86_64-conda-linux-gnu-ar AR_FLAGS: cr Ranlib: /home/conda/feedstock_root/build_artifacts/hdf5_1692561980689/_build_env/bin/x86_64-conda-linux-gnu-ranlib Languages: ---------- C: yes C Compiler: /home/conda/feedstock_root/build_artifacts/hdf5_1692561980689/_build_env/bin/x86_64-conda-linux-gnu-cc CPPFLAGS: -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem /home/path/to/my/envinclude H5_CPPFLAGS: -D_GNU_SOURCE -D_POSIX_C_SOURCE=200809L -DNDEBUG -UH5_DEBUG_API -I/home/conda/feedstock_root/build_artifacts/hdf5_1692561980689/work/src/H5FDsubfiling AM_CPPFLAGS: -I/home/path/to/my/envinclude -I/home/path/to/my/envinclude C Flags: -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/path/to/my/envinclude -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/hdf5_1692561980689/work=/usr/local/src/conda/hdf5-1.14.2 -fdebug-prefix-map=/home/path/to/my/env=/usr/local/src/conda-prefix H5 C Flags: -std=c99 -Wall -Wcast-qual -Wconversion -Wextra -Wfloat-equal -Wformat=2 -Winit-self -Winvalid-pch -Wmissing-include-dirs -Wshadow -Wundef -Wwrite-strings -pedantic -Wno-c++-compat -Wlarger-than=2560 -Wlogical-op -Wframe-larger-than=16384 -Wpacked-bitfield-compat -Wsync-nand -Wno-unsuffixed-float-constants -Wdouble-promotion -Wtrampolines -Wstack-usage=8192 -Wmaybe-uninitialized -Wdate-time -Warray-bounds=2 -Wc99-c11-compat -Wduplicated-cond -Whsa -Wnormalized -Wnull-dereference -Wunused-const-variable -Walloca -Walloc-zero -Wduplicated-branches -Wformat-overflow=2 -Wformat-truncation=1 -Wattribute-alias -Wcast-align=strict -Wshift-overflow=2 -Wattribute-alias=2 -Wmissing-profile -Wc11-c2x-compat -fstdarg-opt -fdiagnostics-urls=never -fno-diagnostics-color -s -Wbad-function-cast -Wcast-align -Wformat -Wimplicit-function-declaration -Wint-to-pointer-cast -Wmissing-declarations -Wmissing-prototypes -Wnested-externs -Wold-style-definition -Wpacked -Wpointer-sign -Wpointer-to-int-cast -Wredundant-decls -Wstrict-prototypes -Wswitch -Wunused-but-set-variable -Wunused-variable -Wunused-function -Wunused-parameter -Wincompatible-pointer-types -Wint-conversion -Wshadow -Wrestrict -Wcast-function-type -Wmaybe-uninitialized -Wno-aggregate-return -Wno-inline -Wno-missing-format-attribute -Wno-missing-noreturn -Wno-overlength-strings -Wno-jump-misses-init -Wstrict-overflow=2 -Wno-suggest-attribute=const -Wno-suggest-attribute=noreturn -Wno-suggest-attribute=pure -Wno-suggest-attribute=format -Wno-suggest-attribute=cold -Wno-suggest-attribute=malloc -O3 AM C Flags: Shared C Library: yes Static C Library: no Fortran: yes Fortran Compiler: /home/conda/feedstock_root/build_artifacts/hdf5_1692561980689/_build_env/bin/x86_64-conda-linux-gnu-gfortran Fortran Flags: H5 Fortran Flags: -std=f2008 -Waliasing -Wall -Wcharacter-truncation -Wextra -Wimplicit-interface -Wsurprising -Wunderflow -pedantic -Wintrinsics-std -Wimplicit-procedure -Wreal-q-constant -Wfunction-elimination -Wrealloc-lhs -Wrealloc-lhs-all -Wno-c-binding-type -Winteger-division -Wfrontend-loop-interchange -fdiagnostics-urls=never -fno-diagnostics-color -s -Wno-unused-dummy-argument -Wno-array-temporaries -O3 AM Fortran Flags: Shared Fortran Library: yes Static Fortran Library: no Module Directory: ${includedir} C++: yes C++ Compiler: /home/conda/feedstock_root/build_artifacts/hdf5_1692561980689/_build_env/bin/x86_64-conda-linux-gnu-c++ C++ Flags: -fvisibility-inlines-hidden -fmessage-length=0 -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/path/to/my/envinclude -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/hdf5_1692561980689/work=/usr/local/src/conda/hdf5-1.14.2 -fdebug-prefix-map=/home/path/to/my/env=/usr/local/src/conda-prefix H5 C++ Flags: -std=c++11 -Wall -Wcast-qual -Wconversion -Wctor-dtor-privacy -Weffc++ -Wextra -Wfloat-equal -Wformat=2 -Winit-self -Winvalid-pch -Wmissing-include-dirs -Wnon-virtual-dtor -Wold-style-cast -Woverloaded-virtual -Wreorder -Wshadow -Wsign-promo -Wundef -Wwrite-strings -pedantic -Wlarger-than=2560 -Wlogical-op -Wframe-larger-than=16384 -Wpacked-bitfield-compat -Wsync-nand -Wdouble-promotion -Wtrampolines -Wstack-usage=8192 -Wmaybe-uninitialized -Wdate-time -Wopenmp-simd -Warray-bounds=2 -Wduplicated-cond -Whsa -Wnormalized -Wnull-dereference -Wunused-const-variable -Walloca -Walloc-zero -Wduplicated-branches -Wformat-overflow=2 -Wformat-truncation=1 -Wattribute-alias -Wcast-align=strict -Wshift-overflow=2 -Wattribute-alias=2 -Wmissing-profile -Wno-deprecated-copy -fstdarg-opt -fdiagnostics-urls=never -fno-diagnostics-color -s -Wcast-align -Wmissing-declarations -Wpacked -Wredundant-decls -Wswitch -Wunused-but-set-variable -Wunused-function -Wunused-variable -Wunused-parameter -Wshadow -O3 AM C++ Flags: Shared C++ Library: yes Static C++ Library: no Java: no Features: --------- Parallel HDF5: no Parallel Filtered Dataset Writes: no Large Parallel I/O: no High-level library: yes Dimension scales w/ new references: no Build HDF5 Tests: yes Build HDF5 Tools: yes Build GIF Tools: yes Threadsafety: yes Default API mapping: v114 With deprecated public symbols: yes I/O filters (external): deflate(zlib),szip(encoder) Map (H5M) API: no Direct VFD: yes Mirror VFD: no Subfiling VFD: no (Read-Only) S3 VFD: yes (Read-Only) HDFS VFD: no Packages w/ extra debug output: none API tracing: no Using memory checker: yes Function stack tracing: no Use file locking: best-effort Strict file format checks: no Optimization instrumentation: no
当 conda env 停用并且仅激活 base env 时,版本为 1.10.7:

SUMMARY OF THE HDF5 CONFIGURATION ================================= General Information: ------------------- HDF5 Version: 1.10.7 Configured on: Wed, 08 Dec 2021 23:33:27 +0000 Configured by: Debian Host system: x86_64-pc-linux-gnu Uname information: Debian Byte sex: little-endian Installation point: /usr Flavor name: serial Compiling Options: ------------------ Build Mode: production Debugging Symbols: no Asserts: no Profiling: no Optimization Level: high Linking Options: ---------------- Libraries: static, shared Statically Linked Executables: LDFLAGS: -Wl,-Bsymbolic-functions -flto=auto -Wl,-z,relro H5_LDFLAGS: -Wl,--version-script,$(top_srcdir)/debian/map_serial.ver AM_LDFLAGS: Extra libraries: -lcrypto -lcurl -lpthread -lsz -lz -ldl -lm Archiver: ar AR_FLAGS: cr Ranlib: x86_64-linux-gnu-ranlib Languages: ---------- C: yes C Compiler: /usr/bin/gcc CPPFLAGS: -Wdate-time -D_FORTIFY_SOURCE=2 H5_CPPFLAGS: -D_GNU_SOURCE -D_POSIX_C_SOURCE=200809L -DNDEBUG -UH5_DEBUG_API AM_CPPFLAGS: C Flags: -g -O2 -ffile-prefix-map=$(top_srcdir)=. -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security H5 C Flags: -std=c99 -Wall -Wcast-qual -Wconversion -Wextra -Wfloat-equal -Wformat=2 -Winit-self -Winvalid-pch -Wmissing-include-dirs -Wno-c++-compat -Wno-format-nonliteral -Wshadow -Wundef -Wwrite-strings -pedantic -Wlarger-than=2560 -Wlogical-op -Wframe-larger-than=16384 -Wpacked-bitfield-compat -Wsync-nand -Wstrict-overflow=5 -Wno-unsuffixed-float-constants -Wdouble-promotion -Wtrampolines -Wstack-usage=8192 -Wmaybe-uninitialized -Wdate-time -Warray-bounds=2 -Wc99-c11-compat -Wduplicated-cond -Whsa -Wnormalized -Wnull-dereference -Wunused-const-variable -Walloca -Walloc-zero -Wduplicated-branches -Wformat-overflow=2 -Wformat-truncation=1 -Wimplicit-fallthrough=5 -Wrestrict -Wattribute-alias -Wcast-align=strict -Wshift-overflow=2 -Wattribute-alias=2 -Wmissing-profile -fstdarg-opt -s -Wno-aggregate-return -Wno-inline -Wno-missing-format-attribute -Wno-missing-noreturn -Wno-overlength-strings -Wno-jump-misses-init -Wno-suggest-attribute=const -Wno-suggest-attribute=noreturn -Wno-suggest-attribute=pure -Wno-suggest-attribute=format -Wno-suggest-attribute=cold -Wno-suggest-attribute=malloc -O3 -Werror=bad-function-cast -Werror=declaration-after-statement -Werror=implicit-function-declaration -Werror=missing-declarations -Werror=missing-prototypes -Werror=nested-externs -Werror=old-style-definition -Werror=packed -Werror=pointer-sign -Werror=pointer-to-int-cast -Werror=redundant-decls -Werror=strict-prototypes -Werror=switch -Wunused-function -Wunused-variable -Wunused-parameter -Wcast-align -Wunused-but-set-variable -Wformat -Werror=incompatible-pointer-types -Werror=shadow -Wcast-function-type -Wmaybe-uninitialized AM C Flags: Shared C Library: yes Static C Library: yes Fortran: yes Fortran Compiler: /usr/bin/gfortran Fortran Flags: -g -O2 -ffile-prefix-map=$(top_srcdir)=. -flto=auto -ffat-lto-objects -fstack-protector-strong H5 Fortran Flags: -std=f2008 -Waliasing -Wall -Wcharacter-truncation -Wextra -Wimplicit-interface -Wsurprising -Wunderflow -pedantic -Warray-temporaries -Wintrinsics-std -Wimplicit-procedure -Wreal-q-constant -Wfunction-elimination -Wrealloc-lhs -Wrealloc-lhs-all -Wno-c-binding-type -Wuse-without-only -Winteger-division -Wfrontend-loop-interchange -s -O3 AM Fortran Flags: Shared Fortran Library: yes Static Fortran Library: yes C++: yes C++ Compiler: /usr/bin/g++ C++ Flags: -g -O2 -ffile-prefix-map=$(top_srcdir)=. -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security H5 C++ Flags: -Wall -Wcast-qual -Wconversion -Wctor-dtor-privacy -Weffc++ -Wextra -Wfloat-equal -Wformat=2 -Winit-self -Winvalid-pch -Wmissing-include-dirs -Wno-format-nonliteral -Wnon-virtual-dtor -Wold-style-cast -Woverloaded-virtual -Wreorder -Wshadow -Wsign-promo -Wundef -Wwrite-strings -pedantic -Wlarger-than=2560 -Wlogical-op -Wframe-larger-than=16384 -Wpacked-bitfield-compat -Wsync-nand -Wstrict-overflow=5 -Wno-unsuffixed-float-constants -Wdouble-promotion -Wtrampolines -Wstack-usage=8192 -Wmaybe-uninitialized -Wdate-time -Wopenmp-simd -Warray-bounds=2 -Wduplicated-cond -Whsa -Wnormalized -Wnull-dereference -Wunused-const-variable -Walloca -Walloc-zero -Wduplicated-branches -Wformat-overflow=2 -Wformat-truncation=1 -Wimplicit-fallthrough=5 -Wrestrict -Wattribute-alias -Wcast-align=strict -Wshift-overflow=2 -Wattribute-alias=2 -Wmissing-profile -fstdarg-opt -s -O3 -Wcast-align -Wmissing-declarations -Werror=packed -Werror=redundant-decls -Werror=switch -Wunused-but-set-variable -Werror=unused-function -Werror=unused-variable -Wunused-parameter -Werror=shadow AM C++ Flags: Shared C++ Library: yes Static C++ Library: yes Java: yes Java Compiler: /usr/bin/java (openjdk 11.0.13 2021-10-19) Features: --------- Parallel HDF5: no Parallel Filtered Dataset Writes: no Large Parallel I/O: no High-level library: yes Build HDF5 Tests: yes Build HDF5 Tools: yes Threadsafety: yes Default API mapping: v18 With deprecated public symbols: yes I/O filters (external): deflate(zlib),szip(encoder) MPE: no Direct VFD: no Mirror VFD: no (Read-Only) S3 VFD: yes (Read-Only) HDFS VFD: no dmalloc: no Packages w/ extra debug output: none API tracing: no Using memory checker: no Memory allocation sanity checks: no Function stack tracing: no Use file locking: best-effort Strict file format checks: no Optimization instrumentation: no
在我的 CMakeLists.txt 中,我使用以下相关行:

find_package(HDF5 REQUIRED COMPONENTS CXX) link_directories(${HDF5_LIBRARY_DIRS}) include_directories(${HDF5_INCLUDE_DIRS}) ... #### COMPILER FLAGS ### set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++2a") set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fopenmp -larmadillo -lhdf5") # required for mlpack set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -ljson") # required for SwarmRL integration set(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS} -O3 -march=native") # optimized set(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS} -g -Wall -Wextra -Wpedantic -Wdouble-promotion -Wformat=2 -Wformat-truncation -Wundef -fno-common -Wconversion -Wmisleading-indentation") # debugging # other flags: -Wshadow
添加 

-lcurl

 标志没有帮助。

我的相关 CMakeCache.txt 部分如下所示,表明设置了版本 1.10 路径(而不是 1.14 路径!):

//HDF5 CXX Wrapper compiler. Used only to detect HDF5 compile // flags. HDF5_CXX_COMPILER_EXECUTABLE:FILEPATH=/usr/bin/h5c++ //Path to a file. HDF5_CXX_INCLUDE_DIR:PATH=/usr/include/hdf5/serial //Path to a library. HDF5_CXX_LIBRARY_crypto:FILEPATH=/usr/lib/x86_64-linux-gnu/libcrypto.so //Path to a library. HDF5_CXX_LIBRARY_curl:FILEPATH=/usr/lib/x86_64-linux-gnu/libcurl.so //Path to a library. HDF5_CXX_LIBRARY_dl:FILEPATH=/usr/lib/x86_64-linux-gnu/libdl.a //Path to a library. HDF5_CXX_LIBRARY_hdf5:FILEPATH=/usr/lib/x86_64-linux-gnu/hdf5/serial/libhdf5.so //Path to a library. HDF5_CXX_LIBRARY_hdf5_cpp:FILEPATH=/usr/lib/x86_64-linux-gnu/hdf5/serial/libhdf5_cpp.so //Path to a library. HDF5_CXX_LIBRARY_m:FILEPATH=/usr/lib/x86_64-linux-gnu/libm.so //Path to a library. HDF5_CXX_LIBRARY_pthread:FILEPATH=/usr/lib/x86_64-linux-gnu/libpthread.a //Path to a library. HDF5_CXX_LIBRARY_sz:FILEPATH=/usr/lib/x86_64-linux-gnu/libsz.so //Path to a library. HDF5_CXX_LIBRARY_z:FILEPATH=/usr/lib/x86_64-linux-gnu/libz.so
康达环境:

name: myenv channels: # strict channel priority enforced! order matters - conda-forge - bioconda - defaults dependencies: - python=3.10 - snakemake=7.32 - matplotlib - pandas - seaborn - scipy - yaml - pyyaml - graphviz - ffmpeg - numba - h5py - pytables - jupyter
我怀疑 cmake 配置 HDF5 版本 (1.10.7) 与由于 conda env (1.14.2) 而处于活动状态的版本不匹配。我想我需要在 CMake 中指定要使用哪些 HDF5 库。
理想情况下,我可以使用 h5py 附带的版本,以实现更好的依赖管理。
我怎样才能解决这个问题(以最优雅的方式)?

谢谢您的帮助!

c++ cmake anaconda hdf5 h5py
1个回答
0
投票
您似乎正在处理系统 HDF5 库(版本 1.10.7)和 conda 环境中包含的 HDF5 库(版本 1.14.2)之间的冲突。要解决此问题,您需要确保 CMake 在构建项目时使用 conda 环境中的 HDF5 库。您可以通过设置适当的 CMake 变量来实现这一点。

具体操作方法如下:

1.设置 HDF5_ROOT CMake 变量以指向 conda 环境中的 HDF5 安装。您可以在运行 CMake 之前设置此变量。 重击

cmake -B./build/ -DHDF5_ROOT=/path/to/your/conda/env

将 /path/to/your/conda/env 替换为 conda 环境的实际路径。

    确保在运行 CMake 并构建项目时激活 conda 环境。这可确保 CMake 从激活的环境中找到并使用 HDF5 库和标头。 这是完整的工作流程:
Activate your conda environment* conda activate myenv *Run CMake with the HDF5_ROOT variable set* cmake -B./build/ -DHDF5_ROOT=/path/to/your/conda/env * Build your project* cmake --build ./build/

此方法应确保 CMake 使用 conda 环境中的 HDF5 库,而不是系统 HDF5 库。

如果遇到任何其他问题,您可能需要调整其他 CMake 变量,例如 HDF5_CXX_INCLUDE_DIR 和 HDF5_CXX_LIBRARY,以指向 conda 环境中的正确位置。

通过适当设置这些变量,您可以确保 CMake 使用 conda 环境提供的 HDF5 版本,从而使您能够更有效地管理依赖关系。

© www.soinside.com 2019 - 2024. All rights reserved.