无法解决 Rcpp 代码中的 Valgrind 内存问题

问题描述 投票：0回答：1

我已经在 CRAN 上发布了我的包，并收到了 Valgrind 的检查结果，显示了一些内存泄漏（链接）。不幸的是，我无法重现这些错误。因此，我有一些假设为什么会出现这些错误但无法对其进行测试。所以我需要一个建议，我是否在朝着正确的方向前进，以及如何修复相应的错误。

Valgrind 指出了很多问题，但其中一些问题在某种意义上是相似的，因为它们指的是相同或相似的代码行。因此，下面我只描述了一些对其他问题有代表性的问题（错误消息）。其他信息可以通过上面提到的链接找到。

问题一

运行以下示例代码（R 调用使用 Rcpp 代码的函数，即某些地方的 C++ 函数）：

# ... some preceding code with data preparation
pgn <- mvoprobit(z ~ w1 + w2,
                 data = data,
                 marginal = list("PGN" = 3))

Valgrind 产生错误信息：

==2563572== Invalid read of size 8
==2563572==    at 0x52B841: STRING_ELT (svn/R-devel/src/main/memory.c:3923)
==2563572==    by 0x175F1952: begin (R-devel/site-library/Rcpp/include/Rcpp/vector/string_proxy.h:167)
==2563572==    by 0x175F1952: operator!= (R-devel/site-library/Rcpp/include/Rcpp/vector/string_proxy.h:194)
==2563572==    by 0x175F1952: pmnorm(Rcpp::Vector<14, Rcpp::PreserveStorage>, Rcpp::Vector<14, Rcpp::PreserveStorage>, Rcpp::Vector<14, Rcpp::PreserveStorage>, Rcpp::Vector<14, Rcpp::PreserveStorage>, Rcpp::Matrix<14, Rcpp::PreserveStorage>, Rcpp::Vector<14, Rcpp::PreserveStorage>, int, Rcpp::String, Rcpp::String, bool, bool, bool, bool, bool, bool, Rcpp::Nullable<Rcpp::Vector<19, Rcpp::PreserveStorage> >, int, Rcpp::Nullable<Rcpp::Vector<19, Rcpp::PreserveStorage> >, bool, bool) (/tmp/RtmpvZ7OOy/R.INSTALL331e2639ef54f9/mnorm/src/pmnorm.cpp:384)
==2563572==    by 0x175C2584: _mnorm_pmnorm_try(SEXPREC*, SEXPREC*, SEXPREC*, SEXPREC*, SEXPREC*, SEXPREC*, SEXPREC*, SEXPREC*, SEXPREC*, SEXPREC*, SEXPREC*, SEXPREC*, SEXPREC*, SEXPREC*, SEXPREC*, SEXPREC*, SEXPREC*, SEXPREC*, SEXPREC*, SEXPREC*) (/tmp/RtmpvZ7OOy/R.INSTALL331e2639ef54f9/mnorm/src/RcppExports.cpp:232)
==2563572==    by 0x1767C9B3: mnorm::pmnorm(Rcpp::Vector<14, Rcpp::PreserveStorage>, Rcpp::Vector<14, Rcpp::PreserveStorage>, Rcpp::Vector<14, Rcpp::PreserveStorage>, Rcpp::Vector<14, Rcpp::PreserveStorage>, Rcpp::Matrix<14, Rcpp::PreserveStorage>, Rcpp::Vector<14, Rcpp::PreserveStorage>, int, Rcpp::String, Rcpp::String, bool, bool, bool, bool, bool, bool, Rcpp::Nullable<Rcpp::Vector<19, Rcpp::PreserveStorage> >, int, Rcpp::Nullable<Rcpp::Vector<19, Rcpp::PreserveStorage> >, bool, bool) (R-devel/site-library/mnorm/include/mnorm_RcppExports.h:138)
==2563572==    by 0x176867DE: lnL_mvoprobit(arma::Col<double>, Rcpp::Vector<19, Rcpp::PreserveStorage>, Rcpp::String, int, int) (packages/tests-vg/switchSelection/src/mvoprobit.cpp:399)
==2563572==    by 0x17663D34: _switchSelection_lnL_mvoprobit (packages/tests-vg/switchSelection/src/RcppExports.cpp:75)
==2563572==    by 0x4A268D: R_doDotCall (svn/R-devel/src/main/dotcode.c:880)
==2563572==    by 0x4E126C: bcEval (svn/R-devel/src/main/eval.c:8002)
==2563572==    by 0x4F43CF: Rf_eval (svn/R-devel/src/main/eval.c:1013)
==2563572==    by 0x4F620E: R_execClosure (svn/R-devel/src/main/eval.c:2187)
==2563572==    by 0x4F6F42: Rf_applyClosure (svn/R-devel/src/main/eval.c:2113)
==2563572==    by 0x4E45FD: bcEval (svn/R-devel/src/main/eval.c:7414)

在阅读了一些解释 Valgrind 检查结果的教程后，我得出的结论是，从下往上阅读错误信息是合理的。首先，我看到问题是从 

mvoprobit.cpp
文件中代码的第 399 行开始的，该文件包含函数 
lnL_mvoprobit
。这行代码使用了另一个函数
pmnorm
。因此，我得出结论，问题出现在 
pmnorm.cpp
的第 384 行，它具有以下代码行（Valgrind 提到的 
operator!=
强化了它恰好提到了这行代码的假设）：

if (marginal_names[i] != "normal")

其中

marginal_names
是
CharacterVector
。我认为 
Invalid read of size 8
消息表明我正在尝试阅读 
marginal_names[i]
而它是空的（用 Valgrand 提到的 
begin
来强化这个假设）。然而，深入检查（包括对其他示例的检查，包括其他示例代码和各种适当的替代输入规范）表明，当示例在循环中运行时，
i
marginal_names[i]
occures 始终是 
0
（并且从来没有其他值) 而 
marginal_names
是单个（一个）元素的特征向量。因此我无法理解问题的根源。

问题2

Valgrind 的下一条消息紧接着第一条消息如下：

==2563572==  Address 0x1920c570 is 7,840 bytes inside a block of size 7,960 alloc'd
==2563572==    at 0x484086F: malloc (/builddir/build/BUILD/valgrind-3.18.1/coregrind/m_replacemalloc/vg_replace_malloc.c:381)
==2563572==    by 0x5318D0: GetNewPage (svn/R-devel/src/main/memory.c:972)
==2563572==    by 0x533503: Rf_allocVector3 (svn/R-devel/src/main/memory.c:2812)
==2563572==    by 0x175C7ADA: SEXPREC* Rcpp::internal::primitive_range_wrap__impl__nocast<__gnu_cxx::__normal_iterator<int const*, std::vector<int, std::allocator<int> > >, int>(__gnu_cxx::__normal_iterator<int const*, std::vector<int, std::allocator<int> > >, __gnu_cxx::__normal_iterator<int const*, std::vector<int, std::allocator<int> > >, std::random_access_iterator_tag) [clone .isra.0] (R-devel/site-library/Rcpp/include/Rcpp/internal/wrap.h:116)
==2563572==    by 0x175CBFB2: primitive_range_wrap__impl<__gnu_cxx::__normal_iterator<int const*, std::vector<int, std::allocator<int> > >, int> (R-devel/site-library/Rcpp/include/Rcpp/internal/wrap.h:167)
==2563572==    by 0x175CBFB2: range_wrap_dispatch___impl<__gnu_cxx::__normal_iterator<int const*, std::vector<int, std::allocator<int> > >, int> (R-devel/site-library/Rcpp/include/Rcpp/internal/wrap.h:179)
==2563572==    by 0x175CBFB2: range_wrap_dispatch<__gnu_cxx::__normal_iterator<int const*, std::vector<int, std::allocator<int> > >, int> (R-devel/site-library/Rcpp/include/Rcpp/internal/wrap.h:419)
==2563572==    by 0x175CBFB2: range_wrap<__gnu_cxx::__normal_iterator<int const*, std::vector<int, std::allocator<int> > > > (R-devel/site-library/Rcpp/include/Rcpp/internal/wrap.h:429)
==2563572==    by 0x175CBFB2: wrap<__gnu_cxx::__normal_iterator<int const*, std::vector<int, std::allocator<int> > > > (R-devel/site-library/Rcpp/include/Rcpp/internal/wrap.h:941)
==2563572==    by 0x175CBFB2: operator SEXP (R-devel/site-library/Rcpp/include/Rcpp/api/meat/Dimension.h:26)
==2563572==    by 0x175CBFB2: wrap_dispatch_unknown<Rcpp::Dimension> (R-devel/site-library/Rcpp/include/Rcpp/internal/wrap.h:503)
==2563572==    by 0x175CBFB2: wrap_dispatch_eigen<Rcpp::Dimension> (R-devel/site-library/Rcpp/include/Rcpp/internal/wrap.h:770)
==2563572==    by 0x175CBFB2: wrap_dispatch_unknown_importable<Rcpp::Dimension> (R-devel/site-library/Rcpp/include/Rcpp/internal/wrap.h:787)
==2563572==    by 0x175CBFB2: wrap_dispatch<Rcpp::Dimension> (R-devel/site-library/Rcpp/include/Rcpp/internal/wrap.h:807)
==2563572==    by 0x175CBFB2: wrap<Rcpp::Dimension> (R-devel/site-library/Rcpp/include/Rcpp/internal/wrap_end.h:30)
==2563572==    by 0x175CBFB2: operator=<Rcpp::Dimension> (R-devel/site-library/Rcpp/include/Rcpp/api/meat/proxy.h:33)
==2563572==    by 0x175CBFB2: arma_wrap<arma::Mat<double> > (R-devel/site-library/RcppArmadillo/include/RcppArmadillo/interface/RcppArmadilloWrap.h:33)
==2563572==    by 0x175CBFB2: SEXPREC* Rcpp::wrap<double>(arma::Mat<double> const&) (R-devel/site-library/RcppArmadillo/include/RcppArmadillo/interface/RcppArmadilloWrap.h:72)
==2563572==    by 0x175C7E23: get<arma::Mat<double> > (R-devel/site-library/Rcpp/include/Rcpp/vector/converter.h:80)
==2563572==    by 0x175C7E23: replace_element__dispatch__isArgument<Rcpp::traits::named_object<arma::Mat<double> > > (R-devel/site-library/Rcpp/include/Rcpp/vector/Vector.h:532)
==2563572==    by 0x175C7E23: replace_element__dispatch<Rcpp::traits::named_object<arma::Mat<double> > > (R-devel/site-library/Rcpp/include/Rcpp/vector/Vector.h:525)
==2563572==    by 0x175C7E23: replace_element<Rcpp::traits::named_object<arma::Mat<double> > > (R-devel/site-library/Rcpp/include/Rcpp/vector/Vector.h:514)
==2563572==    by 0x175C7E23: Rcpp::Vector<19, Rcpp::PreserveStorage> Rcpp::Vector<19, Rcpp::PreserveStorage>::create__dispatch<Rcpp::traits::named_object<Rcpp::Matrix<14, Rcpp::PreserveStorage> >, Rcpp::traits::named_object<Rcpp::Matrix<14, Rcpp::PreserveStorage> >, Rcpp::traits::named_object<arma::Mat<double> >, Rcpp::traits::named_object<arma::Mat<double> >, Rcpp::traits::named_object<arma::Mat<double> >, Rcpp::traits::named_object<arma::Mat<double> >, Rcpp::traits::named_object<arma::Mat<double> > >(Rcpp::traits::integral_constant<bool, true>, Rcpp::traits::named_object<Rcpp::Matrix<14, Rcpp::PreserveStorage> > const&, Rcpp::traits::named_object<Rcpp::Matrix<14, Rcpp::PreserveStorage> > const&, Rcpp::traits::named_object<arma::Mat<double> > const&, Rcpp::traits::named_object<arma::Mat<double> > const&, Rcpp::traits::named_object<arma::Mat<double> > const&, Rcpp::traits::named_object<arma::Mat<double> > const&, Rcpp::traits::named_object<arma::Mat<double> > const&) [clone .isra.0] (R-devel/site-library/Rcpp/include/Rcpp/generated/Vector__create.h:378)
==2563572==    by 0x175CAA52: create<Rcpp::traits::named_object<Rcpp::Matrix<14> >, Rcpp::traits::named_object<Rcpp::Matrix<14> >, Rcpp::traits::named_object<arma::Mat<double> >, Rcpp::traits::named_object<arma::Mat<double> >, Rcpp::traits::named_object<arma::Mat<double> >, Rcpp::traits::named_object<arma::Mat<double> >, Rcpp::traits::named_object<arma::Mat<double> > > (R-devel/site-library/Rcpp/include/Rcpp/generated/Vector__create.h:343)
==2563572==    by 0x175CAA52: cmnorm(Rcpp::Vector<14, Rcpp::PreserveStorage>, Rcpp::Matrix<14, Rcpp::PreserveStorage>, Rcpp::Vector<14, Rcpp::PreserveStorage>, Rcpp::Vector<14, Rcpp::PreserveStorage>, Rcpp::Vector<14, Rcpp::PreserveStorage>, bool, bool, Rcpp::Nullable<Rcpp::Vector<19, Rcpp::PreserveStorage> >, int) (/tmp/RtmpvZ7OOy/R.INSTALL331e2639ef54f9/mnorm/src/cmnorm.cpp:261)
==2563572==    by 0x175F38F7: pmnorm(Rcpp::Vector<14, Rcpp::PreserveStorage>, Rcpp::Vector<14, Rcpp::PreserveStorage>, Rcpp::Vector<14, Rcpp::PreserveStorage>, Rcpp::Vector<14, Rcpp::PreserveStorage>, Rcpp::Matrix<14, Rcpp::PreserveStorage>, Rcpp::Vector<14, Rcpp::PreserveStorage>, int, Rcpp::String, Rcpp::String, bool, bool, bool, bool, bool, bool, Rcpp::Nullable<Rcpp::Vector<19, Rcpp::PreserveStorage> >, int, Rcpp::Nullable<Rcpp::Vector<19, Rcpp::PreserveStorage> >, bool, bool) (/tmp/RtmpvZ7OOy/R.INSTALL331e2639ef54f9/mnorm/src/pmnorm.cpp:638)
==2563572==    by 0x175FBAA4: pmnorm(Rcpp::Vector<14, Rcpp::PreserveStorage>, Rcpp::Vector<14, Rcpp::PreserveStorage>, Rcpp::Vector<14, Rcpp::PreserveStorage>, Rcpp::Vector<14, Rcpp::PreserveStorage>, Rcpp::Matrix<14, Rcpp::PreserveStorage>, Rcpp::Vector<14, Rcpp::PreserveStorage>, int, Rcpp::String, Rcpp::String, bool, bool, bool, bool, bool, bool, Rcpp::Nullable<Rcpp::Vector<19, Rcpp::PreserveStorage> >, int, Rcpp::Nullable<Rcpp::Vector<19, Rcpp::PreserveStorage> >, bool, bool) (/tmp/RtmpvZ7OOy/R.INSTALL331e2639ef54f9/mnorm/src/pmnorm.cpp:1343)
==2563572==    by 0x175C2584: _mnorm_pmnorm_try(SEXPREC*, SEXPREC*, SEXPREC*, SEXPREC*, SEXPREC*, SEXPREC*, SEXPREC*, SEXPREC*, SEXPREC*, SEXPREC*, SEXPREC*, SEXPREC*, SEXPREC*, SEXPREC*, SEXPREC*, SEXPREC*, SEXPREC*, SEXPREC*, SEXPREC*, SEXPREC*) (/tmp/RtmpvZ7OOy/R.INSTALL331e2639ef54f9/mnorm/src/RcppExports.cpp:232)
==2563572==    by 0x1767C9B3: mnorm::pmnorm(Rcpp::Vector<14, Rcpp::PreserveStorage>, Rcpp::Vector<14, Rcpp::PreserveStorage>, Rcpp::Vector<14, Rcpp::PreserveStorage>, Rcpp::Vector<14, Rcpp::PreserveStorage>, Rcpp::Matrix<14, Rcpp::PreserveStorage>, Rcpp::Vector<14, Rcpp::PreserveStorage>, int, Rcpp::String, Rcpp::String, bool, bool, bool, bool, bool, bool, Rcpp::Nullable<Rcpp::Vector<19, Rcpp::PreserveStorage> >, int, Rcpp::Nullable<Rcpp::Vector<19, Rcpp::PreserveStorage> >, bool, bool) (R-devel/site-library/mnorm/include/mnorm_RcppExports.h:138)
==2563572==    by 0x17670748: lnL_mnprobit(arma::Col<double>, Rcpp::Vector<19, Rcpp::PreserveStorage>, Rcpp::String, int, int) (packages/tests-vg/switchSelection/src/mnprobit.cpp:287)

因为它提到 

lnL_mnprobit
函数与上述示例代码中的 
mvoprobit
函数无关（而不是 
lnL_mnprobit
与 
mnprobit
函数相关联）我得出结论，尽管错误消息所在的位置它与另一个相关联示例代码的一部分（在我上面发布的示例代码之前）：

# some data preparation code
model <- mnprobit(z ~ w1 + w2,
                  data = data)
summary(model)

使用与之前相同的逻辑，我认为与 cmnorm.cpp 文件的第 261 行相关的问题如下：

  List return_list = List::create(Named("mean") = mean_cond_numeric,
                                  Named("sigma") = sigma_cond_numeric,
                                  Named("s12s22") = s12s22,
                                  Named("sigma_d") = sigma_d,
                                  Named("sigma_g") = sigma_g,
                                  Named("sigma_dg") = sigma_dg,
                                  Named("sigma_g_inv") = sigma_g_inv);

其中

mean_cond_numeric
和
sigma_cond_numeric
是
Rcpp::NumericMatrix
，而
s12s22
，
sigma_d
，
sigma_dg
和
sigma_g_inv
是
arma::mat
。通过运行示例代码，我检查了这些矩阵总是非空的并且包含正确的值。我唯一的假设是 
Rcpp::List
包含 
Rcpp::NumericMatrix
和 
arma::mat
。这可能是内存问题的根源吗？

问题三

报错信息如下：

==2563572== 352 bytes in 1 blocks are possibly lost in loss record 94 of 1,822
==2563572==    at 0x4845464: calloc (/builddir/build/BUILD/valgrind-3.18.1/coregrind/m_replacemalloc/vg_replace_malloc.c:1328)
==2563572==    by 0x401364B: UnknownInlinedFun (/usr/src/debug/glibc-2.33-21.fc34.x86_64/elf/../include/rtld-malloc.h:44)
==2563572==    by 0x401364B: allocate_dtv (/usr/src/debug/glibc-2.33-21.fc34.x86_64/elf/../elf/dl-tls.c:366)
==2563572==    by 0x401364B: _dl_allocate_tls (/usr/src/debug/glibc-2.33-21.fc34.x86_64/elf/../elf/dl-tls.c:612)
==2563572==    by 0x53C8008: pthread_create@@GLIBC_2.2.5 (in /usr/lib64/libpthread-2.33.so)
==2563572==    by 0x5395283: ??? (in /usr/lib64/libgomp.so.1.0.0)
==2563572==    by 0x538C2D0: GOMP_parallel (in /usr/lib64/libgomp.so.1.0.0)
==2563572==    by 0x17607F70: normcdf_helper<arma::Mat<double>, arma::Gen<arma::Mat<double>, arma::gen_zeros>, arma::Gen<arma::Mat<double>, arma::gen_ones> > (R-devel/site-library/RcppArmadillo/include/armadillo_bits/fn_normcdf.hpp:67)
==2563572==    by 0x17607F70: arma::enable_if2<arma::is_real<arma::Mat<double>::elem_type>::value, arma::Mat<arma::Mat<double>::elem_type> >::result arma::normcdf<arma::Mat<double> >(arma::Base<arma::Mat<double>::elem_type, arma::Mat<double> > const&) (R-devel/site-library/RcppArmadillo/include/armadillo_bits/fn_normcdf.hpp:153)
==2563572==    by 0x175F4C5B: pmnorm(Rcpp::Vector<14, Rcpp::PreserveStorage>, Rcpp::Vector<14, Rcpp::PreserveStorage>, Rcpp::Vector<14, Rcpp::PreserveStorage>, Rcpp::Vector<14, Rcpp::PreserveStorage>, Rcpp::Matrix<14, Rcpp::PreserveStorage>, Rcpp::Vector<14, Rcpp::PreserveStorage>, int, Rcpp::String, Rcpp::String, bool, bool, bool, bool, bool, bool, Rcpp::Nullable<Rcpp::Vector<19, Rcpp::PreserveStorage> >, int, Rcpp::Nullable<Rcpp::Vector<19, Rcpp::PreserveStorage> >, bool, bool) (/tmp/RtmpvZ7OOy/R.INSTALL331e2639ef54f9/mnorm/src/pmnorm.cpp:725)
==2563572==    by 0x175C2584: _mnorm_pmnorm_try(SEXPREC*, SEXPREC*, SEXPREC*, SEXPREC*, SEXPREC*, SEXPREC*, SEXPREC*, SEXPREC*, SEXPREC*, SEXPREC*, SEXPREC*, SEXPREC*, SEXPREC*, SEXPREC*, SEXPREC*, SEXPREC*, SEXPREC*, SEXPREC*, SEXPREC*, SEXPREC*) (/tmp/RtmpvZ7OOy/R.INSTALL331e2639ef54f9/mnorm/src/RcppExports.cpp:232)
==2563572==    by 0x1767C9B3: mnorm::pmnorm(Rcpp::Vector<14, Rcpp::PreserveStorage>, Rcpp::Vector<14, Rcpp::PreserveStorage>, Rcpp::Vector<14, Rcpp::PreserveStorage>, Rcpp::Vector<14, Rcpp::PreserveStorage>, Rcpp::Matrix<14, Rcpp::PreserveStorage>, Rcpp::Vector<14, Rcpp::PreserveStorage>, int, Rcpp::String, Rcpp::String, bool, bool, bool, bool, bool, bool, Rcpp::Nullable<Rcpp::Vector<19, Rcpp::PreserveStorage> >, int, Rcpp::Nullable<Rcpp::Vector<19, Rcpp::PreserveStorage> >, bool, bool) (R-devel/site-library/mnorm/include/mnorm_RcppExports.h:138)
==2563572==    by 0x17670748: lnL_mnprobit(arma::Col<double>, Rcpp::Vector<19, Rcpp::PreserveStorage>, Rcpp::String, int, int) (packages/tests-vg/switchSelection/src/mnprobit.cpp:287)
==2563572==    by 0x17662A54: _switchSelection_lnL_mnprobit (packages/tests-vg/switchSelection/src/RcppExports.cpp:47)
==2563572==    by 0x4A268D: R_doDotCall (svn/R-devel/src/main/dotcode.c:880)

我认为与来自

pmnorm.cpp
的725行代码相关的问题：

arma::vec prob0 = (arma::normcdf(upr1) - arma::normcdf(lwr1)) %
                  (arma::normcdf(upr2) - arma::normcdf(lwr2));

不幸的是，我不知道为什么会出现错误消息，因为 

arma::normcdf
的参数总是定义明确。

如果能帮助我理解上述问题的根源以及如何解决这些问题，我将非常感激！

r valgrind rcpp cran rcpparmadillo
1个回答
0
投票

“在阅读了一些解释 Valgrind 检查结果的教程后，我得出的结论是，从下往上阅读错误信息是合理的。”

我不建议这样做。

  1. 最初不要打开泄漏检查。
  2. 自上而下修复所有错误。
  3. 当你没有更多的错误时，打开泄漏检查。
  4. 从下往上修复漏洞（最大的漏洞在最后）。

假设

==2563572== Invalid read of size 8
==2563572==    at 0x52B841: STRING_ELT (svn/R-devel/src/main/memory.c:3923)

确实是指

if (marginal_names[i] != "normal")

这可能意味着

i
为负数或大于
marginal_names
被分配容纳的元素数。 memcheck 错误应该提供更多信息，说明无效写入在数组之前或之后有多少字节。

最新问题
© www.soinside.com 2019 - 2024. All rights reserved.