PyList_New() 导致分段错误“free():invalidpointer”

Question

我正在尝试将 2D C 数组转换为 2D Python 列表。但是当代码尝试调用 PyList_New() 时，我遇到分段错误。行和列的大小通常固定为 50。这种情况发生在特定索引 idxR=39 时。 PyList_New() 正确分配最多 38 个索引，并且总是在第 39 个索引处崩溃。我不知道如何调试这种极其奇怪的行为。

Get2DList 使用不同的数据值重复调用大约 5 次，并且在索引 idxR = 39 处第 5 次总是失败

这是参考代码

/*
Function to convert a C 2D array into a PythonAPI 2D List object
This object is passed onto the python interpretor,
* which then passes it onto the appropriate function.
* The PythonAPI List object is converted to a python list by the interpretor.
* /
PyObject* Get2DList (double** arr,size_t rows, size_t cols)
{
    PyObject* pyList2D=PyList_New(0);
    PyObject* tmpLst = NULL;
    for (int idxR=0; idxR<rows ; idxR++)
    {
        tmpLst=PyList_New(0) ;
        for (int idxC=0; idxC<cols; idxC++)
        {
            PyList_Append (tmpLst, PyFloat_FromDouble (arr[idxR][idxC])) ;
            PyList_Append (pyList2D, tmpLst) ;
            Py_XDECREF (tmpLst);
        }
    }
    return pyList2D;
}

我尝试附加 GDB，并获取堆栈跟踪，但我无法准确理解问题。

这是供参考的堆栈跟踪：

#3
0x000015555109320 in malloc_printerr(str=str@entry=0×1555511be4c1"free():invalidpointer")atmalloc.c:5347
#4
0x000015555109fb5c in
#5
_int_free (av=<optimized out>, p=<optimized out>, have_lock=0) at malloc.c:4173
0x00001554910322d4 in google::protobuf::Map<std::
-cxx11::basic_string<char, std:: char_traits<char>, std::allocator<char> >, tensorf
low::AttrValue>::~Map() () from /home/./.local/lib/python3.8/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so
#6
0x000015548b3081e5 in tensorflow::FunctionDef::~FunctionDef()
from /home/./.local/lib/python3.8/site-packages/tensorflow/python/../libtensorflow_framework.so.2
#7
0x0000155490cf26b3 in TF DeleteFunction ()
from /home/./.local/lib/python3.8/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so
#8 0x0000155527e8cd4f in pybind11:: cp_function::initialize<void (*&) (TF_Function*), void, TF_Function*, pybind11::name, pybind11::scop e, pybind11::sibling, pybind11::call_guard<pybind11::gil_scoped_release> >(void (*&) (TF_Function*), void (*) (TF_Function*), pybind11::na me const&, pybind11::scope const&, pybind11::sibling const&, pybind11::call_guard<pybind11::gil_scoped_release> const&):: flambda(pybind1
1::detail::function_ call&#3}::_ FUN(pybind11::detail::function_call&) (
#9
from /home/./.local/lib/python3.8/site-packages/tensorflow/python/client/_pywrap_tf_session.so
0x0000155527e8fb2f in pybind11::cp_function::dispatcherl_object*, _object*, _object*) () from /home/./.local/lib/python3.8/site-packages/tensorflow/python/client/_pywrap_tf_session.so
#10 0X0000155555065748 in ?? () from /lib/X86_64-1inux-gnu/libpython3.8.so.1.0 #11 0x0000155555065b26 in
_PyObject_MakeTpCall () from /lib/X86_64-1inux-gnu/libpython3.8.so.1.0
#12 0X0000155554631df3 in 7? () from /]ib/×86_64-1inux-gnu/libpython3.8.so.1.0
#13 0x0000155554639ef6 in _PyEval_EvalFrameDefault () from /lib/×86_64-linux-gnu/libpython3.8.so.1.0
#14 0X000015555403d06b in ?? () from /1ib/×86_64-1inux-gnu/libpython3.8.so.1.0 #15 0×0000155555000c36 in
??
() from /lib/X86_64-1inux-gnu/libpython3.8.so.1.0
#16 0X0000155554eebce in ?? () from /lib/×86_64-1inux-gnu/libpython3.8.so.1.0
#17 0x0000155554f29994 in _Pyobject_GC_Malloc()from/lib/×86_64-linux-gnu/libpython3.8.so.1.0
#18 0x0000155554f29c47 in
_PyObject_GC_New () from /lib/X86_64-1inux-gnu/libpython3.8.so.1.0
#19 0X0000155555044f4c in PyList_New () from /lib/X86_64-linux-gnu/libpython3.8.so.1.0
#20 0×0000555555557819 in Get2DList (arr=0×5555599d4c30, rows=50, cols=50) at model.c:22 #21 0x0000555555557a89 in predict (v10=0×55555 f229190, pvo=0×55555a0b8990, avo=0×55555a0bbce, rh=0x555559dc4b40, sIp=0×5555599d4c30)
at model.c:80
#22 0x0000555555568764 in triggerMain (
runPath=0×555555624670"/home/./My_files/Research/WRF_Installation/Dummy_WRF/WRF/test/em_real",params=0×55555563dc60, simulatedTimestep=0x55555c066d00"wrfout_d01_2022-05-05_00:00:08",cur_shmid=2981889)attrigger.c:642
#23 0x0000555555569518 in monitor
--Type <RET> for more, a to quit, c to continue without paging--
runpath=0x555555624070"/home/./My_files/Research/WRF_Installation/Dummy_WRF/WRF/test/em_real",mainArgs=0×5555555f8810, params=0×55555563dc60) at monitor.c:215
#24 0x900055555556abc4 in main (arg=2, argv=0x7fffffffd1c8) at main.c:95

第 20 帧上的当地人信息：

(gdb) info locals 
idxR = 39
pyList2D = 0×1550d4f74980
tmpLst = 0×1550d4f15380

数组数据完全正常，并且“arr”中没有无效的内存访问。 我的 Py_Initialise() 很好，如果我认为初始化有任何问题，它不应该正确运行 4 次迭代。该代码将 2D C 数组转换为 Python 列表，并将数据传递到使用 Tensorflow 的 ML 模型。

Answer 1

我没有测试过这个，但你有一个逻辑错误。

下面是你的内循环。在这里，您正在使用特定 row=idxR 处的数组中的值构建内部列表

tmpLst

。在此内部循环结束时，您应该将整个内部列表分配给外部列表 pyList2D。

for (int idxC=0; idxC<cols; idxC++)
{
    PyList_Append (tmpLst, PyFloat_FromDouble (arr[idxR][idxC])) ;
    PyList_Append (pyList2D, tmpLst) ;
    Py_XDECREF (tmpLst);
}

但是，您将

tmpList

附加到每列的外部列表

pyList2D

中。相反，将第二个附加移动到内部循环之外。

for (int idxC=0; idxC<cols; idxC++)
{
    PyList_Append (tmpLst, PyFloat_FromDouble (arr[idxR][idxC])) ;
}
PyList_Append (pyList2D, tmpLst) ;
Py_XDECREF (tmpLst);