我当前在尝试使用 GStreamer 中的 appsrc 元素解码 H264 帧时遇到问题。我在我的应用程序中构建了以下管道：
appsrc 名称=mysource ！视频/x-h264，对齐=au， 流格式=字节流！ h264解析！视频/x-h264，对齐=au， 流格式=字节流！ omxh264dec ！视频/x-raw， 格式=NV16_10LE32，宽度=1920，高度=1080，帧率=30/1！文件接收器 位置=ball.raw
为了向 appsrc 提供帧，我正在逐帧读取 H264 文件，因为我将每个帧及其大小存储在一个结构中。然后，我使用以下函数将框架推送到 appsrc 中：
void VDRE2GStreamerDecodingDisplaySinkTask::PushBufferToGStreamer(uint8_t* encoded_frame, uint32_t frame_size)
{
GstFlowReturn ret;
GstMapInfo map;
GstBuffer *buffer = gst_buffer_new_and_alloc(frame_size);
gst_buffer_map(buffer, &map, GST_MAP_WRITE);
map.data = encoded_frame;
gst_buffer_unmap(buffer, &map);
/* Push the buffer into the appsrc */
g_signal_emit_by_name(m_appsrc, "push-buffer", buffer, &ret);
/* Free the buffer now that we are done with it */
gst_buffer_unref(buffer);
if (ret != GST_FLOW_OK) {
g_print("PushBufferToGStreamer: Error!\n");
}
}
但是，当我执行管道并设置 GST_DEBUG=4 时，遇到以下错误消息：
Decode Thread: Frame to Process : 1 :s 246
Decode Thread: Frame to Process : 2 :s 232
Decode Thread: Frame to Process : 3 :s 854
Decode Thread: Frame to Process : 4 :s 952
Decode Thread: Frame to Process : 5 :s 350
Decode Thread: Frame to Process : 6 :s 1038
Decode Thread: Frame to Process : 7 :s 1447
Decode Thread: Frame to Process : 8 :s 1316
Decode Thread: Frame to Process : 9 :s 1297
Decode Thread: Frame to Process : 10 :s 1205
0:00:02.370465675 6548 0xffff8c01ff60 WARN h264parse gsth264parse.c:1562:gst_h264_parse_handle_frame:<myparser> broken/invalid nal Type: 0 Unknown, Size: 29 will be dropped
0:00:02.370549896 6548 0xffff8c01ff60 WARN h264parse gsth264parse.c:1562:gst_h264_parse_handle_frame:<myparser> broken/invalid nal Type: 0 Unknown, Size: 1140 will be dropped
Decode Thread: Frame to Process : 11 :s 1222
Decode Thread: Frame to Process : 12 :s 1189
Decode Thread: Frame to Process : 13 :s 1094
Decode Thread: Frame to Process : 14 :s 930
Decode Thread: Frame to Process : 15 :s 1000
Decode Thread: Frame to Process : 16 :s 888
Decode Thread: Frame to Process : 17 :s 901
Decode Thread: Frame to Process : 18 :s 788
Decode Thread: Frame to Process : 19 :s 833
Decode Thread: Frame to Process : 20 :s 764
Decode Thread: Frame to Process : 21 :s 727
Decode Thread: Frame to Process : 22 :s 616
Decode Thread: Frame to Process : 23 :s 728
Decode Thread: Frame to Process : 24 :s 622
Decode Thread: Frame to Process : 25 :s 609
Decode Thread: Frame to Process : 26 :s 657
Decode Thread: Frame to Process : 27 :s 651
Decode Thread: Frame to Process : 28 :s 621
Decode Thread: Frame to Process : 29 :s 534
Decode Thread: Frame to Process : 30 :s 2989
0:00:03.033182378 6548 0xffff8c01ff60 WARN h264parse gsth264parse.c:1562:gst_h264_parse_handle_frame:<myparser> broken/invalid nal Type: 0 Unknown, Size: 29 will be dropped
0:00:03.033258709 6548 0xffff8c01ff60 WARN h264parse gsth264parse.c:1562:gst_h264_parse_handle_frame:<myparser> broken/invalid nal Type: 0 Unknown, Size: 117 will be dropped
0:00:03.033309439 6548 0xffff8c01ff60 WARN h264parse gsth264parse.c:1562:gst_h264_parse_handle_frame:<myparser> broken/invalid nal Type: 0 Unknown, Size: 29 will be dropped
0:00:03.033369800 6548 0xffff8c01ff60 WARN h264parse gsth264parse.c:1562:gst_h264_parse_handle_frame:<myparser> broken/invalid nal Type: 0 Unknown, Size: 1468 will be dropped
Decode Thread: Frame to Process : 31 :s 411
Decode Thread: Frame to Process : 32 :s 465
Decode Thread: Frame to Process : 33 :s 514
Decode Thread: Frame to Process : 34 :s 520
Decode Thread: Frame to Process : 35 :s 519
Decode Thread: Frame to Process : 36 :s 476
Decode Thread: Frame to Process : 37 :s 448
Decode Thread: Frame to Process : 38 :s 544
Decode Thread: Frame to Process : 39 :s 523
Decode Thread: Frame to Process : 40 :s 454
Decode Thread: Frame to Process : 41 :s 561
Decode Thread: Frame to Process : 42 :s 409
Decode Thread: Frame to Process : 43 :s 506
Decode Thread: Frame to Process : 44 :s 330
Decode Thread: Frame to Process : 45 :s 2180
0:00:03.530095934 6548 0xffff8c01ff60 WARN h264parse gsth264parse.c:1562:gst_h264_parse_handle_frame:<myparser> broken/invalid nal Type: 0 Unknown, Size: 29 will be dropped
0:00:03.530192075 6548 0xffff8c01ff60 WARN h264parse gsth264parse.c:1562:gst_h264_parse_handle_frame:<myparser> broken/invalid nal Type: 0 Unknown, Size: 117 will be dropped
0:00:03.530251666 654
然后我分析了encoded_data以检查是否存在有效的NAL单元：
分析了encoded_frame数据以确定它是否包含有效的NAL单元。以下是您对前五个数据集的发现摘要：
Encoded data values in hexadecimal: 0, size: 3241
00 00 00 01 27 7A 00 28 B6 D0 0C E8 07 80 22 7E 27 01 6A 02 04 04 80 00 00 03 00 80 00 00 1E 19 A8 00 3D 09 00 01 6E 37 FF FE 05 00 00 00 01 28 EE 3C
Encoded data values in hexadecimal: 1, size: 246
00 00 00 01 21 E2 01 08 42 DF 00 00 03 00 00 03 00 00 03 00 00 03 00 00 03 00 00 03 00 00 03 00 00 09 36 B5 84 05 B0 C9 8C DA C9 6F 9D F0 00
Encoded data values in hexadecimal: 2, size: 232
00 00 00 01 21 E4 02 08 42 DF 00 00 03 00 00 03 00 00 03 00 00 03 00 00 03 00 00 03 00 00 03 00 00 03 00 16 D0 00 00 01 21 00 1F E3 90 08 21
Encoded data values in hexadecimal: 3, size: 854
00 00 00 01 21 E6 03 08 8B 7F 00 00 03 00 00 03 00 00 03 00 00 03 00 00 03 00 00 03 00 00 03 00 00 03 00 06 2C 00 00 01 21 00 1F E3 98 0C 22
Encoded data values in hexadecimal: 4, size: 952
00 00 00 01 21 E8 04 0E DF 00 00 03 00 00 03 00 00 03 00 00 03 00 00 03 00 00 03 00 00 03 00 00 03 00 00 97 80 00 00 01 21 00 1F E3 A0 10 3B
正如预期的那样，分析的数据似乎包含有效的 NAL 单元。我尝试使用文件源进行类似的测试，并使用 gst-launch 启动管道，如下所示：
gst-launch-1.0 -e filesrc 位置=ball2.h264 ！视频/x-h264， 对齐=au，宽度=1920，高度=1080，帧率=30/1！ h264解析！ omxh264dec ！视频/x-raw，格式=NV16_10LE32，宽度=1920，高度=1080， 帧率=30/1！文件接收器位置=ball_gst.raw
在此测试期间，我没有遇到任何 GST_DEBUG=4 的问题或错误消息。此外，原始文件填充了字节，与 appsrc 版本不同，appsrc 版本的内容始终显示为空（大概是因为它在任何时候都没有被解码）。
我什至尝试在 filesrc 和 h264parse 之间使用 identity dump=true element 并且我也有相同的字节注入到 appsrc 所以我不知道有什么区别
我通过删除盖子规格来修复它！！
如果有人想知道如何做这个管道。这是创建它的最终代码 （最后我添加了一个fakesink以避免生成大的原始文件）
m_pipeline = gst_pipeline_new("mypipeline");
m_appsrc = gst_element_factory_make("appsrc", "mysource");
GstElement *h264_parser = gst_element_factory_make("h264parse", "myparser");
GstElement *decoder = gst_element_factory_make("omxh264dec", "mydec");
GstElement *fakesink = gst_element_factory_make("fakesink", "mysink");
/* g_object_set(G_OBJECT (filesink), "location", "ball_inside.raw",
NULL); */
// Check if all elements were created
if (!m_pipeline && m_appsrc && h264_parser && decoder && fakesink)
{
g_print("Could not gst_element_factory_make, terminating\n");
return;
}
// Create gstreamer loop
loop = g_main_loop_new(NULL, FALSE);
// add a message handler
bus = gst_pipeline_get_bus(GST_PIPELINE(m_pipeline));
bus_watch_id = gst_bus_add_watch(bus, bus_call, loop);
gst_object_unref(bus);
// add all elements into pipeline
gst_bin_add_many(GST_BIN(m_pipeline), m_appsrc, h264_parser, decoder, fakesink, NULL);
// link elements into pipe: appsrc -> jpegenc -> appsink
gst_element_link(m_appsrc, h264_parser);
gst_element_link(h264_parser, decoder);
gst_element_link(decoder, fakesink);
GstPad *encoder_pad;
encoder_pad = gst_element_get_static_pad(decoder, "src");
gst_pad_add_probe(encoder_pad, GST_PAD_PROBE_TYPE_BUFFER, (GstPadProbeCallback)probe_source_callback, NULL, NULL);
encoder_pad = gst_element_get_static_pad(decoder, "sink");
gst_pad_add_probe(encoder_pad, GST_PAD_PROBE_TYPE_BUFFER, (GstPadProbeCallback)probe_sink_callback, NULL, NULL);
gst_object_unref (encoder_pad);
g_print("Setting g_main_loop_run to GST_STATE_PLAYING\n");
// Start pipeline so it could process incoming data
gst_element_set_state(m_pipeline, GST_STATE_PLAYING);
我还添加了这些垫来检查解码器两侧是否都有缓冲区。