将 OpenCV Mat 作为 RTP 流式传输到 GStreamer 管道

Question

我在 OpenCV 中有一系列灰度图像，我想通过 GStreamer 流式传输它。
首先，为了检查 gstreamer 是否正常工作，我通过运行一个简单的管道并使用 VLC 接收它来对其进行测试。 （平台是Windows 10。）

# Runs test pipeline
gst-launch-1.0 videotestsrc ! openh264enc ! h264parse ! rtph264pay config-interval=10 pt=96 ! udpsink host=127.0.0.1 port=10000

# receive.sdp
v=0
m=video 10000 RTP/AVP 96
c=IN IP4 127.0.0.1
a=rtpmap:96 H264/90000

在 VLC 中运行

receive.sdp

 文件工作正常。

根据测试的管道，我编写了以下管道，其中源为

cv::Mat

，类型为

CV_8UC1

。

cv::VideoWriter writer;
writer.open(
  "appsrc ! "
  "video/x-raw,format=GRAY8,width=2448,height=2048,framerate=1/1 ! "
  "videoconvert ! "
  "openh264enc ! "
  "h264parse ! "
  "rtph264pay config-interval=10 pt=96 ! "
  "udpsink host=127.0.0.1 port=10000 ",
  cv::CAP_GSTREAMER,
  0, // fourcc
  1, // fps
  cv::Size(2448, 2048), 
  false // isColor
);
// This writer is opened well, and 
// pushing cv::Mat to this writer 
// returns no error or warning code.

...并在 VLC 中运行 receive.sdp

，它什么也没显示。

在管道中处理灰度可能是问题，所以我将管道更改为另存为视频文件以便检查它。

下面的代码将管道保存为文件，效果很好。

cv::VideoWriter writer;
writer.open(
  "appsrc ! "
  "video/x-raw,format=GRAY8,width=2448,height=2048,framerate=1/1 ! "
  "videoconvert ! "
  "openh264enc ! "
  "h264parse ! "
  "mp4mux ! filesink location=test.mp4 ", 
  cv::CAP_GSTREAMER,
  0, // fourcc
  1, // fps
  cv::Size(2448, 2048), 
  false // isColor
);

...以下视频在

writer.release();

之后保存。

我错过了什么？

---

编辑

我发现 VLC 在运行几秒钟后仅显示一次帧。

以下是VLC的日志。

 main debug: VLC media player - 3.0.16 Vetinari
main debug: Copyright ⓒ 1996-2021 the VideoLAN team
main debug: revision 3.0.16-0-g5e70837d8d
main debug: configured with /home/jenkins/workspace/vlc-release/windows/vlc-release-win32-x64/extras/package/win32/../../../configure '--enable-update-check' '--enable-lua' '--enable-faad' '--enable-flac' '--enable-theora' '--enable-avcodec' '--enable-merge-ffmpeg' '--enable-dca' '--enable-mpc' '--enable-libass' '--enable-schroedinger' '--enable-realrtsp' '--enable-live555' '--enable-shout' '--enable-goom' '--enable-sse' '--enable-mmx' '--enable-libcddb' '--enable-zvbi' '--disable-telx' '--enable-nls' '--host=x86_64-w64-mingw32' '--with-contrib=../contrib/x86_64-w64-mingw32' '--with-breakpad=https://win.crashes.videolan.org' '--enable-qt' '--enable-skins2' '--enable-dvdread' '--enable-caca' 'host_alias=x86_64-w64-mingw32' 'CFLAGS= ' 'CXXFLAGS= ' 'PKG_CONFIG=pkg-config' 'PKG_CONFIG_LIBDIR=/usr/x86_64-w64-mingw32/lib/pkgconfig:/usr/lib/x86_64-w64-mingw32/pkgconfig'
h264 warning: waiting for SPS/PPS
h264 warning: waiting for SPS/PPS
h264 warning: waiting for SPS/PPS
h264 warning: waiting for SPS/PPS
h264 warning: waiting for SPS/PPS
h264 warning: waiting for SPS/PPS
h264 warning: waiting for SPS/PPS
main error: ES_OUT_SET_(GROUP_)PCR is called too late (pts_delay increased to 1851 ms)
h264 warning: waiting for SPS/PPS
main debug: ES_OUT_RESET_PCR called
main debug: Buffering 0%
main debug: Buffering 54%
h264 warning: waiting for SPS/PPS
h264 warning: waiting for SPS/PPS
main debug: Stream buffering done (3000 ms in 3000 ms)
main error: buffer deadlock prevented
main debug: Decoder wait done in 0 ms
h264 warning: waiting for SPS/PPS
h264 warning: waiting for SPS/PPS
h264 warning: waiting for SPS/PPS
h264 warning: waiting for SPS/PPS
h264 warning: waiting for SPS/PPS
h264 warning: waiting for SPS/PPS
h264 warning: waiting for SPS/PPS
h264 warning: waiting for SPS/PPS
h264 warning: waiting for SPS/PPS
h264 warning: waiting for SPS/PPS
h264 debug: found NAL_SPS (sps_id=0)
h264 debug: found NAL_PPS (pps_id=0 sps_id=0)
main debug: restarting module due to input format change
main debug: removing module "avcodec"
main debug: looking for video decoder module matching "any": 19 candidates
avcodec debug: using ffmpeg Lavc58.6.103
avcodec debug: CPU flags: 0x000fd3db
avcodec debug: allowing 6 thread(s) for decoding
avcodec debug: codec (h264) started
avcodec debug: using frame thread mode with 6 threads
main debug: using video decoder module "avcodec"
avcodec debug: available hardware decoder output format 119 (cuda)
avcodec debug: available hardware decoder output format 53 (dxva2_vld)
avcodec debug: available hardware decoder output format 118 (d3d11va_vld)
avcodec debug: available hardware decoder output format 174 (d3d11)
avcodec debug: available software decoder output format 0 (yuv420p)
avcodec debug: trying format d3d11va_vld
main debug: looking for text renderer module matching "any": 3 candidates
freetype debug: Using DWrite backend
freetype debug: DWrite_GetFamily(): family name: Arial
freetype debug: DWrite_ParseFamily(): using font at index 0 with weight 400 for bold: 0, italic: 0
freetype debug: DWrite_ParseFamily(): using font at index 1 with weight 700 for bold: 1, italic: 0
freetype debug: DWrite_ParseFamily(): using font at index 2 with weight 400 for bold: 0, italic: 1
freetype debug: DWrite_ParseFamily(): using font at index 3 with weight 700 for bold: 1, italic: 1
main debug: using text renderer module "freetype"
main debug: looking for video converter module matching "any": 24 candidates
swscale debug: 32x32 (32x32) chroma: YUVA -> 16x16 (16x16) chroma: RGBA with scaling using Bicubic (good quality)
main debug: using video converter module "swscale"
main debug: looking for video converter module matching "any": 24 candidates
yuvp debug: YUVP to YUVA converter
main debug: using video converter module "yuvp"
main debug: Deinterlacing available
main debug: deinterlace -1, mode auto, is_needed 0
main debug: looking for vout window module matching "qt,any": 3 candidates
qt debug: requesting video window...
main debug: resized to 667x466
main debug: resized to 2448x2048
main debug: using vout window module "qt"
main debug: looking for inhibit module matching "any": 1 candidates
main debug: using inhibit module "inhibit"
main debug: Opening vout display wrapper
main debug: looking for vout display module matching "any": 12 candidates
direct3d11 debug: creating Vout EventThread
direct3d11 debug: Win32VoutCreateWindow
main debug: VoutDisplayEvent 'resize' 2448x2048
qt debug: Logical video size: 2448x2048
main debug: resized to 2448x2048
main debug: VoutDisplayEvent 'resize' 2448x2048
direct3d11 debug: inclinometer not found. (hr=0x80070490)
direct3d11 debug: created video sub-window
direct3d11 debug: Vout EventThread running
direct3d11 debug: Created the D3D11 device type 1 level b100.
direct3d11 debug: different 0 device p vs P
direct3d11 warning: failed to read the 1 Display Adapter registry key (2)
direct3d11 warning: can't find adapter in registry
direct3d11 debug: supports colorspace RGB Rec.709 gamma:22 range:FULL
direct3d11 debug: supports colorspace RGB Rec.709 gamma:22 range:STUDIO
direct3d11 debug: supports colorspace RGB Rec.2020 gamma:22 range:STUDIO
direct3d11 debug: supports colorspace RGB Rec.2020 gamma:22 range:FULL
direct3d11 debug: supports colorspace RGB Rec.2020 gamma:2084 range:FULL
direct3d11 debug: supports colorspace RGB Rec.2020 gamma:2084 range:STUDIO
direct3d11 debug: Output max luminance: 270.0, colorspace RGB Rec.709 gamma:22 range:FULL, bits per pixel 8
direct3d11 debug: using colorspace RGB Rec.709 gamma:22 range:FULL
direct3d11 debug: Using pixel format VA_NV12 for chroma DX11
main debug: VoutDisplayEvent 'resize' 2470x2104
direct3d11 debug: using GPU render fence
direct3d11 debug: Direct3D11 resources created
direct3d11 debug: Direct3D11 device adapter successfully initialized
direct3d11 debug: Direct3D11 Open Succeeded
main debug: using vout display module "direct3d11"
direct3d11 debug: D3D11 pool succeed with 40 surfaces (2560x2048) context 0x0000021fe01aed70
main debug: original format sz 2448x2048, of (0,0), vsz 2448x2048, 4cc DX11, sar 1:1, msk r0x0 g0x0 b0x0
main debug: Adding 'gradient' as interactive
main debug: looking for video filter module matching "gradient,chain": 46 candidates
gradient error: Unsupported input chroma (DX11)
chain debug: Trying to use chroma I420 as middle man
main debug: looking for video converter module matching "any": 24 candidates
main debug: using video converter module "d3d11_filters"
main debug: Filter 'd3d11_filters' (0000021fdd1aee90) appended to chain
main debug: looking for video filter module matching "gradient,chain": 46 candidates
main debug: using video filter module "gradient"
main debug: Filter 'gradient' (0000021fdd1b06a0) appended to chain
main debug: using video filter module "chain"
main debug: Filter 'gradient' (0000021fdd1ad9f0) appended to chain
main debug: Adding 'sepia' as interactive
main debug: looking for video filter module matching "sepia,chain": 46 candidates
main debug: using video filter module "sepia"
main debug: Filter 'sepia' (0000021fdd1add60) appended to chain
main debug: Adding a filter to compensate for format changes
main debug: looking for video converter module matching "any": 24 candidates
d3d11_filters debug: different 0 device p vs P
d3d11_filters warning: failed to read the 1 Display Adapter registry key (2)
d3d11_filters warning: can't find adapter in registry
main debug: looking for video converter module matching "any": 24 candidates
main debug: using video converter module "i420_nv12"
main debug: using video converter module "d3d11_filters"
main debug: Filter 'd3d11_filters' (0000021fdd1af570) appended to chain
main debug: removing module "chain"
main debug: removing module "d3d11_filters"
main debug: Filter 0000021fdd1aee90 removed from chain
main debug: removing module "gradient"
main debug: Filter 0000021fdd1b06a0 removed from chain
main debug: Filter 0000021fdd1ad9f0 removed from chain
main debug: removing module "sepia"
main debug: Filter 0000021fdd1add60 removed from chain
main debug: removing module "d3d11_filters"
main debug: removing module "i420_nv12"
main debug: Filter 0000021fdd1af570 removed from chain
main debug: Adding 'gradient' as interactive
main debug: looking for video filter module matching "gradient,chain": 46 candidates
gradient error: Unsupported input chroma (DX11)
chain debug: Trying to use chroma I420 as middle man
main debug: looking for video converter module matching "any": 24 candidates
main debug: using video converter module "d3d11_filters"
main debug: Filter 'd3d11_filters' (0000021fdd1af570) appended to chain
main debug: looking for video filter module matching "gradient,chain": 46 candidates
main debug: using video filter module "gradient"
main debug: Filter 'gradient' (0000021fdd1ad680) appended to chain
main debug: using video filter module "chain"
main debug: Filter 'gradient' (0000021fdd1b06a0) appended to chain
main debug: Adding 'sepia' as interactive
main debug: looking for video filter module matching "sepia,chain": 46 candidates
main debug: using video filter module "sepia"
main debug: Filter 'sepia' (0000021fdd1b0a10) appended to chain
main debug: Adding a filter to compensate for format changes
main debug: looking for video converter module matching "any": 24 candidates
d3d11_filters debug: different 0 device p vs P
d3d11_filters warning: failed to read the 1 Display Adapter registry key (2)
d3d11_filters warning: can't find adapter in registry
main debug: looking for video converter module matching "any": 24 candidates
main debug: using video converter module "i420_nv12"
main debug: using video converter module "d3d11_filters"
main debug: Filter 'd3d11_filters' (0000021fdd1aee90) appended to chain
main debug: removing module "freetype"
main debug: looking for text renderer module matching "any": 3 candidates
main debug: VoutDisplayEvent 'resize' 2448x2048
direct3d11 debug: Detected size change 2448x2048
freetype debug: Using DWrite backend
freetype debug: DWrite_GetFamily(): family name: Arial
freetype debug: DWrite_ParseFamily(): using font at index 0 with weight 400 for bold: 0, italic: 0
freetype debug: DWrite_ParseFamily(): using font at index 1 with weight 700 for bold: 1, italic: 0
freetype debug: DWrite_ParseFamily(): using font at index 2 with weight 400 for bold: 0, italic: 1
freetype debug: DWrite_ParseFamily(): using font at index 3 with weight 700 for bold: 1, italic: 1
main debug: using text renderer module "freetype"
main debug: looking for hw decoder module matching "any": 2 candidates
d3d11va debug: different 0 device p vs P
d3d11va warning: failed to read the 1 Display Adapter registry key (2)
d3d11va warning: can't find adapter in registry
d3d11va debug: Reusing Direct3D11 device
d3d11va debug: CreateDevice succeed
d3d11va debug: - 'MPEG-2 variable-length decoder' is supported
d3d11va debug: - 'MPEG-2 inverse discrete cosine transform' is supported
d3d11va debug: - 'VC-1 variable-length decoder' is supported
d3d11va debug: - 'VC-1 variable-length decoder 2 (Intel)' is supported
d3d11va debug: - 'VC-1 variable-length decoder (Intel)' is supported
d3d11va debug: - 'Windows Media Video 9 IDCT' is supported
d3d11va debug: - 'VC-1 inverse discrete cosine transform' is supported
d3d11va debug: - 'H.264 variable-length decoder, no film grain technology' is supported
d3d11va debug: - 'H.264 stereo high profile, mbs flag set' is supported
d3d11va debug: - 'H.264 stereo high profile' is supported
d3d11va debug: - 'H.264 multiview high profile' is supported
d3d11va debug: - 'Unknown decoder 0xc528916c-0xc0af-0x4645-0x8cb2372b6d4adc2a' is supported
d3d11va debug: - 'Unknown decoder 0x91cd2d6e-0x897b-0x4fa1-0xb0d751dc88010e0a' is supported
d3d11va debug: - 'Unknown decoder 0x97688186-0x56a8-0x4094-0xb543fc9daaa49f4b' is supported
d3d11va debug: - 'Unknown decoder 0x1424d4dc-0x7cf5-0x4bb1-0x9cd7b63717a72a6b' is supported
d3d11va debug: - 'Unknown decoder 0xc346e8a3-0xcbed-0x4d27-0x87cca70eb4dc8c27' is supported
d3d11va debug: - 'Unknown decoder 0xffc79924-0x5eaf-0x4666-0xa73606190f281443' is supported
d3d11va debug: - 'Unknown decoder 0x464bdb3c-0x91c4-0x4e9b-0x896f225496ac4ed6' is supported
d3d11va debug: - 'Unknown decoder 0x9b31316b-0xf204-0x455d-0x8a8c9345dca77c01' is supported
d3d11va debug: - 'Unknown decoder 0xafe4285c-0xab63-0x4b2d-0x8278e6baacea2ce9' is supported
d3d11va debug: - 'Unknown decoder 0x277de9c5-0xed83-0x48dd-0xab8fac2d24b22943' is supported
d3d11va debug: - 'Unknown decoder 0x04c5ba10-0x4e9a-0x4b8e-0x8dbf4f4b48afa27c' is supported
d3d11va debug: - 'Unknown decoder 0x0acef8bc-0x285f-0x415d-0xab227bf2527a3d2e' is supported
d3d11va debug: - 'Unknown decoder 0x24d19fca-0xc5a2-0x4b8e-0x9f93f8f6ef15c890' is supported
d3d11va debug: - 'Unknown decoder 0x353aca91-0xd945-0x4c13-0xae7e469060fac8d8' is supported
d3d11va debug: - 'Unknown decoder 0x28566328-0xf041-0x4466-0x8b148f5831e78f8b' is supported
d3d11va debug: - 'Unknown decoder 0x6b4a94db-0x54fe-0x4ae1-0x9be47a7dad004600' is supported
d3d11va debug: - 'Unknown decoder 0xb8b28e0c-0xecab-0x4217-0x8c82eaaa9755aaf0' is supported
d3d11va debug: - 'Unknown decoder 0x8732ecfd-0x9747-0x4897-0xb42ae534f9ff2b7a' is supported
d3d11va debug: - 'Unknown decoder 0xe139b5ca-0x47b2-0x40e1-0xaf1cad71a67a1836' is supported
d3d11va debug: - 'Unknown decoder 0x056a6e36-0xf3a8-0x4d00-0x96637e9430358bf9' is supported
d3d11va debug: - 'Unknown decoder 0x5415a68c-0x231e-0x46f4-0x878b5e9a22e967e9' is supported
d3d11va debug: - 'Unknown decoder 0x161be912-0x44c2-0x49c0-0xb61ed946852b32a1' is supported
d3d11va debug: - 'Unknown decoder 0xd6d6bc4f-0xd51a-0x4712-0x97e8750917c860fd' is supported
d3d11va debug: - 'Unknown decoder 0x7fef652d-0x3233-0x44df-0xacf7ecfb584dab35' is supported
d3d11va debug: - 'Unknown decoder 0x580da148-0xe4bf-0x49b1-0x943b4214ab05a6ff' is supported
d3d11va debug: - 'Unknown decoder 0xcee393ab-0x1030-0x4f7b-0x8dbc55629c72f17e' is supported
d3d11va debug: - 'Unknown decoder 0x87b2ae39-0xc9a5-0x4c53-0x86b8a52d7edba488' is supported
d3d11va debug: - 'Unknown decoder 0x10e19ac8-0xbf39-0x4443-0xbec31b0cbfe4c7aa' is supported
d3d11va debug: - 'Unknown decoder 0x2dec00c7-0x21ee-0x4bf8-0x8f0e773f11f126a2' is supported
d3d11va debug: - 'Unknown decoder 0xc35153a0-0x23c0-0x4a81-0xb3bb6a1326f2b76b' is supported
d3d11va debug: - 'Unknown decoder 0xa33fd0ec-0xa9d3-0x4c21-0x9276c241cc90f6c7' is supported
d3d11va debug: - 'Unknown decoder 0x310e59d2-0x7ea4-0x47bb-0xb319500e78855336' is supported
d3d11va debug: - 'Unknown decoder 0x8c56eb1e-0x2b47-0x466f-0x8d337dbcd63f3df2' is supported
d3d11va debug: - 'HEVC Main profile' is supported
d3d11va debug: - 'Unknown decoder 0x75fc75f7-0xc589-0x4a07-0xa25b72e03b0383b3' is supported
d3d11va debug: - 'HEVC Main 10 profile' is supported
d3d11va debug: - 'Unknown decoder 0xe484dcb8-0xcac9-0x4859-0x99f55c0d45069089' is supported
d3d11va debug: - 'Unknown decoder 0x41a5af96-0xe415-0x4b0c-0x9d03907858e23e78' is supported
d3d11va debug: - 'Unknown decoder 0x6a6a81ba-0x912a-0x485d-0xb57fccd2d37b8d94' is supported
d3d11va debug: - 'Unknown decoder 0xe4e3cf5b-0x97d2-0x4658-0xaacb366e3ee2ceee' is supported
d3d11va debug: - 'Unknown decoder 0xfd9d9559-0x0fd3-0x4917-0xa9a707e714ee9ef9' is supported
d3d11va debug: - 'Unknown decoder 0x0e4bc693-0x5d2c-0x4936-0xb125aefe32b16d8a' is supported
d3d11va debug: - 'Unknown decoder 0x2f08b5b1-0xdbc2-0x4d48-0x883a4e7b8174cff6' is supported
d3d11va debug: - 'Unknown decoder 0x5467807a-0x295d-0x445d-0xbd2ecba8c2457c3d' is supported
d3d11va debug: - 'Unknown decoder 0xae0d4e15-0x2360-0x40a8-0xbf82028e6a0dd827' is supported
d3d11va debug: - 'Unknown decoder 0x8ff8a3aa-0xc456-0x4132-0xb6ef69d9dd72571d' is supported
d3d11va debug: - 'Unknown decoder 0xc23dd857-0x874b-0x423c-0xb6e082ceaa9b118a' is supported
d3d11va debug: - 'Unknown decoder 0x5b08e35d-0x0c66-0x4c51-0xa6f189d00cb2c197' is supported
d3d11va debug: - 'Unknown decoder 0x07cfaffb-0x5a2e-0x4b99-0xb62ae4ca53b6d5aa' is supported
d3d11va debug: - 'VP9 profile 0' is supported
d3d11va debug: - 'VP9 profile 2' is supported
d3d11va debug: - 'VP9 profile Intel' is supported
d3d11va debug: - 'Unknown decoder 0x68a21c7b-0xd58f-0x4e74-0x9993e4b8172b19a0' is supported
d3d11va debug: - 'Unknown decoder 0x80a3a7bd-0x89d8-0x4497-0xa2b82126af7e6eb8' is supported
d3d11va debug: - 'Unknown decoder 0x1d5c4d76-0xb55a-0x4430-0x904c3383a7ae3b16' is supported
d3d11va debug: - 'Unknown decoder 0xa7f759dd-0x5f54-0x4d7f-0x829142e883c546fe' is supported
d3d11va debug: - 'Unknown decoder 0xf34fa92f-0xdc79-0x474c-0xb0dbb7bd4522df77' is supported
d3d11va debug: - 'Unknown decoder 0xb8be4ccb-0xcf53-0x46ba-0x8d59d6b8a6da5d2a' is supported
d3d11va debug: - 'Unknown decoder 0xca44afc5-0xe1d0-0x42e6-0x9154b127186d4d40' is supported
d3d11va debug: - 'Unknown decoder 0xf9a16190-0x3fb4-0x4dc5-0x9846c8751f83d6d7' is supported
d3d11va debug: - 'Unknown decoder 0x50925b7b-0xe931-0x4978-0xa12a586630f095f9' is supported
d3d11va debug: - 'Unknown decoder 0xb69c20e0-0x2508-0x8790-0x0305875499e0a2d0' is supported
d3d11va debug: - 'Unknown decoder 0x49761bec-0x4b63-0x4349-0xa5ff87ffdf088466' is supported
d3d11va debug: Trying to use 'H.264 variable-length decoder, no film grain technology' as input
d3d11va debug: NV12 output is supported for decoder H.264 variable-length decoder, no film grain technology.
d3d11va debug: Using output format NV12 for decoder H.264 variable-length decoder, no film grain technology
d3d11va debug: va_pool_SetupDecoder id 27 2448x2048 count: 24
d3d11va debug: using external surface pool
d3d11va debug: ID3D11VideoDecoderOutputView succeed with 24 surfaces (2448x2048)
d3d11va debug: we got 9 decoder configurations
d3d11va debug: configuration[0] ConfigBitstreamRaw 2
d3d11va debug: configuration[1] ConfigBitstreamRaw 1
d3d11va debug: configuration[2] ConfigBitstreamRaw 2
d3d11va debug: configuration[3] ConfigBitstreamRaw 2
d3d11va debug: configuration[4] ConfigBitstreamRaw 2
d3d11va debug: configuration[5] ConfigBitstreamRaw 2
d3d11va debug: configuration[6] ConfigBitstreamRaw 2
d3d11va debug: configuration[7] ConfigBitstreamRaw 1
d3d11va debug: configuration[8] ConfigBitstreamRaw 1
d3d11va debug: DxCreateDecoderSurfaces succeed
main debug: using hw decoder module "d3d11va"
avcodec info: Using D3D11VA (Intel(R) UHD Graphics 750, vendor 8086(Intel), device 4c8a, revision 4) for hardware decoding
qt debug: Logical video size: 2448x2048
main debug: resized to 2448x2048
main debug: VoutDisplayEvent 'resize' 2448x2048
main debug: auto hiding mouse cursor
main warning: picture is too late to be displayed (missing 3891 ms)
main warning: picture is too late to be displayed (missing 6906 ms)
main warning: picture is too late to be displayed (missing 4938 ms)
main warning: picture is too late to be displayed (missing 4936 ms)
avcodec error: more than 5 seconds of late video -> dropping frame (computer too slow ?)
avcodec error: more than 5 seconds of late video -> dropping frame (computer too slow ?)
avcodec error: more than 5 seconds of late video -> dropping frame (computer too slow ?)
avcodec error: more than 5 seconds of late video -> dropping frame (computer too slow ?)
avcodec error: more than 5 seconds of late video -> dropping frame (computer too slow ?)
main warning: picture is too late to be displayed (missing 8106 ms)
main warning: picture is too late to be displayed (missing 7228 ms)
main warning: picture is too late to be displayed (missing 8023 ms)
main warning: picture is too late to be displayed (missing 8041 ms)
main warning: picture is too late to be displayed (missing 7972 ms)
avcodec error: more than 5 seconds of late video -> dropping frame (computer too slow ?)
avcodec error: more than 5 seconds of late video -> dropping frame (computer too slow ?)
avcodec error: more than 5 seconds of late video -> dropping frame (computer too slow ?)
avcodec error: more than 5 seconds of late video -> dropping frame (computer too slow ?)
avcodec error: more than 5 seconds of late video -> dropping frame (computer too slow ?)
avcodec error: more than 5 seconds of late video -> dropping frame (computer too slow ?)
main warning: picture is too late to be displayed (missing 11379 ms)
main warning: picture is too late to be displayed (missing 8455 ms)
main warning: picture is too late to be displayed (missing 11198 ms)
main warning: picture is too late to be displayed (missing 9201 ms)
main warning: picture is too late to be displayed (missing 10076 ms)
avcodec error: more than 5 seconds of late video -> dropping frame (computer too slow ?)
avcodec error: more than 5 seconds of late video -> dropping frame (computer too slow ?)
avcodec error: more than 5 seconds of late video -> dropping frame (computer too slow ?)
avcodec error: more than 5 seconds of late video -> dropping frame (computer too slow ?)
avcodec error: more than 5 seconds of late video -> dropping frame (computer too slow ?)
avcodec error: more than 5 seconds of late video -> dropping frame (computer too slow ?)
main warning: picture is too late to be displayed (missing 11264 ms)
main warning: picture is too late to be displayed (missing 9342 ms)
main warning: picture is too late to be displayed (missing 11444 ms)
main warning: picture is too late to be displayed (missing 9518 ms)
main warning: picture is too late to be displayed (missing 9556 ms)
avcodec error: more than 5 seconds of late video -> dropping frame (computer too slow ?)
avcodec error: more than 5 seconds of late video -> dropping frame (computer too slow ?)
avcodec error: more than 5 seconds of late video -> dropping frame (computer too slow ?)
avcodec error: more than 5 seconds of late video -> dropping frame (computer too slow ?)
avcodec error: more than 5 seconds of late video -> dropping frame (computer too slow ?)
avcodec error: more than 5 seconds of late video -> dropping frame (computer too slow ?)
avcodec error: more than 5 seconds of late video -> dropping frame (computer too slow ?)
main warning: picture is too late to be displayed (missing 9699 ms)
main warning: picture is too late to be displayed (missing 9582 ms)
main warning: picture is too late to be displayed (missing 9579 ms)
main warning: picture is too late to be displayed (missing 10576 ms)
main warning: picture is too late to be displayed (missing 14580 ms)
avcodec error: more than 5 seconds of late video -> dropping frame (computer too slow ?)
avcodec error: more than 5 seconds of late video -> dropping frame (computer too slow ?)
avcodec error: more than 5 seconds of late video -> dropping frame (computer too slow ?)
avcodec error: more than 5 seconds of late video -> dropping frame (computer too slow ?)
avcodec error: more than 5 seconds of late video -> dropping frame (computer too slow ?)
main warning: picture is too late to be displayed (missing 8595 ms)
main warning: picture is too late to be displayed (missing 8544 ms)
main warning: picture is too late to be displayed (missing 15550 ms)
main warning: picture is too late to be displayed (missing 9535 ms)
main warning: picture is too late to be displayed (missing 9546 ms)
avcodec error: more than 5 seconds of late video -> dropping frame (computer too slow ?)
avcodec error: more than 5 seconds of late video -> dropping frame (computer too slow ?)
avcodec error: more than 5 seconds of late video -> dropping frame (computer too slow ?)
avcodec error: more than 5 seconds of late video -> dropping frame (computer too slow ?)
avcodec error: more than 5 seconds of late video -> dropping frame (computer too slow ?)
avcodec error: more than 5 seconds of late video -> dropping frame (computer too slow ?)
main warning: picture is too late to be displayed (missing 9482 ms)
main warning: picture is too late to be displayed (missing 9532 ms)
main warning: picture is too late to be displayed (missing 9488 ms)
main warning: picture is too late to be displayed (missing 9533 ms)
main warning: picture is too late to be displayed (missing 9454 ms)
avcodec error: more than 5 seconds of late video -> dropping frame (computer too slow ?)
avcodec error: more than 5 seconds of late video -> dropping frame (computer too slow ?)
avcodec error: more than 5 seconds of late video -> dropping frame (computer too slow ?)
avcodec error: more than 5 seconds of late video -> dropping frame (computer too slow ?)
avcodec error: more than 5 seconds of late video -> dropping frame (computer too slow ?)
main warning: picture is too late to be displayed (missing 8475 ms)
main warning: picture is too late to be displayed (missing 9420 ms)
main warning: picture is too late to be displayed (missing 7577 ms)
main warning: picture is too late to be displayed (missing 10465 ms)
main warning: picture is too late to be displayed (missing 7465 ms)
avcodec error: more than 5 seconds of late video -> dropping frame (computer too slow ?)
avcodec error: more than 5 seconds of late video -> dropping frame (computer too slow ?)
avcodec error: more than 5 seconds of late video -> dropping frame (computer too slow ?)
avcodec error: more than 5 seconds of late video -> dropping frame (computer too slow ?)
avcodec error: more than 5 seconds of late video -> dropping frame (computer too slow ?)
avcodec error: more than 5 seconds of late video -> dropping frame (computer too slow ?)

...我仍然不知道到底是什么原因导致了这个问题，所以我将源尺寸调整得更小，最后效果很好。（2448x2048 到 200x100）

cv::VideoWriter writer;
writer.open(
  "appsrc ! "
  "video/x-raw,format=GRAY8,width=200,height=100,framerate=1/1 ! "
  "videoconvert ! "
  "openh264enc ! "
  "h264parse ! "
  "rtph264pay config-interval=10 pt=96 ! "
  "udpsink host=127.0.0.1 port=10000 ",
  cv::CAP_GSTREAMER,
  0, // fourcc
  1, // fps
  cv::Size(200, 100), 
  false // isColor
);

Answer 1

尝试了一些方法后，简单地提高 FPS 就解决了我的问题。

更大的图像尺寸不是问题。
以下代码片段是增加 fps（至 20）的版本，具有将原始灰度流式传输到 RTP 的最小元素（即使删除
h264parse

 元素也与结果无关。）。

cv::VideoWriter writer;
writer.open(
  "appsrc ! "
  "video/x-raw,format=GRAY8,width=2448,height=2048,framerate=20/1 ! "
  "videoconvert ! "
  "openh264enc ! "
  "rtph264pay ! "
  "udpsink host=127.0.0.1 port=10000 ",
  cv::CAP_GSTREAMER,
  0, // fourcc
  20, // fps
  cv::Size(2448, 2048), // Inputting cv::Mat size 
  false // isColor
);

仍然不明白为什么小 fps（在我的例子中是 1）可能是一个问题。