Gstreamer WebRTC 被叫方无法发送视频到远程

问题描述 投票：0回答：1

我们正在编写 webrtc 的示例代码，以使用 GStreamer 向/从浏览器传输/接收视频/音频。我们正在引用演示：https://github.com/centricular/gstwebrtc-demos/blob/master/sendrecv/gst/webrtc_sendrecv.py。它工作正常。

但是我们cloud side的代码有一点不同。在 

gstwebrtc
演示代码中，
webrtcbin
创建一个报价，浏览器响应一个答案。在我们的项目中，浏览器创建一个报价，
webrtcbin
回复一个答案。

我们修改上面的demo代码来实现逆流(browser offer -> webrtcbin)。但是浏览器端没有视频流。浏览器中的 

onTrack
回调永远不会被调用。

我的问题是，当

webrtcbin
作为被调用者时，如何向浏览器发送视频/音频？顺便说一句，我们正在使用
gstreamer 1.20.3
.

附上代码：

import paho.mqtt.client as mqtt
import ssl
import json
import time
import sys

import gi
gi.require_version('Gst', '1.0')
from gi.repository import Gst
gi.require_version('GstWebRTC', '1.0')
from gi.repository import GstWebRTC
gi.require_version('GstSdp', '1.0')
from gi.repository import GstSdp

PIPELINE_DESC = '''
webrtcbin name=sendrecv bundle-policy=max-bundle stun-server=stun://turn.jujiucloud.com:3478,
 v4l2src ! video/x-raw,format=NV12,width=1280,height=720,framerate=30/1 ! videoconvert ! queue ! vp8enc deadline=1 ! rtpvp8pay !
 queue ! application/x-rtp,media=video,encoding-name=VP8,payload=97 ! sendrecv.
 audiotestsrc is-live=true wave=red-noise ! audioconvert ! audioresample ! queue ! opusenc ! rtpopuspay !
 queue ! application/x-rtp,media=audio,encoding-name=OPUS,payload=96 ! sendrecv.
'''

MQTT_URL = 'mqtt.abceefg.com'
MQTT_USERNAME = 'abceefg'
MQTT_PASSWORD = 'abceefg'
MQTT_IPC_TOPIC_PREFIX = 'abceefg'
  
USN = 'abceefg'

webrtc = None
peers = {}

def check_plugins():
  needed = ['opus', 'vpx', 'nice', 'webrtc', 'dtls', 'srtp', 'rtp', 'rtpmanager', 'videotestsrc', 'audiotestsrc']
  missing = list(filter(lambda p: Gst.Registry.get().find_plugin(p) is None, needed))
  if len(missing):
    print('Missing gstreamer plugins:', missing)
    return False
  return True

def on_set_description(promise, webrtc, client, msg_obj):
  print('on_set_description')
  promise.wait()
  promise = Gst.Promise.new_with_change_func(on_answer_created, webrtc, client, msg_obj)
  webrtc.emit('create-answer', None, promise)

def on_answer_created(promise, webrtc, client, msg_obj):
  print('on_answer_created')
  promise.wait()
  reply = promise.get_reply()
  answer = reply.get_value('answer')
  if not answer:
    print('no answer created.')
    return
  promise = Gst.Promise.new()
  webrtc.emit('set-local-description', answer, promise)
  promise.interrupt()

  mqtt_answer = {
    'timestamp': time.time(),
    'source': msg_obj['destination'],
    'destination': msg_obj['source'],
    'type': 'webrtc_signal',
    'subtype': 'answer',
    'answer': {
      'type': 'answer',
      'sdp': answer.sdp.as_text()
    },
  }
  # print(mqtt_answer)
  client.publish(msg_obj['source'], json.dumps(mqtt_answer))

def on_negotiation_needed(element):
  print('on_negotiation_needed')
  

def send_ice_candidate_message(_, mlineindex, candidate, userdata):
  (client, msg_obj) = userdata
  mqtt_ice = {
    'timestamp': time.time(),
    'source': msg_obj['destination'],
    'destination': msg_obj['source'],
    'type': 'webrtc_signal',
    'subtype': 'candidate',
    'candidate': {
      'candidate': candidate,
      'sdpMLineIndex': mlineindex,
      'sdpMid': '0'
    },
  }
  # print(send_ice_candidate_message, mqtt_ice)
  client.publish(msg_obj['source'], json.dumps(mqtt_ice))

def on_connection_state_notify(webrtc, state):
  print(webrtc.get_property('connection-state'))
  return

def on_incoming_stream(_, pad):
  print('on_incoming_stream', pad.direction)
  caps = pad.get_current_caps()
  name = caps.to_string()
  print('on_incoming_stream', name)

def on_connect(client, userdata, flags, rc, properties):
  print('mqtt connected')
  client.subscribe(MQTT_IPC_TOPIC_PREFIX + USN)

def on_disconnect(client, userdata, rc, properties):
  print('mqtt disconnect')

def on_subscribe(client, userdata, mid, rc, properties):
  print('Subscribed.')

def on_message(client, userdata, msg):
  msg_obj = json.loads(msg.payload)
  # print('mqtt received', msg_obj)

  if msg_obj['type'] != 'webrtc_signal':
    print('Unknown type: %s' % msg_obj['type'])
    return
  
  if msg_obj['subtype'] == 'offer':
    print('process offer')
    pipe = Gst.parse_launch(PIPELINE_DESC)
    webrtc = pipe.get_by_name('sendrecv')
    webrtc.connect('on-negotiation-needed', on_negotiation_needed)
    webrtc.connect('on-ice-candidate', send_ice_candidate_message, (client, msg_obj))
    webrtc.connect('pad-added', on_incoming_stream)
    webrtc.connect('notify::connection-state', on_connection_state_notify)
    pipe.set_state(Gst.State.PLAYING)

    peers[msg_obj['source']] = webrtc
    print(peers)

    offer_obj = msg_obj['offer']
    assert(offer_obj['type'] == 'offer')
    sdp = offer_obj['sdp']
    res, sdpmsg = GstSdp.SDPMessage.new()
    GstSdp.sdp_message_parse_buffer(bytes(sdp.encode()), sdpmsg)
    offer = GstWebRTC.WebRTCSessionDescription.new(GstWebRTC.WebRTCSDPType.OFFER, sdpmsg)
    promise = Gst.Promise.new_with_change_func(on_set_description, webrtc, client, msg_obj)
    webrtc.emit('set-remote-description', offer, promise)
    promise.wait()
  elif msg_obj['subtype'] == 'candidate':
    webrtc = peers[msg_obj['source']]
    if not webrtc:
      print('Unknown source of candidate')
      return
    print('Process ice...')
    candidate_obj = msg_obj['candidate']
    if not candidate_obj:
      return
    print(candidate_obj)
    webrtc.emit('add-ice-candidate', candidate_obj['sdpMLineIndex'], candidate_obj['candidate'])
  else:
    print('Unknown subtype: %s' % msg_obj['subtype'])
    return

Gst.init(None)
if not check_plugins():
  sys.exit(1)
print(Gst.version())

client = mqtt.Client(client_id='test_tinker_board_2_webrtc', protocol=mqtt.MQTTv5)
client.username_pw_set(MQTT_USERNAME, MQTT_PASSWORD)
client.on_connect = on_connect
client.on_disconnect = on_disconnect
client.on_subscribe = on_subscribe
client.on_message = on_message
client.connect(MQTT_URL)
client.loop_forever()

浏览器发送的报价：

v=0
o=- 1510238263320950742 2 IN IP4 127.0.0.1
s=-
t=0 0
a=group:BUNDLE 0 1
a=extmap-allow-mixed
a=msid-semantic: WMS a5532e2e-1efa-46cf-a66c-8c4a100d716f
m=audio 9 UDP/TLS/RTP/SAVPF 111 63 9 0 8 13 110 126
c=IN IP4 0.0.0.0
a=rtcp:9 IN IP4 0.0.0.0
a=ice-ufrag:YJn3
a=ice-pwd:QbCvGKsjzFPxFBvQYPRDJdSr
a=ice-options:trickle
a=fingerprint:sha-256 AA:57:14:42:E4:6B:D5:3E:DB:8E:E9:38:DE:63:9D:39:79:08:BA:99:7D:1D:96:CF:1F:96:58:8F:9E:41:16:CF
a=setup:actpass
a=mid:0
a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=extmap:4 urn:ietf:params:rtp-hdrext:sdes:mid
a=sendrecv
a=msid:a5532e2e-1efa-46cf-a66c-8c4a100d716f c0a53b85-cfb3-43d0-aefe-742abcf5ac88
a=rtcp-mux
a=rtpmap:111 opus/48000/2
a=rtcp-fb:111 transport-cc
a=fmtp:111 minptime=10;useinbandfec=1
a=rtpmap:63 red/48000/2
a=fmtp:63 111/111
a=rtpmap:9 G722/8000
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:13 CN/8000
a=rtpmap:110 telephone-event/48000
a=rtpmap:126 telephone-event/8000
a=ssrc:3745948409 cname:Q2GFJXf+QUwZmcpV
a=ssrc:3745948409 msid:a5532e2e-1efa-46cf-a66c-8c4a100d716f c0a53b85-cfb3-43d0-aefe-742abcf5ac88
m=video 9 UDP/TLS/RTP/SAVPF 45 46 47 48 35 36 37 38 100 101 98 99 96 97 102 103 104 105 106 107 108 109 127 125 39 40 41 42 43 44 112 113 114 115 116 49
c=IN IP4 0.0.0.0
a=rtcp:9 IN IP4 0.0.0.0
a=ice-ufrag:YJn3
a=ice-pwd:QbCvGKsjzFPxFBvQYPRDJdSr
a=ice-options:trickle
a=fingerprint:sha-256 AA:57:14:42:E4:6B:D5:3E:DB:8E:E9:38:DE:63:9D:39:79:08:BA:99:7D:1D:96:CF:1F:96:58:8F:9E:41:16:CF
a=setup:actpass
a=mid:1
a=extmap:14 urn:ietf:params:rtp-hdrext:toffset
a=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:13 urn:3gpp:video-orientation
a=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=extmap:5 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay
a=extmap:6 http://www.webrtc.org/experiments/rtp-hdrext/video-content-type
a=extmap:7 http://www.webrtc.org/experiments/rtp-hdrext/video-timing
a=extmap:8 http://www.webrtc.org/experiments/rtp-hdrext/color-space
a=extmap:4 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:10 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
a=extmap:11 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id
a=recvonly
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:45 AV1/90000
a=rtcp-fb:45 goog-remb
a=rtcp-fb:45 transport-cc
a=rtcp-fb:45 ccm fir
a=rtcp-fb:45 nack
a=rtcp-fb:45 nack pli
a=rtpmap:46 rtx/90000
a=fmtp:46 apt=45
a=rtpmap:47 AV1/90000
a=rtcp-fb:47 goog-remb
a=rtcp-fb:47 transport-cc
a=rtcp-fb:47 ccm fir
a=rtcp-fb:47 nack
a=rtcp-fb:47 nack pli
a=fmtp:47 profile=1
a=rtpmap:48 rtx/90000
a=fmtp:48 apt=47
a=rtpmap:35 VP9/90000
a=rtcp-fb:35 goog-remb
a=rtcp-fb:35 transport-cc
a=rtcp-fb:35 ccm fir
a=rtcp-fb:35 nack
a=rtcp-fb:35 nack pli
a=fmtp:35 profile-id=1
a=rtpmap:36 rtx/90000
a=fmtp:36 apt=35
a=rtpmap:37 VP9/90000
a=rtcp-fb:37 goog-remb
a=rtcp-fb:37 transport-cc
a=rtcp-fb:37 ccm fir
a=rtcp-fb:37 nack
a=rtcp-fb:37 nack pli
a=fmtp:37 profile-id=3
a=rtpmap:38 rtx/90000
a=fmtp:38 apt=37
a=rtpmap:100 VP9/90000
a=rtcp-fb:100 goog-remb
a=rtcp-fb:100 transport-cc
a=rtcp-fb:100 ccm fir
a=rtcp-fb:100 nack
a=rtcp-fb:100 nack pli
a=fmtp:100 profile-id=2
a=rtpmap:101 rtx/90000
a=fmtp:101 apt=100
a=rtpmap:98 VP9/90000
a=rtcp-fb:98 goog-remb
a=rtcp-fb:98 transport-cc
a=rtcp-fb:98 ccm fir
a=rtcp-fb:98 nack
a=rtcp-fb:98 nack pli
a=fmtp:98 profile-id=0
a=rtpmap:99 rtx/90000
a=fmtp:99 apt=98
a=rtpmap:96 VP8/90000
a=rtcp-fb:96 goog-remb
a=rtcp-fb:96 transport-cc
a=rtcp-fb:96 ccm fir
a=rtcp-fb:96 nack
a=rtcp-fb:96 nack pli
a=rtpmap:97 rtx/90000
a=fmtp:97 apt=96
a=rtpmap:102 H264/90000
a=rtcp-fb:102 goog-remb
a=rtcp-fb:102 transport-cc
a=rtcp-fb:102 ccm fir
a=rtcp-fb:102 nack
a=rtcp-fb:102 nack pli
a=fmtp:102 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42001f
a=rtpmap:103 rtx/90000
a=fmtp:103 apt=102
a=rtpmap:104 H264/90000
a=rtcp-fb:104 goog-remb
a=rtcp-fb:104 transport-cc
a=rtcp-fb:104 ccm fir
a=rtcp-fb:104 nack
a=rtcp-fb:104 nack pli
a=fmtp:104 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42001f
a=rtpmap:105 rtx/90000
a=fmtp:105 apt=104
a=rtpmap:106 H264/90000
a=rtcp-fb:106 goog-remb
a=rtcp-fb:106 transport-cc
a=rtcp-fb:106 ccm fir
a=rtcp-fb:106 nack
a=rtcp-fb:106 nack pli
a=fmtp:106 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f
a=rtpmap:107 rtx/90000
a=fmtp:107 apt=106
a=rtpmap:108 H264/90000
a=rtcp-fb:108 goog-remb
a=rtcp-fb:108 transport-cc
a=rtcp-fb:108 ccm fir
a=rtcp-fb:108 nack
a=rtcp-fb:108 nack pli
a=fmtp:108 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42e01f
a=rtpmap:109 rtx/90000
a=fmtp:109 apt=108
a=rtpmap:127 H264/90000
a=rtcp-fb:127 goog-remb
a=rtcp-fb:127 transport-cc
a=rtcp-fb:127 ccm fir
a=rtcp-fb:127 nack
a=rtcp-fb:127 nack pli
a=fmtp:127 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=4d001f
a=rtpmap:125 rtx/90000
a=fmtp:125 apt=127
a=rtpmap:39 H264/90000
a=rtcp-fb:39 goog-remb
a=rtcp-fb:39 transport-cc
a=rtcp-fb:39 ccm fir
a=rtcp-fb:39 nack
a=rtcp-fb:39 nack pli
a=fmtp:39 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=4d001f
a=rtpmap:40 rtx/90000
a=fmtp:40 apt=39
a=rtpmap:41 H264/90000
a=rtcp-fb:41 goog-remb
a=rtcp-fb:41 transport-cc
a=rtcp-fb:41 ccm fir
a=rtcp-fb:41 nack
a=rtcp-fb:41 nack pli
a=fmtp:41 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=f4001f
a=rtpmap:42 rtx/90000
a=fmtp:42 apt=41
a=rtpmap:43 H264/90000
a=rtcp-fb:43 goog-remb
a=rtcp-fb:43 transport-cc
a=rtcp-fb:43 ccm fir
a=rtcp-fb:43 nack
a=rtcp-fb:43 nack pli
a=fmtp:43 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=f4001f
a=rtpmap:44 rtx/90000
a=fmtp:44 apt=43
a=rtpmap:112 H264/90000
a=rtcp-fb:112 goog-remb
a=rtcp-fb:112 transport-cc
a=rtcp-fb:112 ccm fir
a=rtcp-fb:112 nack
a=rtcp-fb:112 nack pli
a=fmtp:112 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=64001f
a=rtpmap:113 rtx/90000
a=fmtp:113 apt=112
a=rtpmap:114 red/90000
a=rtpmap:115 rtx/90000
a=fmtp:115 apt=114
a=rtpmap:116 ulpfec/90000
a=rtpmap:49 flexfec-03/90000
a=rtcp-fb:49 goog-remb
a=rtcp-fb:49 transport-cc
a=fmtp:49 repair-window=10000000


WebRTCBin
回复的答案：

v=0
o=- 1510238263320950742 2 IN IP4 0.0.0.0
s=-
t=0 0
a=group:BUNDLE 0 1
m=audio 9 UDP/TLS/RTP/SAVPF 111
c=IN IP4 0.0.0.0
a=ice-ufrag:iNopjlp4xw5BzSeD57e9gG50EzJgCtCq
a=ice-pwd:1FDGS7Bl7vZVt9ND6t3qBSVg4o4zbUAO
a=mid:0
a=rtcp-mux
a=setup:active
a=rtpmap:111 OPUS/48000/2
a=rtcp-fb:111 transport-cc
a=fmtp:111 minptime=10;useinbandfec=1
a=recvonly
a=fingerprint:sha-256 14:23:DD:E9:61:77:1C:74:25:A4:DD:3B:FD:5D:4B:B3:E6:9D:8B:2C:6B:D3:C2:59:62:6D:BB:F1:35:81:31:89
m=video 9 UDP/TLS/RTP/SAVPF 45
c=IN IP4 0.0.0.0
a=ice-ufrag:iNopjlp4xw5BzSeD57e9gG50EzJgCtCq
a=ice-pwd:1FDGS7Bl7vZVt9ND6t3qBSVg4o4zbUAO
a=mid:1
a=rtcp-mux
a=setup:active
a=rtpmap:45 AV1/90000
a=rtcp-fb:45 nack pli
a=rtcp-fb:45 ccm fir
a=rtcp-fb:45 transport-cc
a=inactive
a=fingerprint:sha-256 14:23:DD:E9:61:77:1C:74:25:A4:DD:3B:FD:5D:4B:B3:E6:9D:8B:2C:6B:D3:C2:59:62:6D:BB:F1:35:81:31:89

当我启用 gstreamer 的调试信息时，我看到这条信息：

webrtcbin gstwebrtcbin.c:4569:_create_answer_task:<sendrecv> did not find compatible transceiver for offer caps application/x-rtp, media=(string)video, payload=(int)96, clock-rate=(int)90000, encoding-name=(string)VP8, rtcp-fb-goog-remb=(boolean)true, rtcp-fb-transport-cc=(boolean)true, rtcp-fb-ccm-fir=(boolean)true, rtcp-fb-nack=(boolean)true, rtcp-fb-nack-pli=(boolean)true; application/x-rtp, media=(string)video, payload=(int)97, clock-rate=(int)90000, encoding-name=(string)RTX, apt=(string)96; application/x-rtp, media=(string)video, payload=(int)98, clock-rate=(int)90000, encoding-name=(string)VP9, profile-id=(string)0, rtcp-fb-goog-remb=(boolean)true, rtcp-fb-transport-cc=(boolean)true, rtcp-fb-ccm-fir=(boolean)true, rtcp-fb-nack=(boolean)true, rtcp-fb-nack-pli=(boolean)true; application/x-rtp, media=(string)video, payload=(int)99, clock-rate=(int)90000, encoding-name=(string)RTX, apt=(string)98; application/x-rtp, media=(string)video, payload=(int)100, clock-rate=(int)90000, encoding-name=(string)VP9, profile-id=(string)2, rtcp-fb-goog-remb=(boolean)true, rtcp-fb-transport-cc=(boolean)true, rtcp-fb-ccm-fir=(boolean)true, rtcp-fb-nack=(boolean)true, rtcp-fb-nack-pli=(boolean)true; application/x-rtp, media=(string)video, payload=(int)101, clock-rate=(int)90000, encoding-name=(string)RTX, apt=(string)100; application/x-rtp, media=(string)video, payload=(int)35, clock-rate=(int)90000, encoding-name=(string)VP9, profile-id=(string)1, rtcp-fb-goog-remb=(boolean)true, rtcp-fb-transport-cc=(boolean)true, rtcp-fb-ccm-fir=(boolean)true, rtcp-fb-nack=(boolean)true, rtcp-fb-nack-pli=(boolean)true; application/x-rtp, media=(string)video, payload=(int)36, clock-rate=(int)90000, encoding-name=(string)RTX, apt=(string)35; application/x-rtp, media=(string)video, payload=(int)37, clock-rate=(int)90000, encoding-name=(string)VP9, profile-id=(string)3, rtcp-fb-goog-remb=(boolean)true, rtcp-fb-transport-cc=(boolean)true, rtcp-fb-ccm-fir=(boolean)true, rtcp-fb-nack=(boolean)true, rtcp-fb-nack-pli=(boolean)true; application/x-rtp, media=(string)video, payload=(int)38, clock-rate=(int)90000, encoding-name=(string)RTX, apt=(string)37; application/x-rtp, media=(string)video, payload=(int)102, clock-rate=(int)90000, encoding-name=(string)H264, packetization-mode=(string)1, profile=(string)baseline, rtcp-fb-goog-remb=(boolean)true, rtcp-fb-transport-cc=(boolean)true, rtcp-fb-ccm-fir=(boolean)true, rtcp-fb-nack=(boolean)true, rtcp-fb-nack-pli=(boolean)true; application/x-rtp, media=(string)video, payload=(int)103, clock-rate=(int)90000, encoding-name=(string)RTX, apt=(string)102; application/x-rtp, media=(string)video, payload=(int)104, clock-rate=(int)90000, encoding-name=(string)H264, packetization-mode=(string)0, profile=(string)baseline, rtcp-fb-goog-remb=(boolean)true, rtcp-fb-transport-cc=(boolean)true, rtcp-fb-ccm-fir=(boolean)true, rtcp-fb-nack=(boolean)true, rtcp-fb-nack-pli=(boolean)true; application/x-rtp, media=(string)video, payload=(int)105, clock-rate=(int)90000, encoding-name=(string)RTX, apt=(string)104; application/x-rtp, media=(string)video, payload=(int)106, clock-rate=(int)90000, encoding-name=(string)H264, packetization-mode=(string)1, profile=(string)constrained-baseline, rtcp-fb-goog-remb=(boolean)true, rtcp-fb-transport-cc=(boolean)true, rtcp-fb-ccm-fir=(boolean)true, rtcp-fb-nack=(boolean)true, rtcp-fb-nack-pli=(boolean)true; application/x-rtp, media=(string)video, payload=(int)107, clock-rate=(int)90000, encoding-name=(string)RTX, apt=(string)106; application/x-rtp, media=(string)video, payload=(int)108, clock-rate=(int)90000, encoding-name=(string)H264, packetization-mode=(string)0, profile=(string)constrained-baseline, rtcp-fb-goog-remb=(boolean)true, rtcp-fb-transport-cc=(boolean)true, rtcp-fb-ccm-fir=(boolean)true, rtcp-fb-nack=(boolean)true, rtcp-fb-nack-pli=(boolean)true; application/x-rtp, media=(string)video, payload=(int)109, clock-rate=(int)90000, encoding-name=(string)RTX, apt=(string)108; application/x-rtp, media=(string)video, payload=(int)127, clock-rate=(int)90000, encoding-name=(string)H264, packetization-mode=(string)1, profile=(string)main, rtcp-fb-goog-remb=(boolean)true, rtcp-fb-transport-cc=(boolean)true, rtcp-fb-ccm-fir=(boolean)true, rtcp-fb-nack=(boolean)true, rtcp-fb-nack-pli=(boolean)true; application/x-rtp, media=(string)video, payload=(int)125, clock-rate=(int)90000, encoding-name=(string)RTX, apt=(string)127; application/x-rtp, media=(string)video, payload=(int)39, clock-rate=(int)90000, encoding-name=(string)H264, packetization-mode=(string)0, profile=(string)main, rtcp-fb-goog-remb=(boolean)true, rtcp-fb-transport-cc=(boolean)true, rtcp-fb-ccm-fir=(boolean)true, rtcp-fb-nack=(boolean)true, rtcp-fb-nack-pli=(boolean)true; application/x-rtp, media=(string)video, payload=(int)40, clock-rate=(int)90000, encoding-name=(string)RTX, apt=(string)39; application/x-rtp, media=(string)video, payload=(int)41, clock-rate=(int)90000, encoding-name=(string)H264, packetization-mode=(string)1, profile=(string)high-4:4:4, rtcp-fb-goog-remb=(boolean)true, rtcp-fb-transport-cc=(boolean)true, rtcp-fb-ccm-fir=(boolean)true, rtcp-fb-nack=(boolean)true, rtcp-fb-nack-pli=(boolean)true; application/x-rtp, media=(string)video, payload=(int)42, clock-rate=(int)90000, encoding-name=(string)RTX, apt=(string)41; application/x-rtp, media=(string)video, payload=(int)43, clock-rate=(int)90000, encoding-name=(string)H264, packetization-mode=(string)0, profile=(string)high-4:4:4, rtcp-fb-goog-remb=(boolean)true, rtcp-fb-transport-cc=(boolean)true, rtcp-fb-ccm-fir=(boolean)true, rtcp-fb-nack=(boolean)true, rtcp-fb-nack-pli=(boolean)true; application/x-rtp, media=(string)video, payload=(int)44, clock-rate=(int)90000, encoding-name=(string)RTX, apt=(string)43; application/x-rtp, media=(string)video, payload=(int)45, clock-rate=(int)90000, encoding-name=(string)AV1, rtcp-fb-goog-remb=(boolean)true, rtcp-fb-transport-cc=(boolean)true, rtcp-fb-ccm-fir=(boolean)true, rtcp-fb-nack=(boolean)true, rtcp-fb-nack-pli=(boolean)true; application/x-rtp, media=(string)video, payload=(int)46, clock-rate=(int)90000, encoding-name=(string)RTX, apt=(string)45; application/x-rtp, media=(string)video, payload=(int)47, clock-rate=(int)90000, encoding-name=(string)AV1, profile=(string)1, rtcp-fb-goog-remb=(boolean)true, rtcp-fb-transport-cc=(boolean)true, rtcp-fb-ccm-fir=(boolean)true, rtcp-fb-nack=(boolean)true, rtcp-fb-nack-pli=(boolean)true; application/x-rtp, media=(string)video, payload=(int)48, clock-rate=(int)90000, encoding-name=(string)RTX, apt=(string)47; application/x-rtp, media=(string)video, payload=(int)112, clock-rate=(int)90000, encoding-name=(string)RED; application/x-rtp, media=(string)video, payload=(int)113, clock-rate=(int)90000, encoding-name=(string)RTX, apt=(string)112; application/x-rtp, media=(string)video, payload=(int)114, clock-rate=(int)90000, encoding-name=(string)ULPFEC; application/x-rtp, media=(string)video, payload=(int)49, clock-rate=(int)90000, encoding-name=(string)FLEXFEC-03, repair-window=(string)10000000, rtcp-fb-goog-remb=(boolean)true, rtcp-fb-transport-cc=(boolean)true, will only receive
gstreamer gstreamer-1.0 python-gstreamer
1个回答
0
投票

经过将近一个星期的调查，我终于让它工作了，只做了两次微不足道的修改。

原代码是这样的，来自https://github.com/centricular/gstwebrtc-demos/blob/master/sendrecv/gst/webrtc_sendrecv.py（video payload 97 and audio payload 96）：

PIPELINE_DESC = '''
webrtcbin name=sendrecv bundle-policy=max-bundle stun-server=stun://stun.l.google.com:19302
 videotestsrc is-live=true pattern=ball ! videoconvert ! queue ! vp8enc deadline=1 ! rtpvp8pay !
 queue ! application/x-rtp,media=video,encoding-name=VP8,payload=97 ! sendrecv.
 audiotestsrc is-live=true wave=red-noise ! audioconvert ! audioresample ! queue ! opusenc ! rtpopuspay !
 queue ! application/x-rtp,media=audio,encoding-name=OPUS,payload=96 ! sendrecv.
'''

这行不通。 GStreamer 抱怨：

webrtcbin gstwebrtcbin.c:4569:_create_answer_task:<sendrecv> did not find compatible transceiver for offer caps ...
.

然后我搜索了很多，找到这段代码：https://github.com/sipsorcery/webrtc-echoes/blob/master/gstreamer/gstreamer-webrtc-echo.c.

#define RTP_CAPS_VP8 "application/x-rtp,media=video,encoding-name=VP8,payload=96"

视频的有效载荷数是96而不是97.

所以我将代码更改为（视频有效载荷 96 和音频有效载荷 97）：

PIPELINE_DESC = '''
webrtcbin name=sendrecv bundle-policy=max-bundle stun-server=stun://stun.l.google.com:19302
 videotestsrc is-live=true pattern=ball ! videoconvert ! queue ! vp8enc deadline=1 ! rtpvp8pay !
 queue ! application/x-rtp,media=video,encoding-name=VP8,payload=96 ! sendrecv.
 audiotestsrc is-live=true wave=red-noise ! audioconvert ! audioresample ! queue ! opusenc ! rtpopuspay !
 queue ! application/x-rtp,media=audio,encoding-name=OPUS,payload=97 ! sendrecv.
'''

获取视频流并显示在浏览器中

总结

我们不想知道这些细节来编写 WebRTC 的应用程序。 GStreamer 应该向我们隐藏这些棘手的信息，让它轻松工作。

最新问题
© www.soinside.com 2019 - 2024. All rights reserved.