I am using twilio video, i want to make a video conference app in android.
Peer to peer connection works fine, but
1) how can i render/handle audio-video feed of more then one participant in a video room?
2) I am rendering the view of first participant as shown in video quickstart sdk and for more then one i tried inflating views by using participant array size. i have tried inflating views when 2nd participant(in a room that has the room creator and a participant) joins then all three mobile devices stuck and become totally unresponsive and i have to kill the app.
3) So how can i render lets say 4 participants(besides the room creator-local participant) and render their feeds accordingly in the order they
join the room and handle their removal or disconnection from the room?
The video quick start sample has a comment stating that currently only one participant is handled in the UI, i couldn't find any demo that shows how to handle multiple participants. Is there a way to do it?
Thanks.
here is my code
视频活动
房间监听器
private Room.Listener roomListener() {
return new Room.Listener() {
@Override
public void onConnected(Room room) {
localParticipant = room.getLocalParticipant();
videoStatusTextView.setText("Connected to " + room.getName());
Snackbar.make(connectActionFab,
"Local Participant name: " + room.getLocalParticipant().getIdentity(),
Snackbar.LENGTH_LONG)
.setAction("Action", null).show();
setTitle(room.getName());
for (RemoteParticipant remoteParticipant : room.getRemoteParticipants()) {
addRemoteParticipant(remoteParticipant);
break;
}
}
@Override
public void onConnectFailure(Room room, TwilioException e) {
videoStatusTextView.setText("Failed to connect");
configureAudio(false);
intializeUI();
}
@Override
public void onDisconnected(Room room, TwilioException e) {
localParticipant = null;
videoStatusTextView.setText("Disconnected from " + room.getName());
VideoActivity.this.room = null;
// Only reinitialize the UI if disconnect was not called from onDestroy()
if (!disconnectedFromOnDestroy) {
configureAudio(false);
intializeUI();
moveLocalVideoToPrimaryView();
}
}
@Override
public void onParticipantConnected(Room room, RemoteParticipant remoteParticipant) {
if (thumbnailVideoView.getVisibility() == View.VISIBLE) {
addRemoteParticipant(remoteParticipant);
Snackbar.make(connectActionFab,
"Participant array size " + room.getRemoteParticipants().size()
+ "Participant " + remoteParticipant.getIdentity() + "connected to room",
Snackbar.LENGTH_LONG)
.setAction("Action", null).show();
} else {
addSecondRemoteParticipant(remoteParticipant);
Snackbar.make(connectActionFab,
"Participant array size " + room.getRemoteParticipants().size()
+ "Second Remote Participant " + remoteParticipant.getIdentity() + "connected to room",
Snackbar.LENGTH_LONG)
.setAction("Action", null).show();
Log.e(TAG, "onSecondParticipantConnected: " + room.getRemoteParticipants().size());
}
}
@Override
public void onParticipantDisconnected(Room room, RemoteParticipant remoteParticipant) {
Snackbar.make(connectActionFab,
"Participant array size " + room.getRemoteParticipants().size()
+ "Participant " + remoteParticipant.getIdentity() + "disconnected to room",
Snackbar.LENGTH_LONG)
.setAction("Action", null).show();
removeRemoteParticipant(remoteParticipant);
}
@Override
public void onRecordingStarted(Room room) {
Log.d(TAG, "onRecordingStarted");
}
@Override
public void onRecordingStopped(Room room) {
Log.d(TAG, "onRecordingStopped");
}
};
}
参加者列表
private RemoteParticipant.Listener remoteParticipantListener() {
return new RemoteParticipant.Listener() {
@Override
public void onAudioTrackPublished(RemoteParticipant remoteParticipant,
RemoteAudioTrackPublication remoteAudioTrackPublication) {
videoStatusTextView.setText("onAudioTrackPublished");
}
@Override
public void onAudioTrackUnpublished(RemoteParticipant remoteParticipant,
RemoteAudioTrackPublication remoteAudioTrackPublication) {
videoStatusTextView.setText("onAudioTrackUnpublished");
}
@Override
public void onDataTrackPublished(RemoteParticipant remoteParticipant,
RemoteDataTrackPublication remoteDataTrackPublication) {
videoStatusTextView.setText("onDataTrackPublished");
}
@Override
public void onDataTrackUnpublished(RemoteParticipant remoteParticipant,
RemoteDataTrackPublication remoteDataTrackPublication) {
videoStatusTextView.setText("onDataTrackUnpublished");
}
@Override
public void onVideoTrackPublished(RemoteParticipant remoteParticipant,
RemoteVideoTrackPublication remoteVideoTrackPublication) {
videoStatusTextView.setText("onVideoTrackPublished");
}
@Override
public void onVideoTrackUnpublished(RemoteParticipant remoteParticipant,
RemoteVideoTrackPublication remoteVideoTrackPublication) {
videoStatusTextView.setText("onVideoTrackUnpublished");
}
@Override
public void onAudioTrackSubscribed(RemoteParticipant remoteParticipant,
RemoteAudioTrackPublication remoteAudioTrackPublication,
RemoteAudioTrack remoteAudioTrack) {
videoStatusTextView.setText("onAudioTrackSubscribed");
}
@Override
public void onAudioTrackUnsubscribed(RemoteParticipant remoteParticipant,
RemoteAudioTrackPublication remoteAudioTrackPublication,
RemoteAudioTrack remoteAudioTrack) {
videoStatusTextView.setText("onAudioTrackUnsubscribed");
}
@Override
public void onDataTrackSubscribed(RemoteParticipant remoteParticipant,
RemoteDataTrackPublication remoteDataTrackPublication,
RemoteDataTrack remoteDataTrack) {
videoStatusTextView.setText("onDataTrackSubscribed");
}
@Override
public void onDataTrackUnsubscribed(RemoteParticipant remoteParticipant,
RemoteDataTrackPublication remoteDataTrackPublication,
RemoteDataTrack remoteDataTrack) {
videoStatusTextView.setText("onDataTrackUnsubscribed");
}
@Override
public void onVideoTrackSubscribed(RemoteParticipant remoteParticipant,
RemoteVideoTrackPublication remoteVideoTrackPublication,
RemoteVideoTrack remoteVideoTrack) {
videoStatusTextView.setText("onVideoTrackSubscribed");
if (thumbnailVideoView.getVisibility() == View.GONE) {
if(!ifFirstParticipantHasVideoFeed){
addRemoteParticipantVideo(remoteVideoTrack);
ifFirstParticipantHasVideoFeed=true;
Snackbar.make(connectActionFab,
"Participant array size " + room.getRemoteParticipants().size()
+ "Participant " + remoteParticipant.getIdentity() + "connected to room",
Snackbar.LENGTH_LONG)
.setAction("Action", null).show();
}
else {
if(ifFirstParticipantHasVideoFeed){
addSecondRemoteParticipantVideo(remoteVideoTrackPublication.getRemoteVideoTrack());
}
}
}
}
@Override
public void onVideoTrackUnsubscribed(RemoteParticipant remoteParticipant,
RemoteVideoTrackPublication remoteVideoTrackPublication,
RemoteVideoTrack remoteVideoTrack) {
videoStatusTextView.setText("onVideoTrackUnsubscribed");
removeParticipantVideo(remoteVideoTrack);
}
@Override
public void onAudioTrackEnabled(RemoteParticipant remoteParticipant,
RemoteAudioTrackPublication remoteAudioTrackPublication) {
}
@Override
public void onAudioTrackDisabled(RemoteParticipant remoteParticipant,
RemoteAudioTrackPublication remoteAudioTrackPublication) {
}
@Override
public void onVideoTrackEnabled(RemoteParticipant remoteParticipant,
RemoteVideoTrackPublication remoteVideoTrackPublication) {
}
@Override
public void onVideoTrackDisabled(RemoteParticipant remoteParticipant,
RemoteVideoTrackPublication remoteVideoTrackPublication) {
}
};
}
要添加第二个远程参与者，我已使用以下方法
private void addSecondRemoteParticipant(RemoteParticipant remoteParticipant) {
secondRemoteParticipantIdentity = remoteParticipant.getIdentity();
videoStatusTextView.setText("Second RemoteParticipant " + remoteParticipantIdentity + " joined");
Log.e(TAG, "addSecondRemoteParticipant: " + remoteParticipant.getIdentity());
if (remoteParticipant.getRemoteVideoTracks().size() > 0) {
RemoteVideoTrackPublication remoteVideoTrackPublication =
remoteParticipant.getRemoteVideoTracks().get(0);
if (remoteVideoTrackPublication.isTrackSubscribed()) {
addSecondRemoteParticipantVideo(remoteVideoTrackPublication.getRemoteVideoTrack());
}
}
remoteParticipant.setListener(remoteParticipantListener());
}
private void addSecondRemoteParticipantVideo(RemoteVideoTrack remoteVideoTrack) {
VideoView video = new VideoView(VideoActivity.this);
video.setLayoutParams(new FrameLayout.LayoutParams(300, 300));
video.setVisibility(View.VISIBLE);
video.setBackgroundColor(getResources().getColor(android.R.color.black));
linearVideoParent.addView(video);
remoteVideoTrack.addRenderer(video);
i have checked in debugging till here, but still i some how video is not getting rendered in the view i created.
Snackbar.make(connectActionFab, "added second participant's video",
Snackbar.LENGTH_LONG).show();
}
我尝试在第二个远程参与者连接时扩大视图运行时间，但无法在其中呈现视频。
似乎之间存在时间间隔1）加入参与者/加入参与者后（我们可以从中收听）参与者的听众）
和
2）订阅参与者的视频曲目时（onVideotrackSubscribed）
我还在twilio的多参与者中面临问题。您可以共享您的代码。