如何使用Camera api 2解决Android视频录制上的视频和音频不同步问题？

Question

我正在使用Camera2 api实现视频录制应用程序。我用过Google samples用于视频录制。但是，某些设备（例如Samsung J5，J6）上存在音频，视频不同步的问题。我已经更改了MediaRecorder.AudioEncoder, MediaRecorder.VideoEncoder, VideoEncodingBitrate，但它无济于事。如何处理音频，视频同步问题？

Answer 1

我从this article找到了解决方案。它可能不是最好的解决方案，但它可以工作。为了解决不同步问题，利用了mp4parser库。首先，视频录制工作流程与普通视频录制相同，但是对于有问题的设备，还有额外的步骤。我将在下面提供我的答案。录像的第一步是准备MediaRecorder，为缩短回答，我将省略一些步骤。

 private void setupMediaRecorder(){
    mMediaRecorder = new MediaRecorder();

    mMediaRecorder.setAudioSource(MediaRecorder.AudioSource.DEFAULT);
    mMediaRecorder.setVideoSource(MediaRecorder.VideoSource.SURFACE);
    mMediaRecorder.setOutputFormat(MediaRecorder.OutputFormat.MPEG_4);

    mMediaRecorder.setOutputFile(mVideoFilePath);
    mMediaRecorder.setVideoSize(mVideoSize.getWidth(), mVideoSize.getHeight());

    CamcorderProfile profile = CamcorderProfile.get(CamcorderProfile.QUALITY_480P);
    mMediaRecorder.setVideoFrameRate(profile.videoFrameRate);
    mMediaRecorder.setVideoSize(profile.videoFrameWidth, profile.videoFrameHeight);
    mMediaRecorder.setVideoEncodingBitRate(profile.videoBitRate);
    mMediaRecorder.setVideoEncoder(MediaRecorder.VideoEncoder.H264);
    mMediaRecorder.setAudioEncoder(MediaRecorder.AudioEncoder.AAC);

    mMediaRecorder.setAudioChannels(2);
    mMediaRecorder.setAudioEncodingBitRate(profile.audioBitRate);
    mMediaRecorder.setAudioSamplingRate(profile.audioSampleRate);

    int rotation = ((WindowManager) mContext.getSystemService(Context.WINDOW_SERVICE)).getDefaultDisplay().getRotation();

    switch (getRotation()) {
        case SENSOR_ORIENTATION_DEFAULT_DEGREES:

            mMediaRecorder.setOrientationHint(ORIENTATIONS.get(rotation));
            break;
        case SENSOR_ORIENTATION_INVERSE_DEGREES:

            mMediaRecorder.setOrientationHint(INVERSE_ORIENTATIONS.get(rotation));
            break;
        case SENSOR_ORIENTATION_DEFAULT_LAND_DEGREES:

            mMediaRecorder.setOrientationHint((ORIENTATIONS.get(rotation)+270)%360);
            break;
        case SENSOR_ORIENTATION_INVERSE_LAND_DEGREES:

            mMediaRecorder.setOrientationHint((INVERSE_ORIENTATIONS.get(rotation)+270)%360);
            break;
    }
    try{
        mMediaRecorder.prepare();
    }catch (IllegalStateException | IOException exc){
        exc.printStackTrace();
    }
}

停止录像

public void stopVideo(){
    //Stop recording
    try {
       mMediaRecorder.stop();
       mMediaRecorder.release();
       parseVideo(mVideoFilePath);
    }catch (RuntimeException e){
    e.printStackTrace();
   }
   closePreviewSession();
   createCameraPreviewSession();
   if (mListener!=null){
    mListener.onPrepareRecorder();
   }
}

最后也是重要的一步是调用parseVideo函数

 private String parseVideo(String mFilePath) {
    try {
        DataSource channel = new FileDataSourceImpl(mFilePath);
        IsoFile isoFile = new IsoFile(channel);
        List<TrackBox> trackBoxes = isoFile.getMovieBox().getBoxes(TrackBox.class);
        boolean isError = false;
        for (TrackBox trackBox : trackBoxes) {
            TimeToSampleBox.Entry firstEntry = trackBox.getMediaBox().getMediaInformationBox().getSampleTableBox().getTimeToSampleBox().getEntries().get(0);
            // Detect if first sample is a problem and fix it in isoFile
            // This is a hack. The audio deltas are 1024 for my files, and video deltas about 3000
            // 10000 seems sufficient since for 30 fps the normal delta is about 3000
            if (firstEntry.getDelta() > 10000) {
                isError = true;
                firstEntry.setDelta(3000);
            }
        }

        if (isError) {
            Movie movie = new Movie();
            for (TrackBox trackBox : trackBoxes) {
                movie.addTrack(new Mp4TrackImpl(channel.toString() + "[" + trackBox.getTrackHeaderBox().getTrackId() + "]", trackBox));
            }
            movie.setMatrix(isoFile.getMovieBox().getMovieHeaderBox().getMatrix());
            Container out = new DefaultMp4Builder().build(movie);

            //delete file first!
            FileChannel fc = new RandomAccessFile(mPostProcessingFilePath, "rw").getChannel();
            out.writeContainer(fc);
            fc.close();
            deleteFile(mVideoFilePath);
            mListener.onVideoStop(mPostProcessingFilePath);
            return mPostProcessingFilePath;
        }
        mListener.onVideoStop(mVideoFilePath);
        return mFilePath;
    }catch (IOException e){
        mListener.onVideoError("");
        return mPostProcessingFilePath;
    }
}

在解析器函数中，它检查增量值，如果它大于10000，它将处理录制的视频并返回结果。如果增量值小于10000，则仅返回视频而不进行处理。有关更多详细信息，请参见link。希望对您有帮助。