javacv合并图像和mp3会创建一个比原始mp3长的mp4

Question

我正在使用以下代码合并mp3和图像

IplImage ipl = cvLoadImage(path2ImageFile);
int height = ipl.height();
int width = ipl.width();
if(height%2!=0) height = height+1;
if(width%2!=0) width = width+1;

OpenCVFrameConverter.ToIplImage grabberConverter = new OpenCVFrameConverter.ToIplImage();  
FFmpegFrameRecorder recorder = new FFmpegFrameRecorder(path2OutputFile,width,height); 
FrameGrabber audioFileGrabber = new FFmpegFrameGrabber(path2AudioFile);
try 
{  
    audioFileGrabber.start();

    recorder.setVideoCodec(avcodec.AV_CODEC_ID_H264 );//AV_CODEC_ID_VORBIS
    recorder.setAudioCodec(avcodec.AV_CODEC_ID_AAC);//AV_CODEC_ID_MP3 //AV_CODEC_ID_AAC

    recorder.setFormat("mp4");  
    recorder.setAudioChannels(2);
    recorder.start();  

    Frame frame = null;
    //audioFileGrabber.getFrameNumber()
    while ((frame = audioFileGrabber.grabFrame())!=null) 
    {   recorder.record(grabberConverter.convert(ipl));  
        recorder.record(frame);
    }

    recorder.stop();  
    audioFileGrabber.stop();
 }  

这会产生mp4，但大约要延长1分半钟。为什么会这样？有什么办法可以解决这个问题？

Answer 1

这是一个很好的解决方案。 IMO，问题似乎是由于两个原因

• 在这种情况下，视频的帧速率必须与音频的帧速率相同，因此不会增加时间
• 正在写入的音频帧应在相同的时间戳记下写入

下面是对我有用的最终代码

import org.bytedeco.ffmpeg.global.*;
import org.bytedeco.javacv.*;


public class AudioExample {
    public static void main(String[] args) {
        String path2ImageFile = "WhatsApp Image 2018-12-29 at 21.54.58.jpeg";
        String path2OutputFile = "test.mp4";
        String path2AudioFile = "GMV_M_Brice_English_Greeting-Menus.mp3";

        FFmpegFrameGrabber imageFileGrabber = new FFmpegFrameGrabber(path2ImageFile);
        FFmpegFrameGrabber audioFileGrabber = new FFmpegFrameGrabber(path2AudioFile);
        try {
            audioFileGrabber.start();
            imageFileGrabber.start();

            FFmpegFrameRecorder recorder = new FFmpegFrameRecorder(path2OutputFile,
                    imageFileGrabber.getImageWidth(), imageFileGrabber.getImageHeight());

            recorder.setVideoCodec(avcodec.AV_CODEC_ID_H264);//AV_CODEC_ID_VORBIS
            recorder.setAudioCodec(avcodec.AV_CODEC_ID_AAC);//AV_CODEC_ID_MP3 //AV_CODEC_ID_AAC
            recorder.setSampleRate(audioFileGrabber.getSampleRate());
            recorder.setFrameRate(audioFileGrabber.getAudioFrameRate());
            recorder.setVideoQuality(1);
            recorder.setFormat("mp4");
            recorder.setAudioChannels(2);
            recorder.start();

            Frame frame = null;

            Frame imageFrame = imageFileGrabber.grab();
            while ((frame = audioFileGrabber.grab()) != null) {
                recorder.setTimestamp(frame.timestamp);
                recorder.record(imageFrame);
                recorder.record(frame);
            }
            recorder.stop();
            audioFileGrabber.stop();
            imageFileGrabber.stop();
        } catch (FrameRecorder.Exception | FrameGrabber.Exception e) {
            e.printStackTrace();
        }
    }
}

我删除了图像的opencv，因为我们也可以使用另一个framegrabber来获取图像作为框架。您还可以看到生成的音频与mp3的大小相同

尝试＃2

由于第一次尝试对视频和音频使用相同的帧速率，这会引起一些问题，所以我考虑了固定帧速率，然后将音频插入顶部的问题

import org.bytedeco.ffmpeg.global.*;
import org.bytedeco.javacv.*;


public class AudioExample {
    public static void main(String[] args) {
        String path2ImageFile = "/Users/tarunlalwani/Downloads/WhatsApp Image 2018-12-29 at 21.54.58.jpeg";
        String path2OutputFile = "/Users/tarunlalwani/Downloads/test.mp4";
        String path2AudioFile = "/Users/tarunlalwani/Downloads/GMV_M_Brice_English_Greeting-Menus.mp3";

        FFmpegFrameGrabber imageFileGrabber = new FFmpegFrameGrabber(path2ImageFile);
        FFmpegFrameGrabber audioFileGrabber = new FFmpegFrameGrabber(path2AudioFile);
        try {
            audioFileGrabber.start();
            imageFileGrabber.start();

            FFmpegFrameRecorder recorder = new FFmpegFrameRecorder(path2OutputFile,
                    imageFileGrabber.getImageWidth(), imageFileGrabber.getImageHeight());

            recorder.setVideoCodec(avcodec.AV_CODEC_ID_H264);//AV_CODEC_ID_VORBIS
            recorder.setAudioCodec(avcodec.AV_CODEC_ID_AAC);//AV_CODEC_ID_MP3 //AV_CODEC_ID_AAC
            recorder.setSampleRate(audioFileGrabber.getSampleRate());
            int frameRate = 30;
            recorder.setFrameRate(frameRate);
            recorder.setFormat("mp4");
            recorder.setAudioChannels(2);
            recorder.start();

            Frame frame = null;

            Frame imageFrame = imageFileGrabber.grab();
            int frames = (int)((audioFileGrabber.getLengthInTime() / 1000000f) * frameRate) + 1;
            int i = 0;

            //create video using fixed images and framerate
            while (i ++ <frames) {
                recorder.record(imageFrame);
            }

            // Add audio to given timestamps
            while ((frame = audioFileGrabber.grabFrame()) != null) {
                recorder.setTimestamp(frame.timestamp);
                recorder.record(frame);
            }

            recorder.stop();
            audioFileGrabber.stop();
            imageFileGrabber.stop();
        } catch (FrameRecorder.Exception | FrameGrabber.Exception e) {
            e.printStackTrace();
        }
    }
}