我使用我编写的 Android 5.2 应用程序从平板电脑的前置摄像头录制了多个视频。我已经存储了每个视频的开始时间戳（以毫秒为单位（Unix 时间））。
不幸的是，每个视频都有不同的帧速率（范围从 20 到 30）。使用 OpenCV，我可以获得每个视频的帧速率：
import cv2
video = cv2.VideoCapture(videoFile)
fps = video.get(cv2.CAP_PROP_FPS)
这效果很好，理论上我可以为视频中的每一帧添加 1000/fps（由于毫秒）。但这假设帧率在整个录制过程中保持稳定。不知道是不是这样。
Python 是否有可能独立于帧速率获取视频中每一帧的时间戳（以毫秒为单位）？
你想要
cv2.CAP_PROP_POS_MSEC编辑：实际上，正如 Dan Mašek 向我指出的那样，当您获取该属性时，看起来 OpenCV 正在完全执行该计算（至少假设您使用的是 FFMPEG）：
case CV_FFMPEG_CAP_PROP_POS_MSEC:
return 1000.0*(double)frame_number/get_fps();
所以看来你总是依赖恒定的帧速率假设。然而，即使假设帧速率恒定，乘以帧数也很重要，而不仅仅是不断相加
1000/fpsimport cv2
cap = cv2.VideoCapture('vancouver2.mp4')
fps = cap.get(cv2.CAP_PROP_FPS)
timestamps = [cap.get(cv2.CAP_PROP_POS_MSEC)]
calc_timestamps = [0.0]
while(cap.isOpened()):
frame_exists, curr_frame = cap.read()
if frame_exists:
timestamps.append(cap.get(cv2.CAP_PROP_POS_MSEC))
calc_timestamps.append(calc_timestamps[-1] + 1000/fps)
else:
break
cap.release()
for i, (ts, cts) in enumerate(zip(timestamps, calc_timestamps)):
print('Frame %d difference:'%i, abs(ts - cts))
第0帧差异：0.0
第 1 帧差异：0.0
第 2 帧差异：0.0
第3帧差异：1.4210854715202004e-14
第4帧差异：0.011111111111091532
第5帧差异：0.011111111111091532
第6帧差异：0.011111111111091532
第7帧差异：0.011111111111119953
第8帧差异：0.022222222222183063
第9帧差异：0.022222222222183063
...
第294帧差异：0.8111111111411446
这当然是以毫秒为单位的，所以也许看起来并没有那么大。但这里我的计算差了近 1 毫秒，而且这只是一个 11 秒的视频。无论如何，使用这个属性更容易。
这是一个简化版本，仅读取视频并打印出帧号及其时间戳。
import cv2
cap = cv2.VideoCapture('path_to_video/video_filename.avi')
frame_no = 0
while(cap.isOpened()):
frame_exists, curr_frame = cap.read()
if frame_exists:
print("for frame : " + str(frame_no) + " timestamp is: ", str(cap.get(cv2.CAP_PROP_POS_MSEC)))
else:
break
frame_no += 1
cap.release()
这给出了如下所示的输出：
for frame : 0 timestamp is: 0.0
for frame : 1 timestamp is: 40.0
for frame : 2 timestamp is: 80.0
for frame : 3 timestamp is: 120.0
for frame : 4 timestamp is: 160.0
for frame : 5 timestamp is: 200.0
for frame : 6 timestamp is: 240.0
for frame : 7 timestamp is: 280.0
for frame : 8 timestamp is: 320.0
for frame : 9 timestamp is: 360.0
for frame : 10 timestamp is: 400.0
for frame : 11 timestamp is: 440.0
for frame : 12 timestamp is: 480.0
...
我对多个库做了一些测试。
import av
import cv2
import json
import os
import shutil
import sys
import subprocess
import time
from decimal import Decimal
from decord import VideoReader
from ffms2 import VideoSource
from moviepy.editor import VideoFileClip
from typing import List
def with_movie_py(video: str) -> List[int]:
"""
Link: https://pypi.org/project/moviepy/
My comments:
The timestamps I get are not good compared to gMKVExtractGUI or ffms2. (I only tried with VFR video)
Parameters:
video (str): Video path
Returns:
List of timestamps in ms
"""
vid = VideoFileClip(video)
timestamps = [
round(tstamp * 1000) for tstamp, frame in vid.iter_frames(with_times=True)
]
return timestamps
def with_cv2(video: str) -> List[int]:
"""
Link: https://pypi.org/project/opencv-python/
My comments:
I don't know why, but the last 4 or 5 timestamps are equal to 0 when they should not.
Also, cv2 is slow. It took my computer 132 seconds to process the video.
Parameters:
video (str): Video path
Returns:
List of timestamps in ms
"""
timestamps = []
cap = cv2.VideoCapture(video)
while cap.isOpened():
frame_exists, curr_frame = cap.read()
if frame_exists:
timestamps.append(round(cap.get(cv2.CAP_PROP_POS_MSEC)))
else:
break
cap.release()
return timestamps
def with_pyffms2(video: str) -> List[int]:
"""
Link: https://pypi.org/project/ffms2/
My comments:
Works really well, but it doesn't install ffms2 automatically, so you need to do it by yourself.
The easiest way is to install Vapoursynth and use it to install ffms2.
Also, the library doesn't seems to be really maintained.
Parameters:
video (str): Video path
Returns:
List of timestamps in ms
"""
video_source = VideoSource(video)
# You can also do: video_source.track.timecodes
timestamps = [
int(
(frame.PTS * video_source.track.time_base.numerator)
/ video_source.track.time_base.denominator
)
for frame in video_source.track.frame_info_list
]
return timestamps
def with_decord(video: str) -> List[int]:
"""
Link: https://github.com/dmlc/decord
My comments:
Works really well, but it seems to only work with mkv and mp4 file.
Important, Decord seems to automatically normalise the timestamps which can cause many issue: https://github.com/dmlc/decord/issues/238
Mp4 file can have a +- 1 ms difference with ffms2, but it is acceptable.
Parameters:
video (str): Video path
Returns:
List of timestamps in ms
"""
vr = VideoReader(video)
timestamps = vr.get_frame_timestamp(range(len(vr)))
timestamps = (timestamps[:, 0] * 1000).round().astype(int).tolist()
return timestamps
def with_pyav(video: str, index: int = 0) -> List[int]:
"""
Link: https://pypi.org/project/av/
My comments:
Works really well, but it is slower than ffprobe.
The big advantage is that ffmpeg does not have to be installed on the computer, because pyav installs it automatically
Parameters:
video (str): Video path
index (int): Stream index of the video.
Returns:
List of timestamps in ms
"""
container = av.open(video)
video = container.streams.get(index)[0]
if video.type != "video":
raise ValueError(
f'The index {index} is not a video stream. It is an {video.type} stream.'
)
av_timestamps = [
int(packet.pts * video.time_base * 1000) for packet in container.demux(video) if packet.pts is not None
]
container.close()
av_timestamps.sort()
return av_timestamps
def with_ffprobe(video_path: str, index: int = 0) -> List[int]:
"""
Link: https://ffmpeg.org/ffprobe.html
My comments:
Works really well, but the user need to have FFMpeg in his environment variables.
Parameters:
video (str): Video path
index (int): Index of the stream of the video
Returns:
List of timestamps in ms
"""
def get_pts(packets) -> List[int]:
pts: List[int] = []
for packet in packets:
pts.append(int(Decimal(packet["pts_time"]) * 1000))
pts.sort()
return pts
# Verify if ffprobe is installed
if shutil.which("ffprobe") is None:
raise Exception("ffprobe is not in the environment variable.")
# Getting video absolute path and checking for its existance
if not os.path.isabs(video_path):
dirname = os.path.dirname(os.path.abspath(sys.argv[0]))
video_path = os.path.join(dirname, video_path)
if not os.path.isfile(video_path):
raise FileNotFoundError(f'Invalid path for the video file: "{video_path}"')
cmd = f'ffprobe -select_streams {index} -show_entries packet=pts_time:stream=codec_type "{video_path}" -print_format json'
ffprobeOutput = subprocess.run(cmd, capture_output=True, text=True)
ffprobeOutput = json.loads(ffprobeOutput.stdout)
if len(ffprobeOutput) == 0:
raise Exception(
f"The file {video_path} is not a video file or the file does not exist."
)
if len(ffprobeOutput["streams"]) == 0:
raise ValueError(f"The index {index} is not in the file {video_path}.")
if ffprobeOutput["streams"][0]["codec_type"] != "video":
raise ValueError(
f'The index {index} is not a video stream. It is an {ffprobeOutput["streams"][0]["codec_type"]} stream.'
)
return get_pts(ffprobeOutput["packets"])
def main():
video = r"WRITE_YOUR_VIDEO_PATH"
start = time.process_time()
movie_py_timestamps = with_movie_py(video)
print(f"With Movie py {time.process_time() - start} seconds")
start = time.process_time()
cv2_timestamps = with_cv2(video)
print(f"With cv2 {time.process_time() - start} seconds")
start = time.process_time()
ffms2_timestamps = with_pyffms2(video)
print(f"With ffms2 {time.process_time() - start} seconds")
start = time.process_time()
decord_timestamps = with_decord(video)
print(f"With decord {time.process_time() - start} seconds")
start = time.process_time()
av_timestamps = with_pyav(video)
print(f"With av {time.process_time() - start} seconds")
start = time.process_time()
ffprobe_timestamps = with_ffprobe(video)
print(f"With ffprobe {time.process_time() - start} seconds")
if __name__ == "__main__":
main()
这是获取 24 分钟 mp4 的时间戳所需的时间。
With Movie py 11.421875 seconds
With cv2 131.890625 seconds
With ffms2 0.640625 seconds
With decord 0.328125 seconds
With av 0.6875 seconds
With ffprobe 0.21875 seconds
我使用 moviepy 来获取单个帧的时间（以秒为单位）
pip install moviepy
import sys
import numpy as np
import cv2
import moviepy.editor as mpy
from matplotlib import pyplot as plt
vid = mpy.VideoFileClip('input_video\\v3.mp4')
for i, (tstamp, frame) in enumerate(vid.iter_frames(with_times=True)):
print(tstamp%60)
plt.imshow(frame)
plt.show()
通常这些相机都有卷帘快门，这意味着图像是逐行扫描的，因此严格来说，不能在图像上打上时间戳。我一直致力于使用精确定时（纳秒级）LED 闪光灯来同步多个卷帘快门相机（iPhone 6）。我发现帧速率是可变的（高速时标称 240 fps，但在 239,something 和 241,something 之间变化。相互同步最多可以完成 1/500000 秒，但这需要特殊设置。如果你有兴趣，我可以给你发送一些文档（恐怕我的软件是在 Matlab 中，所以没有现成的 python 代码）