nokhwa
使用我的 Android 手机摄像头捕获图像。我已通过 USB 将 Android 手机连接到笔记本电脑,并使用 Droidcam 将其用作网络摄像头。
我的手机摄像头以
YU12
格式发送图像,该库本身不支持。我试图通过将字节流从 YU12
转换为 YUYV
来添加对它的支持。
这是尝试执行转换的函数:
pub fn yu12_to_yuyv(resolution: Resolution, data: &[u8]) -> Vec<u8> {
// Calculate the sizes of the Y, U, and V planes
let width = resolution.width_x as usize;
let height = resolution.height_y as usize;
let size = width * height;
let u_size = (width / 2) * (height / 2);
let v_size = u_size;
// Extract Y, U, and V planes from the input data
let y_plane = &data[0..size];
let u_plane = &data[size..(size + u_size)];
let v_plane = &data[(size + u_size)..(size + u_size + v_size)];
// Create a vector to hold the YUYV data
let mut yuyv = Vec::with_capacity(size * 2);
// Iterate over the image in 2x1 pixel blocks
for y in 0..height {
for x in (0..width).step_by(2) {
// Calculate positions in the Y, U, and V planes
let y_index1 = y * width + x;
let y_index2 = y * width + x + 1;
let uv_index = (y / 2) * (width / 2) + (x / 2);
// Read Y, U, and V values
let y1 = y_plane[y_index1];
let y2 = y_plane[y_index2];
let u = u_plane[uv_index];
let v = v_plane[uv_index];
// Append YUYV data (2 pixels)
yuyv.push(y1);
yuyv.push(u);
yuyv.push(y2);
yuyv.push(v);
}
}
yuyv
}
当然,除了看似噪音的部分之外,这看起来基本上是正确的。鉴于图像的大部分看起来都很好,这里可能出了什么问题?有关更多上下文,宽度为
640
,高度为 480
,data
长度为 462848
。我会注意到,data
的长度有点奇怪,因为它预计是460800
,如我运行v4l2-ctl -d /dev/video0 --all
时所述,所以我不确定额外的2048
字节来自哪里。我认为这可能是某种填充,但我不是 100% 确定。
根据本文档,您交换了 V(Cr) 和 U(Cb)。
此其他文档解释了偏移量。
在本例中,with 为 640,可以忽略步幅,但对于其他宽度,我们在迭代平面时应注意步幅。
fn main() {
let width = 640_usize;
let height = 480_usize;
{
println!("~~~~ initial ~~~~");
let size = width * height;
let u_size = (width / 2) * (height / 2);
let v_size = u_size;
println!("y_plane: {} .. {}", 0, size);
println!("u_plane: {} .. {}", size, (size + u_size));
println!(
"v_plane: {} .. {}",
(size + u_size),
(size + u_size + v_size)
);
}
{
println!("~~~~ modified ~~~~");
// https://learn.microsoft.com/en-us/windows/win32/medfound/recommended-8-bit-yuv-formats-for-video-rendering#yv12
// https://developer.android.com/reference/android/graphics/ImageFormat#YV12
let align16 = |p| (p + 15) & !15;
let stride = align16(width); // actually 640
let y_size = stride * height;
let c_stride = align16(stride / 2); // actually 320
let c_size = c_stride * height / 2;
let cr_offset = y_size; // V is Cr
let cb_offset = y_size + c_size; // U is Cb
println!("y_plane: {} .. {}", 0, y_size);
println!("u_plane: {} .. {}", cb_offset, cb_offset + c_size);
println!("v_plane: {} .. {}", cr_offset, cr_offset + c_size);
}
}
/*
~~~~ initial ~~~~
y_plane: 0 .. 307200
u_plane: 307200 .. 384000
v_plane: 384000 .. 460800
~~~~ modified ~~~~
y_plane: 0 .. 307200
u_plane: 384000 .. 460800
v_plane: 307200 .. 384000
*/