使用 javascript 从视频摄像机拍摄时图像被缩小并且质量很差

Question

我创建了一个使用 mediaPipe API 的 Web 应用程序。有了这个应用程序，我使用后置摄像头检测人脸并点击图片。

单击图片后，我将获得我在 Canvas 上编写的 MediaPipe api resultimage 对象。它运作良好，期望图像在画布上缩小并且图像质量也很差。

我在下面添加示例代码。非常感谢您的帮助。

下面是打开视频的代码

const videoHeight = 1080;
const videoWidth = 1920
const mode = 'environment';
const camera = new Camera(videoRef.current, {
    onFrame: async () => {
        await faceDetection.send({ image: videoRef.current });
    },
    facingMode: {
        exact: mode
    },
    width: {
        min: videoWidth
    },
    height: {
        min: videoHeight
    }
});
camera.start();

在画布上绘制图像的代码

const getDimensions = (result) => {
    var maxWidth = videoRef.current.clientWidth; // Max width for the image
    var maxHeight = videoRef.current.clientHeight;    // Max height for the image

    var ratio = 0;  // Used for aspect ratio
    var width = result.image.width;    // Current image width
    var height = result.image.height;  // Current image height

    // Check if the current width is larger than the max
    if (width > maxWidth) {
        ratio = maxWidth / width;   // get ratio for scaling image
        height = height * ratio;    // Reset height to match scaled image
        width = width * ratio;    // Reset width to match scaled image
    }

    // Check if current height is larger than max
    if (height > maxHeight) {
        ratio = maxHeight / height; // get ratio for scaling image
        width = width * ratio;    // Reset width to match scaled image
        height = height * ratio;    // Reset height to match scaled image
    }
    return { width: width, height: height }
}




const canvasElement = canvasImgRef.current;
const clientDims = getDimensions(result);  // result is the mediaPipe result
canvasElement.width = clientDims.width;
canvasElement.height = clientDims.height;
let canvasCtx = canvasElement.getContext('2d');
canvasCtx.save();
canvasCtx.clearRect(0, 0, clientDims.width, clientDims.height);
canvasCtx.drawImage(result.image, 0, 0, clientDims.width, clientDims.height); // resul.image is the mediaPipe image

if (result.detections.length > 0) {
    result.detections.forEach(detection => {
        drawRectangle(canvasCtx, detection.boundingBox, { color: 'black', lineWidth: 2, fillColor: 'black' });
    })
}
canvasCtx.restore();

如果有人让我知道如何控制图像的缩放并提高质量，那将对我有很大帮助。

我尝试通过更改高度、宽度约束、更改纵横比等...

Answer 1

1. 从您的代码中不是很清楚您使用什么来处理相机。如果您使用的是

   @mediapipe/camera_utils

   ，那么您使用的参数不正确，因为根据界面应该是

facingMode?: 'user'|'environment';
width?: number;
height?: number;

2. 您是否需要在画布上绘制函数

   faceDetection

   接收到的视频流中的一些特定帧？如果是这样，请查看此链接https://face-detection-codemichael81.netlify.app/我使用 MediaPipe 做了一个示例，它在画布上显示视频并占据整个屏幕，如果这是您需要的，那么您当窗口大小与图像的比例不同时，还需要正确计算宽度和高度以及偏移量

    function (output, canvasCtx, img) {
        output.height = document.documentElement.clientHeight;
        output.width = document.documentElement.clientWidth;
    
        const heightMargin =
          Math.max((img.width / img.height) * output.height - output.width, 0) / 2;
        const widthMargin =
          Math.max((img.height / img.width) * output.width - output.height, 0) / 2;
    
        canvasCtx.drawImage(
          img,
          0,
          0,
          Math.trunc(output.width + 2 * heightMargin),
          Math.trunc(output.height + 2 * widthMargin)
        );
     }

Answer 2

你好迈克尔，感谢你的帮助。我已经根据您的建议更改了参数并计算了图像尺寸，但是图像仍在缩放并且图像质量仍然很差。下面是修改后的代码

        const videoHeight = 1080;
            const videoWidth = 1920
            const camera = new Camera(videoRef.current, {
                        onFrame: async () => {
                            await faceDetection.send({ image: videoRef.current });
                        },
                        facingMode:  'environment',
                        width: videoWidth,
                        height: videoHeight
            });
            camera.start();
            
            const getDimensions = (result) => {
                var maxWidth = videoRef.current.clientWidth; // Max width for the image
                var maxHeight = videoRef.current.clientHeight;    // Max height for the image
        
                // var ratio = 0;  // Used for aspect ratio
                var width = result.image.width;    // Current image width
                var height = result.image.height;  // Current image height
        
                const heightMargin = Math.max((width / height) * maxHeight - maxWidth, 0) / 2;
                const widthMargin = Math.max((height / width) * maxWidth - maxHeight, 0) / 2;
                width = Math.trunc(maxWidth + 2 * heightMargin);
                height =  Math.trunc(maxHeight + 2 * widthMargin);
                return { width: width, height: height }
            }
const drawPicture = () => {
            if (result) {
                const canvasElement = canvasImgRef.current;
                const clientDims = getDimensions(result);
                canvasElement.width = clientDims.width;
                canvasElement.height = clientDims.height;
                let canvasCtx = canvasElement.getContext('2d');
                canvasCtx.save();
                canvasCtx.clearRect(0, 0, clientDims.width, clientDims.height);
                canvasCtx.drawImage(result.image, 0, 0, clientDims.width, clientDims.height);
                if (result.detections.length > 0) {
                    result.detections.forEach(detection => {
                        drawRectangle(canvasCtx, detection.boundingBox, { color: 'black', lineWidth: 2, fillColor: 'black' });
                    })
                }
                canvasCtx.restore();
            }
        }

Answer 3

1. 我更新了链接，尝试使用我的链接拍照并在需要生成图像时单击屏幕，这将打开一个包含结果的新选项卡。

2. 如果它按你的需要工作，那么我得到这个结果使用直接从视频截图生成照片并再次在其上绘制检测结果而不是通过人脸检测获得的画布。为了更轻松地处理视频和截取视频截图，您可以使用例如这个库https://www.npmjs.com/package/react-webcam，而不是 html video