cocossd上没有边界框

Question

我正试图通过npm使用来自tensorflow模型的cocossd。

目前，该组件的页面上显示了视频，但是很遗憾，没有边框。

该模型是​​经过预先训练的，应该对各种对象进行分类，例如苹果，书籍或猫。

这是反应成分，来自npm的cocossd模型：

const cocoSsd = require('@tensorflow-models/coco-ssd');

class CamBox extends React.Component{

  constructor(props){
    super(props)
    this.videoRef = React.createRef();
    this.canvasRef = React.createRef();

 // we are gonna use inline style

 const detectFromVideoFrame = (model, video) => {
    model.detect(video).then(predictions => {
      this.showDetections(predictions);

      requestAnimationFrame(() => {
        this.detectFromVideoFrame(model, video);
      });
    }, (error) => {
      console.log("Couldn't start the webcam")
      console.error(error)
    });
    const showDetections = predictions => {
      const ctx = this.canvasRef.current.getContext("2d");
      ctx.clearRect(0, 0, ctx.canvas.width, ctx.canvas.height);
      const font = "24px helvetica";
      ctx.font = font;
      ctx.textBaseline = "top";

      predictions.forEach(prediction => {
        const x = prediction.bbox[0];
        const y = prediction.bbox[1];
        const width = prediction.bbox[2];
        const height = prediction.bbox[3];
        // Draw the bounding box.
        ctx.strokeStyle = "#2fff00";
        ctx.lineWidth = 1;
        ctx.strokeRect(x, y, width, height);
        // Draw the label background.
        ctx.fillStyle = "#2fff00";
        const textWidth = ctx.measureText(prediction.class).width;
        const textHeight = parseInt(font, 10);
        // draw top left rectangle
        ctx.fillRect(x, y, textWidth + 10, textHeight + 10);
        // draw bottom left rectangle
        ctx.fillRect(x, y + height - textHeight, textWidth + 15, textHeight + 10);

        // Draw the text last to ensure it's on top.
        ctx.fillStyle = "#000000";
        ctx.fillText(prediction.class, x, y);
        ctx.fillText(prediction.score.toFixed(2), x, y + height - textHeight);
      });
    };
  }
  };


componentDidMount() {
    if (navigator.mediaDevices.getUserMedia) {
      // define a Promise that'll be used to load the webcam and read its frames
      const webcamPromise = navigator.mediaDevices
        .getUserMedia({
          video: true,
          audio: true,
        })
        .then(stream => {
          // pass the current frame to the window.stream
          window.stream = stream;
          // pass the stream to the videoRef
          this.videoRef.current.srcObject = stream;

          return new Promise(resolve => {
            this.videoRef.current.onloadedmetadata = () => {
              resolve();
            };
          });
        }, (error) => {
          console.log("Couldn't start the webcam")
          console.error(error)
        });

      // define a Promise that'll be used to load the model
      const loadlModelPromise = cocoSsd.load();

      // resolve all the Promises
      Promise.all([loadlModelPromise, webcamPromise])
        .then(values => {
          this.detectFromVideoFrame(values[0], this.videoRef.current);
        })
        .catch(error => {
          console.error(error);
        });
    }
  }

  render()

{
  console.log(cocoSsd.version);
  return(
    <div class="CamMonitor">
    <div class="videoJawn" style={{transform:'translate(0px,0px)'}}>
        <video class="innerVideo"
          autoPlay
          muted
          ref={this.videoRef}
          width="450"
          height="360"
        />
      </div>
    </div>
  )
 }
}

export default CamBox

[我以为模型未显示边界框的原因可能是因为我安装了npm，但是我正在运行yarn安装并启动。但是视频正在显示，并且没有其他损坏，因此另一个想法是，它可能是该类中的错误设置变量或其他内容。

一如既往的任何帮助，我们将不胜感激。

Answer 1

在将其srcObject设置为网络摄像头流之前，应创建一个视频元素

this.videoRef.current = document.createElement('video')
this.videoRef.current.srcObject = stream;

另外，应设置width和height

---

[tfjs附带tf.data.webcam会很方便，它将在后台进行所有操作以获取网络摄像头数据。