在OpenCV和Mask_RCNN中使用Java

Question

我在Java方面拥有丰富的经验，并且已经编写了自己的实用程序来处理图像。我意识到有很多关于Python的教程，但是我更倾向于使用Java，因为我想将OpenCV 4.1的某些功能与现有的Java代码集成在一起。我的实验之一是尝试使用可可数据集获取已识别离子图像的对象的蒙版]

// Give the textGraph and weight files for the model
private static final String TEXT_GRAPH = "models/mask_rcnn_inception_v2_coco_2018_01_28.pbtxt";
private static final String MODEL_WEIGHTS = "models/mask_rcnn_inception_v2_coco_2018_01_28/frozen_inference_graph.pb";
private static final String CLASSES_FILE = "models/mscoco_labels.names";

TextFileConsumer consumer = new TextFileConsumer();
File textFile = new File(CLASSES_FILE);
if (textFile.exists()){
    BufferedReader fr1 = new BufferedReader(new FileReader(textFile));
    fr1.lines().forEach(consumer::storeLines);
}
String[] CLASSES = consumer.lines.toArray(new String[0]);
Mat image = Imgcodecs.imread("images/bird.jpg");
Size size = image.size();
int cols = image.cols();
int rows = image.rows();
double h = size.height;
double w = size.width;
int hh = (int)size.height;
int ww = (int)size.width;
Mat blob = org.opencv.dnn.Dnn.blobFromImage(image, 1.0,  new Size(w, h), Scalar.all(0), true, false);

// Load the network
Net net = org.opencv.dnn.Dnn.readNetFromTensorflow(MODEL_WEIGHTS, TEXT_GRAPH);
net.setPreferableBackend(org.opencv.dnn.Dnn.DNN_BACKEND_OPENCV);
net.setPreferableTarget(org.opencv.dnn.Dnn.DNN_TARGET_CPU);
net.setInput(blob);
ArrayList<String> outputlayers = new ArrayList<String>();
ArrayList<Mat> outputMats = new ArrayList<Mat>();
outputlayers.add("detection_out_final");
outputlayers.add("detection_masks");
net.forward(outputMats,outputlayers);
Mat numClasses = outputMats.get(0);
numClasses = numClasses.reshape(1, (int)numClasses.total() / 7);

for (int i = 0; i < numClasses.rows(); ++i) {
    double confidence = numClasses.get(i, 2)[0];
    //System.out.println(confidence);
    if (confidence > 0.2) {
        int classId = (int) numClasses.get(i, 1)[0];
        String label = CLASSES[classId] + ": " + confidence;
        System.out.println(label);
        int left   = (int)(numClasses.get(i, 3)[0] * cols);
        int top    = (int)(numClasses.get(i, 4)[0] * rows);
        int right  = (int)(numClasses.get(i, 5)[0] * cols);
        int bottom = (int)(numClasses.get(i, 6)[0] * rows);
        System.out.println(left + " " + top + " " + right + " " + bottom);
    }
}
Mat numMasks = outputMats.get(1);

到目前为止，一切都很好。我现在的问题是尝试从numMasks Mat中获取分割形状。大小为90x100，总计2025000，类型为-1 * -1 * CV_32FC1，isCont = true，isSubmat = true。我知道我也必须重塑，也许像这样：

numMasks = numMasks.reshape(1, (int)numMasks.total() / 90);

任何帮助将不胜感激。进一步编辑我认为面具是15 x 15。90 x 100 x 15 x 15给出了2025000，这样就给了我我之前看到的总数，因此我需要调整蒙版的大小并与原始图像混合。

Answer 1

非常感谢您的解决方案。重塑为15x15后我有一个问题，如何使遮罩覆盖在原始图像上？

我尝试过mask = cv2.resize（mask，（boxW，boxH），插值= cv2.INTER_NEAREST）

问候