在图像 OpenCV 上搜索图标

Question

我是计算机视觉领域的新手，在图像中找到鱼图标时遇到了挑战。具体来说，我需要从游戏中找到鱼图标并获取其坐标。图标的大小根据其在水中的位置以及鱼被拉动时的颜色而变化。此外，游戏中的效果通常会阻碍搜索图标的过程。 主要问题是图标经常改变其比例。结果，我尝试使用的搜索方法已经丢失或很少产生正常结果。

首先，我尝试使用模板匹配，但如果图标大小不同，看起来不太好。我也尝试过制作多个大小的数组，但它工作起来非常不稳定，并且大大减慢了算法速度。

std::vector<Mat> fishIcon;
const std::vector<double> SCALES = { 0.4, 0.5, 0.6, 0.8, 1.0, 1.2, 1.4 };

//Going through all the scales until you find the right one
for (Mat ic : fishIcon) {
    if (FindObject(ic, screenshotGray, max_loc, result, true, 0.8)) {
        fishIconPosition = Point(max_loc.x, max_loc.y);
        found = true;
        break;
    }
}

//Code for creating multiple masterstabs
for (double scale : SCALES) {
    Mat resIcon;
    resize(sourceIcon, resIcon, Size(0, 0), scale, scale);

    fishIcon.push_back(resIcon);
}

//A method for searching for an icon in an image
bool botActions::FindObject(Mat& templateForFind, Mat& screenshot, Point& maxLoc, Mat& result, bool debug, double confid) {
    matchTemplate(screenshot, templateForFind, result, TM_CCOEFF_NORMED);
    double minVal, maxVal;
    minMaxLoc(result, &minVal, &maxVal, NULL, &maxLoc);

    if (debug) {
        std::cout << maxVal << std::endl;
    }

    if (maxVal >= confid) {
        return true;
    }
    else {
        return false;
    }
}

我尝试了使用 ORB 和直方图的基本搜索方法，这些方法在 OpenCV 网站上显示，但也没有显示出好的结果。

所以我想看看有没有更简单更好的方法？我将不胜感激任何提示

Answer 1

我尝试使用一些图像处理来获取图标。我使用了您提供的 3 个图像以及 python 中的以下解决方案。假设图标边框与示例图像中一样是白色的。注释中提供了每个步骤的说明。希望这有帮助。

import cv2
import numpy as np

fish_images = ["fish1.png","fish2.png","fish3.png"] #Image name list to process

#For loop to process
for img in fish_images:
    #Read image
    image_org = cv2.imread(img)
    #Convert to grayscale
    grayscale_image = cv2.cvtColor(image_org, cv2.COLOR_BGR2GRAY)
    #Use threshold to get the white part in images , this will select the icon part also0 as it's white borders.
    ret, thresholded_image = cv2.threshold(grayscale_image, 200, 255, cv2.THRESH_BINARY)

    #Apply some morphological operatipn sto remove noise
    kernel = np.ones((5, 5), np.uint8)
    thresholded_image = cv2.morphologyEx(thresholded_image, cv2.MORPH_OPEN, kernel)
    thresholded_image = cv2.morphologyEx(thresholded_image, cv2.MORPH_CLOSE, kernel)

    #Dinf contours in the image
    contours, hierarchy = cv2.findContours(thresholded_image, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
    contour_image = np.zeros_like(thresholded_image)

    #This loops go throught the contours and find the ones with area >700 and <1500
    # This is found through trial. You can adjust this accoring to your use
    #The I find the are of contour and bounding rectangle of the contour.
    # For rectangle, the contours the area and rectangular area difference will be the lowest
    selected_ctr = None
    selected_bounding_rect = []
    lowest_area_diff = 100000
    for ctr in contours:
        ctr_area = cv2.contourArea(ctr)
        if ctr_area > 700 and ctr_area < 1500:  # ADJUST THIS ACCORDING TO YOUR NEED.
            bounding_rect = cv2.boundingRect(ctr)
            bounding_rect_area = bounding_rect[2]*bounding_rect[3]
            area_difference = abs(ctr_area - bounding_rect_area)
            if area_difference < lowest_area_diff: #Compare area difference select one with lowest difference to avoid unnecessary areas
                selected_ctr = ctr
                lowest_area_diff = area_difference

    #The selected contour is used to create the mask
    #Using the mask the icon is obtained from image
    if selected_ctr is not None:
        cv2.drawContours(contour_image, [selected_ctr], -1, 255, thickness=-1)
        selected_image = cv2.bitwise_and(image_org, image_org, mask=contour_image)
        #Save result image
        cv2.imwrite(img+"_result.png",selected_image)

输出示例：