如何在不同目标尺寸的大图像上定位图像

Question

我尝试使用

pyautogui.locateOnScreen(btn, confidence=0,8, grayscale=True)

来定位图像，但是如果搜索图像的尺寸不同，则无法定位。

就我而言，我想在屏幕上找到确认按钮。我在A站（小号）上找到按钮的快捷方式，然后在B站找到它，但没有用。

我想检测这个按钮在不同网站上的位置。除了大小之外，所有其他部分都相同，包括颜色、形状和按钮文本。

我尝试过

cv2.matchTemplate(scr_img, search_image, cv2.TM_CCOEFF_NORMED)

但也没成功。

还有别的办法吗？

---

与 pyautogui.locateOnScreen

from PIL import Image

confirm_btn_img = Image.open("./img/confirm_btn.png")
try: 
    l, t, w, h = pyautogui.locateOnScreen(confirm_btn_img, confidence=0.8, grayscale=True)
except Exception as e:
    print("Not Found")

与 cv2.matchTemplate：

import cv2
import numpy as np

confirm_btn_img2 = cv2.imread("./img/confirm_btn.png", 0)
yautogui.screenshot()
big_image = cv2.cvtColor(np.array(big_image), cv2.COLOR_RGB2GRAY)

res = cv2.matchTemplate(big_image, confirm_btn_img2, cv2.TM_CCOEFF_NORMED)
threshold = 0.8

loc = np.where(res >= threshold)

print("loc====", loc)

for pt in zip(*loc[::-1]):
    cv2.rectangle(big_image, pt, (pt[0] + confirm_btn_img2.shape[1], pt[1] + confirm_btn_img2.shape[0]), (0, 0, 255), 2)

cv2.imshow('Detected', big_image)
cv2.waitKey(0)
cv2.destroyAllWindows()

可以找到：

找不到：

目标图片：

示例代码：

import cv2
import numpy as np

confirm_btn_img2 = cv2.imread("./test/ok.png", 0)
big_image = cv2.imread("./test/can_found_tpl.png", 0)

res = cv2.matchTemplate(big_image, confirm_btn_img2, cv2.TM_CCOEFF_NORMED)
threshold = 0.8

loc = np.where(res >= threshold)

print("loc====", loc)

for pt in zip(*loc[::-1]):
    cv2.rectangle(big_image, pt, (pt[0] + confirm_btn_img2.shape[1], pt[1] + confirm_btn_img2.shape[0]), (0, 0, 255), 2)

cv2.imshow('Detected', big_image)
cv2.waitKey(0)
cv2.destroyAllWindows()

Answer 1

我认为，如果尺度变化不是非常随机，那么模板匹配与缩放会有所帮助。 您还可以尝试特征匹配。

对于功能匹配，你可以尝试这个

import cv2
import numpy as np

def feature_matching(big_image_path, template_image_path):
    big_image = cv2.imread(big_image_path, 0)
    template_image = cv2.imread(template_image_path, 0)

    orb = cv2.ORB_create()

    keypoints1, descriptors1 = orb.detectAndCompute(template_image, None)
    keypoints2, descriptors2 = orb.detectAndCompute(big_image, None)

    bf = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck=True)
    matches = bf.match(descriptors1, descriptors2)
    matches = sorted(matches, key=lambda x: x.distance)

    if matches:
        img_matches = cv2.drawMatches(template_image, keypoints1, big_image, keypoints2, matches[:10], None, flags=cv2.DrawMatchesFlags_NOT_DRAW_SINGLE_POINTS)
        cv2.imshow("Matches", img_matches)
        cv2.waitKey(0)
        cv2.destroyAllWindows()

        # Get the coordinates of the matched keypoints
        point = keypoints2[matches[0].trainIdx].pt
        print(f"Found at: {point}")
    else:
        print("Not Found")