如何使用 pyautogui 和 selenium 自动查找元素的位置

Question

我正在编写一个脚本，我需要自动查找元素的光标位置。该脚本将在不同的计算机上运行，所以我希望它自动完成。

假设我想点击 stackoverflow.com 头部的搜索框，使用

selenium

和

pyautogui

找到它的位置。

如何使鼠标位于该位置并退出脚本？

编辑1

假设我希望它点击这个元素：

input

其中

name="q"

和

class="s-input s-input__search js-search-field "

编辑2：

我当前的代码在

Prophet

和

undetected Selenium

的帮助下，但我得到的输出不正确：

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.action_chains import ActionChains
import pyautogui
from time import sleep
from datetime import datetime
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as ec
from selenium.webdriver.support.wait import WebDriverWait

path = r'path'
user_data = r'user'
options = webdriver.ChromeOptions()
options.add_argument(f'--user-data-dir={user_data}')
options.add_argument('--profile-directory=Default')
driver = webdriver.Chrome(executable_path=path, options=options)
actions = ActionChains(driver)
xpath = '//*[@id="search"]/div/input'

url = 'https://stackoverflow.com/'
driver.get(url)
position = WebDriverWait(driver, 20).until(ec.visibility_of_element_located((By.XPATH, xpath))).location_once_scrolled_into_view
# both tried the above position var and the below
# position = WebDriverWait(driver, 20).until(ec.visibility_of_element_located((By.XPATH, xpath))).location
# element = driver.find_element(By.XPATH, xpath)
# actions.move_to_element(element).perform()
# position = pyautogui.position()
file = open('position.txt', 'a')
file.write(f'{position}, {datetime.now()}\n')
driver.close()

# to test the exact or relative position to compare with above code
# while True:
#     sleep(1)
#     print(pyautogui.position())

编辑3：

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.action_chains import ActionChains
import pyautogui
from time import sleep
from datetime import datetime
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.wait import WebDriverWait

path = r'D:\\Downloads\\chromedriver_win32\\chromedriver.exe'
user_data = r'C:\\Users\\Saeed\\AppData\\Local\\Google\\Chrome\\User Data\\Default'
options = webdriver.ChromeOptions()
options.add_argument(f'--user-data-dir={user_data}')
options.add_argument('--profile-directory=Default')
driver = webdriver.Chrome(executable_path=path, options=options)
actions = ActionChains(driver)
xpath = '//*[@id="search"]/div/input'  # `search` box at header
url = 'https://stackoverflow.com//'
driver.get(url)
driver.maximize_window()
sleep(10)
position = WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, xpath))).location
print(position)
file = open('position.txt', 'a')
file.write(f'{position}, {datetime.now()}\n')
# pyautogui.click(793, 9)
driver.close()

# while True:
#     sleep(1)
#     print(pyautogui.position())

我的代码现在如何工作的视频：https://up.mazandhost.com/uploads/1028407950.bandicam-2022-03-27-21-33-27-729.mp4

我的案例的解决方案：

回答https://stackoverflow.com/a/56693949/5790653帮助我。

Answer 1

通过

Selenium

，您可以使用

action_chains

库执行鼠标操作。
一旦您找到带有 selenium 的元素，您就可以将鼠标移动到该元素，如下所示：

from selenium.webdriver.common.action_chains import ActionChains

actions = ActionChains(driver)

element = driver.find_element(By.XPATH, 'the_xpath_locator')
actions.move_to_element(element).perform()

Answer 2

要提取 location 和/或 position 元素，您需要为 visibility_of_element_ located() 引入 WebDriverWait，并且您可以使用以下任一定位器策略：

• 使用 location 属性：

  driver.get('https://www.google.com/')
  print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.NAME, "q"))).location)
  

• 控制台输出：

  {'x': 439, 'y': 184}
  

• 使用 location_once_scrolled_into_view 属性：

  driver.get('https://www.google.com/')
  print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.NAME, "q"))).location_once_scrolled_into_view)
  

• 控制台输出：

  {'x': 439, 'y': 184}
  

• 注意：您必须添加以下导入：

  from selenium.webdriver.support.ui import WebDriverWait
  from selenium.webdriver.common.by import By
  from selenium.webdriver.support import expected_conditions as EC
  

---

这个用例

根据 HTML 详细信息：

输入 name="q" 和 class="s-input s-input__search js-search-field "

要单击 clickable 元素，您需要为 element_to_be_clickable() 引入 WebDriverWait，并且您可以使用以下任一定位器策略：

• 使用CSS_SELECTOR：

  WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input.s-input.s-input__search.js-search-field[name='q']"))).click()
  

• 使用XPATH：

  WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//input[@class='s-input s-input__search js-search-field ' and @name='q']"))).click()
  

---

替代方案

作为替代方案，您还可以使用 ActionChains API 中的 move_to_element() 方法，如下所示：

ActionChains(driver).move_to_element(WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//input[@class='s-input s-input__search js-search-field ' and @name='q']")))).perform()

Answer 3

我是这样做的。

import pyautogui
from random import *

elementToClick = driver.find_element(by=By.CSS_SELECTOR,value='#nav-search-submit-button')

panel_height = driver.execute_script('return window.outerHeight - window.innerHeight;')
abs_x = elementToClick.location['x']
y = elementToClick.location['y']
abs_y = y + panel_height

pyautogui.moveTo(abs_x, abs_y, uniform(0.6, 1.7), pyautogui.easeOutQuad)
pyautogui.click(abs_x, abs_y)