Selenium Driver Web Scraping 在打开多个窗口时无法获取下拉元素

Question

我正在使用以下代码抓取 URL https://ephtracking.cdc.gov/DataExplorer/

options = webdriver.ChromeOptions() 
options.headless = False

options.add_argument("window-size=1920,1080")
options.page_load_strategy = 'none'
options.add_argument("--enable-javascript")
options.add_argument('--ignore-certificate-errors') 

driver = Chrome(options=options, service=chrome_service) 

driver.get(url)
wait= WebDriverWait(driver,20)

step1Content_click = wait.until(ExpectedConditions.presence_of_element_located((
      By.XPATH,'//select[@id="contentArea"]//option[text()="Tornadoes"]')))
driver.execute_script("arguments[0].click();", step1Content_click)

但是，无法从

Tornadoes

 -> 

Step1 Content

 的下拉列表中选择选项 

Select Content Area

。

Answer 1

检查下面的工作代码：

import time

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as ExpectedConditions
from selenium.webdriver.support.ui import Select

options = webdriver.ChromeOptions()
options.add_argument("window-size=1920,1080")
options.page_load_strategy = 'none'
options.add_argument("--enable-javascript")
options.add_argument('--ignore-certificate-errors')

driver = webdriver.Chrome(options=options)
driver.maximize_window()

driver.get("https://ephtracking.cdc.gov/DataExplorer/")
wait= WebDriverWait(driver,60)
select = Select(wait.until(ExpectedConditions.visibility_of_element_located((By.ID, "contentArea"))))
select.select_by_visible_text("Tornadoes")
time.sleep(30)

结果：

Answer 2

要从

html-select 标签中选择带有文本 Tornadoes 的 select-options，您需要为 element_to_be_clickable() 引入 WebDriverWait，并且可以使用以下任一定位器策略：

• 使用

  CSS_SELECTOR：

  driver.get("https://ephtracking.cdc.gov/DataExplorer/")
  WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "select#contentArea[data-qp-data-name='contentAreas']"))).click()
  Select(WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "select#contentArea[data-qp-data-name='contentAreas']")))).select_by_visible_text("Tornadoes")

• 使用

  XPATH：

  driver.get("https://ephtracking.cdc.gov/DataExplorer/")
  WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//select[@id='contentArea' and @data-qp-data-name='contentAreas']"))).click()
  Select(WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//select[@id='contentArea' and @data-qp-data-name='contentAreas']")))).select_by_value('39')

• 注意：您必须添加以下导入：

  from selenium.webdriver.support.ui import WebDriverWait
  from selenium.webdriver.common.by import By
  from selenium.webdriver.support import expected_conditions as EC
  from selenium.webdriver.support.ui import Select

• 浏览器快照：

---

参考文献

您可以在以下位置找到一些相关的详细讨论：

• 如何使用 Selenium 和 Python 从下拉菜单中选择选项