在从网站上获取数据时，每个列表都有一个按钮，当我们单击该按钮时，弹出窗口会打开但不会在Python中关闭

Question

弹出窗口打开但没有关闭，请帮我关闭弹出窗口。唯一的问题是这里的弹出窗口未关闭。

color="accent" class="mat-focus-indicator mat-button mat-button-base mat-accent">... 在点 (684, 847) 处不可点击。其他元素将收到点击：
（会话信息：chrome=124.0.6367.61）解释错误

# Import necessary modules
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from webdriver_manager.chrome import ChromeDriverManager
import csv
import time

# Set up Chrome service
service = webdriver.chrome.service.Service(ChromeDriverManager().install())

# Initialize Chrome WebDriver with the service
driver = webdriver.Chrome(service=service)

# Navigate to the webpage
driver.get("https://mmiconnect.in/app/exhibition/catalogue/ep2023")

# Close the initial popup if present
try:
    # Find and click the close button if it's clickable
    close_button = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.CLASS_NAME, "close-button")))
    close_button.click()
except:
    # If close button not found, pass
    pass

# List to store the scraped data
siteData = []

# Iterate over pages
for nav in range(1, 33):
    for i in range(1, 750):
        try:
            # Find and click on the buttons
            listing = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.CLASS_NAME, 'center')))
            lists = listing.find_elements(By.XPATH, "//div[@class='courses']//button[contains(@class, 'mat-focus-indicator')]")

            # Iterate over the buttons
            for listss in lists:
                try:
                    # Click on the button
                    listss.click()
                    # Wait for the popup to appear
                    time.sleep(2)
                    
                    # Check if the popup is still open
                    if EC.visibility_of_element_located((By.ID, "mat-dialog-3"))(driver):
                        # Wait for the popup to appear
                        popup_element = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.ID, "mat-dialog-3")))
                        
                        # Scrape data from the popup
                        table = popup_element.find_element(By.ID, "customers")
                        rows = table.find_elements(By.TAG_NAME, "tr")
                        table_data = []
                        for row in rows:
                            cells = row.find_elements(By.TAG_NAME, "td")
                            row_data = [cell.text for cell in cells]
                            table_data.append(row_data)

                        # Write table data to CSV
                        with open("mmei.csv", mode='a', newline='') as file:
                            writer = csv.writer(file)
                            writer.writerows(table_data)

                        # Execute JavaScript to click the "Close" button within the popup
                        close_button = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "//button[contains(@aria-label, 'Close dialog')]")))
                        driver.execute_script("arguments[0].click();", close_button)
                    
                except Exception as e:
                    # Print error if any
                    print("Error processing popup:", e)
                    pass
                
        except Exception as e:
            # Print error if any
            print("Error iterating over pages:", e)
            continue

# Quit the driver
driver.quit()

Answer 1

当我运行你的代码时，什么也没有发生。它从未点击第一个“查看详细信息”链接等，也从未输出任何内容，所以我重写了它。

一些建议...

1. 从 Selenium 4.6 开始，您不再需要安装和使用 DriverManager，因为 Selenium 内置了一个 DriverManager。我删除了与 DriverManager 相关的所有不必要的代码。

2. 您在脚本中多次声明了

   WebDriverWait

   。您可以只声明一次并重复使用它，例如

   wait = WebDriverWait(driver, 10)
   wait.until(EC.element_to_be_clickable((By.CLASS_NAME, "close-button"))).click()
   

3. 如果您不打算多次使用某个变量，那么声明它就没有多大意义。删除这些实例将清理您的代码。

    close_button = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.CLASS_NAME, "close-button")))
    close_button.click()
   

   可以

   wait.until(EC.element_to_be_clickable((By.CLASS_NAME, "close-button"))).click()
   

4. 不要对数字进行硬编码以在

   range

   中循环，而是从页面中抓取元素并对其进行计数。

5. 将所有内容都放入

   try-except

   中并不是一个好的做法...在通用异常级别捕获也不是一个好的做法。你最终所做的就是吃掉“所有”异常，隐藏真正出了问题的地方。省略 try-except 并学习阅读异常消息。一旦你的脚本高度完善，然后把

   try-except

   放回去，只捕获你要处理的特定异常，例如 except TimeoutException:

使用
6. time.sleep()

   是一种不好的做法。相反，请像在其他地方一样使用

   WebDriverWait

   。

不要使用
7. 来检查元素是否可见

   if EC.visibility_of_element_located(...):

   而是等待它可见。这样您就可以放心它会在那里。

在一种情况下，您正在等待弹出窗口出现，但正在使用
8. EC.presence_of_element_located()

   。存在意味着元素存在于 DOM 中，而不是可见。而是使用

   EC.visibility_of_element_located()

   。

9. wait.until()

   返回等待的元素...所以使用它。

   listing = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.CLASS_NAME, 'center')))
   lists = listing.find_elements(By.XPATH, "//div[@class='courses']//button[contains(@class, 'mat-focus-indicator')]")

   成为

   lists = wait.until(EC.visibility_of_all_elements_located((By.XPATH, "//div[@class='courses']//button[contains(@class, 'mat-focus-indicator')]")))

您正在循环内打开并写入 CSV。将该代码移到循环之后，当所有抓取完成后，CSV 将被写入一次，从而节省大量时间和硬盘驱动器抖动。
使用此建议，这是工作代码

# Import necessary modules
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import csv

# Initialize Chrome WebDriver with the service
driver = webdriver.Chrome()
driver.maximize_window()

# Navigate to the webpage
driver.get("https://mmiconnect.in/app/exhibition/catalogue/ep2023")

wait = WebDriverWait(driver, 10)
# Close the initial popup if present
try:
    # Find and click the close button if it's clickable
    wait.until(EC.element_to_be_clickable((By.CLASS_NAME, "close-button"))).click()
except:
    # If close button not found, pass
    pass

# List to store the scraped data
site_data = []

# Find all View details links
view_details_links = wait.until(EC.visibility_of_all_elements_located((By.XPATH, "//button/span[text()='View details']")))

# Iterate over the links
for view_details_link in view_details_links:
    # Click on the link
    view_details_link.click()
    
    # Wait for the TABLE in the popup to appear
    table = wait.until(EC.visibility_of_element_located((By.ID, "customers")))

    # Wait for TABLE content to load
    while True:
        text = driver.find_element(By.CSS_SELECTOR, "#customers td.field-content").text
        if text is not "":
            break
    
    # Scrape data from the popup
    rows = table.find_elements(By.TAG_NAME, "tr")
    table_data = []
    for row in rows:
        cells = row.find_elements(By.TAG_NAME, "td")
        row_data = [cell.text for cell in cells]
        print(row_data)
        table_data.append(row_data)
    
    site_data.append(table_data)

    # Close the dialog
    close = driver.find_element(By.XPATH, "//span[text()='Close']")
    close.click()
    wait.until(EC.staleness_of(close))

# Write table data to CSV
with open("mmei.csv", mode='w', newline='') as file:
    writer = csv.writer(file)
    writer.writerows(site_data)

# Quit the driver
driver.quit()