为什么从标签.get_attribute获取的值不能以键值对的形式存储在字典中?

问题描述 投票:0回答:1

我目前无法将对 Selenium 中的标签执行 .get_attribute 获得的值存储到字典中,特别是标题和 href,如下面的代码所示。我正在尝试存储这些值,一个作为键(标题),另一个作为值(href)。

我尝试直接存储它们,如下所示,但它不起作用。

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

# Set up Chrome Options
chrome_options = Options()
chrome_options.add_argument("--headless")  # Enable headless mode to disable displaying of browser window when running the code

# Setting up Selenium Web Driver
service = Service(ChromeDriverManager().install())
driver = webdriver.Chrome(service=service, options=chrome_options)  # Create instance of Chrome WebDriver

driver.get("https://www.volunteer.gov.sg/volunteer/agencies/agency_details/?code=YCS")

# Find the div with id "upcomingOppoList"
upOppLst = driver.find_element(By.ID, "upcomingOppoList")

# Find all elements with class "col-sm-6 col-md-3" within the div
elements = upOppLst.find_elements(By.CLASS_NAME, "col-sm-6.col-md-3")  # find_elements gives list

volOpp = {}

for element in elements:
    # Find the <a> tag within the element
    a_tag = element.find_element(By.TAG_NAME, "a")
    href = a_tag.get_attribute("href")
    
    # Find the <span> tag within the element
    span_tag = element.find_element(By.TAG_NAME, "span")
    title = span_tag.get_attribute("title")
 
    
    # Store href and title in the dictionary directly
    volOpp[title] = href

# Print the dictionary
print(volOpp)

driver.quit()

如果我运行上面的代码,我的字典的输出将为空。

但是,如果我首先打印出标题和 href,然后将值存储在我的字典中,我将按预期存储所有键值对。

print("Title:", title)
print("Href:", href)
 
volOpp[title]=href

为什么会这样?我怎样才能让我的代码工作?

非常感谢您的帮助!

python selenium-webdriver web-scraping
1个回答
0
投票

我认为页面未完全加载,您正在尝试访问 driver.get 之后的 HTML 标记。下面的代码可能会有所帮助。

upOppLst = WebDriverWait(driver, 10).until(EC.visibility_of_element_ located((By.ID, "upcomingOppoList")))

或者只是在 driver.get 之后添加 time.sleep

时间.sleep(随机.randint(10,15))

© www.soinside.com 2019 - 2024. All rights reserved.