Selenium脚本返回null给元素

问题描述 投票:1回答:1

我正在运行这个脚本,在找到一个id之后获取一些内容,这将由AJAX调用填充,它应该转到第二个URL。转到第二个URL后,它找不到任何具有相同ID的内容。

Code snippet is here:

from selenium import webdriver
from selenium.webdriver.remote.webelement import WebElement
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
import json
from selenium.webdriver.support import expected_conditions as EC

path_to_chromedriver = 'D:\Mangilal\Downloads\chromedriver_win32\chromedriver.exe'  #Change path 
as needed.
browser = webdriver.Chrome(executable_path=path_to_chromedriver)
#Sample list of URLs.

lists_of_url = ['http://facebook.com', 'http://twitter.com', 
                'http://google.com', 'http://youtube.com',
                'http://linkedin.com', 'http://wordpress.org', 
                'http://instagram.com', 'http://pinterest.com',
                'http://wikipedia.org', 'http://wordpress.com',
                'http://slideshare.net', 'http://e-recht24.de', 
                'http://washingtonpost.com', 'http://etsy.com',
                'http://eventbrite.com', 'http://archive.org', 
                'http://cpanel.net', 'http://miibeian.gov.cn',
                'http://sourceforge.net', 'http://telegraph.co.uk', 
                'http://ameblo.jp', 'http://amazon.co.uk',
                'http://ebay.com', 'http://fc2.com',
                'http://free.fr', 'http://bing.com']


for i in range(10):
    url = 'https://www.shareaholic.com/sharecounter?url=' + lists_of_url[i]
    browser.get(url)
    element = WebDriverWait(browser, 10).until(EC.presence_of_element_located((By.XPATH, '//*[@id="container"]/div[1]/div[2]/div[3]/div/div/h1/span')))
    str = ''
    #Finding element here.
    count = browser.find_element_by_xpath('//*[@id="container"]/div[1]/div[2]/div[3]/div/div/h1/span')
    str = count.get_attribute('innerHTML')
    print(str)
Output of this is shown in the image, which fetches content for the first URL after that null value:

Image showing the output

python selenium web-scraping
1个回答
1
投票

首先,str是Python的内置函数。不要将它用作变量。 其次,应该编码查询字符串。 第三,你应该等到元素可见。请参阅下面的代码。

from urllib.parse import urlencode
...
for i in range(10):
    url = 'https://www.shareaholic.com/sharecounter?' + urlencode({"url":lists_of_url[i]})
    driver.get(url)
    element = WebDriverWait(driver, 10).until(EC.visibility_of_element_located((By.XPATH, '//*[@id="container"]/div[1]/div[2]/div[3]/div/div/h1/span')))
    txt = ''
    #Finding element here.
    count = driver.find_element_by_xpath('//*[@id="container"]/div[1]/div[2]/div[3]/div/div/h1/span')
    txt = count.get_attribute('innerHTML')
    print(txt)
© www.soinside.com 2019 - 2024. All rights reserved.