鼠标悬停在错误的FirefoxWebElement上,而不是在指定的元素上悬停

问题描述 投票:0回答:1

我正在尝试从Internet档案馆中获取在一定日期范围内每一天的《华盛顿邮报》首页最后快照的时间。问题是,尽管我收集了日期对象列表,但Selenium并不总是选择正确的日期,而且似乎没有错误。例如,硒将从1月31日跳到2月11日,而不是2月1日。

打印输出:

Moving to 31
Date: 2020-01-31 00:00:00
Last snapshot taken at 22:59:03
Moving to 1
Date: 2020-02-11 00:00:00
Last snapshot taken at 23:53:32
Moving to 2
Date: 2020-02-02 00:00:00
Last snapshot taken at 23:59:56

在此输出中,我们可以看到它应该为1,并且日期对象的文本为1,这意味着该日期对象是从页面中正确提取的。硒没有正确地悬停在它上面吗?

完整代码:

urls = ['https://web.archive.org/web/20190901000000*/washingtonpost.com', 
        'https://web.archive.org/web/*/washingtonpost.com']

browser = webdriver.Firefox(executable_path='/usr/local/bin/geckodriver') # brew install chromedriver then see in terminal where it was installed to and paste this
data = {}

for j in range(0, len(urls)):
    browser.get(urls[j])
    calendar_grid = WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.CLASS_NAME, 'calendar-grid')))

    if j == 0: # 2019
        browser.execute_script("window.scrollTo(0, document.body.scrollHeight);")
        time.sleep(10)
        start = len(dates) - 1
        end = 0
        step = -1
    elif j == 1: # 2020
        start = 0
        end = len(dates) 
        step = 1

    dates = calendar_grid.find_elements_by_css_selector('.calendar-day')
    print('Dates on page: ' + str(len(dates)))

    for i in range(start, end, step):
        if j==0 and len(data) == len(desired_ranges[j]): # to end 2019
            break

        # Hover over the date, let popup appear, wait for loader to disappear, select scroll area
        print('Moving to ' + dates[i])
        hov = ActionChains(browser).move_to_element(dates[i])
        hov.perform()
        popup = WebDriverWait(browser, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, '.popup-of-day-content')))
        WebDriverWait(browser, 20).until(EC.invisibility_of_element_located((By.TAG_NAME, 'svg')))
        scroll_area = WebDriverWait(browser, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, '.popup-of-day-content > ul > div')))

        # Get date and check that it is in our range
        date = popup.find_element_by_class_name('day-tooltip-title')
        date_formatted = datetime.datetime.strptime(date.text, '%B %d, %Y')
        print('Date: ' + str(date_formatted))
        if date_formatted not in desired_ranges[j]:
            continue # skip if it is not
        else:
            attempts = 0
            while attempts < 5:
                try:
                    browser.execute_script('arguments[0].scrollTop = arguments[0].scrollHeight', scroll_area)
                    snapshots = popup.find_elements_by_tag_name('a')        
                    last_snapshot = snapshots[len(snapshots) - 1]

                    print('Last snapshot taken at ' + last_snapshot.text)
                    data[date_formatted] = {'link': last_snapshot.get_attribute('href'),
                                                 'time': last_snapshot.text,
                                                 'headlines': []}
                    break
                except StaleElementReferenceException:
                    attempts += 1

用于设置日期范围的附加代码:

start_day = datetime.date(2019, 12, 8)
end_day = datetime.date.today()
days = (end_day - start_day).days 
desired_range = pd.date_range(start_day, periods=days).tolist()
print('Range: ' + str(start_day) + ' to ' + str(end_day))
print('Days: ' + str(days))
desired_range

def time_in_range(start, end, x):
    '''Return true if x is in the range [start, end]'''
    if start <= end:
        return start <= x <= end
    else:
        return start <= x or x <= end

#Get 2019 date range
desired_range_in_2019 = [x for x in desired_range if time_in_range(datetime.date(2019, 1, 1), datetime.date(2019, 12, 31), x)]
desired_range_in_2020 = [x for x in desired_range if time_in_range(datetime.date(2020, 1, 1), datetime.date(2020, 12, 31), x)]
desired_ranges = [desired_range_in_2019, desired_range_in_2020]
print('Dates in 2019: ' + str(len(desired_range_in_2019)))
print('Dates in 2020: ' + str(len(desired_range_in_2020)))

我正在尝试从Internet档案馆中获取在一定日期范围内每一天的《华盛顿邮报》首页最后快照的时间。问题是硒并不总是选择...

python selenium selenium-webdriver web-scraping
1个回答
0
投票

[最有可能发生这种情况是因为ActionChains模拟了用户的鼠标移动,因此,在到达预期元素的途中,光标(在到达适当元素的途中)会偶然发现其他元素。通过网络节流可以可靠地重现该问题。在Developer Tools中将network更改为Regular 3G

热门问题
推荐问题
最新问题