Python+Selenium。页面无法在无头模式下加载

Question

我正在尝试使用该代码加载页面

    from selenium import webdriver
    from selenium.webdriver.chrome.options import Options
    from selenium.webdriver.support.ui import WebDriverWait
    from selenium.webdriver.support import expected_conditions as EC
    from selenium.webdriver.common.by import By
    from selenium.common.exceptions import TimeoutException
    chrome_options = Options()
    chrome_options.add_argument("--headless=new")
    driver = webdriver.Chrome(options=chrome_options)
    driver.get('https://www.chabad.org/calendar/zmanim_cdo/aid/143790/locationid/790/locationtype/1/save/1/tdate/4-15-2024/jewish/Zmanim-Halachic-Times.htm')
    try:
        myElem = WebDriverWait(driver, 20).until(EC.presence_of_element_located((By.ID, 'group_4/15/2024')))
        print("Page is ready!")
    except TimeoutException:
        print("Loading took too much time!")

我遇到了 TimeoutException。但如果我以标准模式（不是无头模式）运行浏览器，它就可以正常工作。 因为我需要从服务器上的页面获取信息，所以我想以无头模式运行浏览器。那么我如何从该页面加载信息呢？

Answer 1

该网站阻止常规 Selenium 无头模式，但您可以在 UC 模式 + 无头模式下使用 https://github.com/seleniumbase/SeleniumBase 来绕过限制。

pip install seleniumbase

，然后使用

python

运行：

import re
from seleniumbase import SB

with SB(uc=True, headless=True) as sb:
    sb.open('https://www.chabad.org/calendar/zmanim_cdo/aid/143790/locationid/790/locationtype/1/save/1/tdate/4-15-2024/jewish/Zmanim-Halachic-Times.htm')
    sb.click("button#button1")
    sb.click("span#month")
    sb.click('div[value="12"]')
    sb.click('[date*="12/31"]')
    text = sb.get_text_content("div.group.active_date")
    text = re.sub(r'\n+', '\n', text).strip()
    print(text)

预期输出：

Dawn (Alot Hashachar)
3:34 AM
Earliest Tallit and Tefillin (Misheyakir)
4:37 AM
Sunrise (Hanetz Hachamah)
5:50 AM
Latest Shema
9:19 AM
Latest Shacharit
10:31 AM
Midday (Chatzot Hayom)
12:54 PM
Earliest Mincha (Mincha Gedolah)
1:31 PM
Mincha Ketanah (“Small Mincha”)
5:06 PM
Plag Hamincha (“Half of Mincha”)
6:36 PM
Sunset (Shkiah)
8:00 PM
Nightfall (Tzeit Hakochavim)
8:40 PM
Midnight (Chatzot HaLailah)
12:53 AM
Tuesday
Shaah Zmanit (proportional hour)
71:42 min.

该脚本可让您设置所需的日期。