我正在网上抓取《芝加哥论坛报》报纸的网站，以获取新闻的日期，但我无法获取。
import time
import requests
from bs4 import BeautifulSoup
from bs4.element import Tag
url = 'https://www.chicagotribune.com/search/dispatcher.front?page={}&sortby=display_time%20descending&target=stories&spell=on&Query=cybersecurity#trb_search'
pages = 10
for page in range(1, pages+1):
res = requests.get(url.format(page))
soup = BeautifulSoup(res.text,"lxml")
for item in soup.find_all("a", {"class": "trb_search_result_title"}, href=True):
_href = item.get("href")
try:
resp = requests.get(_href)
except Exception as e:
try:
resp = requests.get("https://www.chicagotribune.com"+_href)
except Exception as e:
continue
sauce = BeautifulSoup(resp.text,"lxml")
timetag = sauce.find('time',{'class': "trb_search_result_datetime"})
date = None
if isinstance(timetag, Tag):
timetag = timetag.get_text().strip()
print(f'{date}\n')
time.sleep(3)
为什么我没有得到任何结果。
预先感谢。
htmldate模块可在Chicago Tribune网站上使用，我刚刚检查过。
1。安装软件包：
pip/pip3/pipenv (your choice) -U htmldate
2。检索网页，解析并输出日期：
from htmldate import find_date
find_date('https://www.chicagotribune.com/news/criminal-justice/ct-chicago-pastor-meals-for-needy-kids-bentley-20200110-gk4rgc4qdbhqhajk57k67i2xli-story.html')
如果您已经下载了网页，也可以将HTML文档直接传递到提取器：
htmldoc = '<html><body><span class="entry-date">July 12th, 2016</span></body></html>'
find_date(htmldoc)