循环不是抓取多个页面,只是重复从一个页面返回数据

问题描述 投票:0回答:0
import requests
from bs4 import BeautifulSoup
import pandas as pd

headers = {'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/110.0.0.0 Safari/537.36'}

questionlist = []

url = "https://seekingalpha.com/market-news?page=20"

r = requests.get(url, headers=headers)

soup = BeautifulSoup(r.text, 'html.parser')

questions = soup.find_all('article', {'class': 'mT-jA ga-jA Q-b8 R-cS R-df ks-IX R-cG R-dJ ks-IX R-cG R-dJ ks-I0 ks-I0 mT-NM'})

for page in range(1, 10):
    for item in questions:
        question = {
        'title': item.find('h3', {'class': 'km-X R-cw Q-cs km-IM V-gT V-g9 V-hj km-IO V-hY V-ib V-ip km-II R-fZ'}).text,
        'link': 'https://seekingalpha.com/market-news' + item.find('a', {'class': 'hq-ox R-fu'})['href'],
        'date': item.find('span', {'class': 'mU-uO mU-gE'}),
        }
        questionlist.append(question)
    
print(questionlist)

为什么我的循环不工作我正在报废多个页面,但输出多次出现在单页

python html web-scraping web lxml
© www.soinside.com 2019 - 2024. All rights reserved.