有关python requests.get 400错误请求()

问题描述 投票:-1回答:1

我做了一些网络与政治捐款刮和有我从一个页面刮比我这时就需要刮的链接。我可以得到二级链接就好了,但是当我试图发送一个requests.get()调用,HTML从调用返回给了我一个错误请求400错误。

我已经试图通过改变或添加更多的信息,但是似乎没有任何工作改变周围的请求。

headers = {
        "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8",
        "Accept - Encoding": "gzip, deflate",
        "Accept-Language": "en-US,en;q=0.9",
        "Cache - Control": "max - age = 0",
        "Connection": "keep-alive",
        "DNT": "1",
        "Host": "docquery.fec.gov",
        "Referer": "http://www.politicalmoneyline.com/tr/tr_MG_IndivDonor.aspx?tm=3",
        "Upgrade-Insecure-Requests": "1",
        "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36"
    }

params = {
        "14960391627": ""
    }

pdf_page = requests.get(potential_donor[10], headers=headers, params=params)
html = pdf_page.text
soup_donor_page = BeautifulSoup(html, 'html.parser')
print(soup_donor_page)   

注意:该网站的网址应该是这个样子:http://docquery.fec.gov/cgi-bin/fecimg/?14960391627与结尾的数字是不同的

打印(soup_donor_page)的输出是:

400 Bad request

Your browser sent an invalid request.

我需要获得页面的实际HTML为了抢在页面嵌入PDF文件。

python-requests
1个回答
0
投票

我怀疑的原因是issue当它提供的参数没有值出现的requests

尝试建立网址的格式字符串,而不是:

import requests
from bs4 import BeautifulSoup

headers = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36"
}

param = "14960391627"
r = requests.get(f"http://docquery.fec.gov/cgi-bin/fecimg/?{param}", headers=headers)
soup = BeautifulSoup(r.content, "html.parser")

print(soup.find("embed")["src"])

结果:

http://docquery.fec.gov/pdf/859/14960388859/14960388859_002769.pdf#zoom=fit&navpanes=0
© www.soinside.com 2019 - 2024. All rights reserved.