向链接发送 HTTP 请求时无法正确使用字典参数

Question

我正在尝试从此网页中抓取表格内容。我想显示发送带有参数的 GET HTTP 请求后收到的 JSON 响应。

当我从开发人员工具中手动输入参数作为字符串并将它们直接合并到脚本中时，我达到了预期的结果。然而，当我尝试使用参数作为字典时，就像传统方法一样，我遇到了状态 500 错误。

import requests

url = 'https://www.produktdatenbank-get.at/api/products'

# params = {
#     'eq(group)': '337',
#     'eq(instock)': '1',
#     'ne(custom.field_28)': ['R-404A', 'R-407C', 'R-417A', 'R-410A'],
#     'eq(custom.field_133)': '1',
#     'select': 'image,id,name,custom.field_22,custom.field_28,custom.field_133,subsidable,custom.field_78',
#     'sort': '+name'
# }

params = "eq(group,337)&eq(instock,1)&ne(custom.field_28,R-404A)&ne(custom.field_28,R-407C)&ne(custom.field_28,R-417A)&ne(custom.field_28,R-410A)&eq(custom.field_133,1)&select(image,id,name,custom.field_22,custom.field_28,custom.field_133,subsidable,custom.field_78)&sort(+name)"


headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:124.0) Gecko/20100101 Firefox/124.0',
    'Accept': 'application/json',
    'Accept-Encoding': 'identity',
    'Accept-Language': 'en-US,en;q=0.9',
    'Host': 'www.produktdatenbank-get.at',
    'Referer': 'https://www.produktdatenbank-get.at/',
    'X-Requested-With': 'XMLHttpRequest',
}

res = requests.get(url,params=params,headers=headers)
print(res.status_code)
print(res.json())

具体来说，如果我按原样执行脚本，我发现成功，但是当我取消注释字典参数并踢出用作字符串的参数时，我遇到了状态 500 错误。

如何在脚本中使用字典参数并获取 JSON 响应？

Answer 1

看起来您可能必须发布这些参数。

例如：

import requests
import json

url = "https://www.produktdatenbank-get.at/api/products"

params = "eq(group,337)&eq(instock,1)&ne(custom.field_28,R-404A)&ne(custom.field_28,R-407C)&ne(custom.field_28,R-417A)&ne(custom.field_28,R-410A)&eq(custom.field_133,1)&select(image,id,name,custom.field_22,custom.field_28,custom.field_133,subsidable,custom.field_78)&sort(+name)"
with requests.post(url, data=params) as response:
    response.raise_for_status()
    print(json.dumps(response.json(), indent=2))

输出（部分）：

{
    "id": 3148,
    "name": "KNV Topline 1145 - 05",
    "image": "/temp/20120801114144/logo.jpg",
    "fact_sheet": "/temp/20140627092445/KNV_Topline_1145-1245_SW_2014_V2.pdf",
    "group": 340,
    "supplier": 497,
    "manufacturer": 374,
    "instock": 1,
    "subsidable": 0,
    "custom": {
      "field_15": "Topline",
...
"field_125": 5,
      "field_126": "/temp/20160201130951/3_product_fiche_F1145_05_GW_300_200.pdf",
      "field_128": "A++",
      "field_130": 0,
      "field_131": 0,
      "field_132": 177,
      "field_133": 133,
      "field_134": 1,
      "field_135": 0,
      "field_136": 0,
      "field_137": 1,
      "field_138": ""
    }

Answer 2

你必须找到API文档，一些参数可以清楚地作为字典传递，例如equal可以用字典代替：

params = {
    "group": 337,
    "instock": 1,
}

对于非相等和选择，您必须在 API 描述中找到它，如何编写它，或者您可以为该查询字符串创建自己的自定义解析器：

import requests

url = 'https://www.produktdatenbank-get.at/api/products'

headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:124.0) Gecko/20100101 Firefox/124.0',
    'Accept': 'application/json',
    'Accept-Encoding': 'identity',
    'Accept-Language': 'en-US,en;q=0.9',
    'Host': 'www.produktdatenbank-get.at',
    'Referer': 'https://www.produktdatenbank-get.at/',
    'X-Requested-With': 'XMLHttpRequest',
}
params = {
    "eq": {
        "group": 337,
        "instock": 1,
        "custom.field_133": 1,
    },
    "ne": {
        "custom.field_28": ["R-407C", "R-417A", "R-410A"],
    },
    "select": ["image","id","name","custom.field_22","custom.field_28","custom.field_133","subsidable","custom.field_78"],
    "sort": "+name"
}

def makeQuuryString(aDict):
    q = ""
    for key, val in aDict.items():
        if isinstance(val, dict):
            if isinstance(val[next(iter(val))], list):
                for k, vals in val.items():
                    q += "&".join([f"{key}({k},{item})" for item in vals])
            else:
                q += "&".join([f"{key}({k},{v})" for k, v in val.items()])
        elif isinstance(val, list):
            q += f'{key}({",".join(val)})'
        else:
            q += f'{key}({val})'
        q += "&"
    return q[:-1]

res = requests.get(url,params=makeQuuryString(params),headers=headers)
print(res.status_code)
print(res.json())

注意：您选择的自定义字段无法按预期工作。