在预览文件中抓取网页数据

Question

当我检查网页时（例如https://fonds.axa-im.fr/fr/professionnels/fund/axa-pea-rgularit-capitalization-eur/#performanceRisk），我可以访问文件使用预览工具“执行”格式化的 JSON。

但是，当我单击它来访问它时（https://api-eu.kurtosys.app/dataset/execute），我只是有一个空白页面，其中包含错误消息“无法获取/执行”。

我认为它必须处理令牌、密钥和 API。正如你所知，我是这个领域的新手。

但是有没有办法在Python中检索预览文件的数据，或者获得它的访问权限？

由于数据显示在网站和预览文件中，我猜是这样，但我不知道如何。

我尝试了类似的方法，看到如何使用Python和请求从“Fetch/XHR”的“预览”中抓取信息？和无法使用python从url中检索json数据但它不起作用：

import requests


payload = {
    "value": {
        "queries": [
            {
                "key": "nav_price_evolution_filtered",
                "type": "arrayOfObjects",
                "valid": True,
                "columns": [
                    {"name": "date", "nameTranslated": "date"},
                    {"name": "date_month_year", "nameTranslated": "date_month_year"},
                    {"name": "nav_price", "nameTranslated": "nav_price"},
                    {"name": "aum_size", "nameTranslated": "aum_size"},
                    {"name": "month_year", "nameTranslated": "month_year"},
                    {"name": "total", "nameTranslated": "total"},
                    {"name": "startDate", "nameTranslated": "startDate"},
                    {"name": "endDate", "nameTranslated": "endDate"},
                    {"name": "currency", "nameTranslated": "currency"}
                ],
                "results": [
                    {
                        "date": "22/05/2024",
                        "date_month_year": "05-2024",
                        "nav_price": 100.1215,
                        "aum_size": 458439516.26,
                        "currency": "EUR",
                        "endDate": "2024-05-22",
                        "month_year": "May 2024",
                        "startDate": "2002-12-20",
                        "total": 5101
                    },
                ]
            }
        ]
    }
}


api_url = "https://api-eu.kurtosys.app/dataset/execute"
headers = {
    "x-api-key": "b5954b1e-2cfa-453f-b336-75557f19ffae",
    "Origin": "https://fonds.axa-im.fr/fr/professionnels/fund/axa-pea-rgularit-capitalisation-eur/#performanceRisk",
}

data = requests.post(api_url, headers=headers, json=payload).json()
print(data)

Answer 1

看着你的

payload

我认为这是错误的：你试图发布你已经得到的结果。试试这个：

import json
import requests

payload = {
    "code": "nav_price_evolution",
    "inputs": {
        "endDate": "2024-05-23",
        "isin": "FR0000447039",
        "startDate": "2002-12-20",
    },
}

headers = {
    "User-Agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:126.0) Gecko/20100101 Firefox/126.0",
    "X-KSYS-TOKEN": "b5954b1e-2cfa-453f-b336-75557f19ffae",
}

api_url = "https://api-eu.kurtosys.app/dataset/execute"

data = requests.post(api_url, json=payload, headers=headers).json()
print(json.dumps(data, indent=4))

打印：

{
    "value": {
        "queries": [
            {
                "key": "nav_price_evolution",
                "type": "arrayOfObjects",
                "valid": true,
                "columns": [
                    {
                        "name": "date",
                        "nameTranslated": "date"
                    },
                    {
                        "name": "nav_price",
                        "nameTranslated": "nav_price"
                    },
                    {
                        "name": "aum_size",
                        "nameTranslated": "aum_size"
                    },
                    {
                        "name": "month_year",
                        "nameTranslated": "month_year"
                    },
                    {
                        "name": "total",
                        "nameTranslated": "total"
                    },
                    {
                        "name": "startDate",
                        "nameTranslated": "startDate"
                    },
                    {
                        "name": "endDate",
                        "nameTranslated": "endDate"
                    },
                    {
                        "name": "currency",
                        "nameTranslated": "currency"
                    }
                ],
                "results": [
                    {
                        "date": "2024-05-23",
                        "nav_price": 100.1332,
                        "aum_size": 455326392.64,
                        "month_year": "May 2024",
                        "total": 5102,
                        "startDate": "2002-12-20",
                        "endDate": "2024-05-23",
                        "currency": "EUR"
                    },
                    {
                        "date": "2024-05-22",
                        "nav_price": 100.1215,
                        "aum_size": 458439516.26,
                        "month_year": "May 2024",
                        "total": 5102,
                        "startDate": "2002-12-20",
                        "endDate": "2024-05-23",
                        "currency": "EUR"
                    },
                    {
                        "date": "2024-05-21",
                        "nav_price": 100.1103,
                        "aum_size": 457445140.41,
                        "month_year": "May 2024",
                        "total": 5102,
                        "startDate": "2002-12-20",
                        "endDate": "2024-05-23",
                        "currency": "EUR"
                    },
                    {
                        "date": "2024-05-17",
                        "nav_price": 100.0673,
                        "aum_size": 454458232.38,
                        "month_year": "May 2024",
                        "total": 5102,
                        "startDate": "2002-12-20",
                        "endDate": "2024-05-23",
                        "currency": "EUR"
                    },

...