无法通过 Puppeteer 检索 Steam 游戏的捆绑价格

Question

我正在编写一段代码来获取游戏捆绑包的标题和价格，将每个捆绑包的标题和价格分开，但它返回的bundle_price 为空。

import puppeteer from "puppeteer";

async function handleAgeRestriction(p) {
  await p.evaluate(() => {
    const select = document.querySelector("#ageYear");
    const options = select.querySelectorAll("option");
    const selectedOption = [...options].find(
      (option) => option.text === "1900"
    );

    selectedOption.selected = true;
  });
  await p.click("#view_product_page_btn");
}

async function getDataFromGame() {
  const browser = await puppeteer.launch({ headless: false });

  const page = await browser.newPage();

  await page.goto("https://store.steampowered.com/app/271590/");

  await handleAgeRestriction(page);

  await page.waitForSelector(".apphub_AppName");
  // await page.waitForSelector(".game_purchase_price");
  await page.waitForSelector("div.game_area_purchase_game > h1");
  await page.waitForSelector("div.discount_final_price");

  const result = await page.evaluate(() => {
    const data = document.querySelectorAll('.game_area_purchase_game');
    const game = [...data].map((bundle) => {
      const bundle_title = bundle.querySelector('div.game_area_purchase_game > h1').innerText;
      const bundle_price = bundle.querySelector("div.discount_final_price").innerText;

      return {
        bundle_title,
        bundle_price,
      }
    })
    return game;
  });

  console.log(result);

  await browser.close();
}

getDataFromGame();

我发现这个错误很奇怪，因为如果我用“文档”替换“捆绑包”，它会正确检索价格，但它始终是相同的价格

 const result = await page.evaluate(() => {
    const data = document.querySelectorAll('.game_area_purchase_game');
    const game = [...data].map((bundle) => {
      const bundle_title = bundle.querySelector('div.game_area_purchase_game > h1').innerText;
      const bundle_price = document.querySelector("div.discount_final_price").innerText;

      return {
        bundle_title,
        bundle_price,
      }
    })
    return game;
  });

  console.log(result);

（这是一个示例，说明它不会给我带来错误，但所有捆绑包的价格都是相同的。）

Answer 1

错误（

Error [TypeError]: Cannot read properties of null (reading 'textContent')

）的原因是某些元素没有

.discount_final_price

元素。相反，他们有一个下拉菜单，可让您选择具有不同价格的几个选项之一。

因此，不要假设

.discount_final_price

始终存在，而是使用条件来处理它不存在的可能性并获取下拉列表：

const result = await page.$$eval(
  ".game_area_purchase_game",
  els =>
    els.map(bundle => {
      const bundle_title = bundle
        .querySelector("h1")
        ?.textContent.trim();
      const bundle_price = bundle.querySelector(
        ".discount_final_price"
      )?.textContent;

      if (bundle_price) {
        return {
          bundle_title,
          bundle_price,
        };
      }

      bundle
        .querySelector(
          ".game_area_purchase_game_dropdown_selection"
        )
        .click();
      return {
        bundle_title,
        bundle_prices: [
          ...bundle.querySelectorAll(
            ".game_area_purchase_game_dropdown_menu_item_text"
          ),
        ].map(e => e.textContent)
      };
    })
);

输出现在看起来像这样：

[
  {
    bundle_title: 'Buy Shark Cash Cards',
    bundle_prices: [
      'Tiger Shark: GTA$250,000 - $4.99',
      'Bull Shark: GTA$600,000 - $9.99',
      'Great White Shark: GTA$1,500,000 - $19.99',
      'Whale Shark: GTA$4,250,000 - $49.99',
      'Megalodon Shark: GTA$10,000,000 - $99.99'
    ]
  },
  {
    bundle_title: 'Buy Grand Theft Auto V: Premium Edition & Great White Shark Card Bundle',
    bundle_price: '$19.80'
  },
  {
    bundle_title: 'Buy Grand Theft Auto V: Premium Edition & Megalodon Shark Card Bundle',
    bundle_price: '$36.40'
  },
  {
    bundle_title: 'Buy Grand Theft Auto V: Premium Edition',
    bundle_price: '$29.98'
  }
]

您可以进一步处理此列表并使用

e.textContent.split(" - ").at(-1)

从每个下拉项目中提取价格。