为什么不从YouTube获取任何'a'标签？

Question

我正在尝试通过BeautifulSoup从给定的输入频道链接中获取视频的所有链接。我发现视频的所有“ a”标签的ID都为“ video-title”，但以下代码未提供任何输出：

import requests
from bs4 import BeautifulSoup

source = requests.get('https://www.youtube.com/user/TheCraftingLab/featured').text
soup = BeautifulSoup(source, 'html.parser')

container = soup.findAll("a", {id: "video-title"})
for i in container:
    print(i)

怎么了？

Answer 1

随JS一起加载，如果需要，您可以使用this模块，该模块允许在不使用浏览器的情况下使用JS渲染

from requests_html import HTMLSession
from bs4 import BeautifulSoup

URL = "https://www.example.com"

with HTMLSession() as session:
    response = session.get(URL)
    response.html.render()
    soup = BeautifulSoup(response.html.html, 'html.parser')

container = soup.findAll("a", {id: "video-title"})
for i in container:
    print(i)