在YouTube的搜索和返回Python中的所有链接

Question

在YouTube上，我想搜索特定的视频（即Python的视频），并在此之后，我想返回所有视频此搜索返回。现在如果我尝试这条巨蟒不是返回搜索后的页面上的启动页面上的所有视频。

当前的代码：

from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager

driver = webdriver.Chrome(ChromeDriverManager().install())
driver.get("http://youtube.com")
driver.find_element_by_name("search_query").send_keys("Python")
driver.find_element_by_id("search-icon-legacy").click()
links = driver.find_elements_by_id("video-title")
for x in links:
    print(x.get_attribute("href"))

错在这里？

Answer 1

从与关键字作为Python中，你需要将搜索返回的所有视频：

• 将屏幕最大化，因此所有产生的视频链接的HTML DOM内呈现。
• 诱导WebDriverWait为所需的元素中提取的href属性之前是可见的。
• 您可以采用如下方案 代码块：

  from selenium import webdriver
  from selenium.webdriver.common.by import By
  from selenium.webdriver.support.ui import WebDriverWait
  from selenium.webdriver.support import expected_conditions as EC
  
  options = webdriver.ChromeOptions()
  options.add_argument("start-maximized")
  options.add_argument("disable-infobars")
  options.add_argument("--disable-extensions")
  driver=webdriver.Chrome(chrome_options=options, executable_path=r'C:\Utility\BrowserDrivers\chromedriver.exe')
  driver.get("https://www.youtube.com/")
  WebDriverWait(driver, 5).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input#search"))).send_keys("Python")
  driver.find_element_by_css_selector("button.style-scope.ytd-searchbox#search-icon-legacy").click()
  print([my_href.get_attribute("href") for my_href in WebDriverWait(driver, 5).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "a.yt-simple-endpoint.style-scope.ytd-video-renderer#video-title")))])

  控制台输出： ['https://www.youtube.com/watch?v=rfscVS0vtbw', 'https://www.youtube.com/watch?v=7UeRnuGo-pg', 'https://www.youtube.com/watch?v=3cZsjOclmoM', 'https://www.youtube.com/watch?v=f79MRyMsjrQ', 'https://www.youtube.com/watch?v=CtbckFw0pJs', 'https://www.youtube.com/watch?v=Z1Yd7upQsXY', 'https://www.youtube.com/watch?v=kLZuut1fYzQ', 'https://www.youtube.com/watch?v=IZ0IM_T4aio', 'https://www.youtube.com/watch?v=qiSCMNBIP2g', 'https://www.youtube.com/watch?v=N0lxfilGfak', 'https://www.youtube.com/watch?v=N4mEzFDjqtA', 'https://www.youtube.com/watch?v=s3Ejdx6cIho', 'https://www.youtube.com/watch?v=Y8Tko2YC5hA', 'https://www.youtube.com/watch?v=c3FXQU3TyCU', 'https://www.youtube.com/watch?v=yE9v9rt6ziw', 'https://www.youtube.com/watch?v=yvHrNlAF0Y0', 'https://www.youtube.com/watch?v=ZDa-Z5JzLYM']

Answer 2

但最好是使用一个明确的等待这样的：

links = ui.WebDriverWait(driver, 10).until(EC.presence_of_all_elements_located((By.ID, "video-title")))

Reference。

希望它可以帮助你！

Answer 3

按照与@马克的讨论：

看来，在Youtube的第一个页面的元素仍然在DOM ...

我看到的唯一的解决方法就是去搜索网址：

driver.get("http://youtube.com/results?search_query=Python")
# driver.find_element_by_name("search_query").send_keys("Python")
# driver.find_element_by_id("search-icon-legacy").click()

---

您应该使用WebDriverWait睡不着：

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.wait import WebDriverWait

from selenium.webdriver.chrome.options import Options

opt = Options()
opt.add_argument("--incognito")

driver = webdriver.Chrome(executable_path=r'C:\path\to\chromedriver.exe', chrome_options=opt)
driver.get("http://youtube.com")
driver.find_element_by_name("search_query").send_keys("Python")
driver.find_element_by_id("search-icon-legacy").click()
WebDriverWait(driver, 10).until(EC.presence_of_all_elements_located((By.ID, "video-title")))
links = driver.find_elements_by_id("video-title")

for x in links:

    print(x.get_attribute("href"))

输出：

https://www.youtube.com/watch?v=rfscVS0vtbw
https://www.youtube.com/watch?v=f79MRyMsjrQ
https://www.youtube.com/watch?v=kLZuut1fYzQ
https://www.youtube.com/watch?v=N4mEzFDjqtA
https://www.youtube.com/watch?v=Z1Yd7upQsXY
https://www.youtube.com/watch?v=hnDU1G9hWqU
https://www.youtube.com/watch?v=3cZsjOclmoM
https://www.youtube.com/watch?v=f3EbDbm8XqY
https://www.youtube.com/watch?v=2uCXIbkbDSE
https://www.youtube.com/watch?v=HXV3zeQKqGY
https://www.youtube.com/watch?v=JJmcL1N2KQs
https://www.youtube.com/watch?v=qiSCMNBIP2g
https://www.youtube.com/watch?v=7lmCu8wz8ro
https://www.youtube.com/watch?v=25ovCm9jKfA
https://www.youtube.com/watch?v=q6Mc_sAPZ2Y
https://www.youtube.com/watch?v=yE9v9rt6ziw
https://www.youtube.com/watch?v=Y8Tko2YC5hA
https://www.youtube.com/watch?v=G0rQ7AEl5LA
https://www.youtube.com/watch?v=CtbckFw0pJs
https://www.youtube.com/watch?v=sugvnHA7ElY