使用硒分析动态网页

Question

我正在尝试从Amazon刮取图像，这并不容易。

我想我快到了，但是我没有得到结果。

[这里，我正在使用硒来1.打开主图像，然后单击2.在缩略图中单击第二个图像。3.然后获取第二张图片的完整大小的src。

但是失败了，我不知道为什么

这是我写的行。

from urllib.request import urlretrieve
from bs4 import BeautifulSoup
from selenium import webdriver
from selenium.webdriver.common.action_chains import ActionChains
import time

driver = webdriver.Firefox()
url = "https://www.amazon.com/Kraft-Original-Macaroni-Microwaveable-Packets/dp/B005ECO3H0"
driver.get(url)
action = ActionChains(driver)
time.sleep(5)

driver.find_element_by_css_selector('#landingImage').click()
time.sleep(10)

html = driver.page_source
soup = BeautifulSoup(html,"html.parser")

driver.find_element_by_css_selector('#ivImage_1').click()
amazon = soup.select_one(".fullscreen")
imgUrl = amazon.find("img")['src']
print(imgUrl)

我无法理解的一件事是，如果我输入print（amazon），它会给我img标签，但是根据上面代码的结果，imgUrl是'Nonetype'。

[请帮助我找到答案。

Answer 1

你去这里

from urllib.request import urlretrieve
from bs4 import BeautifulSoup
from selenium import webdriver
from selenium.webdriver.common.action_chains import ActionChains
import time

driver = webdriver.Firefox()
url = "https://www.amazon.com/Kraft-Original-Macaroni-Microwaveable- 
Packets/dp/B005ECO3H0"
driver.get(url)
action = ActionChains(driver)
time.sleep(5)

driver.find_element_by_css_selector('#landingImage').click()
time.sleep(5)

html = driver.page_source
soup = BeautifulSoup(html,"html.parser")

driver.find_element_by_css_selector('#ivImage_1').click()
image_url = driver.find_element_by_class_name("fullscreen").get_attribute("src")
print(image_url)

#if you want to download
import requests
resp = requests.get(image_url)
with open("asd.png", "wb")as image:
    image.write(resp.content)