为什么我只能从 url 中抓取某些图像而不是其他图像？

Question

这是我在 stackoverflow 上提出的第一个问题，因此这个过程与我提出的问题一样，都是一种学习体验。关于此的任何提示/技巧也将被应用。

我在

beautifulsoup

中使用

python

从wiki抓取图像。总共有18张图片，但我只能抓到不到那部分。例如，我不能刮

image[0]

或

image[1]

，但我可以刮

image[2]

。我在寻找

KeyError

属性时得到一个

src

，但每个图像都有一个

src

属性。

这是我正在使用的代码。

import requests
import bs4
res = requests.get("https://en.wikipedia.org/wiki/Subaru_Outback")
soup = bs4.BeautifulSoup(res.text,"lxml")
subaru = soup.select('.thumbimage')
image_url = subaru[0]
image_url['src']

当我在

image_url = subaru[0]

和

image_url = subaru[1]

上运行代码时，我得到 keyError: 'src'。但是，当我在

image_url = subaru[2]

上运行代码时，代码运行正常。

这些是 soup.select('.thumbimage')[0,1,2] 的结果：

soup.select('.thumbimage')[0]

soup.select('.thumbimage')[1]

soup.select('.thumbimage')[2]

如您所见，每个图像都有“src”属性。我确实注意到 subaru[0] 和 subaru[1] 在输出的开头有“div”，而 subaru[2] 有“img”。 subaru[2] 实际上是 wiki 文章中的第三张图片。我不明白为什么我会收到一个 KeyError: 'src' 对于具有 'src' 属性的图像。

Answer 1

这里的问题是你的选择器而不是

<img>

在你的前两个例子中只得到

<div>

并且它没有

src

属性所以下面会更接近：

soup.select('img.thumbimage')

来自文档：

KeyError: [attr] - 当标签在 问题没有定义 attr 属性。最常见的错误是 KeyError: 'href' 和 KeyError: 'class'。使用 tag.get('attr') 如果你是 不确定是否定义了 attr，就像使用 Python 字典一样。

你可以选择你的元素也更具体，例如根据他们的部分

src

：

soup.select('img[src*="commons/thumb/"]')

例子

import requests
import bs4
res = requests.get("https://en.wikipedia.org/wiki/Subaru_Outback")
soup = bs4.BeautifulSoup(res.text,"lxml")
for e in soup.select('img[src*="commons/thumb/"]'):
    print(e.get('src'))

输出

//upload.wikimedia.org/wikipedia/commons/thumb/3/32/2020_Subaru_Outback_AWD%2C_front.jpg/280px-2020_Subaru_Outback_AWD%2C_front.jpg
//upload.wikimedia.org/wikipedia/commons/thumb/a/a5/1st_Subaru_Legacy_wagon_--_03-16-2012_front.JPG/200px-1st_Subaru_Legacy_wagon_--_03-16-2012_front.JPG
//upload.wikimedia.org/wikipedia/commons/thumb/e/e3/1st_Subaru_Legacy_wagon_--_03-16-2012_rear.JPG/200px-1st_Subaru_Legacy_wagon_--_03-16-2012_rear.JPG
//upload.wikimedia.org/wikipedia/commons/thumb/b/bf/New_Car_-_Exterior_%286879842774%29.jpg/220px-New_Car_-_Exterior_%286879842774%29.jpg
//upload.wikimedia.org/wikipedia/commons/thumb/7/70/1998_Subaru_Outback_%28BG9_MY98%29_Special_Edition_station_wagon_%282010-09-23%29_01.jpg/200px-1998_Subaru_Outback_%28BG9_MY98%29_Special_Edition_station_wagon_%282010-09-23%29_01.jpg
//upload.wikimedia.org/wikipedia/commons/thumb/1/10/1998_Subaru_Outback_%28BG9_MY98%29_Special_Edition_station_wagon_%282010-09-23%29_02.jpg/200px-1998_Subaru_Outback_%28BG9_MY98%29_Special_Edition_station_wagon_%282010-09-23%29_02.jpg
//upload.wikimedia.org/wikipedia/commons/thumb/1/19/2001_Subaru_Outback_%28BH9_MY02%29_Limited_station_wagon_%282011-04-02%29_01.jpg/200px-2001_Subaru_Outback_%28BH9_MY02%29_Limited_station_wagon_%282011-04-02%29_01.jpg
//upload.wikimedia.org/wikipedia/commons/thumb/2/2f/2001_Subaru_Outback_%28BH9_MY02%29_Limited_station_wagon_%282011-04-02%29_02.jpg/200px-2001_Subaru_Outback_%28BH9_MY02%29_Limited_station_wagon_%282011-04-02%29_02.jpg
//upload.wikimedia.org/wikipedia/commons/thumb/c/c9/2006-2009_Subaru_Outback_2.5i_station_wagon_%282011-11-17%29_01.jpg/200px-2006-2009_Subaru_Outback_2.5i_station_wagon_%282011-11-17%29_01.jpg
//upload.wikimedia.org/wikipedia/commons/thumb/8/85/2006-2009_Subaru_Outback_2.5i_station_wagon_%282011-11-17%29_02.jpg/200px-2006-2009_Subaru_Outback_2.5i_station_wagon_%282011-11-17%29_02.jpg
//upload.wikimedia.org/wikipedia/commons/thumb/c/c0/Subaru_Outback_%28BR%29_manufactured_2010_2457cc_%282%29.jpg/200px-Subaru_Outback_%28BR%29_manufactured_2010_2457cc_%282%29.jpg
//upload.wikimedia.org/wikipedia/commons/thumb/2/25/2011_Subaru_Outback_%28BR9_MY11%29_2.5i_station_wagon_%282015-07-14%29_02.jpg/200px-2011_Subaru_Outback_%28BR9_MY11%29_2.5i_station_wagon_%282015-07-14%29_02.jpg
//upload.wikimedia.org/wikipedia/commons/thumb/3/3b/2018_Subaru_Outback_SE_Premium_Symmetrical_AWD_2.5_Front.jpg/200px-2018_Subaru_Outback_SE_Premium_Symmetrical_AWD_2.5_Front.jpg
//upload.wikimedia.org/wikipedia/commons/thumb/2/27/2018_Subaru_Outback_SE_Premium_Symmetrical_AWD_2.5_Rear.jpg/200px-2018_Subaru_Outback_SE_Premium_Symmetrical_AWD_2.5_Rear.jpg
//upload.wikimedia.org/wikipedia/commons/thumb/c/c1/2021_Subaru_Outback_Limited%2C_front_3.17.21.jpg/200px-2021_Subaru_Outback_Limited%2C_front_3.17.21.jpg
//upload.wikimedia.org/wikipedia/commons/thumb/f/f0/2021_Subaru_Outback_Limited%2C_rear_3.17.21.jpg/200px-2021_Subaru_Outback_Limited%2C_rear_3.17.21.jpg
//upload.wikimedia.org/wikipedia/commons/thumb/0/04/1st-Subaru-Outback-Sport.jpg/220px-1st-Subaru-Outback-Sport.jpg
//upload.wikimedia.org/wikipedia/commons/thumb/2/21/2002-2003_Subaru_Outback_Sport.jpg/220px-2002-2003_Subaru_Outback_Sport.jpg
//upload.wikimedia.org/wikipedia/commons/thumb/3/39/3rd_Subaru_Impreza_Outback_Sport.jpg/220px-3rd_Subaru_Impreza_Outback_Sport.jpg