用python废止网址链接

Question

这里是一个代码。

from selenium import webdriver
from bs4 import BeautifulSoup

driver = webdriver.Firefox()
url = 'https://www.coteur.com/cotes-foot.php'
driver.get(url)

links = driver.find_elements_by_xpath('//a[contains(@href, "match/cotes-")]')

driver.close()

我想从这个网站上搜刮所有与足球比赛相关的url链接。 https:/www.coteur.comcotes-foot.php

我总是刮所有的balise，其中包括足球游戏。但是，我怎么能提取链接到这些足球游戏的网址？

Answer 1

你得到的webelements带有 find_elements_by_xpath 你需要得到 href 从中

from selenium import webdriver

driver = webdriver.Firefox()
url = 'https://www.coteur.com/cotes-foot.php'
driver.get(url)

links = []
for i in driver.find_elements_by_xpath('//a[contains(@href, "match/cotes-")]'):
    links.append(i.get_attribute('href'))

print(links)
driver.close()

Answer 2

我试了一下。

n = 0
while n < len(links):
   links[n] = links[n].text
   n = n + 1

print(links)

driver.findElement(By.linkText(link[1])).click()

这里是输出 :

hao@hao-ThinkPad-T420:~$ ./coteur2.py
['FS METTA/LU - JPFS/FK Spartaks', 'Fc Ararat Erevan - Lori Vanadzor', 'Stabaek If - Mjondalen', 'Viking - FK BODO/GLIMT', 'Aalesund - Molde', 'Odd Ballklubb Grenland - Sandefjord', 'Rosenborg - Kristiansund Bk', 'Ac Horsens - Esbjerg Fb', 'Fk Sutjeska Niksic - Ofk Titograd', 'FK Tukums 2000/Tss - Fk Jelgava', 'Borrusia Monch. - Vfl Wolfsburg', 'Fc Admira Wacker Modling - Scr Altach', 'Skn St. Pölten - Sv Mattersburg', 'Sv Wehen Wiesbaden - 1. Fc Nuremberg', 'Hambourg Sv - Vfl Osnabrück', 'Greuther Furth - 1. Fc Heidenheim 1846', 'Tallinna Jk Legion - Jk Tulevik Viljandi', 'Hnk Hajduk Split - Nk Varteks Varazdin', 'Nk Istra 1961 Pula - Inter Zapresic', 'Sepsi Osk Sfantu Gheorghe - Voluntari', 'Varda Se - Zalaegerszeg Te', 'Nd Mura 05 - Nk Celje', 'Ab Argir - EB/STREYMUR', 'Villarreal - Real Majorque', 'Getafe - Espanyol']
Traceback (most recent call last):
  File "./coteur2.py", line 23, in <module>
    driver.findElement(By.linkText(link[1])).click()
AttributeError: 'WebDriver' object has no attribute 'findElement'

Answer 3

试试这个。

import urllib.request, urllib.error, urllib.parse  #Import required modules
from bs4 import BeautifulSoup
import ssl

ctx=ssl.create_default_context()  #Check certificates, you can skip this for some 
                                   #websites 
ctx.check_hostname=False
ctx.verify_mode=ssl.CERT_NONE

userInput=input("Enter URL: ")
url=userInput if len(userInput)!=0 else "https://www.coteur.com/cotes-foot.php"

html=urllib.request.urlopen(url, context=ctx).read()
soup=BeautifulSoup(html, "html.parser")

tags=soup("a")                       #Find all html "a" tags, and print
for tag in tags:                 #The "a" tag is used to create link
    print(tag.get("href", None))

这个程序会打印出它在页面上找到的所有链接。

如果你只想要足球相关的链接，你可以将最后一行修改为。

if 'soccer' in tag.get("href", None):
    print(tag.get("href", None))