如何使用python硒对具有相同div的no.of元素进行抓取

Question

我正在尝试抓取此网站上的所有匹配项

https://web.bet9ja.com/Sport/OddsToday.aspx?IDSport=590

我需要的是

1。在我共享和抓取数据的链接中单击匹配名称，例如kuttosh kujand，然后再次导航并再次单击另一个匹配名称，应对链接中存在的匹配项进行处理

到目前为止，我已经用自己的代码编写了这样的代码，我能够完成上面提到的比赛过程，但是如何为所有比赛完成该过程

我写的代码：

# Here using selenium for scraping
# importing necessary modules
import selenium.webdriver
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
import mysql.connector
import pymysql
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

# the relevant url
url = 'https://web.bet9ja.com/Sport/OddsToday.aspx?IDSport=590'

# the driver path
driver = webdriver.Chrome(r"c:/Users/SATYA/mysite/chromedriver")
driver.get(url)
driver.implicitly_wait(10) # seconds
buttons = WebDriverWait(driver,15).until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, "div.Event.ng-binding")))
for btn in buttons:
    btn.click()
    headings= [item.text for item in driver.find_elements_by_css_selector("div.SECQ.ng-binding")]
    keys = [item.text for item in driver.find_elements_by_css_selector("div.SEOdd.g1")]
    values = [item.text for item in driver.find_elements_by_css_selector("div.SEOddLnk.ng-binding")]
    driver.execute_script("window.history.go(-1)")
    print(headings,keys,values)

任何人都可以帮我这个忙

在抓取第一个比赛数据后，我使用编写的代码收到此错误

Traceback (most recent call last):
  File "dynamicscrape.py", line 21, in <module>
    btn.click()
  File "C:\Users\SATYA\AppData\Local\Programs\Python\Python37\lib\site-packages\selenium\webdriver\remote\webelement.py", line 80, in click
    self._execute(Command.CLICK_ELEMENT)
  File "C:\Users\SATYA\AppData\Local\Programs\Python\Python37\lib\site-packages\selenium\webdriver\remote\webelement.py", line 633, in _execute
    return self._parent.execute(command, params)
  File "C:\Users\SATYA\AppData\Local\Programs\Python\Python37\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "C:\Users\SATYA\AppData\Local\Programs\Python\Python37\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.StaleElementReferenceException: Message: stale element reference: element is not attached to the page document
  (Session info: chrome=81.0.4044.113)

Answer 1

stale element reference: element is not attached to the page document错误发生，因为自刷新页面以来未将元素附加到您已捕获的页面上。

要解决此问题，您需要重新分配元素以避免陈旧。

buttons = WebDriverWait(driver,15).until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, "div.Event.ng-binding")))
for btn in range(len(buttons)):
    #elements re-assigned again to avoid stale.
    buttons = WebDriverWait(driver, 15).until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, "div.Event.ng-binding")))
    buttons[btn].click()
    headings= [item.text for item in driver.find_elements_by_css_selector("div.SECQ.ng-binding")]
    keys = [item.text for item in driver.find_elements_by_css_selector("div.SEOdd.g1")]
    values = [item.text for item in driver.find_elements_by_css_selector("div.SEOddLnk.ng-binding")]
    driver.execute_script("window.history.go(-1)")
    print(headings,keys,values)