嗨,我想问一下如何同时处理多个Chrome浏览器。我尝试使用多个chromedriver.exe,对于每个我想打开的浏览器,都有一个自己的chromedriver.exe]
driverPlace2 = "C:/Users/M/PycharmProjects/Test/chromedriver2.exe"
driverPlace3 = "C:/Users/M/PycharmProjects/Test/chromedriver3.exe"
driverPlace4 = "C:/Users/M/PycharmProjects/Test/chromedriver4.exe"
driverPlace5 = "C:/Users/M/PycharmProjects/Test/chromedriver5.exe"
driverPlace6 = "C:/Users/M/PycharmProjects/Test/chromedriver6.exe"
driverPlace7 = "C:/Users/M/PycharmProjects/Test/chromedriver7.exe"
driverPlace8 = "C:/Users/M/PycharmProjects/Test/chromedriver8.exe"
第六个之后,它开始关闭第二个,第三个...
它在控制台中什么也没有给出,它只是开始关闭。
我该如何解决?
非常感谢您阅读和帮助我
您可以将程序与Thread :(以下脚本一起使用,打开2个chrome浏览器,它们同时在网上爬行)
from selenium import webdriver
from random import shuffle
from concurrent import futures
from urllib.parse import urlparse
def driver_crawl_web(first_page):
driver = webdriver.Chrome()
driver.get(first_page)
while True:
a_tags = driver.find_elements_by_xpath("//a[@href]")
shuffle(a_tags)
for tag in a_tags:
href = tag.get_attribute('href')
if check_url(href):
selected_link = href
break
if selected_link == driver.current_url:
driver.back()
continue
driver.get(selected_link)
def check_url(url):
parse = urlparse(url)
if "https" in parse.scheme:
if "youtube" not in parse.netloc:
return True
return False
def crawl_with_concurrency(links):
executor = futures.ThreadPoolExecutor(max_workers=2)
executions = executor.map(driver_crawl_web, links)
links = ["https://www.google.com/search?q=python",
"https://www.google.com/search?q=javascript"]
crawl_with_concurrency(links)