beautifulsoup 相关问题

我尝试从此 html 文件获取此处包含的文本值：我尝试从这个 html 文件中获取此处包含的文本值： <div> <div> <div class="py d-flex align-items-start align-items-lg-center css-17435dd row" data-test="salaries-list-item-0" data-brandviews="MODULE:n=salaries-search-salaries-by-company:eid=7927:uid=0:salary_job_title_id=37856" data-triggered-brandview=""><div class="col-12 col-md-3 col-lg-3"><div class="d-flex" data-test="employer-info"><div class="employerLogo mr"><a href="/Salary/Infosys-Project-Manager-United-States-Salaries-EJI_IE7927.0,7_KO8,23_IL.24,37.htm" data-test="salaries-list-item-0-employer-url"><img alt="Infosys" src="https://media.glassdoor.com/sql/7927/infosys-squareLogo-1620208556721.png" class="css-1rc0f2z e1aj7ssy1"></a></div><div class="employerStats"><a href="/Salary/Infosys-Project-Manager-United-States-Salaries-EJI_IE7927.0,7_KO8,23_IL.24,37.htm" data-test="salaries-list-item-0-employer-url" class="css-1ikln7a el6ke052">Infosys</a><div class="d-flex align-items-center mt-xxsm"><span class="css-h9sogr m-0 css-60s9ld el6ke050" color="#0caa41">3.2</span><span data-color-variant="primary" data-size-variant="sm" class="gd-ui-star css-h9sogr ml-xxsm css-ojdxng efdhmtv0" role="presentation"><svg aria-hidden="true" class="css-pq72fl e7xsrz90" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><defs><path id="star_svg__star-multicolor-icon-path" d="m12 2.694 2.119 6.857.109.352h7.294L15.898 14.2l-.277.212.103.333 2.135 6.909-5.556-4.244-.303-.232-.303.232-5.556 4.244 2.135-6.909.103-.333-.277-.212-5.624-4.296h7.295l.108-.352L12 2.694Z"></path><clipPath id="star_svg__star-multicolor-icon-clip"><use href="#star_svg__star-multicolor-icon-path"></use></clipPath></defs><use href="#star_svg__star-multicolor-icon-path" clip-path="url(#star_svg__star-multicolor-icon-clip)" fill="var(--icon-color-a, currentColor)" stroke="var(--icon-color-b, currentColor)" stroke-width="var(--icon-stroke-width, 1px)"></use></svg></span></div><span class="d-flex align-items-middle css-1in2cw4 el6ke050">Project Manager<span class="SVGInline"><svg class="SVGInline-svg" xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24"><path d="M12 9a1 1 0 101 1 1 1 0 00-1-1zm0 3a1 1 0 00-1 1v2a1 1 0 002 0v-2a1 1 0 00-1-1zm0-6a6 6 0 11-6 6 6 6 0 016-6zm0 11a5 5 0 10-5-5 5 5 0 005 5z" fill="currentColor" fill-rule="evenodd"></path></svg></span></span><a class="m-0 d-flex css-259095 e1aj7ssy6" href="/Salary/Infosys-Project-Manager-Salaries-EI_IE7927.0,7_KO8,23.htm" data-test="salaries-list-item-0-salary-count"><span>See 23,491 salaries from this location</span></a><div class="d-block d-lg-none"><a href="/Jobs/Infosys-Jobs-E7927.htm" data-test="salaries-list-item-0-open-jobs-link" class="css-259095 e1aj7ssy6"><span>1,744 open jobs</span></a></div></div></div></div><div class="col-12 col-md-3 col-lg-3 d-none d-lg-block align-self-start"><div class="d-flex" data-test="employer-info"><a href="/Jobs/Infosys-Jobs-E7927.htm" data-test="salaries-list-item-0-open-jobs-link" class="css-259095 e1aj7ssy6"><span>1,744 open jobs</span></a></div></div><div data-test="range-bar" class="col-12 col-lg-6 order-1 order-md-2 order-lg-1 pt-xl pt-lg-0"><div class="d-flex flex-column"><div class="order-0 mt-xl css-1ex4vn2 e1hxjh2q1"><div class="css-1xxwalr e1hxjh2q3"><div class="css-79elbk e13r6qcv0"><div class="mb-xxsm d-flex align-items-baseline css-3tpw1n e13r6qcv1"><h3 class="m-0 css-16zrpia el6ke054">$118,568</h3><span class="m-0 css-1in2cw4 el6ke050"> / yr</span></div></div></div></div><div class="order-2 css-79elbk e1hxjh2q0" style="visibility: visible; min-height: 24px;"><div class="css-osz7jh e13r6qcv4"><div class="css-79elbk e1hxjh2q0"><div class="css-ituc4g e13r6qcv5"><div class="d-flex col css-79elbk e1hxjh2q0"><div class="d-flex flex-column align-items-end css-15o6gsn e13r6qcv5"><span class="m-0 css-1in2cw4 el6ke050">$104K</span></div><div class="d-flex flex-column align-items-end css-1qfy6mj e13r6qcv5"><span class="m-0 css-1in2cw4 el6ke050">$135K</span></div></div></div></div></div></div><div class="order-1 my-xxsm css-psber1 e13r6qcv3"><div class="css-32y0el e13r6qcv2"></div><div class="css-x87ns5 e13r6qcv4"></div></div></div></div></div> 到目前为止，我尝试使用 BS4，但结果值为“无”： source = BeautifulSoup(driver.page_source,'html.parser') data_test_param = 'salaries-list-item-'+str(x)+'-employer-url' a_value = source.find('a', attrs={'data-test': data_test_param}) h3_value = source.find_all('h3',{'class':"m-0 css-16zrpia el6ke054"}) 请您指教一下吗？查看 HTML，您可以执行以下操作： from bs4 import BeautifulSoup html_text = """\ <div> <div> <div class="py d-flex align-items-start align-items-lg-center css-17435dd row" data-test="salaries-list-item-0" data-brandviews="MODULE:n=salaries-search-salaries-by-company:eid=7927:uid=0:salary_job_title_id=37856" data-triggered-brandview=""><div class="col-12 col-md-3 col-lg-3"><div class="d-flex" data-test="employer-info"><div class="employerLogo mr"><a href="/Salary/Infosys-Project-Manager-United-States-Salaries-EJI_IE7927.0,7_KO8,23_IL.24,37.htm" data-test="salaries-list-item-0-employer-url"><img alt="Infosys" src="https://media.glassdoor.com/sql/7927/infosys-squareLogo-1620208556721.png" class="css-1rc0f2z e1aj7ssy1"></a></div><div class="employerStats"><a href="/Salary/Infosys-Project-Manager-United-States-Salaries-EJI_IE7927.0,7_KO8,23_IL.24,37.htm" data-test="salaries-list-item-0-employer-url" class="css-1ikln7a el6ke052">Infosys</a><div class="d-flex align-items-center mt-xxsm"><span class="css-h9sogr m-0 css-60s9ld el6ke050" color="#0caa41">3.2</span><span data-color-variant="primary" data-size-variant="sm" class="gd-ui-star css-h9sogr ml-xxsm css-ojdxng efdhmtv0" role="presentation"><svg aria-hidden="true" class="css-pq72fl e7xsrz90" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><defs><path id="star_svg__star-multicolor-icon-path" d="m12 2.694 2.119 6.857.109.352h7.294L15.898 14.2l-.277.212.103.333 2.135 6.909-5.556-4.244-.303-.232-.303.232-5.556 4.244 2.135-6.909.103-.333-.277-.212-5.624-4.296h7.295l.108-.352L12 2.694Z"></path><clipPath id="star_svg__star-multicolor-icon-clip"><use href="#star_svg__star-multicolor-icon-path"></use></clipPath></defs><use href="#star_svg__star-multicolor-icon-path" clip-path="url(#star_svg__star-multicolor-icon-clip)" fill="var(--icon-color-a, currentColor)" stroke="var(--icon-color-b, currentColor)" stroke-width="var(--icon-stroke-width, 1px)"></use></svg></span></div><span class="d-flex align-items-middle css-1in2cw4 el6ke050">Project Manager<span class="SVGInline"><svg class="SVGInline-svg" xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24"><path d="M12 9a1 1 0 101 1 1 1 0 00-1-1zm0 3a1 1 0 00-1 1v2a1 1 0 002 0v-2a1 1 0 00-1-1zm0-6a6 6 0 11-6 6 6 6 0 016-6zm0 11a5 5 0 10-5-5 5 5 0 005 5z" fill="currentColor" fill-rule="evenodd"></path></svg></span></span><a class="m-0 d-flex css-259095 e1aj7ssy6" href="/Salary/Infosys-Project-Manager-Salaries-EI_IE7927.0,7_KO8,23.htm" data-test="salaries-list-item-0-salary-count"><span>See 23,491 salaries from this location</span></a><div class="d-block d-lg-none"><a href="/Jobs/Infosys-Jobs-E7927.htm" data-test="salaries-list-item-0-open-jobs-link" class="css-259095 e1aj7ssy6"><span>1,744 open jobs</span></a></div></div></div></div><div class="col-12 col-md-3 col-lg-3 d-none d-lg-block align-self-start"><div class="d-flex" data-test="employer-info"><a href="/Jobs/Infosys-Jobs-E7927.htm" data-test="salaries-list-item-0-open-jobs-link" class="css-259095 e1aj7ssy6"><span>1,744 open jobs</span></a></div></div><div data-test="range-bar" class="col-12 col-lg-6 order-1 order-md-2 order-lg-1 pt-xl pt-lg-0"><div class="d-flex flex-column"><div class="order-0 mt-xl css-1ex4vn2 e1hxjh2q1"><div class="css-1xxwalr e1hxjh2q3"><div class="css-79elbk e13r6qcv0"><div class="mb-xxsm d-flex align-items-baseline css-3tpw1n e13r6qcv1"><h3 class="m-0 css-16zrpia el6ke054">$118,568</h3><span class="m-0 css-1in2cw4 el6ke050"> / yr</span></div></div></div></div><div class="order-2 css-79elbk e1hxjh2q0" style="visibility: visible; min-height: 24px;"><div class="css-osz7jh e13r6qcv4"><div class="css-79elbk e1hxjh2q0"><div class="css-ituc4g e13r6qcv5"><div class="d-flex col css-79elbk e1hxjh2q0"><div class="d-flex flex-column align-items-end css-15o6gsn e13r6qcv5"><span class="m-0 css-1in2cw4 el6ke050">$104K</span></div><div class="d-flex flex-column align-items-end css-1qfy6mj e13r6qcv5"><span class="m-0 css-1in2cw4 el6ke050">$135K</span></div></div></div></div></div></div><div class="order-1 my-xxsm css-psber1 e13r6qcv3"><div class="css-32y0el e13r6qcv2"></div><div class="css-x87ns5 e13r6qcv4"></div></div></div></div></div>""" soup = BeautifulSoup(html_text, "html.parser") for h3 in soup.select("h3"): url = h3.find_previous("a")["href"] amount = h3.text print(url) print(amount) print() 打印： /Jobs/Infosys-Jobs-E7927.htm $118,568

python selenium-webdriver beautifulsoup

回答 1 投票 0

抓取 Google 搜索结果 Python BeautifulSoup

我有一个谷歌查询，它显示了8000个带链接的结果，我只想抓取搜索结果中的链接（url），我能够获取第一页链接，有没有什么方法可以抓取下一页。他...

python web-scraping beautifulsoup google-search-api

回答 1 投票 0

Python 导入某些库时出现“非法指令（核心转储）”（beautifulsoup4.yfinance）

我在 Ubuntu 22.04.4 上使用 Python3.10，并且尝试运行我最初在 Windows 11 计算机上编写的代码。每当我运行这个脚本 - main.py - 它总是停止在导入阶段，并且......

python python-3.x ubuntu beautifulsoup yfinance

回答 1 投票 0

在嵌套跨度下的跨度中抓取信息

我想通过网络抓取获取实时天气数据。我正在考虑使用 BeautifulSoup 来实现这一点。我想通过网络抓取获取实时天气数据。我正在考虑使用 BeautifulSoup 来实现此目的。 <span class="Column--precip--3JCDO"> <span class="Accessibility--visuallyHidden--H7O4p">Chance of Rain</span> 3% </span> 我想从这个容器中取出 3%。我已经设法使用此代码片段从网站获取另一部分的数据。 temp_value = soup.find("span", {"class":"CurrentConditions--tempValue--MHmYY"}).get_text(strip=True) 我对 rain_forecast 也做了同样的尝试 rain_forecast = soup.find("span", {"class": "Column--precip--3JCDO"}).get_text(strip=True) 但是我的控制台提供的输出是：“--” for print(rain_forecast)。我能看到的唯一区别是，在应该从跨度获取的“文本”之间，还有另一个跨度。我遇到 stackoverflow 的另一种方法是使用 Selenium，因为数据尚未加载到变量中，因此输出为“--”。但我不知道这对我的应用程序来说是否太过分了，或者是否有更简单的解决方案来解决这个问题。如果您想获取今天的天气预报表，您可以使用此示例： import pandas as pd import requests from bs4 import BeautifulSoup headers = {"User-Agent": "Mozilla/5.0"} url = "https://weather.com/en-IN/weather/today/l/a0e0a5a98f7825e44d5b44b26d6f3c2e76a8d70e0426d099bff73e764af3087a" soup = BeautifulSoup(requests.get(url, headers=headers).content, "html.parser") today_forecast = [] for a in soup.select(".TodayWeatherCard--TableWrapper--globn a"): today_forecast.append( t.get_text(strip=True, separator=" ") for t in a.find_all(recursive=False) ) df = pd.DataFrame( today_forecast, columns=["Time of day", "Degrees", "Text", "Chance of rain"] ) print(df) 打印： Time of day Degrees Text Chance of rain 0 Morning 11 ° Partly Cloudy -- 1 Afternoon 20 ° Partly Cloudy -- 2 Evening 14 ° Partly Cloudy Night Rain Chance of Rain 3% 3 Overnight 10 ° Cloudy Rain Chance of Rain 5% from bs4 import BeautifulSoup # Assuming you have your HTML content in 'html_content' soup = BeautifulSoup(html_content, 'html.parser') # Find the parent span and extract the text, excluding the nested span's text rain_forecast = soup.find("span", {"class": "Column--precip--3JCDO"}).contents[-1].strip() print(rain_forecast)

python web-scraping beautifulsoup

回答 2 投票 0

即使没有错误，WebScraping 也不起作用

我想设计Python网页抓取代码来抓取这些数据（https://www.nyc.gov/site/tlc/about/tlc-trip-record-data.page）。这是代码：导入操作系统导入请求随机导入导入时间 ...

python visual-studio-code web-scraping beautifulsoup python-requests

回答 1 投票 0

如何使用BeautifulSoup从<a href="TextWithUrlBehind">Something</a>中提取url？

我正在尝试从网页中提取 .json 文件中的一些链接和文本。我已经解析了 HTML tbody > tr > td，每个 td 都包含 Something&...

python html beautifulsoup

回答 1 投票 0

如何从需要点击/切换的网页中提取表格？

我正在尝试从此网页中提取表格，但我只能获取投球表。我也想获得命中表，理论上是这个 URL： https://www.cover...

python pandas web-scraping web beautifulsoup

回答 1 投票 0

使用 BeautifulSoup 抓取所有链接

我试图从页面上抓取所有比赛报告链接，但有“加载更多”按钮，而且我不想使用selenium。有没有解决方案可以收集所有不使用硒的链接。谢谢

python selenium-webdriver web-scraping beautifulsoup

回答 1 投票 0

如何将抓取的 HTML 文档转换为数据框？

我正在尝试从网站 FBRef 抓取足球运动员的数据，我从网站获取了 bs4.element.ResultSet 对象的数据。代码：导入请求从 bs4 导入 BeautifulSoup 进口...

python pandas dataframe web-scraping beautifulsoup

回答 2 投票 0

Python 初学者：将从网站抓取的 HTML 文档转换为数据框

我正在尝试从网站 FBRef 抓取足球运动员的数据，我从网站获取了 bs4.element.ResultSet 对象的数据。代码：导入请求从 bs4 导入 BeautifulSoup 进口...

python pandas beautifulsoup

回答 1 投票 0

使用 python 从 rotowire 抓取 MLB 每日阵容

我正在尝试从这里抓取 MLB 每日阵容信息：https://www.rotowire.com/baseball/daily-lineups.php 我正在尝试将 python 与请求、BeautifulSoup 和 pandas 一起使用。我的终极...

python web-scraping beautifulsoup

回答 1 投票 0

如何在网页抓取时将 colspan 包含到表头

我是网络抓取新手，我正在尝试从该网站抓取 2022 年福布斯排行榜 - https://en.wikipedia.org/wiki/List_of_largest_companies_in_India ，但排名栏和福布斯排名...

python pandas web-scraping beautifulsoup jupyter-notebook

回答 1 投票 0

如何在网页抓取时将 colspan 包含到表头中

我是网络抓取的新手，我正在尝试从该网站抓取 2022 年福布斯排行榜 - https://en.wikipedia.org/wiki/List_of_largest_companies_in_India ，但排名栏和福布斯排名

pandas web-scraping beautifulsoup jupyter-notebook

回答 1 投票 0

当我尝试使用 urllib 发送请求时，出现 InvalidURL: URL can't contains control characters

我正在尝试从用作 urllib 请求参数的链接获取 JSON 响应。但它给了我一个错误，它不能包含控制字符。我该如何解决这个问题？开始网址=...

python web-scraping beautifulsoup urllib

回答 8 投票 0

如何在 beautifulsoup 中处理正则表达式

check_regex = re.compile("HO\s?#",re.IGNORECASE) check_ho_number3 = soup.select_one('td:-soup-contains("HO #")+ td') 打印（check_ho_number3）有没有什么办法可以处理...

python regex beautifulsoup

回答 1 投票 0

beautifulsoup 相关问题

最新问题