使用 Selenium 和/或 BS4 获取 a href 文本值和 h3 文本值

Question

我尝试从这个 html 文件中获取此处包含的文本值：

<div>
<div> <div class="py d-flex align-items-start align-items-lg-center css-17435dd row" data-test="salaries-list-item-0" data-brandviews="MODULE:n=salaries-search-salaries-by-company:eid=7927:uid=0:salary_job_title_id=37856" data-triggered-brandview=""><div class="col-12 col-md-3 col-lg-3"><div class="d-flex" data-test="employer-info"><div class="employerLogo mr"><a href="/Salary/Infosys-Project-Manager-United-States-Salaries-EJI_IE7927.0,7_KO8,23_IL.24,37.htm" data-test="salaries-list-item-0-employer-url"><img alt="Infosys" src="https://media.glassdoor.com/sql/7927/infosys-squareLogo-1620208556721.png" class="css-1rc0f2z e1aj7ssy1"></a></div><div class="employerStats"><a href="/Salary/Infosys-Project-Manager-United-States-Salaries-EJI_IE7927.0,7_KO8,23_IL.24,37.htm" data-test="salaries-list-item-0-employer-url" class="css-1ikln7a el6ke052">Infosys</a><div class="d-flex align-items-center mt-xxsm"><span class="css-h9sogr m-0 css-60s9ld el6ke050" color="#0caa41">3.2</span><span data-color-variant="primary" data-size-variant="sm" class="gd-ui-star css-h9sogr ml-xxsm css-ojdxng efdhmtv0" role="presentation"><svg aria-hidden="true" class="css-pq72fl e7xsrz90" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><defs><path id="star_svg__star-multicolor-icon-path" d="m12 2.694 2.119
    6.857.109.352h7.294L15.898 14.2l-.277.212.103.333 2.135 6.909-5.556-4.244-.303-.232-.303.232-5.556 4.244 2.135-6.909.103-.333-.277-.212-5.624-4.296h7.295l.108-.352L12 2.694Z"></path><clipPath id="star_svg__star-multicolor-icon-clip"><use href="#star_svg__star-multicolor-icon-path"></use></clipPath></defs><use href="#star_svg__star-multicolor-icon-path" clip-path="url(#star_svg__star-multicolor-icon-clip)" fill="var(--icon-color-a, currentColor)" stroke="var(--icon-color-b, currentColor)" stroke-width="var(--icon-stroke-width, 1px)"></use></svg></span></div><span class="d-flex align-items-middle css-1in2cw4 el6ke050">Project Manager<span class="SVGInline"><svg class="SVGInline-svg" xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24"><path d="M12 9a1 1 0 101 1 1 1 0 00-1-1zm0 3a1 1 0 00-1 1v2a1 1 0 002 0v-2a1 1 0 00-1-1zm0-6a6 6 0 11-6 6 6 6 0 016-6zm0 11a5 5 0 10-5-5 5 5 0 005 5z" fill="currentColor" fill-rule="evenodd"></path></svg></span></span><a class="m-0 d-flex css-259095 e1aj7ssy6" href="/Salary/Infosys-Project-Manager-Salaries-EI_IE7927.0,7_KO8,23.htm" data-test="salaries-list-item-0-salary-count"><span>See 23,491 salaries from this location</span></a><div class="d-block d-lg-none"><a href="/Jobs/Infosys-Jobs-E7927.htm" data-test="salaries-list-item-0-open-jobs-link" class="css-259095 e1aj7ssy6"><span>1,744 open jobs</span></a></div></div></div></div><div class="col-12 col-md-3 col-lg-3 d-none d-lg-block align-self-start"><div class="d-flex" data-test="employer-info"><a href="/Jobs/Infosys-Jobs-E7927.htm" data-test="salaries-list-item-0-open-jobs-link" class="css-259095 e1aj7ssy6"><span>1,744 open jobs</span></a></div></div><div data-test="range-bar" class="col-12 col-lg-6 order-1 order-md-2 order-lg-1 pt-xl pt-lg-0"><div class="d-flex flex-column"><div class="order-0 mt-xl css-1ex4vn2 e1hxjh2q1"><div class="css-1xxwalr e1hxjh2q3"><div class="css-79elbk e13r6qcv0"><div class="mb-xxsm d-flex align-items-baseline css-3tpw1n e13r6qcv1"><h3 class="m-0 css-16zrpia el6ke054">$118,568</h3><span class="m-0 css-1in2cw4 el6ke050">&nbsp;/&nbsp;yr</span></div></div></div></div><div class="order-2 css-79elbk e1hxjh2q0" style="visibility: visible; min-height: 24px;"><div class="css-osz7jh e13r6qcv4"><div class="css-79elbk e1hxjh2q0"><div class="css-ituc4g e13r6qcv5"><div class="d-flex col css-79elbk e1hxjh2q0"><div class="d-flex flex-column align-items-end css-15o6gsn e13r6qcv5"><span class="m-0 css-1in2cw4 el6ke050">$104K</span></div><div class="d-flex flex-column align-items-end css-1qfy6mj e13r6qcv5"><span class="m-0 css-1in2cw4 el6ke050">$135K</span></div></div></div></div></div></div><div class="order-1 my-xxsm css-psber1 e13r6qcv3"><div class="css-32y0el e13r6qcv2"></div><div class="css-x87ns5 e13r6qcv4"></div></div></div></div></div>

到目前为止，我尝试使用 BS4，但结果值为“无”：

source = BeautifulSoup(driver.page_source,'html.parser')

data_test_param = 'salaries-list-item-'+str(x)+'-employer-url'
a_value = source.find('a', attrs={'data-test': data_test_param})

h3_value = source.find_all('h3',{'class':"m-0 css-16zrpia el6ke054"})

请您指教一下吗？

Answer 1

查看 HTML，您可以执行以下操作：

from bs4 import BeautifulSoup

html_text = """\
<div>
<div> <div class="py d-flex align-items-start align-items-lg-center css-17435dd row" data-test="salaries-list-item-0" data-brandviews="MODULE:n=salaries-search-salaries-by-company:eid=7927:uid=0:salary_job_title_id=37856" data-triggered-brandview=""><div class="col-12 col-md-3 col-lg-3"><div class="d-flex" data-test="employer-info"><div class="employerLogo mr"><a href="/Salary/Infosys-Project-Manager-United-States-Salaries-EJI_IE7927.0,7_KO8,23_IL.24,37.htm" data-test="salaries-list-item-0-employer-url"><img alt="Infosys" src="https://media.glassdoor.com/sql/7927/infosys-squareLogo-1620208556721.png" class="css-1rc0f2z e1aj7ssy1"></a></div><div class="employerStats"><a href="/Salary/Infosys-Project-Manager-United-States-Salaries-EJI_IE7927.0,7_KO8,23_IL.24,37.htm" data-test="salaries-list-item-0-employer-url" class="css-1ikln7a el6ke052">Infosys</a><div class="d-flex align-items-center mt-xxsm"><span class="css-h9sogr m-0 css-60s9ld el6ke050" color="#0caa41">3.2</span><span data-color-variant="primary" data-size-variant="sm" class="gd-ui-star css-h9sogr ml-xxsm css-ojdxng efdhmtv0" role="presentation"><svg aria-hidden="true" class="css-pq72fl e7xsrz90" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><defs><path id="star_svg__star-multicolor-icon-path" d="m12 2.694 2.119
    6.857.109.352h7.294L15.898 14.2l-.277.212.103.333 2.135 6.909-5.556-4.244-.303-.232-.303.232-5.556 4.244 2.135-6.909.103-.333-.277-.212-5.624-4.296h7.295l.108-.352L12 2.694Z"></path><clipPath id="star_svg__star-multicolor-icon-clip"><use href="#star_svg__star-multicolor-icon-path"></use></clipPath></defs><use href="#star_svg__star-multicolor-icon-path" clip-path="url(#star_svg__star-multicolor-icon-clip)" fill="var(--icon-color-a, currentColor)" stroke="var(--icon-color-b, currentColor)" stroke-width="var(--icon-stroke-width, 1px)"></use></svg></span></div><span class="d-flex align-items-middle css-1in2cw4 el6ke050">Project Manager<span class="SVGInline"><svg class="SVGInline-svg" xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24"><path d="M12 9a1 1 0 101 1 1 1 0 00-1-1zm0 3a1 1 0 00-1 1v2a1 1 0 002 0v-2a1 1 0 00-1-1zm0-6a6 6 0 11-6 6 6 6 0 016-6zm0 11a5 5 0 10-5-5 5 5 0 005 5z" fill="currentColor" fill-rule="evenodd"></path></svg></span></span><a class="m-0 d-flex css-259095 e1aj7ssy6" href="/Salary/Infosys-Project-Manager-Salaries-EI_IE7927.0,7_KO8,23.htm" data-test="salaries-list-item-0-salary-count"><span>See 23,491 salaries from this location</span></a><div class="d-block d-lg-none"><a href="/Jobs/Infosys-Jobs-E7927.htm" data-test="salaries-list-item-0-open-jobs-link" class="css-259095 e1aj7ssy6"><span>1,744 open jobs</span></a></div></div></div></div><div class="col-12 col-md-3 col-lg-3 d-none d-lg-block align-self-start"><div class="d-flex" data-test="employer-info"><a href="/Jobs/Infosys-Jobs-E7927.htm" data-test="salaries-list-item-0-open-jobs-link" class="css-259095 e1aj7ssy6"><span>1,744 open jobs</span></a></div></div><div data-test="range-bar" class="col-12 col-lg-6 order-1 order-md-2 order-lg-1 pt-xl pt-lg-0"><div class="d-flex flex-column"><div class="order-0 mt-xl css-1ex4vn2 e1hxjh2q1"><div class="css-1xxwalr e1hxjh2q3"><div class="css-79elbk e13r6qcv0"><div class="mb-xxsm d-flex align-items-baseline css-3tpw1n e13r6qcv1"><h3 class="m-0 css-16zrpia el6ke054">$118,568</h3><span class="m-0 css-1in2cw4 el6ke050">&nbsp;/&nbsp;yr</span></div></div></div></div><div class="order-2 css-79elbk e1hxjh2q0" style="visibility: visible; min-height: 24px;"><div class="css-osz7jh e13r6qcv4"><div class="css-79elbk e1hxjh2q0"><div class="css-ituc4g e13r6qcv5"><div class="d-flex col css-79elbk e1hxjh2q0"><div class="d-flex flex-column align-items-end css-15o6gsn e13r6qcv5"><span class="m-0 css-1in2cw4 el6ke050">$104K</span></div><div class="d-flex flex-column align-items-end css-1qfy6mj e13r6qcv5"><span class="m-0 css-1in2cw4 el6ke050">$135K</span></div></div></div></div></div></div><div class="order-1 my-xxsm css-psber1 e13r6qcv3"><div class="css-32y0el e13r6qcv2"></div><div class="css-x87ns5 e13r6qcv4"></div></div></div></div></div>"""

soup = BeautifulSoup(html_text, "html.parser")

for h3 in soup.select("h3"):
    url = h3.find_previous("a")["href"]
    amount = h3.text

    print(url)
    print(amount)
    print()

打印：

/Jobs/Infosys-Jobs-E7927.htm
$118,568