Scrapy列表选择器

问题描述 投票:0回答:1

我试图在scrapy中迭代一个列表,这是html示例:

<html>
 <head>
  <base href='http://example.com/' />
  <title>Example website</title>
 </head>
 <body>
  <div id='images'>
   <a href='image1.html'>Name: My image 1 <br /><img src='image1_thumb.jpg' /></a>
   <a href='image2.html'>Name: My image 2 <br /><img src='image2_thumb.jpg' /></a>
   <a href='image3.html'>Name: My image 3 <br /><img src='image3_thumb.jpg' /></a>
   <a href='image4.html'>Name: My image 4 <br /><img src='image4_thumb.jpg' /></a>
   <a href='image5.html'>Name: My image 5 <br /><img src='image5_thumb.jpg' /></a>
  </div>
  <ol class="jobs">      
    <li class="job ">
        <div id="logoso-listing">
            <img width="100" height="100" class="alignleft size-thumbnail wp-image-22824">             
        </div>
        <div id="titlo">
            <strong><a href="http://link/">Text1</a></strong>
        </div>
        <div id="type-tag"><span class="jtype permanent">Permanent1</span></div>
        <div id="type-tag-prev"><span class="jtype permanent">Permanent1</span></div>
    </li>
    <li class="job ">
        <div id="logoso-listing">
            <img width="100" height="100" class="alignleft size-thumbnail wp-image-22824">             
        </div>
        <div id="titlo">
            <strong><a href="http://link/">Text2</a></strong>
        </div>
        <div id="type-tag"><span class="jtype permanent">Permanent2</span></div>
        <div id="type-tag-prev"><span class="jtype permanent">Permanent2</span></div>
    </li>
    <li class="job ">
        <div id="logoso-listing">
            <img width="100" height="100" class="alignleft size-thumbnail wp-image-22824">             
        </div>
        <div id="titlo">
            <strong><a href="http://link/">Text3</a></strong>
        </div>
        <div id="type-tag"><span class="jtype permanent">Permanent3</span></div>
        <div id="type-tag-prev"><span class="jtype permanent">Permanent3</span></div>
    </li>
</ol>
 </body>
</html>

这些是scrapy爬虫的命令:

content = response.xpath("//ol[@class = 'jobs']")
job_list = content.xpath("//li[contains(@class,'job')]")
for job in job_list:
    job.xpath("//div[@id = 'titlo']/strong/a/text()").getall()

上面的代码返回:

['Text1', 'Text2', 'Text3']
['Text1', 'Text2', 'Text3']
['Text1', 'Text2', 'Text3']

虽然我的预期输出是:

['Text1']
['Text2']
['Text3']

我没有得到什么?这种垂直迭代,而我想以横向方式迭代response

python web-scraping scrapy
1个回答
0
投票

为什么不使用带有strong标签的a标签的css选择器?

.css("strong a").getall()

你应该添加id

#titlo strong a

对于xpath

//*[@id='titlo']/strong/a
© www.soinside.com 2019 - 2024. All rights reserved.