如何将值附加到全局数组中从不同的解析方法中提取值?

问题描述 投票:0回答:1
import requests
import win32api
import scrapy
from scrapy.spider import BaseSpider
from scrapy.selector import HtmlXPathSelector
from scrapy.http import Request
from array import *
import itertools
import numpy


class firstSpider(BaseSpider):
  name = "vdfgedu"
  allowed_domains = ["de.vdfg.edu"]
  start_urls = ["https://de.vdfg.edu/nephrology/node/1608"]
  globalarray = []
  def parse(self, response):
  #find all the hrefs first
      depts_with_info = []
      for dept_info in response.css('div.pane-content'):
               depts = {}
               links = dept_info.css('table.views-table tbody tr.viewsrow-first td a::attr(href)').extract()
               for link in links:
                       yield scrapy.Request("https://de.vdfg.edu"+link, callback = self.title_info)
                       yield scrapy.Request("https://de.vdfg.edu"+link+"#group-tabs-node-course-default3", callback = self.venue_info)
               def title_info(self, response):
                     titleinfo = {}
                     titleinfo['title'] = response.css('div.field-item h1::text').extract()
                     titleinfo['cmepoints'] = response.css('div.item-list ul.course-credit-list li span::text').extract()[0].strip()
                    self.globalarray.append(titleinfo)
              def venue_info(self, response):
                   venueinfo = {}
                   venueinfo['venue'] = response.css('div.adr span::text').extract()
                   self.globalarray.append(venueinfo)

我想将值附加到一个全局数组,并从两个不同的方法进入各自的hrefs进入excel表。如何声明全局数组?我们可以从多种解析方法中添加vaules吗?

python-2.7 scrapy
1个回答
0
投票

您可以创建数据类成员,该成员可以在类中的所有方法中用作全局值。

class firstSpider(BaseSpider):
    def __intit__(self):
        self.globalarray = []
© www.soinside.com 2019 - 2024. All rights reserved.