如何使用tweepy和twitter REST API获得具有特定主题标签的每条推文

Question

对于数据可视化项目，我需要收集带有特定主题标签的所有tweet（是否有可能？）。为此，我使用下面的代码。它使用Tweepy和REST API。但是，它最多只能下载约2500条以下的推文。我想知道如何解决此限制。是否有专业订阅或我应该购买的其他产品，或者我应该如何修改代码。

#!/usr/bin/python
# -*- coding: utf-8 -*-
# this file is configured for rtl language and farsi characters

import sys
from key import *
import tweepy

#imported from the key.py file
API_KEY =KAPI_KEY
API_SECRET =KAPI_SECRET
OAUTH_TOKEN =KOAUTH_TOKEN
OAUTH_TOKEN_SECRET =KOAUTH_TOKEN_SECRET

auth = tweepy.AppAuthHandler(API_KEY, API_SECRET)

api = tweepy.API(auth, wait_on_rate_limit=True,
                 wait_on_rate_limit_notify=True)

if not api:
    print("Can't Authenticate")
    sys.exit(-1)

def write_unicode(text, charset='utf-8'):
    return text.encode(charset)

searchQuery = "#کرونا"  # this is what we're searching for
maxTweets = 100000  # Some arbitrary large number
tweetsPerQry = 100  # this is the max the API permits
fName = 'Corona-rest8.txt'  # We'll store the tweets in a text file.

sinceId = None

max_id = -1
tweetCount: int = 0
print("Downloading max {0} tweets".format(maxTweets))
with open(fName, "wb") as f:
    while tweetCount < maxTweets:
        try:
            if max_id <= 0:
                if not sinceId:
                    new_tweets = api.search(q=searchQuery, count=tweetsPerQry)
                else:
                    new_tweets = api.search(q=searchQuery, count=tweetsPerQry,
                                            since_id=sinceId)
            else:
                if not sinceId:
                    new_tweets = api.search(q=searchQuery, count=tweetsPerQry,
                                            max_id=str(max_id - 1))
                else:
                    new_tweets = api.search(q=searchQuery, count=tweetsPerQry,
                                            max_id=str(max_id - 1),
                                            since_id=sinceId)
            if not new_tweets:
                print("No more tweets found")
                break
            for tweet in new_tweets:

                #print(tweet._json["created_at"])
                if str(tweet._json["user"]["location"])!="":
                    print(tweet._json["user"]["location"])
                myDict = json.dumps(tweet._json["text"], ensure_ascii=False).encode('utf8')+ "\n".encode('ascii')
                f.write(myDict)

            tweetCount += len(new_tweets)
            print("Downloaded {0} tweets".format(tweetCount))
            max_id = new_tweets[-1].id
        except tweepy.TweepError as e:
            # Just exit if any error
            print("some error : " + str(e))
            break
print("Downloaded {0} tweets, Saved to {1}".format(tweetCount, fName))

Answer 1

tweepy API Reference for api.search()为此提供了一些颜色：

[请注意，Twitter的搜索服务，以及扩展的Search API，并不意味着详尽的Tweets来源。并非所有推文都可以通过搜索界面编入索引或可用。

要直接回答您的问题，不可能从API获取详尽的推文列表（由于一些限制）。但是，可以使用一些基于抓取的Python库来解决这些API限制，例如@taspinar的tweepy。