How do you use TwitterPager in TwitterAPI in Python to make multiple calls?

Question

I'm trying to do a search for tweets in Twitter's premium API. I copied the format in the TwitterAPI documentation into my current code, but I get the following error. What am I missing? I am fairly new to Python. I'm using Twitter's sandbox premium search API to search full history. I probably have a gap in knowledge on how get_iterator() is supposed to be utilized. Thanks in advance!

api = TwitterAPI(key, secretkey, token, secrettoken)

#search tweets #added edit to pass api through Twitter Pager
tweets = TwitterPager(api, 'tweets/search/%s/:%s' % (PRODUCT, LABEL), 
                      {'query': 'pizza',
                       'lang': 'en'
                    })

for item in tweets.get_iterator():
    print(item['text'] if 'text' in item else item)

And I get the following error:

---------------------------------------------------------------------------
TwitterRequestError                       Traceback (most recent call last)
<ipython-input-71-85784cf972fe> in <module>
----> 1 for item in tweets.get_iterator():
      2     print(item)

~\AppData\Local\Continuum\anaconda3\lib\site-packages\TwitterAPI\TwitterPager.py in get_iterator(self, wait, new_tweets)
     45                 start = time.time()
     46                 r = self.api.request(self.resource, self.params)
---> 47                 it = r.get_iterator()
     48                 if new_tweets:
     49                     it = reversed(list(it))

~\AppData\Local\Continuum\anaconda3\lib\site-packages\TwitterAPI\TwitterAPI.py in get_iterator(self)
    204         """
    205         if self.response.status_code != 200:
--> 206             raise TwitterRequestError(self.response.status_code)
    207 
    208         if self.stream:

TwitterRequestError: Twitter request failed (422)

My code works with basic 'api.request(...)' method using the same search parameters. but I need to use TwitterPager and get_iterator() in order to return more results.

Robert Kearns · Answer 1 · 2019-09-19T16:13:17.380

1

I am not familiar with the particular package you are using, but my guess is that you are not actually passing it the api object you have created. Therefore it is trying to perform its actions on your first argument(a string), which should actually be the api instance. Try this:

api = TwitterAPI(key, secretkey, token, secrettoken)

#search tweets
tweets = TwitterPager(api, 'tweets/search/%s/:%s' % (PRODUCT, LABEL), # added the api as first arg
                      {'query': 'pizza',
                       'lang': 'en'
                    })

for item in tweets.get_iterator():
    print(item['text'] if 'text' in item else item)

Edit

It looks like in these docs that you must send the api as the first argument to the pager.

edited Sep 19 '19 at 16:13

answered Sep 19 '19 at 16:08

Robert Kearns

1,631
1
8
15

Thanks, I didn't notice that detail in my code. However, I tried that and now receive the error: "Twitter Request failed (422)" – mochiroar Sep 19 '19 at 16:16
Impossible to say why without seeing what exactly you are sending to twitter. But based on twitters official API docs 422 usually aligns with a bad format in your request. Make sure everything is formatted exactly how it should be. – Robert Kearns Sep 19 '19 at 16:32
I should add that my code works with basic 'api.request(...)' method using the same search parameters. but I need to use TwitterPager and get_iterator() in order to return more results. – mochiroar Sep 19 '19 at 16:35
@mochiroar remove the `lang` parameter. It is not supported by Premium Search. See https://developer.twitter.com/en/docs/tweets/search/api-reference/premium-search for valid parameters. – Jonas Sep 25 '19 at 23:12

score 0 · Answer 2 · answered Sep 20 '22 at 03:11

See https://geduldig.github.io/TwitterAPI/paging.html

The example given shows:

r = TwitterPager(api, 'search/tweets', {'q':'pizza', 'count':100})
for item in r.get_iterator():
    if 'text' in item:
        print item['text']
    elif 'message' in item and item['code'] == 88:
        print 'SUSPEND, RATE LIMIT EXCEEDED: %s\n' % item['message']

You'll have to adjust the endpoint (i.e. search/tweets) and parameter dictionary (i.e., {'q':'pizza', 'count':100}) to fit what you want to call with the premium api

How do you use TwitterPager in TwitterAPI in Python to make multiple calls?

2 Answers2

Edit