parallel-python-twitter's Introduction

parallel-python-twitter

A client that distributes Twitter API requests across multiple keys. Built for Python 3.6.

Getting Started

First, get your API credentials together and get a list of twitter.Api objects:

import parallel_twitter
TWITTER_API_CONSUMER_KEY = ...
TWITTER_API_CONSUMER_SECRET = ...
OAUTHS = [
    {
        'oauth_token': ...,
        'oauth_token_secret': ...
    },
    {
        'oauth_token': ...,
        'oauth_token_secret': ...
    },
    ...
]
apis = parallel_twitter.oauth_dicts_to_apis(
    oauth_dicts=OAUTHS,
    api_consumer_key=TWITTER_API_CONSUMER_KEY,
    api_consumer_secret=TWITTER_API_CONSUMER_SECRET
)

Next, try out some of the examples:

user_ids = [
    561808704, # @neeljsomani
    813286, # @BarackObama
    17919972 # @taylorswift13
]
# Get a list of posts that these users liked
parallel_twitter.examples.pull_users_likes(
    users=user_ids,
    apis=apis
)
# Get a list of these users' posts + number of likes
parallel_twitter.examples.pull_users_posts(
    users=user_ids,
    apis=apis
)

Comparison with Twint

Twint is a Python library for scraping data from Twitter.

From Twint's documentation: "Twitter limits scrolls while browsing the user timeline. This means that with .Profile or with .Favorites you will be able to get ~3200 tweets." There are no such limits if you distribute your requests using parallel-python-twitter.
Twitter will block your requests if you scrape enough (ex: twintproject/twint#682). I've tested parallel-python-twitter up to 100s of megabytes.
Technically, twint violates Twitter's Terms of Service, since scraping is not permitted in general.

parallel-python-twitter's People

Contributors

Stargazers

Watchers

parallel-python-twitter's Issues

GetFollowersID

Hi, I've read this code and found that a function is not defined:

class GetFollowerIDs(TwitterOp):
    """
    An operator to get the IDs of the users that are following the requested
    user.
    """
    def _invoke(self,
                user_id: Optional[int] = None,
                screen_name: Optional[str] = None,
                cursor: int = -1,
                max_count: Optional[int] = None) -> Tuple[int, int, List[int]]:
        """
        Return the users that are following the specified user.
        Parameters
        ----------
        user_id : Optional[int]
            The Twitter ID of the specified user
        screen_name : Optional[str]
            The Twitter handle of the specified user
        cursor : int
            Cursor to identify the page to pull, starting at -1
        max_count : Optional[int]
            The maximum number of friends to return. Defaults to 5000.
        """
        return self.api.GetFollowerIDsPaged(
            user_id=user_id,
            screen_name=screen_name,
            cursor=cursor,
            count=max_count
        )

I am not sure where is GetFollowerIDsPaged defined. Also I tried run get_followers and could not get a large list.

Recommend Projects