tweetl:crown:rd

tweetlord is an open source Twitter profile dumper (downloader) with the on-the-fly account swapping support for bypassing the rate limit restrictions. It is written in Python 3, uses the Twitter API and generates .xlsx files at the output containing comprehensive information about the given profile.

Table of Contents:

• Screenshots
• Dependencies
  • DEB Packages
  • PIP Packages
• Installation
• Usage
• Platform

Screenshots

From the table above it is seen why the progress bars in the 1st screenshot were not filled to the end: the actual number of items in each of the sections is less than it was specified in the arguments when running the tool. It's not a bug but a feature 😉 (one API request returns no more than 200 items so the pbar step = 200, btw).

Dependencies

DEB Packages

• python3.x (or newer) interpreter

PIP Packages

tweetlord makes use of the following external modules:

• tweepy
• simplejson
• xlsxwriter
• tqdm
• termcolor

Resolve all Python dependencies with the pip one-liner:

$ python3 -m pip install -r requirements.txt

Installation

The order of use is pretty straightforward:

1. ⚠️ First, you want to set your API keys (could be found here) in the credentials.py file for every Twitter account you want to involve in the procedure. It is needless to say that the more accounts you specify, the faster the dumping process will be (but nevertheless you can specify only one account). If a mistake is made when filling the credentials, the script will terminate with an unhandled tweepy exception, so keep that in mind.
2. Hmm... that's actually it! Feel free to run tweetlord as shown in the next section.

Usage

tweetlord.py [-h] (-u USER | -l) [-fr FRIENDS] [-fo FOLLOWERS]
             [-fa FAVORITES] [-ti TIMELINE] [-o OUTPUT] [-w] [-e] [-d]

required arguments:
  -u USER, --user USER    set the user profile you want to dump: <USER> could be a screen name or an account ID (if it is an ID, you should start the string with the "id" prefix, e. g. "id859377203242426368")
OR
  -l, --show-limits       show the rate limit status (total → remaining → time_to_wait_till_reset) for each of the accounts you set when configuring the tool

optional arguments:
  -fr N, --friends N      set the number of friends to be dumped (if N == -1 then tweetlord will try to dump all friends)
  -fo N, --followers N    set the number of followers to be dumped (if N == -1 then tweetlord will try to dump all followers)
  -fa N, --favorites N    set the number of favorite tweets to be dumped (if N == -1 then tweetlord will try to dump all favorite tweets)
  -ti N, --timeline N     set the number of tweets from user's timeline to be dumped (if N == -1 then tweetlord will try to dump all timeline tweets)
  -a, --all               dump ALL the sections with ALL the items in each of them
  -o NAME, --output NAME  set the output filename (".xlsx" ending will be added)
  -w, --wait-on-limit     sleep if the rate limit is exceeded (the sleeping time will be printed)
  -e, --tweet-extended    get the whole tweet text but not only the first 140 chars
  -d, --debug             debug mode (extra info messages will be show when exceptions are caught)
  -h, --help              show help

If there's no rate limit left and you have specified the -w flag, you can press Ctrl+C during the sleeping (waiting) process, but only during the waiting process (otherwise, you'll terminate the main program), to skip dumping current section and continue with the next one.

See more about the Twitter Rate Limiting.

Platform

tweetlord works great both on Windows and GNU/Linux systems, but the resulting .xlsx dump files look prettier when opened in MS Excel app rather than in LibreOffice (and this is quite obvious because LibreOffice doesn't fully support .xlsx format anyhow 😒).