cuducos / twitter-cleanup Goto Github PK

View Code? Open in Web Editor NEW

288.0 12.0 24.0 69 KB

🛁 Clean-up inactive accounts and bots from your Twitter

Home Page: https://pypi.org/project/twitter-cleanup/

License: GNU General Public License v3.0

Python 100.00%

twitter bots maintenance social-media social-network

twitter-cleanup's Introduction

Twitter Clean-up

🇧🇷 Versão em português do Brasil

Tiny script to clean-up your Twitter account:

Removing users that have not been tweeting for a while
Soft-blocking bots (blocks and immediately unblocks the account, so it stops following you)

Requirements

Python 3.6+
Set environment variables with your Twitter API keys and with Botometer API key:
- TWITTER_CONSUMER_KEY
- TWITTER_CONSUMER_SECRET
- TWITTER_ACCESS_TOKEN_KEY
- TWITTER_ACCESS_TOKEN_SECRET
- BOTOMETER_MASHAPE_KEY

Installing

Install the package with:

$ pip install twitter-cleanup

Usage

Run the CLI with twitter-cleanup --help and follow the on screen instructions.

For example, unfollow everyone that hasn't tweeted in the last 30 days with:

$ twitter-cleanup inactive 30

Or soft-block every bot with:

$ twitter-cleanup bots

Contributing

Please, write and run tests locally, and format your code with Black:

$ python setup.py test
$ black .

twitter-cleanup's People

Contributors

Stargazers

Watchers

twitter-cleanup's Issues

soft_block_bots error: account has no tweet published

>>> import cleanup
>>> abc = cleanup.Cleanup()
>>> abc.soft_block_bots()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/avelino/src/github.com/cuducos/twitter-cleanup/cleanup.py", line 129, in soft_block_bots
    if user.is_bot():
  File "/home/avelino/src/github.com/cuducos/twitter-cleanup/cleanup.py", line 72, in is_bot
    if self.protected or not self.botometer_result:
  File "/home/avelino/src/github.com/cuducos/twitter-cleanup/cleanup.py", line 67, in botometer_result
    result = self.botometer.check_account(self.id)
  File "/home/avelino/.virtualenvs/cuducos.twitter-cleanup/lib/python3.6/site-packages/botometer/__init__.py", line 128, in check_account
    raise NoTimelineError(payload['user'])
botometer.NoTimelineError: user '{'id_str': '994214676652347393', 'screen_name': 'eryk_azevedo'}' has no tweets in timeline

Write tests

Probably a good way to learn abouk unittest.mock (for Portuguese speakers, I have a YouTube video about mocks).

Make it pip-installable and a CLI

As described in comments in #4, this would be a nice way to install and run this package:

$ pip install twitter-cleanup
$ twitter-cleanup --days 30 --botometer 0.68 --yes-for-all

This would avoid Python interactive mode and make it seamless to be run automatically from times to times (as a cronjob, for example).

unfollow_inactive_for error: AttributeError: 'User' object has no attribute 'status'

[y/n] y
Unfollowed planet_lisp
Confirm unfollow clojuredconf?ts

Last tweet was 2 months ago:

Phew, :clojureD 2018 is now really over. All photos and videos published. All bills payed. Website reseted.
We defi… https://t.co/NKHANgNpZ5


[y/n] n
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/avelino/src/github.com/cuducos/twitter-cleanup/cleanup.py", line 105, in unfollow_inactive_for
    if user.last_status_before(**kwargs):
  File "/home/avelino/src/github.com/cuducos/twitter-cleanup/cleanup.py", line 57, in last_status_before
    if not self.status:
AttributeError: 'User' object has no attribute 'status'
>>>

SyntaxError: invalid syntax ao iniciar

Ao tentar iniciar a ferramenta, sou apresentado ao seguinte erro:

File "cleanup.py", line 91
yield from (User.parse(self.api, user._json) for user in users)
^
SyntaxError: invalid syntax

Vale ressaltar que ao executar pip install -r requirements.txt eu tive que ignorar o pacote six. Estou tentando rodar no MacOS High Sierra, com Python 2.7.10

Create a new release and publish to PyPI

The last PyPI release for this package was over a year ago and doesn't work for me currently (#23). A new release would be appreciated 😃

Syntax error calling twitter-cleanup --help

twitter-cleanup --help
Traceback (most recent call last):
  File "C:\Users\klauer\scoop\apps\python\current\scripts\twitter-cleanup-script.py", line 11, in <module>
    load_entry_point('twitter-cleanup==0.0.2', 'console_scripts', 'twitter-cleanup')()
  File "c:\users\klauer\scoop\apps\python\3.7.0\lib\site-packages\pkg_resources\__init__.py", line 480, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "c:\users\klauer\scoop\apps\python\3.7.0\lib\site-packages\pkg_resources\__init__.py", line 2693, in load_entry_point
    return ep.load()
  File "c:\users\klauer\scoop\apps\python\3.7.0\lib\site-packages\pkg_resources\__init__.py", line 2324, in load
    return self.resolve()
  File "c:\users\klauer\scoop\apps\python\3.7.0\lib\site-packages\pkg_resources\__init__.py", line 2330, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "c:\users\klauer\scoop\apps\python\3.7.0\lib\site-packages\twitter_cleanup\__init__.py", line 3, in <module>
    from tweepy import API, Cursor
  File "c:\users\klauer\scoop\apps\python\3.7.0\lib\site-packages\tweepy\__init__.py", line 17, in <module>
    from tweepy.streaming import Stream, StreamListener
  File "c:\users\klauer\scoop\apps\python\3.7.0\lib\site-packages\tweepy\streaming.py", line 358
    def _start(self, async):
                         ^
SyntaxError: invalid syntax

change "iactive" to "inactive"

twitter-cleanup/setup.py

Line 18 in f44698f

description="Clean-up iactive accounts and bots from your Twitter",

tweepy limit rate error

It would be interesting to deal with the error, I believe this to be a common error for accounts that follows many users

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/avelino/src/github.com/cuducos/twitter-cleanup/cleanup.py", line 104, in unfollow_inactive_for
    for user in self.following:
  File "/home/avelino/src/github.com/cuducos/twitter-cleanup/cleanup.py", line 90, in following
    for users in Cursor(self.api.friends).pages():
  File "/home/avelino/.virtualenvs/cuducos.twitter-cleanup/lib/python3.6/site-packages/tweepy/cursor.py", line 49, in __next__
    return self.next()
  File "/home/avelino/.virtualenvs/cuducos.twitter-cleanup/lib/python3.6/site-packages/tweepy/cursor.py", line 75, in next
    **self.kargs)
  File "/home/avelino/.virtualenvs/cuducos.twitter-cleanup/lib/python3.6/site-packages/tweepy/binder.py", line 250, in _call
    return method.execute()
  File "/home/avelino/.virtualenvs/cuducos.twitter-cleanup/lib/python3.6/site-packages/tweepy/binder.py", line 232, in execute
    raise RateLimitError(error_msg, resp)
tweepy.error.RateLimitError: [{'message': 'Rate limit exceeded', 'code': 88}]

Environmental Variables not found

After Installation I set up the variables as environmental variables in system properties, as well as in the .env file. ( I might have misplaced the .env file in the wrong directory)

I am getting this error:
C:\Users\Jorge>twitter-cleanup --help Traceback (most recent call last): File "C:\Users\Jorge\AppData\Local\Programs\Python\Python36\Scripts\twitter-cleanup-script.py", line 11, in <module> load_entry_point('twitter-cleanup==0.0.5', 'console_scripts', 'twitter-cleanup')() File "c:\users\jorge\appdata\local\programs\python\python36\lib\site-packages\pkg_resources\__init__.py", line 487, in load_entry_point return get_distribution(dist).load_entry_point(group, name) File "c:\users\jorge\appdata\local\programs\python\python36\lib\site-packages\pkg_resources\__init__.py", line 2728, in load_entry_point return ep.load() File "c:\users\jorge\appdata\local\programs\python\python36\lib\site-packages\pkg_resources\__init__.py", line 2346, in load return self.resolve() File "c:\users\jorge\appdata\local\programs\python\python36\lib\site-packages\pkg_resources\__init__.py", line 2352, in resolve module = __import__(self.module_name, fromlist=['__name__'], level=0) File "c:\users\jorge\appdata\local\programs\python\python36\lib\site-packages\twitter_cleanup\__init__.py", line 5, in <module> from twitter_cleanup.authentication import authentication File "c:\users\jorge\appdata\local\programs\python\python36\lib\site-packages\twitter_cleanup\authentication.py", line 37, in <module> authentication = Authentication() File "c:\users\jorge\appdata\local\programs\python\python36\lib\site-packages\twitter_cleanup\authentication.py", line 12, in __init__ self.consumer_key = config("TWITTER_CONSUMER_KEY") File "c:\users\jorge\appdata\local\programs\python\python36\lib\site-packages\decouple.py", line 197, in __call__ return self.config(*args, **kwargs) File "c:\users\jorge\appdata\local\programs\python\python36\lib\site-packages\decouple.py", line 85, in __call__ return self.get(*args, **kwargs) File "c:\users\jorge\appdata\local\programs\python\python36\lib\site-packages\decouple.py", line 70, in get raise UndefinedValueError('{} not found. Declare it as envvar or define a default value.'.format(option)) decouple.UndefinedValueError: TWITTER_CONSUMER_KEY not found. Declare it as envvar or define a default value.

Retry Botometer on HTTP error

It is not rare to have the bot detection flow interrupted by HTTP error such as:

Traceback (most recent call last):
  File "/Users/cuducos/.virtualenvs/twitter-cleanup/bin/twitter-cleanup", line 11, in <module>
    load_entry_point('twitter-cleanup', 'console_scripts', 'twitter-cleanup')()
  File "/Users/cuducos/.virtualenvs/twitter-cleanup/lib/python3.6/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/Users/cuducos/.virtualenvs/twitter-cleanup/lib/python3.6/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/Users/cuducos/.virtualenvs/twitter-cleanup/lib/python3.6/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/cuducos/.virtualenvs/twitter-cleanup/lib/python3.6/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/cuducos/.virtualenvs/twitter-cleanup/lib/python3.6/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/Users/cuducos/.virtualenvs/twitter-cleanup/lib/python3.6/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/Users/cuducos/Dropbox/Projects/twitter-cleanup/twitter_cleanup/__main__.py", line 36, in bots
    context.obj["cleanup"].soft_block_bots(threshold=threshold / 100)
  File "/Users/cuducos/Dropbox/Projects/twitter-cleanup/twitter_cleanup/__init__.py", line 91, in soft_block_bots
    should_soft_block = user.is_bot()
  File "/Users/cuducos/Dropbox/Projects/twitter-cleanup/twitter_cleanup/user.py", line 35, in is_bot
    return self.botometer_result.probability > threshold
  File "/Users/cuducos/Dropbox/Projects/twitter-cleanup/twitter_cleanup/botometer.py", line 45, in probability
    self._get_result()
  File "/Users/cuducos/Dropbox/Projects/twitter-cleanup/twitter_cleanup/botometer.py", line 32, in _get_result
    result = self.botometer.check_account(self.user_id)
  File "/Users/cuducos/.virtualenvs/twitter-cleanup/lib/python3.6/site-packages/botometer/__init__.py", line 132, in check_account
    bom_resp.raise_for_status()
  File "/Users/cuducos/.virtualenvs/twitter-cleanup/lib/python3.6/site-packages/requests/models.py", line 940, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 502 Server Error: Proxy Error for url: https://osome-botometer.p.mashape.com/2/check_account

In addition to cache (so we could restart the process without re-requesting Botometer's probability for users we have already requested), we could automatically retry this self.botometer.check_account(self.user_id) in case of a server error, since it seams to be an error typical from instability and not a request error.

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "cleanup.py", line 130, in soft_block_bots
    if user.is_bot():
  File "cleanup.py", line 72, in is_bot
    if self.protected or not self.botometer_result:
  File "cleanup.py", line 67, in botometer_result
    result = self.botometer.check_account(self.id)
  File "/Users/mateus.pontes/.virtualenvs/twitter-cleanup/lib/python3.6/site-packages/botometer/__init__.py", line 128, in check_account
    raise NoTimelineError(payload['user'])
botometer.NoTimelineError: user '{'id_str': '1013973434743885824', 'screen_name': 'ElieneO85015636'}' has no tweets in timeline
>>> cleanup.soft_block_bots(threshold=0.68)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "cleanup.py", line 130, in soft_block_bots
    `threshold` (defaults to 0.75 in User class)."""
  File "cleanup.py", line 72, in is_bot
    self._botometer_result = result.get("cap", {}).get("universal")
  File "cleanup.py", line 67, in botometer_result
    try:
  File "/Users/mateus.pontes/.virtualenvs/twitter-cleanup/lib/python3.6/site-packages/botometer/__init__.py", line 128, in check_account
    raise NoTimelineError(payload['user'])
botometer.NoTimelineError: user '{'id_str': 'XXXXXXXXXX', 'screen_name': 'XXXXXXXXXX'}' has no tweets in timeline

AttributeError: 'NoneType' object has no attribute 'tweepy'

Fresh installation, with the environmental variables set:

❯ twitter-cleanup inactive 365
Traceback (most recent call last):
  File "/usr/local/bin/twitter-cleanup", line 8, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1256, in invoke
    Command.invoke(self, ctx)
  File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/click/decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/twitter_cleanup/__main__.py", line 14, in cli
    context.obj = {"cleanup": TwitterCleanup(assume_yes=yes)}
  File "/usr/local/lib/python3.8/dist-packages/twitter_cleanup/__init__.py", line 16, in __init__
    self.api = API(auth.tweepy, wait_on_rate_limit=True)
AttributeError: 'NoneType' object has no attribute 'tweepy'

It seems that auth is not an object, but just None.