notatallshaw / arandabot Goto Github PK
View Code? Open in Web Editor NEWA youtube to reddit bot based on youtube API v3
A youtube to reddit bot based on youtube API v3
It's a little unreliable, instead keep a list of videos IDs already posted.
Things to consider:
I think sometimes the Google API classes just stop responding as apposed to throwing an exception. The idea to this would to be put the calls in to a separate thread / subprocess and kill the thread after a set amount of time if nothing happens.
Maybe measuring "nothing happens" could be via a queue which is populated by the loops in the method.
I've been running the bot for a while and come across all these exceptions
lots of logging
config goes somewhere in settings
either push in to the clipboard or write to a file the user can open. Or use the functionality to open the browser directly if you can get that to work properly.
Once added logging add option to keep updating to show script is still working.
Allow API v2 (if it really isn't decommed), APIv3, RSS Feeds, to all run in parallel to get data fastest possible way
A users subscriptions may be updated while the script is running. Should allow to check if this has happened.
Currently got an error like this and not sure where it came from:
Some unexpected exception occured in arandabot backing off for 5 mins and trying
again:
'charmap' codec can't encode character u'\u2019' in position 38: character maps
to
Currently dependencies are noted in the readme.md
of this project, the more 'pythonic' and maintainable way to do this would be to pipe the requirements into a requirements.txt
file.
You generate this by running the following command:
pip freeze >requirements.txt
commit the requirements file to your repo.
Then anyone can install all your dependencies by running:
pip install -r requirements.txt
Add in to the settings
Message looks like this:
<HttpError 403 when requesting https://www.googleapis.com/youtube/v3/search?channelId=UCBa659QWEk1AI4Tg--mrJ2A&maxResults=50&safeSearch=none&part=snippet&alt=json&type=video&order=date returned "Quota Exceeded">
Back off for some time when this happens, print to screen why backing off for some time
Will need to depreciate password login to reddit in favor of ouath 2.0
Make it much easier to set up for the first time by adding an interactive walk through script, allowing user to put in username and password and channel configuration details of for the script to handle the saving to json
Low Priority
Noticed an issue where script just isn't pulling some subscribed videos when it's looping for a long time, need to investigate
This would add more resiliency as it seems less flaky than oauth
This will be used to populate all the default values and be kept in the git repository. If settings.json is not in the directory of the bot it will create it based of settings.tempate.json and user input from various questions.
settings.json will be added to the git ignore file so people's secret details (such as reddit username/password) aren't accidentally uploaded)
Use the new API v3 method: https://developers.google.com/youtube/v3/docs/commentThreads
Can use activities list and set home = true:
https://developers.google.com/youtube/v3/docs/activities/list
I don't 100% trust this will be updated with all channel content but it might be worth merging this feed in to the requests in case it's more performant in general?
Password based authentication is being removed:
https://www.reddit.com/comments/2ujhkr/
Add a timer or an input capture to allow user to read instructions
As per this SO answer, it is apparently much faster to use the search functionality than to use playlist lookup: http://stackoverflow.com/questions/29204271/are-upload-playlists-on-youtube-api-v3-purpsofully-slow-to-be-updated/29441811#29441811
Add more output capabilities, twitter?
Affects both main and search branch:
Traceback (most recent call last):
File "main.py", line 19, in
main()
File "main.py", line 16, in main
arandabot.arandabot(settings=settings)
File "C:\Users\Damian\Documents\arandabot - prod - search\arandabot.py", line46, in arandabot
yt.getNewestVideos()
File "C:\Users\Damian\Documents\arandabot - prod - search\ytvideos.py", line 251, in getNewestVideos
self.youtube = self.initilize_youtube(self.set)
File "C:\Users\Damian\Documents\arandabot - prod - search\ytvideos.py", line 120, in initilize_youtube
http=credentials.authorize(httplib2.Http()))
File "C:\Python27\lib\site-packages\oauth2client\util.py", line 132, in positional_wrapper
return wrapped(_args, *kwargs)
File "C:\Python27\lib\site-packages\apiclient\discovery.py", line 192, in build
resp, content = http.request(requested_url)
File "C:\Python27\lib\site-packages\oauth2client\util.py", line 132, in positional_wrapper
return wrapped(args, *kwargs)
File "C:\Python27\lib\site-packages\oauth2client\client.py", line 490, in new_request
redirections, connection_type)
File "C:\Python27\lib\site-packages\httplib2__init.py", line 1593, in request
(response, content) = self.request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
File "C:\Python27\lib\site-packages\httplib2__init.py", line 1335, in request
(response, content) = self.conn_request(conn, request_uri, method, body, headers)
File "C:\Python27\lib\site-packages\httplib2__init.py", line 1263, in _conn_request
raise ServerNotFoundError("Unable to find the server at %s" % conn.host)
httplib2.ServerNotFoundError: Unable to find the server at www.googleapis.com
Add lots of unit tests to help stop regressions.
Traceback (most recent call last):
File "main.py", line 19, in
main()
File "main.py", line 16, in main
arandabot.arandabot(settings=settings)
File "C:\Users\Damian\Documents\arandabot - prod\arandabot.py", line 46, in arandabot
yt.getNewestVideos()
File "C:\Users\Damian\Documents\arandabot - prod\ytvideos.py", line 246, in getNewestVideos
self.youtube = self.initilize_youtube(self.set)
File "C:\Users\Damian\Documents\arandabot - prod\ytvideos.py", line 119, in initilize_youtube
http=credentials.authorize(httplib2.Http()))
File "C:\Python27\lib\site-packages\oauth2client\util.py", line 132, in positional_wrapper
return wrapped(_args, *kwargs)
File "C:\Python27\lib\site-packages\apiclient\discovery.py", line 192, in build
resp, content = http.request(requested_url)
File "C:\Python27\lib\site-packages\oauth2client\util.py", line 132, in positional_wrapper
return wrapped(args, *kwargs)
File "C:\Python27\lib\site-packages\oauth2client\client.py", line 490, in new_request
redirections, connection_type)
File "C:\Python27\lib\site-packages\httplib2__init.py", line 1593, in request
(response, content) = self.request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
File "C:\Python27\lib\site-packages\httplib2__init.py", line 1335, in request
(response, content) = self.conn_request(conn, request_uri, method, body, headers)
File "C:\Python27\lib\site-packages\httplib2__init.py", line 1291, in _conn_request
response = conn.getresponse()
File "C:\Python27\lib\httplib.py", line 1061, in getresponse
raise ResponseNotReady()
httplib.ResponseNotReady
where the settings file is, where the client_secrets file is where to write the oauth file to etc...
Currently assumption is channel name or id will always exist, this could cause bot to get caught in long loop trying resolve non-existent name
Add a post:true post:false for each channel in the JSON, allowing not to just manually add channels but to blacklist certain channels from the subscription list.
Some channels don't have user names, e.g. https://www.youtube.com/channel/UCaVwpbqM8dkhQvbL8XileAA/featured
And make sure it's using the newest version of the code
A common cient_secrets.json will be kept in the git repository. It will be used unless there exists a user cient_secret*.json in the bot directory.
The code will ignore provided settings when using common client_secrets.json, it will limit caling youtube to once every 60 seconds. There will be no obfuscation, if some user chooses to break it, that is up to them.
Once the number of subscription channel requests has got very large it is very slow to loop around getting all the user upload playlists. Should add option to multi-thread this step.
Enable universal flag for python
o see if there is a file named in the style that you download from Google and if there's just 1 try that.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.