qut-digital-observatory / youte Goto Github PK
View Code? Open in Web Editor NEWCommand-line utility to help researchers collect video metadata from Youtube API
Home Page: https://youte.readthedocs.io
License: MIT License
Command-line utility to help researchers collect video metadata from Youtube API
Home Page: https://youte.readthedocs.io
License: MIT License
Add a --limit
to specify how many pages of search results to retrieve
I thought it was a bit puzzling I have to specify --tidy-to and -o. They seem exclusive. Or at least if writing to the raw json is required, then one might as well make a tidy command which converts the json to the desired format.
At any rate, the following command just locked up Youte for 20 minutes at 100% CPU (until I Ctrl-C'd it) after downloading comments for 50 odd videos. It finished the download, it seems, filled comments_untidied.json and created comments_tidied.json but wrote nothing in it.
youte comments --by-video-id --file-path video_ids.txt --format json -o comments_untidied.json --tidy-to comments_tidied.json
The relatedToVideoId
parameter retrieves a list of videos that are related to the video that the parameter value identifies. The parameter value must be set to a YouTube video ID and, if you are using this parameter, the type parameter must be set to video.
Note that if the relatedToVideoId parameter is set, the only other supported parameters are part, maxResults, pageToken, regionCode, relevanceLanguage, safeSearch, type (which must be set to video), and fields.
Requirements list in setup.cfg is out of date, and in trying to find and install the requirements, I've discovered there's a problem with the dotenv package, which is sadly no longer installable and seems to be unmaintained?
Here are some things I'd recommend to sort through this and make similar issues easier to catch in future:
youtupy >= 0.0.1
) - this helps with troubleshooting. Just start with the versions you have currently installedI have an error issue...
So I try refresh a new API (v3)
.../pipx/venvs/youte/lib/python3.11/site-packages/pydantic/_internal/_config.py:317: UserWarning: Valid config keys have changed in V2:
* 'orm_mode' has been renamed to 'from_attributes'
warnings.warn(message, UserWarning)
INFO | Getting API key from config file.
INFO | Getting page 1
^C
Aborted!
Do i need to upgrade pydantic
?
My youte version...
youte --version
youte, version 2.4.1
Thanks
I noticed that the like_count for some channels and videos was NA, and it turns out this is correct (channels have like count hidden, and videos don't seem to have like count showing on YouTube), but it might confuse people. I have a minor suggestion to put a note about this in the documentation just so people understand what's happening ๐ธ
I have a video.id
text file with the following ID as the only line: 1BCmx_ICbRU
, corresponding to a video with 163 comments: https://www.youtube.com/watch?v=1BCmx_ICbRU
If I run the following command, I just get the same page of 100 comments retrieved over and over again, with no termination:
youtupy list-comments video.id knitting_1BCmx_ICbRU.json -v
I would expect this command to return very quickly as there should only be 2 pages of results in total.
CSV, Excel, TSV
Add an option to export each of processed tables, or join them together
especially when the date range is large and there are potential more results than can be contained in YouTube's standard 13 result pages
list-comments
and hydrate
currently require item IDs to be in a text file. Making it possible to manually add the IDs straight into the terminal would make it easier to quickly hydrate and list comments without putting IDs in a text file
Make it easier to know which history db is associated with which file by default, to make resuming clearer.
something like archive
to extract search results, video and channel metadata, and comments, and put all in an SQL database
First, thank you for all your incredible work on youte - it's been really useful.
I wanted to flag a minor issue with video IDs that start from dashes (for example, -Q7G5zfSal8
). I would like to collect videos related to this video, but youte interprets it as an argument. I tried adding another dash or using quotation marks but could not get it to be treated as a value for the first positional argument. Instead, I always get an error Error: No such option: -Q
.
Any help or advice would be much appreciated!
The allThreadsRelatedToChannelId parameter instructs the API to return all comment threads associated with the specified channel. The response can include comments about the channel or about the channel's videos.
The channelId parameter instructs the API to return comment threads containing comments about the specified channel. (The response will not include comments left on videos that the channel uploaded.)
Currently, when using the --get-id
flag of the search
function to retrieve IDs only, a .json
or .jsonl
file will be returned. It would be a great to have the IDs exported/saved as .txt
or .csv
so that this file could then be used later with the hydrate
function.
Allow the ability to filter searches by language and location
The chart
parameter identifies the chart that you want to retrieve.
Acceptable values are:
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.