mitchelljy / trending-youtube-scraper Goto Github PK

Python script that scrapes the currently trending YouTube videos in a variety of countries

License: BSD 2-Clause "Simplified" License

Python 100.00%

trending-youtube-scraper's Introduction

Trending YouTube Video Scraper

Originally used to build this dataset on Kaggle, which has about 6 months worth of trending YouTube videos on it. This script will scrape the most relevant information from videos that are currently trending on YouTube in a specified set of countries. You can find example output files in the output directory.

Trending YouTube videos for whatever country you are in currently can be found here.

Prerequisites

API Key

In order to use this script, you will need a valid API key for the YouTube Data API. It is free and the instructions for doing so are here. It is slightly awkward to get a key, but if you follow the instructions you should be ok.

Once you have the key, put it inside a text file named api_key.txt in the same directory as the script, or if it's not in the same directory you can target it with the --key_path parameter.

Modules

The only module needed that is not in the standard library is the requests module.

Country Codes

In order to run, the script needs country codes for the countries to collect trending videos from. These are 2 letter country abbreviations according to ISO 3166-1. A list of all existing ones can be found here, however not all of these are assured to work with the YouTube API. This project comes with a list of 10 inside the country_codes.txt file.

Running the Script

The script is fairly simple to run, it takes the following optional parameters:

--key_path which takes a path argument that targets the text file containing your API key. By default this is api_key.txt in the current directory.
--country_code_path which takes a path argument that targets the text file containing the list of country codes to target. By default this is country_codes.txt in the current directory.
--output_dir which takes a path argument that specifies the folder to create the output CSV files for each country. By default this is output/ in the current directory.

License

This project is licensed under the BSD 2-Clause License - see the LICENSE.md file for details

trending-youtube-scraper's People

Contributors

Stargazers

Watchers

Forkers

xiaoyan3938 aryans09 sundarsv2 dmohanty2 luminfan ryanamsden wassil-dev lilinformat clayll nu-ree marcusrb furkantasel kobyo qizhu8 raihan2108 101001110 chen8023 wosow kyleyamamoto smdp2000 martyp233 mudmoh mokoo2 mjserrato wangtianjiao502 devianpctek geekbeans xxxtlin yfviar rockinhumingbird pawel717-minadzd sophiejjjj fedialined ryzbaka mauricio-urrego ppcmaverick gasbarroni8 jakcieliao methawiphokhai aditchanchal ismayilofficial umw0lverine homestylegravy venkat1017 9harshit sleepyeepanda wsunshine nmichellenguyen danmgs bazickoff petersimeth haziq9978 jyothiramilla cafew leenadamle rpmullig acjin21 pragyakarki fdcaut anjanibhattar codetcode dhanushthakkalapally divyanshi-bansal mlinhlinh711 dsstudent1 anantaramesh seovalue krishnapsrinivasan ight84 bhumi1810 shinroo akarsh17 arseniysky pcwin12 chloengnguyen nickrwu ayoo5 wooodhead smileyfresh cyshen11 jerrytaohz prject20 minxee howitdube geo750 mario-kart-felix zahidna user1-ds matveynikon msdiheb15 miggy070498 gokmen-54 adriz-nf sipa91 hckynumber9 samnar2017 lookhei saturn1101 5l1v3r1 joploume

trending-youtube-scraper's Issues

Data for CategoryId by Country

What script, if any, did you use to compile VideoCategories, based on Youtube's CategoryId, which are said to be Region or Country Specific?

Would appreciate if that could also be included on future release, or commits.
Thank you.

Can't figure out how to pull trending data of a past date

As the title explains, I am trying to pull the trending data for each day of 2019, but I have yet to figure out how to use the API for it. Any thoughts?

Data not being written to csv file

The JSON files being requested appear to be outdated. I am receiving no data written to these files, aside from default variables. Please see the attached image for reference. Image of Issue

CSV Output Files are Empty

I tried running the script today and while it did complete successfully, all the CSVs that it created were empty except for the headers.. I put my API key in the provided text file. The country codes are the same with the addition of PH for the Philippines.

Blank Output Files

Hi,

I came across this repo from my colleague. After I obtained a YouTube API Key and placed it in api_key.txt and ran the script, I got blank output files. However, I am not sure which part I did wrong because it did not return any error message.

thanks.

Dataset from Italy

I tried to get a dataset from Italy, but it seems not working. Can you help me please? It's for a project in my university

How to get deal with new "like" and "dislike" from youtube?

Now the scraper can't find the likeCount nor the dislikeCount in any video from YouTube...
I guess this is happening due to the new version of YouTube in 2023, as the scraper can't find the likeCount and dislikeCount in statistics.

There is any fix for this issue?
Thanks in advance!

Can't add content to the file

I've run this script on my Ubuntu but the output file just shows headers.
I have also changed my locale by setting hl=vi&regionCode=VN.
What need i do?

"No filter selected"

Hi,

I get this when using print(request.json()):

{u'error': {u'code': 400, u'message': u'No filter selected. Expected one of: myRated, idParam, id, chart', u'errors': [{u'locationType': u'parameter', u'domain': u'youtube.parameter', u'message': u'No filter selected. Expected one of: myRated, idParam, id, chart', u'reason': u'missingRequiredParameter', u'location': u''}]}}

Any idea what to do?

Scrap trending videos from the past

Hi,

I came across this youtube scraper from kaggle and think it's the perfect script to get relevant information about current trending videos on youtube. However, I was wondering if there's a way to get trending videos from the past? The kaggle dataset is not up to date and I would like to create a new dataset for a project that I'm doing for school. Is there any way to do that with this script?

How to get the json file as well

I ran the script and I got the csv files, but I wonder if I can get the json files as well?