Code Monkey home page Code Monkey logo

youtube-dl-server's Introduction

youtube-dl-server

(A modified version of manbearwiz's youtube-dl-server. This project is still in the beta stages, so deploy it at your own risk.)

As of 2020/08/27, I will only be working on this project occasionally. I still want to update it, but the new school year brings new (educationally oriented) different projects that will take up more of my time. If you would like to help out, you can try to add code to this! Thanks! Updates will be coming out occasionally, and hopefully this will be done soon.

What is new in this version?

  • You can now specify where to download the videos on the server you are downloading to, which helps simplifiy adding videos to media servers such as Plex or Jellyfin.
  • Built in metadata tagging. The downloader will now apply the appropriate metadata to media you download (artist/author/title) so that you dont need to deal with tagging everything once its downloaded. This also helps simplify adding videos to media servers.
  • All of the files are hosted locally. Previously, youtube-dl-server reached out to CDNs on the internet for web assets, however this new version has everything included locally. This means that you don't need to be reliant on external CDNs and you are in full control of your files.
  • This new version has error pages. The old version had no error pages, which can lead to some user confusion in the instance of a server side error.
  • /q has been renamed to /queue so that users know what page they are at, instead of being confused by what "q" means.
  • A download progess page where you can view the status of pending, current, past, and failed downloads (the history page at /history). This page refreshes live, so the status of your downloads will update every 3 seconds.
  • Authentication by local user accounts, so that you can restrict who is using the application and prevent unauthorized people from using your bandwidth to download videos.
  • Specific error messages and better error handling.
  • Added the ability to "subscribe" to channels/playlists, where the subscription daemon that runs in the background every x hours will download new videos in the playlist/channel. The subscriptions can be added through the /subscriptions webpage and it can be configured to run every x hours by using crontab, or another scheduling program.
  • You can use proxies for downloading videos at the home page. Currently, you can add as many proxies as you want, however I haven't been able to test this feature out. This is an experimental feature.

What is coming?

In the works right now:

  • Transferring all the form inputs to ajax requests.

In the future:

  • Docker images (planning on supporting Raspis).
  • The ability to administrate the program via Systemctl.
  • Support for downloading videos with captions (currently having issues with this, help would be appreciated).
  • Support for (pleasant) mobile device viewing (currently the only mobile devices that this app works on are tablets).

How do I set this up?

Required Programs (the method of installation vaires by distro):

Install the required packages with apt-get:

sudo apt-get install ffmpeg python3 python3-pip

Install the requrired modules with pip:

python3 -m pip install -r requirements.txt

Non-Docker Install Instructions:

Pre setup-warning: The user this program is running under should have r/w access to EVERY directory that is being referenced. That means the current directory you are installing in, and any external server download directories you are downloading to.

  1. Run setup.py with Python>=3.6 (below 3.6 isn't tested yet). Make sure to use a strong password for your admin account, to ensure that nobody can log on without your permission.
  2. Once you have ran the setup program, without an error, run the Flask application by running gunicorn3 --workers 4 --threads 4 --bind 0.0.0.0:8080 wsgi:app. You can change the host to 127.0.0.1 if you only want the application to work on your computer, but running it as 0.0.0.0 allows others to access the app. You can also change the port from 8080 to something else; 8080 is just the default (warning: port 80 may already be taken by your Apache installation).

Docker Install Instructions:

Use the provided docker-compose.yml file (with your modifications or run directly from the command line).
Docker-compose absolutely sucks and is causing heaps of issues. You'll have to manually build the docker image and run that.

  1. docker build -t katznboyz1/youtube-dl-server:latest .
  2.  docker run -d -p 8080:8080 \
     -v /path/to/downloaded/vids:/app/downloads \
     -v /path/to/database:/app/db \
     -e APPNAME=YDS \
     -e ADMINUSER=admin \
     -e PASSWORD=youtube \
     -e TZ=Australia/Melbourne \
     katznboyz1/youtube-dl-server:latest
    

The environment variables are:

  • APPNAME - the name you want the application to have (default: YDS)
  • ADMINUSER - the name of the admin user (default: admin)
  • PASSWORD - the password for the admin user (default: youtube)
  • TZ - for the timezone. Not actually sure if this is going to work.

And the volumes are:
# Where downloaded videos should go
- /opt/youtubedownloads:/app/downloads
# Where the database is
- /opt/database:/app/db

The docker image exposes port 8080 for the webserver.

Having an issue?

Leave an issue on the official repo!

Known issues:

  • Downloading certain videos with the ultra high format will lead to a missing codec error if you play it on windows.

Want to contribute?

There are some things that still need to be added, and if you want to help out it would be appreciated! Here is a list of some of the needed bits:

  • A new favicon for the application
  • Known issue fixes
  • Unknown issue fixes

Disclaimers

  • I am not a network security professional. If you run this application exposed to the internet, then you run it at your own risk. Do not use common/reused passwords for this application. I am a singular person, and there may be bugs in this program. Do not allow it to fail badly by not following common sense.
  • I am not in charge of how people use this application. I created this application for people to use, however the way that people may use it does not reflect on my original intentions/beliefs.
  • Any damages caused by this application to any party are not the responsibility of me as the creator, and they are either the responsibility of the person hosting the app, or the user of the app.
  • READ THE LICENSE

youtube-dl-server's People

Contributors

flatline-84 avatar plaidstallion avatar renamedquery avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

youtube-dl-server's Issues

[BUG] Sites other than YouTube not supported albeit supported by YouTube-dl

So I ran the server as my service user from terminal to try and see what’s happening as I noticed only YouTube vids download.

Had a look at the code earlier and wondered if you catered for other sites..

See three log snippets,
YouTube vid:

[youtube]_ CgLMOq3-mn4: Downloading webpage
(11, '3 Ways to Prevent Hurricanes (Maybe)', 'https://www.youtube.com/watch?v=CgLMOq3-mn4', 2, 1596146837.317174, 'best', './downloads', './downloads', '#none')
[youtube] CgLMOq3-mn4: Downloading webpage
[download] Destination: ./downloads/1596146837.38775.mp4

[download] 0.0% of 53.79MiB at 179.62KiB/s ETA 05:08

Vimeo vid:

[vimeo] 294427028: Downloading webpage
(10, 'funny moments', 'https://vimeo.com/294427028', 2, 1596146635.494384, 'best', './downloads', './downloads', '#none')
[vimeo] 294427028: Extracting information
[vimeo] 294427028: Downloading JSON metadata
[vimeo] 294427028: Downloading JSON metadata
[vimeo] 294427028: Checking Original video URL
[vimeo] 294427028: Downloading akfire_interconnect_quic m3u8 information
[vimeo] 294427028: Downloading akfire_interconnect_quic m3u8 information
[vimeo] 294427028: Downloading akfire_interconnect_quic MPD information
[vimeo] 294427028: Downloading akfire_interconnect_quic MPD information
[2020-07-31 00:04:12 +0200] [9241] [INFO] Handling signal: winch
[2020-07-31 00:04:15 +0200] [9241] [INFO] Handling signal: int
[2020-07-31 00:04:15 +0200] [9247] [INFO] Worker exiting (pid: 9247)
[2020-07-31 00:04:15 +0200] [9246] [INFO] Worker exiting (pid: 9246)
[2020-07-31 00:04:15 +0200] [9245] [INFO] Worker exiting (pid: 9245)
[2020-07-31 00:04:15 +0200] [9244] [INFO] Worker exiting (pid: 9244)
Error downloading video id 10.
Error parsing the directory "".
Error parsing the directory "".
[2020-07-31 00:04:16 +0200] [9241] [INFO] Shutting down: Master

YouTube-dL direct from terminal same Vimeo (interrupted download once started):

youtube-dl https://vimeo.com/294427028
[vimeo] 294427028: Downloading webpage
[vimeo] 294427028: Extracting information
[vimeo] 294427028: Downloading JSON metadata
[vimeo] 294427028: Downloading JSON metadata
[vimeo] 294427028: Checking Original video URL
[vimeo] 294427028: Downloading akfire_interconnect_quic m3u8 information
[vimeo] 294427028: Downloading akfire_interconnect_quic m3u8 information
[vimeo] 294427028: Downloading akfire_interconnect_quic MPD information
[vimeo] 294427028: Downloading akfire_interconnect_quic MPD information
[download] Sleeping 30 seconds...
[dashsegments] Total fragments: 10
[download] Destination: /downloads/sorted/youtube-dl/funny moments.fdash-akfire_interconnect_quic_sep-video-1119207424.mp4
[download] 40.0% of ~24.32MiB at 2.91MiB/s ETA 00:10^C
ERROR: Interrupted by user

Thanks for the help thus far. I’d find the bug and post a fix but I’m kinda waiting for week-end to work on side projects.

[INFO] Missing application dependencies

Please add the requirement for the following to get it working: (its mentioned in passing and i didn't see an auto dependency retriever thing so worth mentioning instead of trying to read why its failing)

  • gunicorn3
    apt install gunicorn3

  • flask_session
    pip3 install flask_session

Also, user executing needs write access to app and download folders (but this is obvious, for setting up as a service).

Use yt-dlp instead of youtube-dl

How can I switch the download from youtube-dl to yt-dlp?
Is there a re-call into the code to change the directory where the .py search the youtube-dl to download videos/songs?

[Feature]Manually edit metadata

Hey, would it be possible to have the ability to edit the metadata such as author/title etc. before downloading? It could even populate the fields with metadata gathered from the video, then allow the user to edit as necessary. This would be especially helpful with videos from other sites that do not have metadata to get.

[Bug] exec: "./startup.sh": permission denied: unknown.

After building, I try to run the container and no matter the permissions or if I run docker as sudo I get the following error:

docker: Error response from daemon: OCI runtime create failed: container_linux.go:370: starting container process caused: exec: "./startup.sh": permission denied: unknown.

I did a git clone to /opt and then cd'd in to youtube-dl-server and ran the build command. I did a chmod +x startup.sh and then tried to run the container but no luck. Any suggestions would be appreciated.

My run command is as follows:

sudo docker run -dit --name yds --rm -p 8010:8080 \
 -v /opt/youtube-dl-server/downloads:/app/downloads \
 -v /opt/youtube-dl-server/db:/app/db \
 -e APPNAME=YDS \
 -e ADMINUSER=$USER \
 -e PASSWORD=$PW \
 -e TZ=$TZ \
 katznboyz1/youtube-dl-server:latest

[Improvement] Improve subscription-daemon.py by adding a flag to youtube-dl object

Subscription-daemon.py uses youtube_dl.extract_info to get all the videos info in a playlist. The data is redundant for downloaded videos.

https://github.com/katznboyz1/youtube-dl-server/blob/1b958d63afd82c4aa29539540b35a0d37c12ee24/subscription-daemon.py#L81

We can improve the process by adding 'extract_flat': True flag to the youtube-dl options
https://github.com/l1ving/youtube-dl/blob/ab6ade71e5ef7060db209dd89a139f655fcfa328/youtube_dl/YoutubeDL.py#L226

import youtube_dl
ydl_opts = {
    'default_search':'youtube',
    'extract_flat': True,
}
youtubeDLObject = youtube_dl.YoutubeDL(ydl_opts)
playlistOrChannelData = youtubeDLObject.extract_info('https://www.youtube.com/playlist?list=PLOJU8YJjFwGOeOx32q45CjrAq4hAOpxBk', download=False)
print(playlistOrChannelData)

[BUG] Gunicorn install issue

Hey, Just an FYI really. I recently got this running in a docker container, however I had to install gunicorn via pip rather than apt-get so that it could find the other modules required (listed in the requirements.txt file.) Not sure if others had any issue, but thought you may want to update the install instructions.
Cheers

[Feature] Pass youtube-dl commands

It would be nice to have a config file for youtube-dl. Passing cookies for example allows downloading age restricted videos, and ignoring errors is another important one.

One issue I also found was Flask timeouts which I fixed with --threads 4 (threads and workers must be same).

[BUG] Mobile site menu item hides behind Title

So when you visit site on chrome mobile (ios) and expand hamburger menu on the history page, it hides behind the title page. Also the floating menu doesn't have a solid colour so it looks like it bleeds into background which can make it hard to read.

This appear to happen for the History page which has a table header. For the other pages, the page shifts down (iOS -Phone & iPad with Chrome,)

[Feature] Failure log / info stored per failed download and accessible from web

Is it possible to capture a log of the download and store in history if the download fails?

So i tried a few downloads and worked.. then i tried a few that failed (not youtube) but couldn't tell why. Would be great for troubleshooting as not all video sites work with Youtube-dl and you where the default fails, you need to supply add a extension.

Additionally if the site requires a username/password, you won't know that the login fails as there is no log to indicate it.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.