Code Monkey home page Code Monkey logo

peerstohttp's Introduction

peerstohttp

FOSSA Status Go Report Card

Simple torrent proxy to http stream controlled over REST-like api

Built top on anacrolix/torrent lib.

HTTP API

URL Parameters
  • playlist - output file format, one of these values: m3u,html,json
  • hash - torrent info hash. Example: 08ada5a7a6183aae1e09d831df6748d566095a10
  • extsWhitelist - list of whitelisted file extensions. Possible values: "-" (any) or list extension names divided by comma. Examples: "-", "mp3,mp4a"
  • tagsBlacklist - list of blacklisted tags, extracted from file names. Possible values: "-" (no filter) or list tags divided by comma. See /playlist/tags.go for full list of possible tags. Examples: "-", "remix,interview"

Get list of files by magnet uri:

GET http://localhost/list/{playlist}/{extsWhitelist}/{tagsBlacklist}/magnet/{magnetURI}

Get list of files by info hash:

GET http://localhost/list/{playlist}/{extsWhitelist}/{tagsBlacklist}/hash/{hash}

Download file:

GET http://localhost/content/{hash}/{filePath}

Examples

Get HTML links list for Sintel by torrent hash:

http://localhost/list/html/mp4/-/hash/08ada5a7a6183aae1e09d831df6748d566095a10

or by magnet URI:

http://localhost/list/html/mp4/-/magnet/magnet:?xt=urn:btih:08ada5a7a6183aae1e09d831df6748d566095a10&dn=Sintel&tr=udp%3A%2F%2Fexplodie.org%3A6969&tr=udp%3A%2F%2Ftracker.coppersurfer.tk%3A6969&tr=udp%3A%2F%2Ftracker.empire-js.us%3A1337&tr=udp%3A%2F%2Ftracker.leechers-paradise.org%3A6969&tr=udp%3A%2F%2Ftracker.opentrackr.org%3A1337&tr=wss%3A%2F%2Ftracker.btorrent.xyz&tr=wss%3A%2F%2Ftracker.fastcast.nz&tr=wss%3A%2F%2Ftracker.openwebtorrent.com&ws=https%3A%2F%2Fwebtorrent.io%2Ftorrents%2F&xs=https%3A%2F%2Fwebtorrent.io%2Ftorrents%2Fsintel.torrent

Then watch Sintel.mp4 using VLC video player:

vlc http://localhost/content/08ada5a7a6183aae1e09d831df6748d566095a10/Sintel/Sintel.mp4

Or open m3u playlist in VLC video player:

vlc http://localhost/list/m3u/mp4/-/hash/08ada5a7a6183aae1e09d831df6748d566095a10

Generate m3u files from list of magnets using wget and bash script:

makelist.sh
#!/bin/bash

magnets="magnets.txt"
playlist_dir="playlists" 

host="localhost"
port="8484"

exts="mp3,mp4,m4a,m4,flac"
exclude_tags="live,interlude,demo,orchestral,orch,remix,mix,acoustic,instr,instrumental,synthesis,bonus,heavy"

rm -f $playlist_dir/*m3u*

while read line; do
     wget --content-disposition --directory-prefix=$playlist_dir --tries=3 "http://$host:$port/list/m3u/$exts/$exclude_tags/magnet/$line"
done < $magnets

Docker

Run container with exposed 8484 http port:

docker run --volume downloads:/downloads -it -p 8484:80 winpooh32/peerstohttp:latest -dir=/downloads

Build steps

utp dependency requires C compiler, then install it:

Download:

go get -v -u github.com/WinPooh32/peerstohttp

Build in vendor mode:

go build -mod=vendor -o peerstohttp

Install:

go install -i github.com/WinPooh32/peerstohttp/cmd

Run:

go run github.com/WinPooh32/peerstohttp/cmd -port=8484 -dir="/path/to/download"
  • Usually, $GOPATH is "~/go";
  • Default dir value is system tmp folder, for listing all possible options run $ ./peerstohttp -help.

License

FOSSA Status

peerstohttp's People

Contributors

a5r0n avatar fossabot avatar pataquets avatar winpooh32 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

peerstohttp's Issues

Add Heroku Deploy Button

hey it will be nice to have heroku deploy button, so we can test application using your preferred parameters

TOR/Proxy Support?

Hi there. Super nice repo you've made! Just wondering if there's anything implemented for running it through TOR or a Proxy? If not, I will have a go at implementing it and could make a PR or feature branch if you or anyone's interested. Would just need to test first if running thru TOR would make it too slow and therefore unusable.

Cheers,

Feature request: Append configurable fixed string to added magnet URIs (e.g. trackers)

Since trackers come and go and some magnet URIs don't even have trackers built-in, adding some well-known (or even private) trackers would speed up stream bootstrap and discovery.

This could be accomplished at add time by just automatically appending some fixed string passed at service start, containing some bootstraping, well-known trackers encoded in a URI parameter string. As you can't anticipate which URIs will need it and which trackers each user would find best, it would be a nice addition to let the user configure their own string with some sort of -magnetAddString &tr=... switch.

Notice I'm requesting a fixed generic string, which can be of use for adding trackers (which somewhat overlaps #22), but it might be used for other purposes, as long as it is kept generic enough. It could also be used to bootstrap the torrent from a well-known peer by adding a x.pe URI parameter, for instance (see BEP 9).

"no Go files in" error

how to build this properly , i get no go files

go get -v -u github.com/WinPooh32/peerstohttp
github.com/WinPooh32/peerstohttp (download)
package github.com/WinPooh32/peerstohttp: no Go files in /home/voo/go/src/github.com/WinPooh32/peerstohttp

3 more concurrent downloads cause problems

Hi, I had been downloading 3 different files separately at the same time. It printed as follow

2020/12/17 10:15:14 http: Accept error: accept tcp 116.102.186.113:443: accept4: too many open files; retrying in 5ms
2020/12/17 10:15:14 http: Accept error: accept tcp 116.102.186.113:443: accept4: too many open files; retrying in 10ms
2020/12/17 10:15:14 http: Accept error: accept tcp 116.102.186.113:443: accept4: too many open files; retrying in 20ms
2020/12/17 10:15:14 http: Accept error: accept tcp 116.102.186.113:443: accept4: too many open files; retrying in 40ms

2020-12-17T10:16:02+0800 NONE torrent.go:1831: disallowing data download
2020-12-17T10:16:02+0800 NONE torrent.go:1831: disallowing data download
2020-12-17T10:16:02+0800 NONE torrent.go:1831: disallowing data download
2020-12-17T10:16:02+0800 NONE torrent.go:1831: disallowing data download
2020-12-17T10:16:02+0800 NONE torrent.go:1831: disallowing data download
2020-12-17T10:16:02+0800 NONE torrent.go:1831: disallowing data download
2020-12-17T10:16:02+0800 NONE torrent.go:1831: disallowing data download
2020-12-17T10:16:02+0800 NONE torrent.go:1831: disallowing data download
2020-12-17T10:16:02+0800 NONE torrent.go:1831: disallowing data download
2020-12-17T10:16:02+0800 NONE torrent.go:1831: disallowing data download
2020-12-17T10:16:02+0800 NONE torrent.go:1831: disallowing data download

Feature request: expose default added trackers or allow disabling them

Looks like there is a hardcoded list of trackers that are always added to torrents here.

Some thoughts:

  • It would be great to be able to customize them, so you can worry only adding torrent hashes and they'll get eventually discovered always. I always add some reliable trackers and even for some period of time, one which I ran locally, too.
  • If not customizable, the "local trackers if available" might be tunned for some more commonplace, such as udp://localhost:6969, udp://localhost:2710 or the likes.
  • (Not sure, needs more reading of spec) It might present privacy/disclosure problems for private torrents which AFAIK, should not be announced anywhere.

Also, not sure about how would work customization, since everything is done in command line and passing the list might be cumbersome, but would work anyway, I guess. Depending on the requirements, it might be "low hanging fruit".

html5 video tag

hi, i'm trying to directly embed the url magnet format without going into list mode and copying the link , how could i automatically output the filename to video tag.

something like this

> <video width="320" height="240" controls>
>   <source src="http://localhost/list/html/mp4/-/magnet/etc " type="video/mp4">
> Your browser does not support the video tag.
> </video>
> 

instead of using the generated output linke example
http://localhost/content/08ada5a7a6183aae1e09d831df6748d566095a10/Sintel/Sintel.mp4

cpu load almost 100% all the time

I kept on querying data by using the list interface, after about 20 more , I opened 3 windows to play different videos. I found the cpu load was 100%. I closed all the 3 video players. the cpu load was still 100%. Then I waited for an hour, It still kept the high level. And it could not go down to normal again.

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1596 root 20 0 9116272 2.5g 17268 S 726.7 8.0 283:20.38 cmd

dies randomly

I've been having a problem were it just dies while "streaming" a video, How can I run this as a daemon and have it restart if/when it dies?

SQLite3 dependency fails to build for GOOS=windows GOARCH=386

#!/bin/bash -eo pipefail
GOOS=windows GOARCH=386 go build -mod=vendor -o /tmp/artifacts/peerstohttp.win.i386.exe ./cmd/...

# zombiezen.com/go/sqlite
vendor/zombiezen.com/go/sqlite/session.go:68:20: undefined: sqlite3.Xsqlite3session_create
vendor/zombiezen.com/go/sqlite/session.go:90:2: undefined: sqlite3.Xsqlite3session_delete
vendor/zombiezen.com/go/sqlite/session.go:725:37: undefined: sqlite3.SQLITE_CHANGESET_DATA
vendor/zombiezen.com/go/sqlite/session.go:726:37: undefined: sqlite3.SQLITE_CHANGESET_NOTFOUND
vendor/zombiezen.com/go/sqlite/session.go:727:37: undefined: sqlite3.SQLITE_CHANGESET_CONFLICT
vendor/zombiezen.com/go/sqlite/session.go:728:37: undefined: sqlite3.SQLITE_CHANGESET_CONSTRAINT
vendor/zombiezen.com/go/sqlite/session.go:729:37: undefined: sqlite3.SQLITE_CHANGESET_FOREIGN_KEY
vendor/zombiezen.com/go/sqlite/session.go:761:33: undefined: sqlite3.SQLITE_CHANGESET_OMIT
vendor/zombiezen.com/go/sqlite/session.go:765:34: undefined: sqlite3.SQLITE_CHANGESET_ABORT
vendor/zombiezen.com/go/sqlite/session.go:779:36: undefined: sqlite3.SQLITE_CHANGESET_REPLACE
vendor/zombiezen.com/go/sqlite/session.go:90:2: too many errors

Exited with code exit status 2

CircleCI received exit code 2

https://app.circleci.com/pipelines/github/WinPooh32/peerstohttp/67/workflows/31b5bb5f-e0ca-4a15-9d02-54f6d9474ca0/jobs/64/parallel-runs/0/steps/0-104

Feature request: understand special file names for torrents (eg. 'largest') in request path

When downloading video torrents, you have to get the files list first, and later figure out the appropriate file name (which will be different on every torrent).
As I used go-peerflix back in the day, I remembered it automatically picked the largest file from the torrent, which most of the time worked fine.
I wonder if some 'special' file names could be understood in the request path, such as :largest or similar for the example above, so no need to figure it manually and let the program apply some smartness.

Also, smells like to me that other special use cases might be solved with these "smart aliases":

  • Files by index
  • First file with some extension (playlists?)
  • ...

magnet URL issue ?

hello friends, i am trying to load some magnet urls i am unsure what might be the problem, i tried other magnet urls from other sites they all appear to work.

/list/html/mp4,mkv/-/magnet/magnet:?xt=urn:btih:YE6DIFKFJFCLBVORSSQNLZAXXCQ7A54M&dn=%5BSubsPlease%5D%20Spy%20Kyoushitsu%20-%2004%20%28480p%29%20%5B49DA9ACC%5D.mkv&xl=123692828&tr=http%3A%2F%2Fnyaa.tracker.wf%3A7777%2Fannounce&tr=udp%3A%2F%2Ftracker.coppersurfer.tk%3A6969%2Fannounce&tr=udp%3A%2F%2Ftracker.opentrackr.org%3A1337%2Fannounce&tr=udp%3A%2F%2F9.rarbg.to%3A2710%2Fannounce&tr=udp%3A%2F%2F9.rarbg.me%3A2710%2Fannounce&tr=udp%3A%2F%2Ftracker.leechers-paradise.org%3A6969%2Fannounce&tr=udp%3A%2F%2Ftracker.internetwarriors.net%3A1337%2Fannounce&tr=udp%3A%2F%2Ftracker.cyberia.is%3A6969%2Fannounce&tr=udp%3A%2F%2Fexodus.desync.com%3A6969%2Fannounce&tr=udp%3A%2F%2Ftracker3.itzmx.com%3A6961%2Fannounce&tr=udp%3A%2F%2Ftracker.torrent.eu.org%3A451%2Fannounce&tr=udp%3A%2F%2Ftracker.tiny-vps.com%3A6969%2Fannounce&tr=udp%3A%2F%2Fretracker.lanta-net.ru%3A2710%2Fannounce&tr=http%3A%2F%2Fopen.acgnxtracker.com%3A80%2Fannounce&tr=wss%3A%2F%2Ftracker.openwebtorrent.com

with error Bad Request

a example link from the site

magnet:?xt=urn:btih:UH2ASS6KN25SYJAVQNSINB4H2O3N4R6X&dn=%5BSubsPlease%5D%20Revenger%20-%2004%20%28480p%29%20%5BCC1067A2%5D.mkv&xl=378292348&tr=http%3A%2F%2Fnyaa.tracker.wf%3A7777%2Fannounce&tr=udp%3A%2F%2Ftracker.coppersurfer.tk%3A6969%2Fannounce&tr=udp%3A%2F%2Ftracker.opentrackr.org%3A1337%2Fannounce&tr=udp%3A%2F%2F9.rarbg.to%3A2710%2Fannounce&tr=udp%3A%2F%2F9.rarbg.me%3A2710%2Fannounce&tr=udp%3A%2F%2Ftracker.leechers-paradise.org%3A6969%2Fannounce&tr=udp%3A%2F%2Ftracker.internetwarriors.net%3A1337%2Fannounce&tr=udp%3A%2F%2Ftracker.cyberia.is%3A6969%2Fannounce&tr=udp%3A%2F%2Fexodus.desync.com%3A6969%2Fannounce&tr=udp%3A%2F%2Ftracker3.itzmx.com%3A6961%2Fannounce&tr=udp%3A%2F%2Ftracker.torrent.eu.org%3A451%2Fannounce&tr=udp%3A%2F%2Ftracker.tiny-vps.com%3A6969%2Fannounce&tr=udp%3A%2F%2Fretracker.lanta-net.ru%3A2710%2Fannounce&tr=http%3A%2F%2Fopen.acgnxtracker.com%3A80%2Fannounce&tr=wss%3A%2F%2Ftracker.openwebtorrent.com

none of the magnet links working with peerstohttp, anime magnet links

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.