Code Monkey home page Code Monkey logo

red_oxide's Introduction

red_oxide

CLI to help uploading to REDacted, inspired by REDBetter.

Installing

  1. Install intermodal and add it to your PATH
  2. Install lame, sox & flac and add them to your PATH
  3. download the latest release from here

Usage

CLI

Transcode subcommand (red_oxide transcode)

You have to specify api-key, torrent-directory, content-directory, transcode-directory & spectrogram-directory either via the config file or via the CLI

Transcode FLACs to other co-existing formats

Usage: red_oxide transcode [OPTIONS] [URLS]...

Arguments:
  [URLS]...  The Perma URLs (PL's) of torrents to transcode

Options:
      --debug
          If debug logs should be shown
  -a, --automatic-upload
          If the upload should be done automatically
      --concurrency <CONCURRENCY>
          How many tasks (for transcoding as example) should be run in parallel, defaults to your CPU count
      --api-key <API_KEY>
          The Api key from Redacted to use there API with
      --content-directory <CONTENT_DIRECTORY>
          The path to the directory where the downloaded torrents are stored
      --transcode-directory <TRANSCODE_DIRECTORY>
          The path to the directory where the transcoded torrents should be stored
      --torrent-directory <TORRENT_DIRECTORY>
          The path to the directory where the torrents should be stored
      --spectrogram-directory <SPECTROGRAM_DIRECTORY>
          The path to the directory where the spectrograms should be stored
  -c, --config-file <CONFIG_FILE>
          The path to the config file
  -f, --allowed-transcode-formats <ALLOWED_TRANSCODE_FORMATS>
          List of allowed formats to transcode to, defaults to all formats if omitted [possible values: flac24, flac, mp3320, mp3-v0]
  -m, --move-transcode-to-content
          If the transcode should be moved to the content directory, useful when you want to start seeding right after you upload
      --skip-hash-check
          If the hash check of the original torrent should be skipped, defaults to false, not recommended and if enabled done at own risk!
      --skip-spectrogram
          If the spectrogram check of the original torrent should be skipped, defaults to false, not recommended and if enabled done at own risk!
  -d, --dry-run
          If this is a dry run, no files will be uploaded to Redacted
  -h, --help
          Print help

Config file

This is useful if you don't want a super long CLI command and your configs do not change often, note that all the options can be specified via the CLI as well and are fully optional in this config file (will be merged with the CLI options if specified)

There are multiple default locations where the config file will be searched for, in this order (once found it will not look for the config file in the other locations):

  1. The path specified via the --config-file CLI option
  2. ./red_oxide.config.json (In the same folder as the red_oxide executable)
  3. %APPDATA%/red_oxide/red_oxide.config.json (only on Windows)
  4. $XDG_CONFIG_HOME/red_oxide/red_oxide.config.json
  5. HOME/.config/red_oxide/red_oxide.config.json
  6. HOME/red_oxide.config.json

HOME is determined by these environment variables on Windows in this order:

  1. %HOME%
  2. %USERPROFILE%
  3. %HOMEDRIVE%\%HOMEPATH%

HOME is determined by these environment variables on Linux in this order:

  1. $HOME
{
  "api_key": "YOUR_API_KEY",
  "torrent_directory": "FULL_PATH_WHERE_TORRENT_FILES_WILL_BE_STORED",
  "content_directory": "FULL_PATH_WHERE_CONTENT_IS_LOCATED",
  "transcode_directory": "FULL_PATH_WHERE_TRANSCODED_CONTENT_WILL_BE_PUT",
  "spectrogram_directory": "FULL_PATH_WHERE_SPECTROGRAMS_WILL_BE_PUT",
  "move_transcode_to_content": true,
  "automatic_upload": true,
  "skip_hash_check": false,
  "skip_spectrogram": false,
  "allowed_transcode_formats": ["Flac", "Mp3320", "Mp3V0"],
  "concurrency": 16
}

Notes for people using sox under windows

if you use the binaries from here, and you want utf-8 support for paths (this is needed for Japanese/Chinese/Korean names in paths for example) you have to download the files from here and follow the steps below

  1. Extract the files from the zip
  2. Run the PreferExternalManifest.reg file and let it overwrite the registry entry
  3. Copy the sox.exe.manifest file to the folder where sox.exe is located
  4. Enjoy sox working with utf-8 paths :)

Built With

  • Rust - The language used
  • clap - CLI Framework
  • tokio - Async runtime
  • reqwest - HTTP client
  • serde - Serialization/Deserialization
  • intermodal - Used for Torrent Hash checking & creation
  • audiotags - Reading/Writing Audio Metadata

Contributing

Please read CONTRIBUTING.md for details on our code of conduct, and the process for submitting pull requests to us.

Versioning

We use Semantic Versioning for versioning. For the versions available, see the tags on this repository.

Authors

See also the list of contributors who participated in this project.

License

This project is licensed under the MIT See the LICENSE.md file for details

red_oxide's People

Contributors

devyukine avatar edgeworn avatar screxyscroo avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

red_oxide's Issues

Unable to start.

[✅] Logged in as xxxx on the Redacted API
thread 'main' panicked at 'called Option::unwrap() on a None value', src/main.rs:177:37
note: run with RUST_BACKTRACE=1 environment variable to display a backtrace

Skipping due to encountered error: Failed to create torrent

I spent hours setting up this script and it's not working despite all the efforts I've tried. First thing I want to point out is that --debug flag is useless and doesn't provide a debug.log file or doesn't show debug info in shell either. It makes it very complicated and frustrating to troubleshoot.

I'm running the script from my home folder at /home/***/Data/red_oxide

All of the files are in that folder except the torrent directory which is on an NFS share. I made sure that my user have read and write permissions to that share by creating and deleting a folder.

Here's my configuration file:

{
  "api_key": "***",
  "torrent_directory": "/home/***/Data/red_oxide/torrent",
  "content_directory": "/mnt/red_oxide", # NFS SHARE
  "transcode_directory": "/home/***/Data/red_oxide/transcode",
  "spectrogram_directory": "/home/***/Data/red_oxide/spectrogram",
  "move_transcode_to_content": true,
  "automatic_upload": true,
  "skip_hash_check": false,
  "skip_spectrogram": false,
  "allowed_transcode_formats": ["Flac", "Mp3320", "Mp3V0"],
  "concurrency": 16
}

Here's the shell output with the Failed to create torrent:

***@ubuntu:~/Data/red_oxide$ ./red_oxide transcode https://redacted.ch/torrents.php?id\=***\&torrentid\=***\#torrent***
[] Logged in as *** on the Redacted API
[] Got torrent *** from group ***
[] Found missing format(s) MP3 320,MP3 V0, for torrent *** in group ***
[] Local file torrent hash check succeeded for torrent *** in group ***
[⏸️] Created Spectrograms at /home/***/Data/red_oxide/spectrogram/***, please manual check if FLAC is lossless before continuing!
Do those spectrograms look good? yes
[] Transcoding Done!
[] Skipping due to encountered error: Failed to create torrent

I got intermodal, sox, flac and lame installed and added to path. I confirmed this by entering their name in console and I could see their version and information for each app.

I believe the script is broken. My friend who used the script successfully tried for the past 3 hours and wasn't able to figure out why I'm getting this error.

Please help me out, thank you!

Can't get it past 'Logged in as X on the Redacted API'

Running on Windows, installed dependencies
Trying to run it with --config-file=config.json -d --debug
All I get is a green tick next to Logged in as X and then it exits, no error no debug, nothing to go with
Tried feeding parameters rather than config file but same result
Any ideas?

`Flac` transcode target is ignored when `skip_existing_formats_check` is `true` and source is `Flac24`

I'm working my way through the source and was curious if this behaviour is intentional or not?

If you are attempting to:

  • transcode from Flac24
  • allowed_transcode_formats contains Flac
  • skip_existing_formats_check is false

then I would have expected that Flac would be included in transcode_formats but this is explicitly ignored:

&& (release_type != Flac || torrent.format != "FLAC")

Is this the expected behaviour?

Albums with multiple discs

Many torrents of albums with multiple discs have a separate folder for each disc. Disc 1, Disc 2, so on. When oxide tries to transcode one of these torrents, it mixes up all tracks into one folder, like this:
2024-02-04 (2)
That's ugly and confusing (and against RED's rules).
It would be best if it copied the folder structure of the original torrent, so the converted files would end up in its respective folders.

Error: Access is denied. (os error 5)

Keep getting this error when trying to run red_oxide.exe transcode -c [PATH_TO_CONFIG]

I have tried running this program in elevated command prompt and powershell.

Any ideas?

`transcode --dry-run` is confusing and/or redundant

The implementation of --dry-run differed from my expectation

Typically a --dry-run would go through the steps without actually doing anything.

However, the implementation here is spectrograms and transcodes are still generated and the only step that is skipped is uploading.

This behaviour is already handled by --automatic-upload false - which in fact does a better job because it prints the data that would be uploaded to the terminal.

Invalid characters on folder names

I've encountered this on two ocasions:
Error: Redacted API returned an error while uploading a torrent: {"status":"failure","error":"The file or folder in the torrent that has the name 'Big | Brave - Nature Morte [2023] (CD - MP3 - 320)' contains the forbidden character '|'. Please recheck all files and folders in the directory and rename them as necessary and recreate the torrent.<br \/><br \/>\nNote: The complete list of characters that are disallowed are shown below:<br \/>\n\t\t : ? < > \\ * | \" \/\/ (Leading Space)"}

Error: Redacted API returned an error while uploading a torrent: {"status":"failure","error":"The file or folder in the torrent that has the name 'Therapy? - Hard Cold Fire [2023] (CD - MP3 - V0)' contains the forbidden character '?'. Please recheck all files and folders in the directory and rename them as necessary and recreate the torrent.<br \/><br \/>\nNote: The complete list of characters that are disallowed are shown below:<br \/>\n\t\t : ? < > \\ * | \" \/\/ (Leading Space)"}

Can you add for it to automatically replace those either just remove them completely or add an underscore (_) like they do on site.

Thank you.

Skipping due to encountered error: The system cannot find the path specified. (os error 3)

No matter what config settings for pathing I change in the JSON it throws this error on Windows through PS.

Skipping due to encountered error: The system cannot find the path specified. (os error 3)

"torrent_directory": "Z:/music/torrents",
"content_directory": "Z:/music/content",
"transcode_directory": "Z:/music/transcode",
"spectrogram_directory": "Z:/music/spectrogram",

Orpheus support

Would you be interested in adding support for this tool on Orpheus? The API shouldn't be that different.

I can even take a swing at it if you want.

Add support to default config file

It would be nice if there was a config file in a fixed place, like $XDG_CONFIG_HOME/red_oxide/config.json, specifying the config every time is a bit tiring

Respect subfolders when transcoding

Multi-CD releases where each CD is in a different subfolder get their subfolders combined, so after transcoding you get one folder with tracks:

Folder

01 aaa
01 bbb
02 aaa
02 bbb
...

Instead of the existing:

Folder

Disc 1

01 aaa
02 aaa
...

Disc 2

01 bbb
02 bbb
...

If there is artwork in a folder, it already gets copied over in the correct subfolder (Disc 1/cover.jpg for example), can the same be done for the tracks?

If that's not possible, maybe there could be a setting to wait after transcoding so you can manually do the subfolders, then press Y to create the torrent and upload it, but this wouldn't be ideal.

thread 'main' panicked error

Hello I'm having issues running red_oxide on debian (openmediavault). Below is the command that I ran with the config.json as well. I have flac, lame, sox, and imdl all installed within my red_oxide folder.

root@omv:/home/red_oxide# ./red_oxide_Linux_x86_64-unknown-linux-gnu transcode --config-file /home/red_oxide/config.json https://LINKTOMY.PL
[✅] Logged in as MYUSERNAME on the Redacted API
thread 'main' panicked at 'called `Option::unwrap()` on a `None` value', src/main.rs:177:37
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
{
  "api_key": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
  "torrent_directory": "/srv/MUSIC/downloads/Torrents/RED",
  "content_directory": "/srv/MUSIC/downloads/FLAC",
  "transcode_directory": "/srv/MUSIC/downloads/Torrents",
  "spectrogram_directory": "/srv/MUSIC/downloads/Specs",
  "move_transcode_to_content": false,
  "automatic_upload": true,
  "skip_hash_check": false,
  "skip_spectrogram": false,
  "allowed_transcode_formats": ["Flac", "Mp3320", "Mp3V0"],
  "concurrency": 16
}

Copying thumbnails causes OS error 1 over SMB

Hi,
I'm running red_oxide v0.8.2 on WSL Ubuntu with a SMB share drive mounted. All of my directories are on the shared drive.
The transcoding finishes on all of the files, but the program stops with OS error 1.
I have identified that fs::copy in src/transcode/util.rs is causing the error when it tries to copy the thumbnail to the new transcoded folder.
Using a drop-in copy function from ChatGPT that doesn't use fs::copy works, but I'm not sure how you would want to fix it. I can do a PR with the change that works for me if you would like.

Thanks!

thread panic

I'm not sure how to begin troubleshooting this...

thread 'main' panicked at 'called Option::unwrap() on a None value', src\main.rs:177:37
stack backtrace:
0: 0x7ff7d7e8e962 -
1: 0x7ff7d7eb384b -
2: 0x7ff7d7e89a4a -
3: 0x7ff7d7e8e6ab -
4: 0x7ff7d7e91279 -
5: 0x7ff7d7e90efb -
6: 0x7ff7d7e919a8 -
7: 0x7ff7d7e9185b -
8: 0x7ff7d7e8f629 -
9: 0x7ff7d7e91550 -
10: 0x7ff7d7ed8955 -
11: 0x7ff7d7ed8a0c -
12: 0x7ff7d7a5a4a6 -
13: 0x7ff7d7a4c9ab -
14: 0x7ff7d7aba94a -
15: 0x7ff7d7a104b9 -
16: 0x7ff7d79f6338 -
17: 0x7ff7d7ab4c16 -
18: 0x7ff7d7aa699a -
19: 0x7ff7d7e824de -
20: 0x7ff7d79f64bc -
21: 0x7ff7d7ec8b5c -
22: 0x7ff90b707614 - BaseThreadInitThunk
23: 0x7ff90cc226b1 - RtlUserThreadStart

invalid_track_number_vinyl - Why Disable Automatic Upload?

When a vinyl with invalid track numbers is detected, why do you then set cmd.automatic_upload to false?

Looking at your main.rs, it will always run the prompt to fix the tags if invalid_track_number_vinyl is true, and the uploading portion of the code is below that.

Therefore, the user will get the prompt to fix the tags, and can answer either yes or no that the tags are fixed. If they answer yes, then I think automatic upload should proceed as normal. Unless I'm missing something.

Protection against too long file names

Sometimes the naming of the folder gets too long and the torrent upload fails with the API returning:
{"status":"failure","error":"The torrent contained one or more files with too long a name: [name]"}

A check should be added to see if the path is too long.

Unable to run program past the identifying formats/transcoding stages

Trying to run the program on a couple of torrents seems to exit almost all of them, I've only had luck with one torrent while others hang up on either Error: The system cannot find the path specified. (os error 3) (prints after [✅] Found missing format(s) MP3 320,MP3 V0, for torrent x in group x) or Error: program not found (prints after [➡️] Transcoding...)

I've ran all of these so far with red_oxide_Windows_x86_64-pc-windows-msvc.exe transcode --debug -c "G:\RustRED\config.json" "URL"
and my config.json looks like this
{ "api_key": "", "torrent_directory": "G:\\RustRED\\torrent", "content_directory": "G:\\RustRED\\content", "transcode_directory": "G:\\RustRED\\transcodes", "spectrogram_directory": "G:\\RustRED\\spectrograms", "move_transcode_to_content": true, "automatic_upload": true }

and if it helps, i'm running this on Windows 11 (22621.1413)

24-16 fake transcode error

If the original 24b is actually a 16b fake upload, the script will create a folder with cover pix in it and upload it anyway, which should be prevented.

HTML entities in release field don't get encoded

I'm using the gnu linux version, and releases that have edition titles that contain special characters (kanji, "&" ...) Have the HTML entities in the folder title, instead of the special characters.

Edition title: "Limited Edition / 期間生産限定盤"

Limited Edition / 期間生産限定盤

Edition title: "Russia & UK"

Russia & UK

Track meta data tagging

I will be honest I do not know much of anything about coding. If it were not for a ton of googling I do not think I would have go this code to work at all, but after hours of trying ( yeaaa go me) I actually got it to work which was dope. wanna go ahead and say thanks for the hard work you put into this software I really do appreciate you.

My only issue was every time I transcoded a file I noticed the tagging was incorrect. It seems to always list each track as part of another disk. The disk number is always the number of the corresponding track.

I do not know if its just me being a noob or if it is something that has just gone over looked but I wanted to bring it to your attention.

Other than that everything else works like a charm for me! A little bit more info I am using windows 10 used both the pre compiled exe's and ran the build cargo build function in power shell to compile an exe. I had all the prerequisite programs added to path too.

Thanks again in advance let me know if I am just being a goober or if you need anymore info.

Unable to use. This error persists,

Error: Redacted API returned an error while uploading a torrent:

<title>500 Internal Server Error</title>

500 Internal Server Error


nginx

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.