Code Monkey home page Code Monkey logo

Comments (9)

clinton-hall avatar clinton-hall commented on May 23, 2024

Multiple categories can be supported.

By default, in NZBGet ui you can only specify 1 category for each media type in nzbToMedia.py (e.g. you can set 1 movie category, 1 tv category and 1 music category). You can manually edit the autoProcessMedia.cfg to add more.

The other way of doing this is just set the multiple categories in NZBGet to call nzbToNZBDrone.py directly. When configuring this script it doesn't ask for the category, and just passes through the category that calls the script. So this will allow you to call the same script (with the same config settings) for multiple categories.

from nzbtomedia.

ulrick65 avatar ulrick65 commented on May 23, 2024

Thanks for the reply. I found a post you made someplace that explained the second category...which worked once I added it. Now I updated it to a single category like you said and it still works.

I am getting an error (has been there since the start, but everything appears to work OK). I will paste that at the end here...let me know please what it means and if I should care about it. As a side note, I was hoping you could either point to a good explanation or provide one here as to how it all works from a high level. What I mean is, I would like to know a bit about how the communication between the 3 pieces works:

Sonarr sends the request through an API directly to NzbGet...Nzbget talks directly with Sonarr as far as I can tell (Sonarr gets updates as the download progresses, so must be). At this point, nzbToMedia is not even in the picture right?

Once the download completes, NzbGet unpacks it from the interim folder to the final download folder, then runs a few scripts to clean things up (and I have one for language correction, etc.). Then Nzbget calls nzbToNzbDrone.py and then what happens? Does nzbToMedia talk to Sonarr or only back to Nzbget once it is done?

Here's the log file, note the error in the middle there about the fork. Thanks very much for the help.

2023-06-13 02:09:13 INFO ::MAIN: The /opt/nzbget/scripts/nzbToMedia/nzbToNzbDrone.py script completed successfully.
2023-06-13 02:10:04 INFO ::MAIN: Python v3.8 will reach end of life in 489 days.
2023-06-13 02:10:04 INFO ::MAIN: Checking database structure...
2023-06-13 02:10:04 INFO ::MAIN: Checking if git needs an update
2023-06-13 02:10:05 INFO ::MAIN: No update needed
2023-06-13 02:10:05 INFO ::MAIN: nzbToMedia Version:c4cc554ea1e9b342538853a8108964bde7929e2f Branch:master (Linux 4.15.0-212-generic)
2023-06-13 02:10:05 WARNING ::MAIN: Failed to locate 7zip. Transcoding of disk images and extraction of .7z files will not be possible!
2023-06-13 02:10:05 WARNING ::MAIN: Failed to locate par2. Repair and rename using par files will not be possible!
2023-06-13 02:10:05 INFO ::MAIN: #########################################################
2023-06-13 02:10:05 INFO ::MAIN: ## ..::[nzbToMedia.py]::.. ##
2023-06-13 02:10:05 INFO ::MAIN: #########################################################
2023-06-13 02:10:05 INFO ::MAIN: Script triggered from NZBGet Version 21.1.
2023-06-13 02:10:05 INFO ::MAIN: Auto-detected SECTION:NzbDrone
2023-06-13 02:10:05 INFO ::MAIN: Calling NzbDrone:sonarr-tv to post-process:Halo.Nightfall.2014.S01.720p.BluRay.DD5.1.x264.Hifi
2023-06-13 02:10:05 INFO ::MAIN: Attempting to verify sonarr-tv fork
2023-06-13 02:10:05 WARNING ::MAIN: Connection to NzbDrone:sonarr-tv failed! Check your configuration
2023-06-13 02:10:05 INFO ::MAIN: NzbDrone:sonarr-tv fork set to default
2023-06-13 02:10:05 INFO ::MAIN: NzbDrone:sonarr-tv Could not create a fork object for default. Probaly class not added yet.
2023-06-13 02:10:05 INFO ::MAIN: FLATTEN: Flattening directory: /disk3/nzbget-dl/Halo.Nightfall.2014.S01.720p.BluRay.DD5.1.x264.Hifi
2023-06-13 02:10:05 INFO ::TRANSCODER: Checking [Halo.Nightfall.2014.S01.720p.BluRay.DD5.1.x264.Hifi.mkv] for corruption, please stand by ...
2023-06-13 02:10:05 INFO ::TRANSCODER: SUCCESS: [Halo.Nightfall.2014.S01.720p.BluRay.DD5.1.x264.Hifi.mkv] has no corruption.
2023-06-13 02:10:05 POSTPROCESS::NZBDRONE: SUCCESS: The download succeeded, sending a post-process request
2023-06-13 02:12:05 INFO ::MAIN: The /opt/nzbget/scripts/nzbToMedia/nzbToNzbDrone.py script completed successfully.

from nzbtomedia.

clinton-hall avatar clinton-hall commented on May 23, 2024

2 warnings in the log are around par2, and 7zip. These can be installed to get rid of the errors, but these are only required if you need to do extra par repair checks (these are done by nzbget as standard) and/or want to do transcoding of disk images.

The other warning is around a failed connection to sonarr. This could be the api key, port, host, or could even be that the version on Sonarr isn't compatible with the version of nzbToMedia... but this doesn't really matter as per the below:

The way the communication works is largely as you stated;

  • Sonarr sends the request through an API directly to NzbGet
  • Sonarr gets updates from NZBget as the download progresses... so this is Sonarr communicating with NZBget. NZBGet doesn't initiate any communication with Sonarr.... but that is really just a technicality.
  • At this point, nzbToMedia is not in the picture.
  • On completion of downloading (and after any postprocess scripts have finished running) Sonarr will ask NZBGet for the status, identify the download has completed, and do any renaming/moving.

So with this, and indeed with any of the other media management tools (SickBeard, CouchPotato etc) the nzbToMedia scripts are not needed, and the whole management can certainly be done without these scripts.

The reason these scripts started was to reduce the workload that would cause a NAS (or low processing power system) to slow down when trying to download using SABnzbd and having SickBeard/CouchPotato constantly asking SABnzbd what the status was on each download.... Basically the python processes used by SAB were pushing the limits a bit and every time something would interrupt SABnzbd to ask for the status, the systems would get overloaded and slow right down...
These scripts stopped that process and instead would tell SickBeard/CouchPotato when the download was completed and have the files processed. The key benefit here is only realised by actually stopping the constant update request (or significantly reducing the frequency of this).

Since then other additional features were added that can be beneficial (but not necessarily needed).

  • Transcoding (converting the files to another format that is playable on a specific device)
  • Failed download handing (now standard on most SickBeard/Sonarr/Radarr etc)
  • Corruption checks (actually uses ffmpeg/ffprobe to verify that the media is a valid playable file)
  • safe renaming (checks for the rename .sh/.bat files and rebuilds the rename commands to rename the download file... found with obfuscated file names) . The key here is that it will only rename the download files using valid rename commands and is NOT asking you to actually run a downloaded .sh/.bat file which can be very dangerous. Even here, this is usually not a requirement when Sonarr knows what download was requested and therefore knows what to rename to...
  • Download additional Subs (this can also now be achieved through other avenues)
  • Failed/corrupt notification back to the indexer (dognzb.cr)

The script itself will do these particular functions and then tell Sonarr, via api, that the download is complete and where it is (if you used transcoder this will likely be a different location to what NZBGet downloaded to).

Even if this communication to Sonarr fails, when Sonarr asks for the status and see the download is complete it will find the files are rename them (unless you used transcoding options as above).

So. if you don't need/want the extra features and you have enough processing power, removing nzbToMedia might make it a cleaner process for you.

If you have low processing power, or like to keep your usage low, these scripts can reduce the workload, but you also need to "tune" your Sonarr to stop/reduce the polling of NZBGet.

Otherwise these scripts can add some additional features as above, but there are also some other scripts that will do some of these features as well before allowing Sonarr to do its thing. e.g. https://github.com/clinton-hall/GetScripts, https://github.com/nzbget/FakeDetector, https://github.com/nzbget/FailureLink

from nzbtomedia.

ulrick65 avatar ulrick65 commented on May 23, 2024

from nzbtomedia.

clinton-hall avatar clinton-hall commented on May 23, 2024

The version you have should be compatible with Sonarr V4.

are you able to edit the autoprocessmedia.cfg and set

[General]
log_debug = 1

then the next time the script runs there should be more details in the log. Hopefully these will help identify the issue.

from nzbtomedia.

ulrick65 avatar ulrick65 commented on May 23, 2024

Will do. I also noticed that no matter what I set the check for update flag to in the config, it always checks for an update.

One other thing I will mention is that in the autoprocessconfig file, there is always a section called tv. I reinstalled and then reset the config in nzbget, changing the name to sonarr-tv in the category...thinking that would start fresh with only one section in there...but it just added it to the bottom as a new section. Just wanted to be sure that was normal.

I will post back with the log file.

from nzbtomedia.

ulrick65 avatar ulrick65 commented on May 23, 2024

Here's the log file. Looks like the API is working fine at the end there...so perhaps the errors listed about the fork are inconsequential?

I did a test and removed nzbToMedia script from the category and it did process fine...but it seems at the end, it takes longer as it has to wait for Sonarr to poll the folder.

Let me know your thoughts. I am also wondering why it checks for update required, even though I have that set to not do that in the nzbget setting for nzbtodrone. Its not a huge deal, more curious than anything. But I am processing a lot of my stuff right now, so no need to keep checking for updates at the moment.

I removed the IP address, but it is the same in all places in the log file:

2023-06-15 12:25:47 INFO ::MAIN: Python v3.8 will reach end of life in 487 days.
2023-06-15 12:25:47 INFO ::MAIN: Checking database structure...
2023-06-15 12:25:47 DEBUG ::MAIN: Checking Initial Schema database upgrade
2023-06-15 12:25:47 DEBUG ::MAIN: InitialSchema upgrade not required
2023-06-15 12:25:47 DEBUG ::MAIN: Checking if we can use git commands: git version
2023-06-15 12:25:47 DEBUG ::MAIN: Executing git version with your shell in /opt/nzbget/scripts/nzbToMedia
2023-06-15 12:25:47 DEBUG ::MAIN: git version : returned successful
2023-06-15 12:25:47 DEBUG ::MAIN: Using: git
2023-06-15 12:25:47 DEBUG ::MAIN: Executing git symbolic-ref -q HEAD with your shell in /opt/nzbget/scripts/nzbToMedia
2023-06-15 12:25:47 DEBUG ::MAIN: git symbolic-ref -q HEAD : returned successful
2023-06-15 12:25:47 INFO ::MAIN: Checking if git needs an update
2023-06-15 12:25:47 DEBUG ::MAIN: Executing git rev-parse HEAD with your shell in /opt/nzbget/scripts/nzbToMedia
2023-06-15 12:25:47 DEBUG ::MAIN: git rev-parse HEAD : returned successful
2023-06-15 12:25:47 DEBUG ::MAIN: Executing git fetch origin with your shell in /opt/nzbget/scripts/nzbToMedia
2023-06-15 12:25:47 DEBUG ::MAIN: git fetch origin : returned successful
2023-06-15 12:25:47 DEBUG ::MAIN: Executing git rev-parse --verify --quiet '@{upstream}' with your shell in /opt/nzbget/scripts/nzbToMedia
2023-06-15 12:25:47 DEBUG ::MAIN: git rev-parse --verify --quiet '@{upstream}' : returned successful
2023-06-15 12:25:47 DEBUG ::MAIN: Executing git rev-list --left-right '@{upstream}'...HEAD with your shell in /opt/nzbget/scripts/nzbToMedia
2023-06-15 12:25:48 DEBUG ::MAIN: git rev-list --left-right '@{upstream}'...HEAD : returned successful
2023-06-15 12:25:48 DEBUG ::MAIN: cur_commit = c4cc554 % (newest_commit)= c4cc554, num_commits_behind = 0, num_commits_ahead = 0
2023-06-15 12:25:48 INFO ::MAIN: No update needed
2023-06-15 12:25:48 INFO ::MAIN: nzbToMedia Version:c4cc554ea1e9b342538853a8108964bde7929e2f Branch:master (Linux 4.15.0-212-generic)
2023-06-15 12:25:48 WARNING ::MAIN: Failed to locate 7zip. Transcoding of disk images and extraction of .7z files will not be possible!
2023-06-15 12:25:48 WARNING ::MAIN: Failed to locate par2. Repair and rename using par files will not be possible!
2023-06-15 12:25:48 INFO ::MAIN: #########################################################
2023-06-15 12:25:48 INFO ::MAIN: ## ..::[nzbToMedia.py]::.. ##
2023-06-15 12:25:48 INFO ::MAIN: #########################################################
2023-06-15 12:25:48 DEBUG ::MAIN: Options passed into nzbToMedia: ['/opt/nzbget/scripts/nzbToMedia/nzbToNzbDrone.py']
2023-06-15 12:25:48 INFO ::MAIN: Script triggered from NZBGet Version 21.1.
2023-06-15 12:25:48 DEBUG ::MAIN: Adding NZB download info for directory /disk3/nzbget-dl/Hey.Arnold.S05E01.Monkeyman.Buses.Bikes.Subways.480p.HULU.WEB-DL.AAC2.0.H.264-ETHiCS to database
2023-06-15 12:25:48 INFO ::MAIN: Auto-detected SECTION:NzbDrone
2023-06-15 12:25:48 INFO ::MAIN: Calling NzbDrone:sonarr-tv to post-process:Hey.Arnold.S05E01.Monkeyman.Buses.Bikes.Subways.480p.HULU.WEB-DL.AAC2.0.H.264-ETHiCS
2023-06-15 12:25:48 DEBUG ::SERVER: Attempting to connect to server at http://IPREMOVED:8989
2023-06-15 12:25:48 DEBUG ::SERVER: Server responded at http://IPREMOVED:8989
2023-06-15 12:25:48 INFO ::MAIN: Attempting to verify sonarr-tv fork
2023-06-15 12:25:48 WARNING ::MAIN: Connection to NzbDrone:sonarr-tv failed! Check your configuration
2023-06-15 12:25:48 INFO ::MAIN: NzbDrone:sonarr-tv fork set to default
2023-06-15 12:25:48 DEBUG ::MAIN: Create object for fork default
2023-06-15 12:25:48 INFO ::MAIN: NzbDrone:sonarr-tv Could not create a fork object for default. Probaly class not added yet.
2023-06-15 12:25:48 INFO ::MAIN: FLATTEN: Flattening directory: /disk3/nzbget-dl/Hey.Arnold.S05E01.Monkeyman.Buses.Bikes.Subways.480p.HULU.WEB-DL.AAC2.0.H.264-ETHiCS
2023-06-15 12:25:48 DEBUG ::MAIN: Checking for empty folders in:/disk3/nzbget-dl/Hey.Arnold.S05E01.Monkeyman.Buses.Bikes.Subways.480p.HULU.WEB-DL.AAC2.0.H.264-ETHiCS
2023-06-15 12:25:48 DEBUG ::MAIN: calling command: /usr/bin/ffprobe -v quiet -print_format json -show_format -show_streams -show_error /opt/nzbget/scripts/nzbToMedia/tests/test.mp4
2023-06-15 12:25:48 INFO ::TRANSCODER: Checking [Hey.Arnold.S05E01.Monkeyman.Buses.Bikes.Subways.480p.HULU.WEB-DL.AAC2.0.H.264-ETHiCS.mkv] for corruption, please stand by ...
2023-06-15 12:25:48 DEBUG ::MAIN: calling command: /usr/bin/ffprobe -v quiet -print_format json -show_format -show_streams -show_error /disk3/nzbget-dl/Hey.Arnold.S05E01.Monkeyman.Buses.Bikes.Subways.480p.HULU.WEB-DL.AAC2.0.H.264-ETHiCS/Hey.Arnold.S05E01.Monkeyman.Buses.Bikes.Subways.480p.HULU.WEB-DL.AAC2.0.H.264-ETHiCS.mkv
2023-06-15 12:25:48 INFO ::TRANSCODER: SUCCESS: [Hey.Arnold.S05E01.Monkeyman.Buses.Bikes.Subways.480p.HULU.WEB-DL.AAC2.0.H.264-ETHiCS.mkv] has no corruption.
2023-06-15 12:25:48 POSTPROCESS::NZBDRONE: SUCCESS: The download succeeded, sending a post-process request
2023-06-15 12:25:48 DEBUG ::NZBDRONE: path: /disk3/nzbget-dl/Hey.Arnold.S05E01.Monkeyman.Buses.Bikes.Subways.480p.HULU.WEB-DL.AAC2.0.H.264-ETHiCS
2023-06-15 12:25:48 DEBUG ::NZBDRONE: Opening URL: http://IPREMOVED:8989/api/v3/command with data: {"name": "DownloadedEpisodesScan", "path": "/disk3/nzbget-dl/Hey.Arnold.S05E01.Monkeyman.Buses.Bikes.Subways.480p.HULU.WEB-DL.AAC2.0.H.264-ETHiCS", "downloadClientId": "7c4fedd04c044b89aa60d243505f9e1f", "importMode": "Move"}
2023-06-15 12:25:48 DEBUG ::NZBDRONE: Scan started with id: 92445
2023-06-15 12:25:58 DEBUG ::NZBDRONE: The Scan command return status: completed
2023-06-15 12:25:58 DEBUG ::NZBDRONE: The directory /disk3/nzbget-dl/Hey.Arnold.S05E01.Monkeyman.Buses.Bikes.Subways.480p.HULU.WEB-DL.AAC2.0.H.264-ETHiCS has been removed. Renaming was successful.
2023-06-15 12:25:58 INFO ::MAIN: The /opt/nzbget/scripts/nzbToMedia/nzbToNzbDrone.py script completed successfully.

from nzbtomedia.

clinton-hall avatar clinton-hall commented on May 23, 2024

I think from memory (without delving into the code) that the version check still happens even when the updater is disabled.
This part can probably be disabled by setting version_notify = 0

The tv subsection is copied from the default .cfg template. so this will re-create but since it doesn't match a category that calls the script it isn't an issue so much as untidy config... something I should ideally try and clean-up at my end.

Yes, processing seems to be working. The fork warning is a result of the need to continue with creating sub-modules for the script (again, more tidy-up at my end)

from nzbtomedia.

ulrick65 avatar ulrick65 commented on May 23, 2024

Great, thanks for all the info. Seems to be working well. I appreciate the work on this. I am going to try and tweak Sonarr to stop the polling...I think that is a simple database edit, but then it overwrites on updates (which for me in Sev, is often) so not sure its worth it....it is pretty quick to act now, but the polling is taking processing power

from nzbtomedia.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.