Problem when using real time monitoring in library about shokofin HOT 7 CLOSED

You need Shoko Desktop to select the renamer, save the script and mark the saved script to run on import. You can select the active renamer directly in the v3 REST API, but the rest (pun intended) is still exclusive to the mostly-undocumented-except-in-source-code v1 API.

from shokofin.

I got some help with enabling the renamer in the issue I created for LuaRenamer. I haven't set everything up yet but I got rename previews working with the v1 API. Enabling the renamer is almost exactly the same, so that should also work now. I just haven't tried it yet.

I also included some Linux scripts for convenience. They might get added to the documentation later. Just wanted to share it here, in case anyone else is looking for information on how to enable renaming in Linux.

from shokofin.

In my personal opinion than this issue is the result of a sub-optimal usage of Shoko Server together with Jellyfin (or any other media server for that matter). It cannot be mitigated without changing your setup and it's not a bug in the plugin as the plugin is working as intended. To find out why, read on.

A suggestion

I have both Shoko and the Jellyfin library for my anime configured with real-time monitoring ("Watch for new files" in the import folder in Shoko and "Enable real time monitoring" in the library in Jellyfin). This usually works quite well as I don't have to scan the whole library when I add new files. Unfortunately, since Shoko and Jellyfin are both processing these files when they are added to the filesystem, Shokofin will almost certainly find no metadata for those files. Due to hashing in Shoko, it practically always takes longer. So Jellyfin will have added the files to the library but without any metadata. To fix this, I have to refresh the metadata on the entire show, after Shoko is finished. If possible, I'd rather have Shokofin wait for the metadata to become available, ideally checking its progress so it can start right after it's finished.

You are correct in your observation that Shokofin cannot find the metadata about the files for Jellyfin if you add them to both Shoko Server and Jellyfin at the same time, and you can use the move/rename feature of Shoko Server to fix the conflict. To enable moving (and optionally renaming) of files you either navigate in the settings either in Shoko Desktop or the Web UI, or navigate to yourJSON configuration file — The setting name(s) for moving (and renaming) should be self explanatory enough to find regardless of where you go. Afterwards you need to ensure you have at least two import folders in Shoko. One is used as your ingress folder — you drop the files you want Shoko to manage in here — and one for your media library folder — where Shoko will place the files (optionally after it have renamed them to your preferred name convention). In a normal setup using either the default legacy renamer or a "normal" plugin renamer (like Mik1ll's latest plugin available here) then Shoko will only move the files from your ingress folder to your media library folder when the file is recognised by Shoko — either because it was indexed on anidb and automatically linked or because you manually linked it . Since Jellyfin will only know about your media library folder and not your ingress folder, then you ensure that any and all files Jellyfin sees are always recognised by Shoko when it uses Shokofin to ask about metadata.

Surprise?

If possible, I'd rather have Shokofin wait for the metadata to become available, ideally checking its progress so it can start right after it's finished.

and

Alternative

If it's not possible to monitor the queue in Shoko, Shokofin could also just wait a predetermined amount of time before trying to get the metadata again. However, since hashing may take a long time and many files could have been added at the same time, it's hard to predict how long would be reasonable to wait. Because it may also just be the case that Shoko has no metadata at all and is not working on it. In which case you don't want Shokofin to wait at all.

It seems like you know, but it would be devastating to wait till the metadata is available and you encounter an unrecognised file that's not indexed on anidb (and neither is manually linked). The plugin would potentially wait forever unless we specify a timeout, but that would defeat the purpose of waiting for the metadata to be available, in my personal opinion at least. Also, waiting for the metadata to be available would also significantly slow down the library scans if it either encounters a new file (using a setup similar to the one you have right now) or it finds an older unrecognized file.

I don't know if it would benefit you but there is a planned feature — no ETA at the moment but soon™ — to listen to the file added/updated/deleted events and the series added/updated events from Shoko Server to trigger tasks in Jellyfin (e.g. library scan, update series, update season, update episode, etc.), which can be used to either replace and/or complement the existing "Watch for new files" feature native to Jellyfin (that relies on the file system and cannot be used on SMB shares in linux 🥲) by a similar feature that instead relies on Shoko Server as an event backend. The greatest downside (I can think of) to the feature would be that it would require a continuous connection between Jellyfin and Shoko Server (i.e. a web socket) — some people might not want to run Shoko 24/7, others think it a presumption to run it 24/7 — and that it would be extra work for implementing a similar feature to something that already exists. The greatest upsides (I can think of) would be that it would work regardless of how you mount your library in Jellyfin (e.g. using local discs, NTS share, SMB share, etc.) and that it would always keep your entries up to date instead of waiting for the next internet refresh (minimum 30 days in-between) or needing to manually trigger the replace-all refresh library scan or update series/season/episode tasks.

Exactly when the feature will land will depend on a lot of different factors, most of them unrelated to Shoko and programming and more on my irl stuff.

No comment

Additional information

With the reproduction, you may run into another problem, with Shoko itself. If you watch for new files in Shoko, it seems to start processing them as soon as the file entry is added to the file system. At this time, the file may still be transferring to the file system. So with large files or slower transfers (like over a network connection) Shoko may be hashing a partially-transferred file. This will obviously not match anything and it will be considered unrecognized. A workaround for this is to first transfer the file(s) to the same disk/partition as the import folder but outside of that folder. If you then move it to the import folder (move, not copy), it should be an instant transfer and Shoko will hash the whole file.

It seems you already know the workaround so I won't comment more on it here.

Closing words

I'm going to close this issue for now, but feel free to ask more about how to configure your setup for a more optimal experience either in here or in our discord server (it's a lot faster to get help in the discord server, but the choice is yours for how you want to communicate).

from shokofin.

Thanks for the quick response. I did initially try to set it up in the way you described. I agree that that is the better setup. But my problem was that I couldn't figure out how to configure the renamer to use a custom pattern. I have a specific naming scheme that I want to use. But all the documentation seemed to lead to using Shoko Desktop, which you're also suggesting. However, I'm on Linux. So I can't use that. But you mentioned a JSON configuration file. If that's a server configuration file, do let me know which one it I should look for. And what to change for the renamer. I couldn't figure it out when I was setting it up but I was also just starting to use Shoko so I wasn't too familiar with it.

I will try again to change my setting to the scan-rename-then-move setup since that's what I prefer as well. I do hope you can add some Linux-friendly documentation. A lot of the advice that I found assumes that you also use Shoko Desktop, which is just not possible in Linux (and I'd prefer no to use it since I don't need it for anything else). It's fine if it's a bit more complicated but that makes it especially important if there's some good guidance on how to do it.

from shokofin.

For now you need to use Shoko Desktop for some parts (until it's all migrated to the Web UI), so here is a community made guide for how to get Shoko Desktop working on linux; https://docs.shokoanime.com/server/linux-desktop/

from shokofin.

It also works in Bottles

from shokofin.

Thanks for the information. Luckily I don't need to use much of the functionality. If I can get this renaming to work without Shoko Desktop, that should already be enough for my needs. I got quite a way with LuaRenamer. Hopefully I can still do that last bit (for which I created an issue). Otherwise, I might try your suggestion.

from shokofin.