What's your take on it? Is it really a NFS problem; or rather a timing problem that could happen to any copy process if said process was actually slow enough?
(Sorry for asking, but in my setup here, though somewhat close, it'd be quite cumbersome to try.)

This 'wait some time' bothers me. Stuff like that tends to fail. We'd better be waiting for a 'ready' flag, like 'process finished'. Controlling sequence instead of arbitrary (and presumably often unnecessary) wait times.

To me it looks like permissions are set only after finishing copying; while the import method inotify starts recognizing / indexing / whatever once the first inode signals a change. I could be very wrong here, though that's what i understand.

from gerbera.

First of all Permission Denied is quite clear in it's meaning.

The only way we can solve your problem, is to delay after it happened. This could be controlled by some configurable retry count on the autoscan directory. I'm not sure about the effect on the regular import, which does not bother me as long as it's disabled by default.

from gerbera.

Hope, I got it right. Maybe you can try it out before it's merged.

from gerbera.

It really seems that the permissions are set afterwards.

Here a 415ms NFS copy operation seen on the client:

$ time cp -a ~/Music/neu/ogg/Erhard\ Dietl/Die\ Olchis\ und\ der\ schwarze\ Pirat .
real	0m0,415s

The directory list on the server looks that way (nanoseconds after the -):

09:05:29-343432811
drwx------ 2 cweiske cweiske 4.0K May 21 09:05 Die Olchis und der schwarze Pirat

09:05:29-672953081
drwx------ 2 cweiske cweiske 4.0K May 21 09:05 Die Olchis und der schwarze Pirat

09:05:29-783264982
drwxr-xr-x 2 cweiske cweiske 4.0K Sep 13  2023 Die Olchis und der schwarze Pirat

At the end of the 400ms the permissions are set.

Gerbera does not listen to permission changes via inotify:

events = IN_CLOSE_WRITE | IN_CREATE | IN_MOVED_FROM | IN_MOVED_TO | IN_DELETE | IN_DELETE_SELF | IN_MOVE_SELF | IN_UNMOUNT;

It would be possible via IN_ATTRIB: https://www.man7.org/linux/man-pages/man7/inotify.7.html

from gerbera.

Thanks for confirming my point.

It was to be expected to work like this, since the file copying ought to have some semaphore about it to avoid race conditions etc.
Therefore finishing the copy process; that is setting access permissions, is probably the most suitable trigger to start indexing.
Waiting for arbitrary times is not very sensible; and running circles until it works isn't much better. Rather, it is prone to secondary bugs: Imagine something goes wrong - and anything that can go wrong will go wrong one day - it would run in circles for eternity if the copy process fails at some point. Or, again, a limited arbitrary number of rounds: again unconvincing.
Preferably is a singular flag 'copying finished' to initiate the import process, like what you described.
And once import fails at this moment, informing the user is a suitable thing to do.

from gerbera.

It would be possible via IN_ATTRIB: https://www.man7.org/linux/man-pages/man7/inotify.7.html

I was considering that already. But the change is even more complex and in normal operation we get updates without change of file content. On the other side, you do not change your permissions regularly?

from gerbera.

Sorry, but I'm not into the depths of the copying process at all. Just looking from an overall perspective. It is probably obvious that the preferred procedure is "copying successfully finished" -> "start indexing". If this isn't easy to do, tant pis!

from gerbera.

The problem does not occur when using cp -r, only when using cp -a.

A "normal" cp -r creates the directory and closes it, while cp -a does the same, but afterwards changes the attributes.

cp -r

cp -r immediately sets the correct permissions:

drwxr-xr-x 2 cweiske cweiske 4.0K May 21 14:48 ExampleAlbum

# inotifywait --monitor -r /data/media/tmp/
/data/media/tmp/ CREATE,ISDIR SmallAlbum
/data/media/tmp/ OPEN,ISDIR SmallAlbum
/data/media/tmp/ ACCESS,ISDIR SmallAlbum
/data/media/tmp/ CLOSE_NOWRITE,CLOSE,ISDIR SmallAlbum
/data/media/tmp/SmallAlbum/ CREATE 1.ogg
/data/media/tmp/SmallAlbum/ ATTRIB 1.ogg
/data/media/tmp/SmallAlbum/ ATTRIB 1.ogg
/data/media/tmp/SmallAlbum/ MODIFY 1.ogg
/data/media/tmp/SmallAlbum/ CLOSE_WRITE,CLOSE 1.ogg

cp -a

cp -a on the other hand first sets permissions for the user only:

drwx------ 2 cweiske cweiske 4.0K May 21 14:56 ExampleAlbum

and later changes them:

drwxr-xr-x 2 cweiske cweiske 4.0K May 21 14:48 ExampleAlbum

# inotifywait --monitor -r /data/media/tmp/
/data/media/tmp/ CREATE,ISDIR SmallAlbum
/data/media/tmp/ OPEN,ISDIR SmallAlbum
/data/media/tmp/ ACCESS,ISDIR SmallAlbum
/data/media/tmp/ CLOSE_NOWRITE,CLOSE,ISDIR SmallAlbum
/data/media/tmp/SmallAlbum/ CREATE 1.ogg
/data/media/tmp/SmallAlbum/ ATTRIB 1.ogg
/data/media/tmp/SmallAlbum/ ATTRIB 1.ogg
/data/media/tmp/SmallAlbum/ MODIFY 1.ogg
/data/media/tmp/SmallAlbum/ ATTRIB 1.ogg
/data/media/tmp/SmallAlbum/ ATTRIB 1.ogg
/data/media/tmp/ ATTRIB,ISDIR SmallAlbum
/data/media/tmp/SmallAlbum/ ATTRIB,ISDIR 
/data/media/tmp/ ATTRIB,ISDIR SmallAlbum
/data/media/tmp/SmallAlbum/ ATTRIB,ISDIR 
/data/media/tmp/ ATTRIB,ISDIR SmallAlbum
/data/media/tmp/SmallAlbum/ ATTRIB,ISDIR 
/data/media/tmp/SmallAlbum/ CLOSE_WRITE,CLOSE 1.ogg

from gerbera.

This behavior of cp -a is fully intended: https://git.savannah.gnu.org/cgit/coreutils.git/tree/src/cp.c

/* Used by do_copy, make_dir_parents_private, and re_protect
   to keep a list of leading directories whose protections
   need to be fixed after copying. */
struct dir_attr
[...]
make_dir_parents_private
              /* If the ownership or special mode bits might change,
                 omit some permissions at first, so unauthorized users
                 cannot nip in before the file is ready.  */
[...]
                  /* Make the new directory searchable and writable.
                     The original permissions will be restored later.  */
[...]
/* Ensure that parents of CONST_DST_NAME have correct protections, for
   the --parents option.  This is done after all copying has been
   completed, to allow permissions that don't include user write/execute.
static bool re_protect

from gerbera.

Beat me to it, well done, wanted to look at the same source tonight!
However, can you confirm the '-a'? I think it is just a concatenation of some parameters with some '-r/R'? I mean for the change of title, and the inherent logic?

from gerbera.

Thank you so much @KarlStraussberger!

from gerbera.