septag / dmon Goto Github PK

File update (echo test >> test_file) and move events are recorgnized as create event.

Only create and remove events are recognized stable, sometimes update event is receiveed.

./dmon_test .

rm 1 && touch 1 && sleep 1 && echo test > 1

On Windows, _DMON_WINAPI_STR gives me gibberish. I think the memory is simply not set. Replacing it with the following works:

# define _DMON_WINAPI_STR(name, size) wchar_t _##name[size]; mbstowcs(_##name, name, size)

kqueue has been available since the early 2000s and is available on older MacOS systems as well as all BSD variants I am aware of.
This would be a nice fallback for pre-FSnotify Mac's and BSDs

I have got an assertion fail in the function dmon__fsevent_callback for the macOS implementation. It happens in the line there:

        // strip the root dir
        DMON_ASSERT(strstr(abs_filepath, watch->rootdir) == abs_filepath);
        dmon__strcpy(ev.filepath, sizeof(ev.filepath), abs_filepath + strlen(watch->rootdir));

The reason is that although abs_filepath actually begins with watch->rootdir but they have a different case. I think if the directory for which an event is reported has some uppercase part it is reported with lowercase anyway.

I did some research and on macOS the filesystems can be case sensitive or not and it depends on the volume. In practice I think that most commonly volumes are case insensitive. In theory we should compare with a case sensitivity adapted on the volume but I am perplexed.

If you modify a file in a watched directory, it is reported as a "create" event. I used a textedit file and changed some text, and saw that this was reported as a "create" event.

Possibly related: #25, emcrisostomo/fswatch#279

When a new directory is created inside a folder we watch recursively the directory creation is correctly reported but on linux any change inside the new directory is not reported. The same things works correctly on Windows.

Example:

1. watch recursively directory /tmp
2. mkdir /tmp/foo => directory creation correctly reported
3. touch /tmp/foo/test.txt => file creation not reported

I think that on linux we just need to add a watch on each newly created directory if the recursive flag is enabled.

Hey,
I've just noticed an inconsistency between linux and windows, windows seems to just have the filename in the callback filepath, where linux has the full file path.
What is the desired behaviour?
Thanks,
Wayne

I am beginning to integrate dmon in our project lite-xl, a programming editor.

The library is amazing, thank you for publishing this work as open source, this is great.

I stumbled on a problem on linux (on windows it was okay). When watching recursively a directory and you create or remove a directory no notification is sent at all. For example:

• watch /tmp (RECURSIVE flag on)
• touch /tmp/test.txt => ok, get a notification
• mkdir /tmp/foo => no notification
• touch /tmp/foo/test.txt => no notification
• rm /tmp/foo/test.txt => no notification
• rmdir /tmp/foo => no notification

It can be verified with the "test" application included in the repository.

I am using the latest version of dmon as I am creating this issue.

The same scenario works perfectly on windows.

I didn't investigate but it looks like if the "recursive" option is not applied on linux.

Thank you in advance for any help.

I would need the function dmon_unwatch so that I can unwatch a directory and watch another one.

The function essentially already exists in the current implementation but it is private.

There are some remnants of sx_ functions and defines. They are very easy to replace (strlen, min, platform defines) and would make the library independent.

Hi - and thanks for a useful repository 🙂,

I'm on Linux and currently unable to monitor a directory for directory changes:
I.e. if I watch /tmp/wt and do mkdir /tmp/wt/test_dir it's not picked up, but touch /tmp/wt/test_file is.

Is this expected? (If yes - is it on your roadmap to add such a feature?)

Hi,

First, thanks for sharing your work. Maybe i missed something in the documentation but dmon raise an assert when unwatching dirs in the same order that they were supplied, maybe will happen if unwatching randomly too. Below is a small test programm to illustrate this behaviour.

#define DMON_IMPL
#include "dmon.h"

#include <assert.h>

void cb( dmon_watch_id id,  dmon_action action, const char* dir, const char* filename, const char* old_filename, void* opaque )
{
}

void main()
{
    static const char* dirs[] = { "/bin/", "/home", "/tmp", "/usr", "/usr/share", "/var" };
    dmon_watch_id ids[6];
    dmon_init();

    for( int i = 0; i < 6; ++i )
    {
        ids[i] = dmon_watch( dirs[i] , cb, 0, NULL );
        assert( ids[i].id != 0 );
    }

   for( int i = 0; i < 6 ; ++i )
       dmon_unwatch( ids[i] );
   dmon_deinit();
}

Not super familiar with system APIs but based on my understanding of dmon's docs, it seems that directory monitoring should be nonrecursive by default. However, with dmon_watch(dir_.c_str(), watch_callback, 0, data_.get()), I do seem to get recursive notifications on directories.

On macOS 14. Mostly curious if this is known behaviour/intended. I can work around it so it's not really a big deal.

Great library, by the way!

Sanitizer reports incorrect usage of a the quit bool accross threads, which seems to be a correct positive afaics.

I tried adding atomic to the quit bool but then dmon refuse to compile.

==================
150: WARNING: ThreadSanitizer: data race (pid=7868)
150:   Write of size 1 at 0x55aff3fd05f0 by main thread:
150:     #0 dmon_deinit /__w/f3d/f3d/source/external/dmon/dmon.h:1150:16 (f3d+0x172842) (BuildId: 2788a4d430378bd83bc4b94da37bf9dff66918f4)
150:     #1 F3DStarter::~F3DStarter() /__w/f3d/f3d/source/application/F3DStarter.cxx:196:3 (f3d+0x1738f7) (BuildId: 2788a4d430378bd83bc4b94da37bf9dff66918f4)
150:     #2 main /__w/f3d/f3d/source/application/main.cxx:14:3 (f3d+0x17e7fa) (BuildId: 2788a4d430378bd83bc4b94da37bf9dff66918f4)
150: 
150:   Previous read of size 1 at 0x55aff3fd05f0 by thread T1:
150:     #0 _dmon_thread(void*) /__w/f3d/f3d/source/external/dmon/dmon.h:1055:19 (f3d+0x171231) (BuildId: 2788a4d430378bd83bc4b94da37bf9dff66918f4)
150: 
150:   Location is global '_dmon' of size 840 at 0x55aff3fd02b0 (f3d+0x15c25f0)
150: 
150:   Thread T1 (tid=7872, running) created by main thread at:
150:     #0 pthread_create <null> (f3d+0x64e4d) (BuildId: 2788a4d430378bd83bc4b94da37bf9dff66918f4)
150:     #1 dmon_init /__w/f3d/f3d/source/external/dmon/dmon.h:1137:13 (f3d+0x1736e4) (BuildId: 2788a4d430378bd83bc4b94da37bf9dff66918f4)
150:     #2 F3DStarter::F3DStarter() /__w/f3d/f3d/source/application/F3DStarter.cxx:189:3 (f3d+0x1736e4)
150:     #3 main /__w/f3d/f3d/source/application/main.cxx:12:16 (f3d+0x17e7e1) (BuildId: 2788a4d430378bd83bc4b94da37bf9dff66918f4)
150: 
150: SUMMARY: ThreadSanitizer: data race /__w/f3d/f3d/source/external/dmon/dmon.h:1150:16 in dmon_deinit
150: ==================

I noticed the dmon_extra.h wrap the declarations of the functions dmon_watch_add/rm with the directive #if DMON_OS_LINUX but this doesn't work because DMON_OS_LINUX is only defined when DMON_IMPL is activated so it doesn't work for simple declarations.

I propose to declare dmon_watch_add/rm unconditionally or otherwise ensure that DMON_OS_LINUX and the other variables are always initialized even outside of the implementation.

Hi,
Thanks for the useful library, I'm using it on linux, but i've come up to an issue where dmon will use a high amount of CPU on Ubuntu 18.04.

I've looked at the code, it seems to be an issue with the use of select().
select() will set the timeout to the amount of time not slept on return, which is going to be 0 if no filesystem events occurred. The timeout doesn't ever get reinitialised, so subsequent calls to select return immediately and we end up in a high speed loop situation.

I observed different behaviour in debug mode, select returned -1 and errno was 22 invald argument, I don't know if this is something specific to my program.

There is another thing I noticed that could be done more efficiently.
At the moment each watch is checked individually in a loop. It should be possible to populate rfds with all the current watches and use a single select to wait for changes rather than 1 each. This will cause delays of up to num_watches * 0.1s for responding to changes.

If you agree with these issues I'm happy to look at doing PR's.
Thanks again :)

It would be nice to have a generic POSIX implementation using poll(2) or select(2)

I propose the addition of a new function to add a directory to watch into an already existing watch_id.

The inotify interface let you add an additional directory to watch on an existing inotify file descriptor. The current dmon API doesn't currently allow this kind of behavior. I propose here is to add a public function reproducing the same behavior.

The reason I am doing this proposition is to address a real problem I encountered using dmon. Let's say you want to monitor a directory but, when including all the sub-directories recursively, the number of files is too big to be something you can watch recursively. For example imagine something like the "home" directory or something like that.

In a case like this you cannot watch the directory recursively because there are too many files. On the other side your application don't want to index all the files either but will probably show a shallow listing of the files in the top directory and will show the files inside the subdirectories only on demand, when required by an explicit user action. In this case you can watch the top directory non-recursively but what happen if you add a sub-directory ? Well you can add another watch but if every time you get a new watch id thing get unmanageable pretty quick. In reality you want to watch the sub-directories under the same watch id to have something sane on the application side. In addition inotify let you do this natively and efficiently by using an already file description so I advice doing the same using an existing watch_id.

In practice it would be like when you watch a directory recursively but you only add sub-directories on-demand, when required. This would be a refinement of the current approach that force the watch to be either shallow or completely recursive.

I am chasing a difficult to reproduce bug in my application using dmon.

It seems to happen when I get a lot of accumulated files' events when the application is unfocused, for example when I switch to the terminal and I delete recursively a directory with a lot of files in a place monitored by dmon. I suspect a memory corruptions starting in dmon but I am not sure about that.

So I am asking, if I want to investigate in the dmon implementation for Windows what things may cause a buffer overflow or something similar when a lot of events are reported ?

Just for completeness: on windows I use dmon recursively on the directory to monitor and I watch only a single directory.