roma-glushko / notifykit Goto Github PK

A quickest possible approach to reuse the current sync API in asyncio-based apps

Returns events in batches in order we have detected them.

Event though we are using notify-rs (that supports inotify-like watches across various platforms like macOS, Linux, Windows, etc) library in inotifykit, it's still tricky to get fully cross-platform API.

The problem is in events/information that different platform support. For example,

• some platforms support file metadata change events and some not
• some platforms emits rename as a separate event and some emits two events instead (old file deletion + new file creation)
• etc.

Because of this discrepancies in how events are dispatched, it still requires some work to smooth that experience to make it resemble a real platform-agnostic API.

It would be great to put that heavy-lifting on inotifykit, so our users could really focus on their business logic.

Solution

It seems like a way to go is to find a common ground between all supported file watcher's API and unify it on the library level.

One ramification of this is that we would need to hide some of the events or information that is not widely supported by all watchers. it seems like a reasonable limitation as if you had an access to that information, it would be impossible to run that code on the platform that didn't support it (so we would break our cross-platform promise).

This behavior should be optional, so people that needs missing info and don't need true cross-platform support would be able to have that.

I'm thinking of incorporating the full debouncer logic to extend it a way notifykit usecases require.

For now, there are a few benefits from that:

• keep one single thread to process raw notify-rs events
• bring better async support
• ability to address bugs like #10
• expose more information in notifykit events potentially useful for end consumers (e.g. detection time, inode ID, etc.)

We are using sync approach to listening to events which works but there is one caveat. We had to interrupt after some time to see if we had not received a termination signal yet:

https://github.com/roma-glushko/inotifykit/blob/e3097ce41826908ec8f33c51c5b27afb33f02567/src/lib.rs#L38

So essentially, we are polling signal info instead of being notified when it happened.

Let's try to explore asynchronous approach to event watching/signal handling to save even more CPU cycles.

Right now, it's not match statement ignores Event classes when trying to match them there.

Also, we need to allowlist attributes like path, access_mode, etc, to be matched by the statement. It's not disallowed by default.

Here is a way to reproduce that on MacOS:

touch vi.com && mv vi.com vi.co

Instead of correctly stitching two rename events, debouncer produces two Rename(Any) events.

Reference:

• https://pyo3.rs/v0.20.0/ecosystem/logging