Comments (3)
Yeah, I can't think of a simpler alternative that does not degrade performance. Also considering that the code we have in place seems kind of stable and is not creating a lot of maintenance burden. Keeping it around for another while should be cheap.
Probably the only place we'd need to modify in order to add libuv+io_ring support would be src/uv_writer.c
. The src/uv_writer.h
interface would remain the same, so the rest of the code will work unmodified, but internally in UvWriter
we'd have a new code path using libuv+io_uring.
from dqlite.
Just opening an issue so that we have a place to discuss this. The libuv maintainers recently merged support for using io_uring to implement uv_fs APIs on sufficiently recent Linux kernels (the first release with this code is v1.45.0). It would be great to replace our hand-rolled async file I/O code with some simple calls to uv_fs, but there are some things to think about:
- We probably want to support libuv versions that predate the io_uring implementations
- Even if we didn't, we definitely want to support Linux kernels that don't have io_uring
As I see it, we can either use uv_fs_* unconditionally and live with the consequences on old libuv/Linux, or to keep some handwritten fallback code around that does something other than blocking syscalls on the thread pool. I suspect the first option is not acceptable, and so the question becomes: what will the fallback code look like?
- We could keep all our or current code, so the decision tree looks like: if io_uring, then use uv_fs; else if RWF_NOWAIT, then use that implementation; else, run it on the thread pool.
Since we already have the implementation and performance-wise it's going to be better than libuv's threadpool native fallback, I'd say let's keep it around for a while and once we don't need to support anymore older kernels/libuv we can get rid of it.
- We could have a simpler fallback, perhaps without KAIO in the mix.
What would that simpler fallback look like? It feels that simplest fallback would be libuv's native fallback itself (threadpool), which we would get for free, and performance wise that's the max we could get because the only async alternative to io_uring on linux is KAIO.
from dqlite.
What would that simpler fallback look like?
I'm not sure either, that's why I didn't elaborate :). No objection to keeping the KAIO around.
from dqlite.
Related Issues (20)
- Fix Coverity
- Use execinfo.h instead of libbacktrace HOT 3
- Set up tmate action
- Set up a canary PPA to get feedback from LP builders before merge HOT 2
- no known leader error forever
- Node.js bindings HOT 1
- Fix cases where the server can return a FAILURE message with result code 0
- Investigate what causes LXD's dqlite to create lots of singleton closed segments
- Flaky test: cluster/restart HOT 5
- 1.16.4: build fails on linking with missing `raft_register_state_cb` symbol HOT 1
- posix_fallocate extremely slow on ext3 file system HOT 1
- 1.16.5: build fails with `-Wimplicit-function-declaration` HOT 9
- Documentation links are broken HOT 3
- dqlite requires a large stack HOT 3
- MicroOVN built with latest Dqlite crashes on cluster formation. HOT 5
- Investigate CPU spikes when performing role adjustment
- Term increment is skipped in one-node clusters
- Tracking issue for improved observability
- Recursion in EXEC_SQL request handling
- build_mac & build_windows
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dqlite.