Comments (7)
The reason is that libuv's disk I/O uses a threadpool and hence is not fully asynchronous.
from dqlite.
But what implications does that have exactly, apart from (maybe) performance?
I'm interested in adding support for windows, but from what I'm seeing the AIO API is tightly coupled with the rest of the API without much abstraction. This makes it difficult to build out support for other platforms, despite the invitation to do so.
I'm still new to the code base and I haven't fully learned the internals as yet, but my first instinct would be to leverage the libuv's disk I/O abstraction for windows support. However, I'm not sure if this is practical.
from dqlite.
Yes, the only implication is performance.
And yeah you're right, unfortunately the implementation is currently a little bit coupled. I have planned to introduce an abstraction (also to move to the new io_uring
Linux API, which is better than AIO), but didn't have time yet.
I think practically speaking the easiest way would be to fork and modify uv_writer.c
to use libuv's disk I/O facilities instead of kernel AIO. That file is where most of the AIO-dependant code is.
After you get it working, we can see how to best abstract the two implementations and merge it.
from dqlite.
Actually, now that I think of it, we don't even need an interface. Basically the abstraction that is already in place is the raft_io
interface, for which the project provides a stock implementation based on libuv
. The work to do would be to detect the build host at compile time and if it's windows fallback to libuv
's support for disk I/O. Those #ifdef
's should live in uv_writer.c
.
from dqlite.
Introduction
These are a few of my thoughts/observation based on my analysis of the code.
Naming conflict
The following function, defined in heap.h
, conflicts with a function of the same name, defined in Windows' heapapi.h
:
void HeapFree(void *ptr);
Obviously, this problem only occurs when targeting Windows, but the naming scheme of the functions defined in heap.h
should probably be reviewed. For now, I've renamed them to MyHeapxxx() to avoid conflict, but I won't PR that change. I'll leave it up to you to decide on the best strategy here.
Progress
I haven't had much free time lately, but I've tried doing some work in implementing windows support - mostly in uv_writer.c
as you said. However, there were some other things that needed to be addressed, like the naming conflict above and appropriate includes for uv_ip.h
.
So far, I've been able to successfully complete the build process using:
autoreconf -i
./configure --host=x86_64-w64-mingw32
make
But the examples still needs some work. Also I haven't tested the resulting libraries, so my changes are likely buggy. All changes are on my branch.
from dqlite.
Hello folks, working on canonical/raft#173 to continue this effort in MacOS direction, and found that during raft_add
applyChange
is running before the actual set of leader_state.change to run raftChangeCb
and return something - that causing the second node to hang.
Particularly this part is bugging me: https://github.com/canonical/raft/blob/v0.9.25/src/client.c#L201-L207 - on Linux it's working just fine, but on MacOS the applyChange
is actually executed before the r->leader_state.change = req;
line. I tried to move it before the clientChangeConfiguration
call (like here: https://github.com/canonical/raft/blob/v0.9.25/src/client.c#L277-L288 ) - but ended up with segfault... Maybe someone could suggest the proper way to fix that?
from dqlite.
Ok, finally found the actual issue and now PR contains the working patch.
from dqlite.
Related Issues (20)
- Multi threading bug in tracef() HOT 1
- Jepsen: assertion failure in vfs.c HOT 1
- Nested barriers
- 1.16.0: test suite fails HOT 17
- Method to be informed about latest change in table HOT 2
- Several singular nodes joined later together HOT 2
- Run dqlite tests against most recent SQLite release. HOT 1
- Clarification on dqlite read operations HOT 1
- 1.16.2: build fails with raft 0.22.0 HOT 3
- Fix Coverity
- Use execinfo.h instead of libbacktrace HOT 3
- Set up tmate action
- Set up a canary PPA to get feedback from LP builders before merge HOT 2
- no known leader error forever
- Node.js bindings HOT 1
- Fix cases where the server can return a FAILURE message with result code 0
- Investigate what causes LXD's dqlite to create lots of singleton closed segments
- Flaky test: cluster/restart HOT 5
- 1.16.4: build fails on linking with missing `raft_register_state_cb` symbol HOT 1
- posix_fallocate extremely slow on ext3 file system HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dqlite.