C or C++ example about dqlite HOT 5 CLOSED

I am thinking that the App process can call the C API in dqlite.h to create a node, get the handle (sqlite3*) to local sqlite object, and execute the sql commands. The replication will handled by dqlite using raft back-end.

Unfortunately this kind of model (you get a sqlite3* handle and use it with normal SQLite C function calls) would not work. The reason is basically mentioned in the Architecture document you have linked:

The SQLite model is serverless and there’s no network communication involved when running queries: your application process simply invokes SQLite library functions which under the hood read and write data directly from and to the local disk.

The Dqlite model is more similar to a “regular” database with client/server network communication: each of the application processes in your cluster spaws a Dqlite server thread and then uses a Dqlite network client which under the hood connects to the particular Dqlite server thread that happens to be the current cluster leader. This means that the SQL queries issued by an application process using the Dqlite network client might end up being processed by the Dqlite server thread of that very same process (if that node is currently the leader) or by the Dqlite server thread of another application process.There is an important difference between the two models above though.

The SQLite model is serverless and there’s no network communication involved when running queries: your application process simply invokes SQLite library functions which under the hood read and write data directly from and to the local disk.

The Dqlite model is more similar to a “regular” database with client/server network communication: each of the application processes in your cluster spaws a Dqlite server thread and then uses a Dqlite network client which under the hood connects to the particular Dqlite server thread that happens to be the current cluster leader. This means that the SQL queries issued by an application process using the Dqlite network client might end up being processed by the Dqlite server thread of that very same process (if that node is currently the leader) or by the Dqlite server thread of another application process.

In a nutshell, if you had a plain sqlite3* handle connected to the local node, that node might not be the current leader and therefore you wouldn't be able to commit database changes. We could implement what you describe by having the network part (e.g. contacting the leader) happen under the hood inside some modified version of SQLite which maintains the same C API, but that would be beneficial mostly for C clients only (because for other languages it would be better to have native clients that speak the dqlite wire protocol). Also, such a modified version of SQLite would effectively become a C dqlite client, that just happens to have the same API as SQLite. At that point you can probably see that it makes more sense to leave SQLite alone and have a dedicated C dqlite client, where you would have a dqlite* handle and a set of APIs similar to SQLite, but that speak the dqlite wire protocol under the hood. As for now, we don't have plans of writing such C client, but contributions would be welcome.

from dqlite.

There's currently no C/C++ client for dqlite, that I'm aware of. However the protocol is documented here, in case one is willing to write such a client.

from dqlite.

I was looking for the same, I am glad there is already a thread on it. I am planning to switch from a home grown distributed embedded DB to dqlite, to take advantage of the powerful SQL features of sqlite without losing redundancy. I have looked into dqlite.h header file, so there is a C API. Just no examples of how to use it.

After seeing the architecture, I was under the impression that dqlite is an embedded DB like sqlite, but this wire protocol (over socket) is confusing me. If you look at the second diagram in the architecture, I only have "Your App Process" and no "Your App Client".

I am thinking that the App process can call the C API in dqlite.h to create a node, get the handle (sqlite3*) to local sqlite object, and execute the sql commands. The replication will handled by dqlite using raft back-end.

Is this not so? Is there any example code for this?

from dqlite.

Ok, understood. I had assumed that you had a custom sqlite where you re-implemented sqlite3_exec() function to re-direct the execution to the leader. But you are saying dqlite is above sqlite layer, and accessing sqlite directly will bypass the dqlite functionality. And dqlite layer only provides wire protocol.

Now, whoever wants to write a C/C++ client has two choices. Over the socket using wire protocol or just an api that directly calls the server side of wire protocol implementation, skipping the socket part. I think both will be useful in different cases.

from dqlite.

Filed #291 which should cover this.

from dqlite.