Not Leader failure response about dqlite HOT 5 CLOSED

But, the client shouldn't be expected to book keep who the leader is, should it? Maybe I am misunderstanding some basic concepts.

Yes, currently this is the client's responsibility. go-dqlite has logic to forward requests to the current leader, and so does the WIP high-level C client library. It's definitely a bit inconvenient if you're writing your own client.

Note that any client of a distributed system must have knowledge of at least some of the nodes that are part of the system. That is necessary because any of those nodes might be down and in that case the client will need to try with another node. That logic might look like:

for (i = 0; i < n; i++) {
    node = all_nodes_the_client_knows_about[i];
    rv = try_connect(node);
    if (rv == 0) {
        break;
   }
}

Since that type of logic must be in place in the client no matter what, it should only be necessary to modify it slightly, like this:

for (i = 0; i < n; i++) {
    node = all_nodes_the_client_knows_about[i];
    rv = try_connect(node);
    if (rv == 0 && node->is_leader) {
        break;
   }
}

All that being said, implementing request forwarding so that the client can connect to any node is surely possible. Note though that it will increase latency (there will be an extra network hop, like a proxy), so all in all perhaps it's better to have the client do a bit of extra work.

from dqlite.

But, the client shouldn't be expected to book keep who the leader is, should it? Maybe I am misunderstanding some basic concepts.

Yes, currently this is the client's responsibility. go-dqlite has logic to forward requests to the current leader, and so does the WIP high-level C client library. It's definitely a bit inconvenient if you're writing your own client.

Note that any client of a distributed system must have knowledge of at least some of the nodes that are part of the system. That is necessary because any of those nodes might be down and in that case the client will need to try with another node. That logic might look like:

for (i = 0; i < n; i++) {
    node = all_nodes_the_client_knows_about[i];
    rv = try_connect(node);
    if (rv == 0) {
        break;
   }
}

Since that type of logic must be in place in the client no matter what, it should only be necessary to modify it slightly, like this:

for (i = 0; i < n; i++) {
    node = all_nodes_the_client_knows_about[i];
    rv = try_connect(node);
    if (rv == 0 && node->is_leader) {
        break;
   }
}

All that being said, implementing request forwarding so that the client can connect to any node is surely possible. Note though that it will increase latency (there will be an extra network hop, like a proxy), so all in all perhaps it's better to have the client do a bit of extra work.

So I tried doing something similar to this, but what I am seeing is that (going back to my diagram above) instance B's copy of the database does not match instance A's once instance A has closed and handed leadership to instance B.

To give some more detail:

Instance A initializes it's dqlite_server with:

  const char *server_address = "127.0.0.1:8080";
  int err;

  err = dqlite_server_create(aDir, aServer);
  err = dqlite_server_set_address(*aServer, server_address);
  err = dqlite_server_set_auto_bootstrap(*aServer, true);
  err = dqlite_server_start(*aServer);

and Instance B initializes its dqlite_server with:

  const char *addrs[] = {"127.0.0.1:8080"};
  const char *server_address = "127.0.0.1:8082";
  int err;
  int thisClientID = 1234;

  err = dqlite_server_create(aDir, aServer);
  err = dqlite_server_set_address(*aServer, server_address);
  err = dqlite_server_set_auto_join(*aServer, addrs, 1);
  err =
      dqlite_server_set_connect_func(*aServer, endpointConnect, &thisClientID);
  err = dqlite_server_start(*aServer);

I excluded endpointConnect() but it definitely connects to instance A.

• After everything is initialized, instance A successfully creates a database, adds some content via an INSERT, and reads it back confirming it's there via a SELECT.
• At this point, instance A pauses at which point instance B is started and attempts to connect to instance A's dqlite_server, which it does successfully.
• Instance B then successfully reads one of the tables from instance A's dqlite_server via a SELECT command.
• At this point, I manually continue instance A, and it gracefully closes, and calls dqlite_server_handover() before instance A exits.
• I then continue instance B, and it attempts to make another SELECT call, this time failing (because its connection to instance A's dqlite_server no longer exists). It then successfully connects to its own instance of dqlite_server.
• It then is required to send the single world containing the protocol version, the open database message (containing the same database name that instance A created), and finally the SELECT call, which it does successfully.
• Instance B's dqlite_server then responds with a Failure Response message containing "no such table: <table name that was created by instance A>".

From this, it appears that information applied to the dqlite_server leader is not being propagated to other dqlite_servers.

What am I doing incorrectly?

Edit: Got it working. Found a test case that outlines how this should be done, and followed that.

from dqlite.

But, the client shouldn't be expected to book keep who the leader is, should it? Maybe I am misunderstanding some basic concepts.

Yes, currently this is the client's responsibility. go-dqlite has logic to forward requests to the current leader, and so does the WIP high-level C client library. It's definitely a bit inconvenient if you're writing your own client.

from dqlite.

But, the client shouldn't be expected to book keep who the leader is, should it? Maybe I am misunderstanding some basic concepts.

Yes, currently this is the client's responsibility. go-dqlite has logic to forward requests to the current leader, and so does the WIP high-level C client library. It's definitely a bit inconvenient if you're writing your own client.

Speaking of that work in progress C client library. Any update as to when that will be completed?

from dqlite.

Speaking of that work in progress C client library. Any update as to when that will be completed?

I apologize for the delay -- I put #525 aside in favor of other work that we saw as higher-priority. I intend to return to it soon.

from dqlite.