Etcd ignores error code and exits prematurely on Watch about libkv HOT 8 CLOSED

(From @springi99)

Until the moving is done related to endpoint top directory (or KeyPrefix):

--- a/vendor/src/github.com/docker/libnetwork/store.go
+++ b/vendor/src/github.com/docker/libnetwork/store.go
@@ -87,7 +87,8 @@ func (c *controller) deleteNetworkFromStore(n *network) error {
        if err := cs.DeleteObjectAtomic(n); err != nil {
                return err
        }
-
+       tmpEp := endpoint{network: n}
+       cs.KVStore().DeleteTree(datastore.Key(tmpEp.KeyPrefix()...))
        return nil
 }

solves the problem. Sorry about veth pairs they are not belonging to docker.

from libkv.

@springi99 I think the default behavior of List exiting right away is the right one, this avoids launching a goroutine inappropriately because the failure on List might be related to a network connectivity issue or a wrong key.

It is to the users of the lib (in this case libnetwork) to deal with the Watch/WatchTree failure and catch the appropriate error as well as retry on failure if the Watch should be restarted.

The missing catch on error 100 still holds though so if you are interested in submitting a patch I'll be happy to review an merge :)

Same goes for the changes you made on libnetwork, it will definitely help on this side as well :)

from libkv.

Hi,

The problem with the WatchTree is the following: networks for defaults (zero, host, bridge) are also creating this watch and it fails with etcd and not with consul. if you compare the consul.go and etcd.go you can see that in case of failure etcd returns with an error and consule not. This is why i thought it is enough to remove that return. For other networks (overlay and custom/remote e.g. openvswitch from shettyg/ovn-docker) creates the right tree structure. for remote networking some features are still missing, but you can substitute them with hand made scripts.

Related to commiting: do you have any process description for that? Unfortunately i am a newbie and I don't know how can I do it officially.

I have other questions related to networking, if you can provide me resources to learn it would be nice:

• the difference between endpoint and service: i am a little bit confused as there are rest apis for both of them but on cli only services can be attached. Currently it seems to me that they are equivalaent but maybe they will mean different things in the future. E.g. on cli i can use only 1 publish and it is splitted up by the code and attach "endpoint" according to the result. If there are more publishes in the cli only the last one is taken. There were some words about "service container" but i did not catch the point. I think endpoint is somehow lighter by concept, mean multiple interfaces to the containers, but there is no cli for that.

Thanks your help!

from libkv.

@springi99 Oh I see, you are right! I think the Get and List calls of Watch/WatchTree are just misplaced for etcd and should be placed inside the goroutine so that the return actually closes the watch channel (like it's done with Consul). Just tested on the Leader Election recovery of Swarm and it works just fine, so I'd expect it to work as well with libnetwork. I will submit the PR for this as we also need it for docker swarm.

On the whole process of contributing I think that this contains all you need to get started:
https://docs.docker.com/project/make-a-contribution/

If you need some help on getting started with your first PR, do not hesitate to ping us on our IRC channel: #docker-swarm

Regarding your question on libnetwork, I think you can ask this on the docker/libnetwork repository, opening a new Issue. They will provide you with a more in-depth answer that I can provide you and this will also help other users with the same question. Don't want to make a mistake and mislead you on the notion of endpoint and service as things are moving fast on this side :)

Thanks for reporting the issue in the first place though, helps a lot to have users testing things around not only on this repository but on the interaction with other projects! This is a contribution on its own.

from libkv.

Hi,

sorry for the delay... I tried to follow the process, the result can be found here:

https://github.com/springi99/libkv/tree/etcd-fixes

if you think it is ok then tell me what to do. Thanks, robert.

from libkv.

Hi @springi99, sorry for the delay. If you go to your fork (https://github.com/springi99/libkv/) you should see a button "Create pull request" that appears on top. Click on that, and then click "Create pull request" on that new page. Don't worry putting a description there, it's fine for this PR. Let me know if you have any trouble. I can still create it for you and you'll get credit for your work anyway. Thanks!

from libkv.

Hi,

I created it, please check it. Thanks in advance!

from libkv.

Closed by #35

from libkv.