Add support for etcd API v2 about python-etcd HOT 13 CLOSED

http://docs.etcd.apiary.io/ is the reference docs;

I see that a lot of things have been improved, e.g. recursive fetch has been implemented on the server. My idea is to add an optional parameter on the constructor of the client indicating if one version of the api should be forcibly used, and to raise a NotImplementedError for methods called in on v1 that are not implemented by the API.

I'll start working on this as soon as I have some slack time :)

from python-etcd.

Didn't know about those docs, thanks! I've been looking at the etcd code, all the request and the new stuff seem to be at:
https://github.com/coreos/etcd/blob/0.2/server/server.go

and
https://github.com/coreos/etcd/tree/0.2/server/v2

I also like the removal of watch in favor of a GET+wait (less redundant) and the 'prevExist' for the PUT method. I was looking for alternatives to create distributed locks using etcd, and needed that atomic construct.

I think we can do this without disturbing too much the actual API, so I set this for 0.4.0, no rush though, I'd wait until they settle the API before releasing it.

As for the NotImplementedError, I'd ask on the constructor the minimal API version required, check the version on the server (there seems to be an undocumented "/version" that we could probe) and break on anything lower than the required, we'd save lots of checks later.

from python-etcd.

Etcd 0.2 moved to the v2 api; I'll try to work on this during the weekend, If I come up with something decent I'll create a pull request.

from python-etcd.

We have a bit of a problem regarding support for the v2 API: responses have changed significantly between v1 and v2, in particular when you deal with directories.

This puts us in the unpleasant situation where we may need to break the API-agnostic idea we had before and probably the wise choice is to break backwards compatibility as I strongly suspect no one will ever use api v1 anymore.

I'm trying to figure out a way to keep compatibility, but if that isn't possible do you agree on dropping v1 support?

from python-etcd.

Hi! Glad to see you found some time :-) I guess It had to happen sooner or later. Probably our best option is to break backwards compatibility, don't think we've got a "userbase" big enough to worry about, anyway :-D

from python-etcd.

@jplana I managed to find some more time today and I finished a first working (meaning: it works and passes all present unit tests) version of python-etcd with working support for the v2 API.

I refactored everything so that old methods ('get','set','watch'...) are calls to the fundamental methods, that for v2 are basically just "write" "read" and "delete".

I tried to keep everything as compatible with the old client as possible, but some inconsistencies are there. Let's say that if you have a simple program using python-etcd, it will work with v2 api (and the new version).

I have still a few rough edges to polish, but it seems like I'm there. If you care to comment, my code is here:

https://github.com/lavagetto/python-etcd/tree/v2_support

I'm also planning to add a sub-module implementing some common pattern like distributed locks and master election, and another with an etcd shell (the only ones around are quite underwhelming, and they do not support v2!).

Are you interested in including them in the master tree?

from python-etcd.

That's great. I'll have a look at the code in your branch.

As for the the submodules I'd love include some basic abstractions in the master tree, as they won't require any other dependencies, but probably it'll be better to maintain the shell (which BTW is a nice idea I hadn't think about) separated, to avoid getting too many unrelated libraries (readline? ncurses?) as a direct dependency to the module. But let me know about your plans to implement the shell, I'd love to help :-)

from python-etcd.

Just pushed some more changes. I moved the unit tests on the requests and api interface to pymox, as it allows stricter controls on the interface you're mocking, and honestly I prefer it compared to mock.

About the shell, you're perfectly right, I'm going to create a separate repo. Not sure when I'll have time though :(

from python-etcd.

BTW, I need to make python-etcd python3-compliant and guess what? Pymox does not work with python 3. I may be switching back to mock... This really saddens me :/, but please hold while I find a solution.

from python-etcd.

Sorry about the late answer. I had a look at the code, and found that some of the commits are actually related to reconnections. I still have #5 pending. Do you agree with the alternative way of handing reconnections? If you do, I'll merge so we can separate concerns.

About pymox vs mock, I'd like to keep mock, at least by now. Not that I really prefer it, but it is already included in Python 3.3 standard library, and we would have to migrate all tests to pymox. Is there any specific issue with mock?

from python-etcd.

Mh, strange. I'm pretty sure I separated all the reconnection work from the v2_support branch that I rebased on your latest commit.

Btw reconnection and v2 support are pretty much separated (apart from a very plain change), so yes I agree with your work in #5 .

Aboyt pymox vs mock: pymox is better in some ways IMO, but I agree with you and I'll rewrite the new tests using mock. Since I'm recovering from a flu, tomorrow may be the good day for that. :)

from python-etcd.

Ah, I got why you've seen reconnection-related commits...

I gave you the wrong url for the branch (it was v2_client and not v2_support, d'oh).

I'm anyway working so that I'm able to achieve the following:

1. have a client that fully supports v2
2. that is compatible with both python 2.7 and 3.3
3. That is "mostly" compatible with the old one.

I should be done by this evening (though I have to rebase and verify that the reconnection code actually works in python3).

from python-etcd.

FYI for mocking, you might also look at httpretty

from python-etcd.