jplana / python-etcd Goto Github PK

I tried to make a simple function to find empty etcd folders like this:

def find_empty(etcd_client, key):
    empty = True
    logging.debug("checking key %s", key)
    for e in etcd_client.read(key).children:
        empty = False
        if e.dir:
            find_empty(etcd_client, e.key)
    if empty:
        logging.debug("Found empty key %s", key)

But this crashes with a maximum recursion depth exception rather than printing the message when it encounters an empty dir.

Any etcd client should support the ability to close the HTTP request after a timeout; this is particularly important for watch and lock primitives.

Example

c = etcd.Client()
c.read('/testkey', wait=True, timeout=5000)
# we should also support this:
c.set_timeout(5000)
c.read('/testkey', wait=True)

• Timeout should be in milliseconds, not seconds (if urllib3 allows that...)
• If no timeout or 0 is provided, then no timeout will be applied.

commit: b4ff9ce

Assuming the value of '/omsstaging/db/name' is oms_staging

pprint(client.read('/omsstaging/db/name').value)
client.write('/omsstaging/db/name', u'oms_staging1')
pprint(client.read('/omsstaging/db/name').value)

This will output

u'oms_staging'
u'oms_staging'

Desired output should be

u'oms_staging'
u'oms_staging1'

EtcdResult has all of the children flattened, and sometimes I really want to traverse the result as a tree (the way that etcd provides them). Is this something you would consider?

Hi,
I've noticed that you run your integration tests in travis, I am trying to do the same (for etcd-clojure) with my project.

I wonder how do you start the etcd server over there.

Thanks for your help

$ python setup.py test
running test
Searching for pyOpenSSL==0.13.1
Reading https://pypi.python.org/simple/pyOpenSSL/
Best match: pyOpenSSL 0.13.1
Downloading https://pypi.python.org/packages/source/p/pyOpenSSL/pyOpenSSL-0.13.1.tar.gz#md5=e27a3b76734c39ea03952ca94cc56715
Processing pyOpenSSL-0.13.1.tar.gz
Writing /var/folders/2j/32g5k66x3db08z030c5nj0sh0000gn/T/easy_install-wreh2Y/pyOpenSSL-0.13.1/setup.cfg
Running pyOpenSSL-0.13.1/setup.py -q bdist_egg --dist-dir /var/folders/2j/32g5k66x3db08z030c5nj0sh0000gn/T/easy_install-wreh2Y/pyOpenSSL-0.13.1/egg-dist-tmp-uy8_8H
warning: no previously-included files matching '*.pyc' found anywhere in distribution

Installed /Users/jplana/src/etcd-python/python-etcd-github-temp2/pyOpenSSL-0.13.1-py2.7-macosx-10.4-x86_64.egg
running egg_info
writing requirements to src/python_etcd.egg-info/requires.txt
writing src/python_etcd.egg-info/PKG-INFO
writing top-level names to src/python_etcd.egg-info/top_level.txt
writing dependency_links to src/python_etcd.egg-info/dependency_links.txt
reading manifest file 'src/python_etcd.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'src/python_etcd.egg-info/SOURCES.txt'
running build_ext
Traceback (most recent call last):
  File "setup.py", line 36, in <module>
    test_suite='tests.unit',
  File "/Users/jplana/.pyenv/versions/2.7.5/lib/python2.7/distutils/core.py", line 152, in setup
    dist.run_commands()
  File "/Users/jplana/.pyenv/versions/2.7.5/lib/python2.7/distutils/dist.py", line 953, in run_commands
    self.run_command(cmd)
  File "/Users/jplana/.pyenv/versions/2.7.5/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/Users/jplana/.pyenv/versions/temp2/lib/python2.7/site-packages/setuptools/command/test.py", line 138, in run
    self.with_project_on_sys_path(self.run_tests)
  File "/Users/jplana/.pyenv/versions/temp2/lib/python2.7/site-packages/setuptools/command/test.py", line 118, in with_project_on_sys_path
    func()
  File "/Users/jplana/.pyenv/versions/temp2/lib/python2.7/site-packages/setuptools/command/test.py", line 164, in run_tests
    testLoader = cks
  File "/Users/jplana/.pyenv/versions/2.7.5/lib/python2.7/unittest/main.py", line 94, in __init__
    self.parseArgs(argv)
  File "/Users/jplana/.pyenv/versions/2.7.5/lib/python2.7/unittest/main.py", line 149, in parseArgs
    self.createTests()
  File "/Users/jplana/.pyenv/versions/2.7.5/lib/python2.7/unittest/main.py", line 158, in createTests
    self.module)
  File "/Users/jplana/.pyenv/versions/2.7.5/lib/python2.7/unittest/loader.py", line 128, in loadTestsFromNames
    suites = [self.loadTestsFromName(name, module) for name in names]
  File "/Users/jplana/.pyenv/versions/2.7.5/lib/python2.7/unittest/loader.py", line 91, in loadTestsFromName
    module = __import__('.'.join(parts_copy))
ImportError: No module named tests

Not that I use it a lot, (travis will execute bin/test created by buildout) but It seems a nice to have.

etcd API provides support for creating directories. Support for that seems missing in the current implementation of this library.

Or am I missing something? I haven't looked at the code in depth.

Only way to install is from source then? Just confirming because I see an etcd module via "pip search etcd".

When using the library now, if you just need to update the TTL of a key, you need to do as follows:

res = c.read('/somekey')

c.write('/somekey', res.key, ttl=res.ttl+100, dir=res.dir)

It would make more sense, IMO, to be allowed to do as follows:

res = c.read('/somekey')
res.ttl += 100
c.write(res)

I find libraries not able to eat their own dogfood a little appalling.

What is your opinion?

I'm starting to work on support for the built-in lock module in etcd 0.2.0. I wanted to get feedback on what the API should look like. Here's my proposal:

    # Acquire a lock (with no value) for "customer1"
    index = client.acquire_lock('/customer1', ttl=60)
    with client.lock('/customer1', ttl=60):
        # lock is released when the `with` block closes
        do_something()

    # Acquire a lock for "customer1" that is associated with the value "bar"
    index = client.acquire_lock('/customer1', ttl=60, value='bar')
    with client.lock('/customer1', ttl=60, value='bar'):
        # lock is released when the `with` block closes
        do_something()

    # Renew the TTL on the "customer1" lock for index 2
    client.renew_lock('/customer1', ttl=60, index=2)

    # Renew the TTL on the "customer1" lock for value "bar"
    client.renew_lock('/customer1', ttl=60, value='bar')

    # Retrieve the current value for the "customer1" lock.
    client.get_lock_value('/customer1')

    # Retrieve the current index for the "customer1" lock
    client.get_lock_index('/customer1')

    # Delete the "customer1" lock with the index 2
    client.unlock('/customer1', index=2)

    # Delete the "customer1" lock with the value "bar"
    client.unlock('/customer1', value='bar')

The comments there are directly from etcd's documentation on the lock module.

My biggest question: would it be preferable to return some sort of EtcdLock object that has some data about the lock?

Keep in mind that etcd only returns the index of the lock when one is acquired - this is odd to me given that setting a normal value returns a lot more info.

With etcd-v2.0.9-linux-amd64.
I run these 2 etcd process on two machine:
/home/idcuser/HATest/etcd-v2.0.9-linux-amd64/etcd -name monitorserver1 -listen-peer-urls http://10.120.73.86:7001 -listen-client-urls http://10.120.73.86:4001 -initial-cluster-token monitorserver-cluster -initial-advertise-peer-urls http://10.120.73.86:7001 -initial-cluster monitorserver1=http://10.120.73.86:7001,monitorserve2=http://10.112.206.10:2380 -initial-cluster-state new &>> /home/idcuser/HATest/etcd.log &

/home/idcuser/HATest/etcd-v2.0.9-linux-amd64/etcd -name monitorserver2 -listen-peer-urls http://10.112.206.10:2380 -listen-client-urls http://10.112.206.10:2379 -initial-cluster-token monitorserver-cluster -initial-advertise-peer-urls http://10.112.206.10:2380 -initial-cluster monitorserver1=http://10.120.73.86:7001,monitorserver2=http://10.112.206.10:2380 -initial-cluster-state new &>> /home/idcuser/HATest/etcd.log &

Then in one machine 10.120.73.86:
I run following command through python:
etcd= etcd.Client(host='10.120.73.86', port=4001)
leader = etcd.read('/ms/leader/name' ).value
etcd.write( '/ms/leader/name', serverName, ttl=TTL )

Exception throws when I calling etcd.write( '/ms/leader/name', serverName, ttl=TTL )
Traceback (most recent call last):
File "/home/idcuser/HATest/HA_monitor2.python", line 118, in
main()
File "/home/idcuser/HATest/HA_monitor2.python", line 110, in main
run()
File "/home/idcuser/HATest/HA_monitor2.python", line 34, in run
leaderSelect()
File "/home/idcuser/HATest/HA_monitor2.python", line 50, in leaderSelect
beLeader()
File "/home/idcuser/HATest/HA_monitor2.python", line 66, in beLeader
etcd.write( '/ms/leader/name', serverName, ttl=TTL ) # value=xxxx
File "/usr/lib/python2.7/site-packages/python_etcd-0.3.2-py2.7.egg/etcd/client.py", line 272, in write
response = self.api_execute(path, method, params=params)
File "/usr/lib/python2.7/site-packages/python_etcd-0.3.2-py2.7.egg/etcd/client.py", line 587, in api_execute
return self.handle_server_response(response)
File "/usr/lib/python2.7/site-packages/python_etcd-0.3.2-py2.7.egg/etcd/client.py", line 603, in handle_server_response
etcd.EtcdError.handle(**r)
File "/usr/lib/python2.7/site-packages/python_etcd-0.3.2-py2.7.egg/etcd/__init.py", line 184, in handle
raise exc(msg, payload)
etcd.EtcdException: Unable to decode server response

The etcd supports recursive watching on directories, but this library does not. Please implement this.

https://coreos.com/docs/distributed-configuration/getting-started-with-etcd/#watching-the-directory

$ etcdctl watch /foo-service --recursive
$ curl -L http://127.0.0.1:4001/v2/keys/foo-service?wait=true\&recursive=true

If I have time to implement it on my own, I'll submit a pull request. But I wouldn't count on me having time anytime soon :(

It looks like you're considering PUT and POST to be analogous, at least in this [recent] pull request:

#14

However, POSTs are now used to enqueue values using the v2 API (see "In-Order Keys" in the etcd documentation). You should probably switch to preferring PUT exclusively.

At the moment we do not have any logging ability and an user has no real way to understand what is going on in the library.

We should introduce some non-intrusive logging (so that at INFO nothing gets logged if all requests succeed).

This issue refers to the recent comment in a PR: #14 (comment)

It was mentioned that I could use your fork, but it didn't help.

I'm using tag v0.2.0 of etcd.

Instead of

install_requires = [
    'urllib3==1.7',
    'pyOpenSSL==0.13.1',
]

Use:

install_requires = [
    'urllib3>=1.7',
    'pyOpenSSL>=0.13.1',
]

Thanks to @roysmith for noticing it.

Our systems usually work by reading all the data for it's relevant subtree at startup and then wait for changes using read with wait and waitIndex. If this particular subtree has changed little then the modifiedIndexes of the elements in the subtree might be so small that passing them as waitIndex will not work. However we know the latest index at the time of the read is included in the http headers of the read. If we use that index as waitIndex to the read we know we will not miss changes and we are fairly certain that etcd will still have the relevant log. However, to do this we need to get this info out of the library

I try to build up etcd cluster according to:
https://github.com/coreos/etcd/blob/master/Documentation/clustering.md
The static way.

I run following command on 2 machines:
/home/idcuser/HATest/etcd-v2.0.9-linux-amd64/etcd -name monitorserver1 -listen-peer-urls http://10.120.73.86:7001 -listen-client-urls http://10.120.73.86:4001 -initial-cluster-token monitorserver-cluster -initial-advertise-peer-urls http://10.120.73.86:7001 -initial-cluster monitorserver1=http://10.120.73.86:7001,monitorserve2=http://10.112.206.10:2380 -initial-cluster-state new &>> /home/idcuser/HATest/etcd.log &

/home/idcuser/HATest/etcd-v2.0.9-linux-amd64/etcd -name monitorserver2 -listen-peer-urls http://10.112.206.10:2380 -listen-client-urls http://10.112.206.10:2379 -initial-cluster-token monitorserver-cluster -initial-advertise-peer-urls http://10.112.206.10:2380 -initial-cluster monitorserver1=http://10.120.73.86:7001,monitorserver2=http://10.112.206.10:2380 -initial-cluster-state new &>> /home/idcuser/HATest/etcd.log &

But when I run client.machines. It will output
machines [u'http://localhost:2379', u'http://localhost:2379', u'http://localhost:4001', u'http://localhost:4001']
Seems it still find cluster machines from localhost.

Under set a key in the docs it's mentioned that you can perform a mkdir operation:

# mkdir
client.write('/nodes/queue', dir=True)

That fails for me with

TypeError: write() takes at least 3 non-keyword arguments (2 given)

We use this library in our company and I noticed this project hasn't had any updates in some time.. can anyone shed some light on if it is still actively maintained and will receive updates to ensure it continues working with future versions of etcd?

If not, can anyone suggest another python library?

PyOpenSSL is only used to create snakeoil certificates during testing. Remove the certificate generation so this dependency, hard to install in the Windows platform, is not needed.

In versions of etcd >= 2.x, /v2/leader is not a valid endpoint anymore.

should we maintain backwards compatibility or should we just drop it?

Per the etcd team, I should be able to get the keys in the root:

curl -s http://localhost:4001/v2/keys/

It's not allowed with python-etcd:

>>> c.get('/')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Library/Python/2.7/site-packages/etcd/client.py", line 285, in get
    return self._result_from_response(response)
  File "/Library/Python/2.7/site-packages/etcd/client.py", line 327, in _result_from_response
    raise etcd.EtcdException('Unable to decode server response')
etcd.EtcdException: Unable to decode server response

Dustin

Hi,

pyOpenSSL library is required only for unit tests, but listed as a requirement for installation. This prevents installing python-etcd on RHEL5 which have outdated OpenSSL libraries.

Perhaps pyOpenSSL could be moved to test dependencies.

Implement support for https://github.com/coreos/etcd#leader-election

fields={'wait':'true'}, timeout=1,)

File "build/bdist.linux-x86_64/egg/urllib3/request.py", line 68, in request
File "build/bdist.linux-x86_64/egg/urllib3/request.py", line 81, in request_encode_url
File "build/bdist.linux-x86_64/egg/urllib3/poolmanager.py", line 153, in urlopen
File "build/bdist.linux-x86_64/egg/urllib3/connectionpool.py", line 581, in urlopen
File "build/bdist.linux-x86_64/egg/urllib3/connectionpool.py", line 581, in urlopen
File "build/bdist.linux-x86_64/egg/urllib3/connectionpool.py", line 581, in urlopen
File "build/bdist.linux-x86_64/egg/urllib3/connectionpool.py", line 561, in urlopen
File "build/bdist.linux-x86_64/egg/urllib3/util/retry.py", line 271, in increment
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='~~~', port=4001): Max retries exceeded with url: /v2/keys/test?wait=true (Caused by ReadTimeoutError("HTTPConnectionPool(host='~~~', port=4001): Read timed out.",))

and retries in urllib3.connectionpool default is urllib3.util.retry.Retry(3)

so, if watching a key with timeout, will fail?

I've been using python-etcd for all the client communication where the etcd is hosted remotely.

The python client connects to the server correctly when tried on a mac, but throws error on ubuntu.

The python code that I ran on both systems is:

import etcd
client = etcd.Client("55.88.55.161", 4001)
client.read("/_etcd/machines")

The error that ubuntu(in a VM and docker) threw was this:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/etcd/client.py", line 158, in __init__
    (self.__prefix, self.__machines))
ValueError: Could not identify given prefix [http://55.88.55.161:4001] among published prefixes: [[u'http://127.0.0.1:4001', None]]

Is there any way that the client treats the OS'es differently? Or is this an etcd quirk?

Trying to install python-etcd on Ubuntu within a Docker container on CoreOS.

Unfortunately, I am running into pyOpenSSL issues.

Prior to installing python-etcd. I installed OpenSSL (not sure if necessary) but failure below occurs with or without this step. Any advice is appreciated...

root@2a86bce32082:/etc/ibm/etech/coreos/python/python-etcd# openssl version
OpenSSL 1.0.1e 11 Feb 2013

root@2a86bce32082:/etc/ibm/etech/coreos/python# git clone https://github.com/jplana/python-etcd.git
Cloning into 'python-etcd'...
remote: Reusing existing pack: 529, done.
remote: Total 529 (delta 0), reused 0 (delta 0)
Receiving objects: 100% (529/529), 96.02 KiB, done.
Resolving deltas: 100% (256/256), done.
root@2a86bce32082:/etc/ibm/etech/coreos/python# cd python-etcd
root@2a86bce32082:/etc/ibm/etech/coreos/python/python-etcd# python setup.py install
running install
running bdist_egg
running egg_info
creating src/python_etcd.egg-info
writing requirements to src/python_etcd.egg-info/requires.txt
writing src/python_etcd.egg-info/PKG-INFO
writing top-level names to src/python_etcd.egg-info/top_level.txt
writing dependency_links to src/python_etcd.egg-info/dependency_links.txt
writing manifest file 'src/python_etcd.egg-info/SOURCES.txt'
reading manifest file 'src/python_etcd.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'src/python_etcd.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
creating build
creating build/lib.linux-x86_64-2.7
creating build/lib.linux-x86_64-2.7/etcd
copying src/etcd/client.py -> build/lib.linux-x86_64-2.7/etcd
copying src/etcd/init.py -> build/lib.linux-x86_64-2.7/etcd
copying src/etcd/lock.py -> build/lib.linux-x86_64-2.7/etcd
creating build/lib.linux-x86_64-2.7/etcd/tests
copying src/etcd/tests/init.py -> build/lib.linux-x86_64-2.7/etcd/tests
creating build/lib.linux-x86_64-2.7/etcd/tests/unit
copying src/etcd/tests/unit/init.py -> build/lib.linux-x86_64-2.7/etcd/tests/unit
copying src/etcd/tests/unit/test_old_request.py -> build/lib.linux-x86_64-2.7/etcd/tests/unit
copying src/etcd/tests/unit/test_client.py -> build/lib.linux-x86_64-2.7/etcd/tests/unit
copying src/etcd/tests/unit/test_request.py -> build/lib.linux-x86_64-2.7/etcd/tests/unit
creating build/lib.linux-x86_64-2.7/etcd/tests/integration
copying src/etcd/tests/integration/test_simple.py -> build/lib.linux-x86_64-2.7/etcd/tests/integration
copying src/etcd/tests/integration/init.py -> build/lib.linux-x86_64-2.7/etcd/tests/integration
copying src/etcd/tests/integration/test_lock.py -> build/lib.linux-x86_64-2.7/etcd/tests/integration
copying src/etcd/tests/integration/helpers.py -> build/lib.linux-x86_64-2.7/etcd/tests/integration
creating build/bdist.linux-x86_64
creating build/bdist.linux-x86_64/egg
creating build/bdist.linux-x86_64/egg/etcd
copying build/lib.linux-x86_64-2.7/etcd/client.py -> build/bdist.linux-x86_64/egg/etcd
copying build/lib.linux-x86_64-2.7/etcd/init.py -> build/bdist.linux-x86_64/egg/etcd
copying build/lib.linux-x86_64-2.7/etcd/lock.py -> build/bdist.linux-x86_64/egg/etcd
creating build/bdist.linux-x86_64/egg/etcd/tests
copying build/lib.linux-x86_64-2.7/etcd/tests/init.py -> build/bdist.linux-x86_64/egg/etcd/tests
creating build/bdist.linux-x86_64/egg/etcd/tests/unit
copying build/lib.linux-x86_64-2.7/etcd/tests/unit/init.py -> build/bdist.linux-x86_64/egg/etcd/tests/unit
copying build/lib.linux-x86_64-2.7/etcd/tests/unit/test_old_request.py -> build/bdist.linux-x86_64/egg/etcd/tests/unit
copying build/lib.linux-x86_64-2.7/etcd/tests/unit/test_client.py -> build/bdist.linux-x86_64/egg/etcd/tests/unit
copying build/lib.linux-x86_64-2.7/etcd/tests/unit/test_request.py -> build/bdist.linux-x86_64/egg/etcd/tests/unit
creating build/bdist.linux-x86_64/egg/etcd/tests/integration
copying build/lib.linux-x86_64-2.7/etcd/tests/integration/test_simple.py -> build/bdist.linux-x86_64/egg/etcd/tests/integration
copying build/lib.linux-x86_64-2.7/etcd/tests/integration/init.py -> build/bdist.linux-x86_64/egg/etcd/tests/integration
copying build/lib.linux-x86_64-2.7/etcd/tests/integration/test_lock.py -> build/bdist.linux-x86_64/egg/etcd/tests/integration
copying build/lib.linux-x86_64-2.7/etcd/tests/integration/helpers.py -> build/bdist.linux-x86_64/egg/etcd/tests/integration
byte-compiling build/bdist.linux-x86_64/egg/etcd/client.py to client.pyc
byte-compiling build/bdist.linux-x86_64/egg/etcd/init.py to init.pyc
byte-compiling build/bdist.linux-x86_64/egg/etcd/lock.py to lock.pyc
byte-compiling build/bdist.linux-x86_64/egg/etcd/tests/init.py to init.pyc
byte-compiling build/bdist.linux-x86_64/egg/etcd/tests/unit/init.py to init.pyc
byte-compiling build/bdist.linux-x86_64/egg/etcd/tests/unit/test_old_request.py to test_old_request.pyc
byte-compiling build/bdist.linux-x86_64/egg/etcd/tests/unit/test_client.py to test_client.pyc
byte-compiling build/bdist.linux-x86_64/egg/etcd/tests/unit/test_request.py to test_request.pyc
byte-compiling build/bdist.linux-x86_64/egg/etcd/tests/integration/test_simple.py to test_simple.pyc
byte-compiling build/bdist.linux-x86_64/egg/etcd/tests/integration/init.py to init.pyc
byte-compiling build/bdist.linux-x86_64/egg/etcd/tests/integration/test_lock.py to test_lock.pyc
byte-compiling build/bdist.linux-x86_64/egg/etcd/tests/integration/helpers.py to helpers.pyc
creating build/bdist.linux-x86_64/egg/EGG-INFO
copying src/python_etcd.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
copying src/python_etcd.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying src/python_etcd.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying src/python_etcd.egg-info/not-zip-safe -> build/bdist.linux-x86_64/egg/EGG-INFO
copying src/python_etcd.egg-info/requires.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying src/python_etcd.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
creating dist
creating 'dist/python_etcd-0.3.0-py2.7.egg' and adding 'build/bdist.linux-x86_64/egg' to it
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
Processing python_etcd-0.3.0-py2.7.egg
creating /usr/local/lib/python2.7/dist-packages/python_etcd-0.3.0-py2.7.egg
Extracting python_etcd-0.3.0-py2.7.egg to /usr/local/lib/python2.7/dist-packages
Adding python-etcd 0.3.0 to easy-install.pth file

Installed /usr/local/lib/python2.7/dist-packages/python_etcd-0.3.0-py2.7.egg
Processing dependencies for python-etcd==0.3.0
Searching for pyOpenSSL==0.13.1
Reading https://pypi.python.org/simple/pyOpenSSL/
Best match: pyOpenSSL 0.13.1
Downloading https://pypi.python.org/packages/source/p/pyOpenSSL/pyOpenSSL-0.13.1.tar.gz#md5=e27a3b76734c39ea03952ca94cc56715
Processing pyOpenSSL-0.13.1.tar.gz
Writing /tmp/easy_install-4A9HDV/pyOpenSSL-0.13.1/setup.cfg
Running pyOpenSSL-0.13.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-4A9HDV/pyOpenSSL-0.13.1/egg-dist-tmp-BXlayj
warning: no previously-included files matching '*.pyc' found anywhere in distribution
OpenSSL/crypto/crypto.c:14:20: fatal error: Python.h: No such file or directory
compilation terminated.
error: Setup script exited with error: command 'gcc' failed with exit status 1

fields={'wait':'true'}, timeout=None)

File "build/bdist.linux-x86_64/egg/urllib3/request.py", line 68, in request
File "build/bdist.linux-x86_64/egg/urllib3/request.py", line 81, in request_encode_url
File "build/bdist.linux-x86_64/egg/urllib3/poolmanager.py", line 153, in urlopen
File "build/bdist.linux-x86_64/egg/urllib3/connectionpool.py", line 530, in urlopen
File "build/bdist.linux-x86_64/egg/urllib3/response.py", line 284, in from_httplib
File "build/bdist.linux-x86_64/egg/urllib3/response.py", line 104, in init
File "build/bdist.linux-x86_64/egg/urllib3/response.py", line 214, in read
urllib3.exceptions.ProtocolError: ('Connection broken: IncompleteRead(0 bytes read)', IncompleteRead(0 bytes read))

run in python2.7:
Python 2.7.8

As pointed out in the discussion for PR #34 we need a better name for EtcdResult.children; also, some users find it useful to be able to retreive the directory nodes as well.

Etcd v 0.3 introduced the "prevNode" property that is handled strangely for dirs; if we update a dir we obtain a strange result returned:

c = etcd.Client()
r = c.write('/foo', None, dir=True, ttl=50)
print(r.dir) #True
r2 = c.write('/foo', None, dir=True, ttl=120, prevExist=True)
print(r2.dir) #False

We should somehow use what is in prevNode to understand if the node is a dir.

I strongly suspect this is not the only case in which the write() response does something funny.

Getting below error on ubuntu 14.04

No package 'libffi' found
Package libffi was not found in the pkg-config search path.
Perhaps you should add the directory containing libffi.pc' to the PKG_CONFIG_PATH environment variable No package 'libffi' found Package libffi was not found in the pkg-config search path. Perhaps you should add the directory containinglibffi.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libffi' found
compiling '_configtest.c':
__thread int some_threadlocal_variable_42;

x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wa -Wstrict-prototypes -fPIC -c _configtest.c -o _configtest.o
success!
removing: _configtest.c _configtest.o
c/_cffi_backend.c:13:17: fatal error: ffi.h: No such file or directory
#include <ffi.h>
^
compilation terminated.
error: Setup script exited with error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

This block of code will correctly fail if the /test node is modified by something else between reading and writing:

x = client.read('/test')
time.sleep(5)
client.write('/test', int(x.value) + 1, prevIndex = x.modifiedIndex)

But this code succeeds, and overwrites the value that was set elsewhere:

x = client.read('/test')
time.sleep(5)
x.value = int(x.value) + 1
client.update(x)

In 0.3.3, passing True or False for the recursive parameter of watch() works as expected but omitting it defaults to None, which gets stringified and passed to the server, resulting in a bad request.

I presume it should be eternal_watch() (no h)

https://github.com/jplana/python-etcd/blob/master/src/etcd/client.py#L483

See https://travis-ci.org/jplana/python-etcd/jobs/16167499

This is because the command line options have changed. We should change the tests to allow those to work..

If you want to connect a cluster etcd,
you need to write the localhost's IP like 'localhost' as follow:

client = etcd.Client(
host=(
('localhost', 4001), //localhost IP
('192.168.1.33', 4001), //other machine'IP of the cluster
allow_reconnect=True,
)

you can‘t write the localhost's IP like '127.0.0.1' as follow, or the python-etcd will raise a exception:

client = etcd.Client(
host=(
('127.0.0.1', 4001),
('192.168.1.33', 4001),
allow_reconnect=True,
)

the problem is in python-etcd/client.py line 586:
self._machine_cache = self.machines
because the self.machines function return 'localhost' instead of '127.0.0.1' if any machine of the cluster is on your host

Any idea to solve the problem? I can write some code to fix it if anyone have a good idea.
Or at least mention that in the doc

I have a proof-of-concept version of eternal_watch() that uses the streaming API, which I could clean up as a pull request. The streaming API is much more efficient, especially over TLS. Unfortunately, I think that API is 2.0.x+ only.

Are you interested in such a contribution? Do I need to make sure it supports older versions than 2.0.x?

We need to do the following:

• Support calls to /stats (not sure it's fundamental, though)
• Review docs
• Model integration tests on the structure of the documented etcd API as much as possible

Then I'd say we're ready for a new release, as a ton of bugs have been resolved.

I tried using this package with Python 2.6 and it didn't work. Is it not supported? Tests also fail against Python 2.6.

Test results for 2.7:
http://pastebin.mozilla.org/4062963

Test results for 2.6:
http://pastebin.mozilla.org/4062966

If you send a tuple list to host parameter, if the first host fails the urllib3.exceptions.TimeoutError is not caught and the code fails.
Should be added around line 527 in client.py

I am trying the following as described here :

r = client.watch('/test_folder')

and then

client.set('/test_folder/test', 'abc')

Althought the value is set correctly the watch command never returned.

I tried to use

client.read('/test_folder', wait=True)

but I got the same results. :-(

Any suggestions?
Thank you.

We may as well catch if a path does not begin with a / and raise an error before performing the request.

Referred to, here: #14 (comment)

I try to wait on a directory key with a timeout of, say 1s. If no updates to the key occur then the call times out after ~4s (not 1s as expected) and the exception is "EtcdException: No more machines in the cluster
" rather than a dedicated Timeout exception.

>>> import etcd
>>> c = etcd.Client(read_timeout=10)
>>> c.read("/foobar",waitIndex=88,wait=True,recursive=True,timeout=1)
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/etcd/client.py", line 344, in read
    self.key_endpoint + key, self._MGET, params=params, timeout=timeout)
  File "/usr/local/lib/python2.7/dist-packages/etcd/client.py", line 576, in api_execute
    self._base_uri = self._next_server()
  File "/usr/local/lib/python2.7/dist-packages/etcd/client.py", line 534, in _next_server
    raise etcd.EtcdException('No more machines in the cluster')
EtcdException: No more machines in the cluster

$ pip list | grep python-etcd
python-etcd (0.3.2)
$ pip list | grep urllib
urllib3 (1.10.2)

Hi,

I have trouble when writing data to etcd although I can read data without any problem. Would anyone know where the problem could be?

I have following simple code:

#!/usr/bin/python
import etcd
                                                                                                                                                                   client = etcd.Client(host = '127.0.0.1', port = 5000)

print client.read('/coreos_apps/marathon/dev')
print client.write('/writetest', 'TESTSTRING')

Here is my output after I interrupt program with ctrl+C

-> % python scripts/etcd_test.py<class 'etcd.EtcdResult'>({'newKey': False, 'raft_index': 64071, '_children': [], 'createdIndex': 6684, 'modifiedIndex': 6684, 'value': u'{"ssh_port": 224, "count": 1, "version": "latest", "ssh_support": true}', 'etcd_index': 16626, 'expiration': None, 'key': u'/coreos_apps/marathon/dev', 'ttl': None, 'action': u'get', 'dir': False})




^CTraceback (most recent call last):
  File "scripts/etcd_test.py", line 9, in <module>
    print client.write('/writetest', 'TESTSTRING')
  File "/usr/lib/python2.7/site-packages/python_etcd-0.3.2-py2.7.egg/etcd/client.py", line 272, in write
    response = self.api_execute(path, method, params=params)
  File "/usr/lib/python2.7/site-packages/python_etcd-0.3.2-py2.7.egg/etcd/client.py", line 575, in api_execute
    redirect=self.allow_redirect)
  File "build/bdist.linux-x86_64/egg/urllib3/request.py", line 135, in request_encode_body
  File "build/bdist.linux-x86_64/egg/urllib3/poolmanager.py", line 174, in urlopen
  File "build/bdist.linux-x86_64/egg/urllib3/poolmanager.py", line 153, in urlopen
  File "build/bdist.linux-x86_64/egg/urllib3/connectionpool.py", line 516, in urlopen
  File "build/bdist.linux-x86_64/egg/urllib3/connectionpool.py", line 308, in _make_request
  File "/usr/lib64/python2.7/httplib.py", line 973, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib64/python2.7/httplib.py", line 1007, in _send_request
    self.endheaders(body)
  File "/usr/lib64/python2.7/httplib.py", line 969, in endheaders
    self._send_output(message_body)
  File "/usr/lib64/python2.7/httplib.py", line 829, in _send_output
    self.send(msg)
  File "/usr/lib64/python2.7/httplib.py", line 791, in send
    self.connect()
  File "build/bdist.linux-x86_64/egg/urllib3/connection.py", line 154, in connect
  File "build/bdist.linux-x86_64/egg/urllib3/connection.py", line 133, in _new_conn
  File "build/bdist.linux-x86_64/egg/urllib3/util/connection.py", line 78, in create_connection
  File "/usr/lib64/python2.7/socket.py", line 224, in meth
    return getattr(self._sock,name)(*args)
KeyboardInterrupt

Every time it seems it hangs here:

  File "/usr/lib/python2.7/site-packages/python_etcd-0.3.2-py2.7.egg/etcd/client.py", line 272, in write
    response = self.api_execute(path, method, params=params)

Could it be due to connection problem? Weird is that Read works without any problem.

I have no problems writing to database using curl.

My Setup:
python --version
Python 2.7.5

etcd version 0.4.6

Would you need any info for debugging ?

I was trying to write some test cases for a bug I'm fixing and noticed that the TestClientApiInterface test cases were being run twice, with my new tests failing on the second run. I finally tracked the problem down to TestClientRequest inheriting from TestClientApiInterface, meaning that the TestClientApiInterface tests were being rerun as part of TestClientRequest.

Is TestClientRequest meant to being inheriting from TestClientApiBase instead?