Code Monkey home page Code Monkey logo

guv's People

Contributors

craigkelly avatar icook avatar lvella avatar thedrow avatar veegee avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

guv's Issues

Correctly patch threading and threading.Event

threading is currently being patched, but since this code has been ported from eventlet, it needs a very thorough review as to what exactly it is patching. threading.Event, on the other hand, is not being patched but I believe it should be in order to be greenlet-friendly.

Setup error with Ubuntu 16.04, PyPy3.3 v5.2, and guv 0.35.2

Here's the rundown:

I am running the 64-bit Ubuntu 16.04 with libuv1 installed and also have build-essential installed from before. I also downloaded PyPy3.3 v5.2 from https://bitbucket.org/pypy/pypy/downloads/pypy3.3-v5.2.0-alpha1-linux64.tar.bz2 and extracted to /opt/pypy3/ with the whole directory chowned by me.

I had no problems getting pip3 installed via downloading https://bootstrap.pypa.io/get-pip.py to the /opt/pypy3/bin/ directory and running ./pypy3 get-pip.py

Now when I run ./pip3 install guv, I get this error:

Collecting guv
  Downloading guv-0.35.2.tar.gz (86kB)
    100% |████████████████████████████████| 92kB 1.7MB/s 
    Complete output from command python setup.py egg_info:
    pyuv_cffi/__pycache__/_cffi__g5b01d074xf9520033.c:57:16: fatal error: uv.h: No such file or directory
    compilation terminated.
    CompileError: command 'cc' failed with exit status 1

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-rdoa5o/guv/

I even tried converting it to a wheel in another attempt to look for errors by running ./pip3 wheel --wheel-dir=/opt/pypy3/packages/ /opt/pypy3/packages/guv-0.35.2.tar.gz

Processing /opt/pypy3/packages/guv-0.35.2.tar.gz
    Complete output from command python setup.py egg_info:
    pyuv_cffi/__pycache__/_cffi__g5b01d074xf9520033.c:57:16: fatal error: uv.h: No such file or directory
    compilation terminated.
    CompileError: command 'cc' failed with exit status 1

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-kh0is1-build/

Is this a known issue with PyPy3 or this version of it? In the documentation for guv, it said that support for PyPy3 in addition to CPython 3 is there. If any of you have fixes/tips that would be great. :-)

Thread does not support setDaemon()

I'm using a library that's calling setDaemon() on the thread class, but that throws an error when green/Thread is patched in, because that method is missing.

gunicorn worker errors out when quitting (ctrl-c)

When serving using gunicorn with more than 2 workers (why does this only happen when using more than 2 workers?), quitting the program with ctrl-c prints this traceback and sometimes hangs for a minute:

[veegee@server] ~/projects/python/guv/examples git:(develop) !
➜ gunicorn -w 3 -b 127.0.0.1:8001 -k guv.GuvWorker wsgi_app:app
2014-12-05 15:24:06 guv      DEBUG    Start runloop
2014-12-05 15:24:06 guv      DEBUG    Start runloop
2014-12-05 15:24:06 guv      DEBUG    Start runloop
^C

2014-12-05 15:24:12 guv      DEBUG    Abort loop
2014-12-05 15:24:12 guv      DEBUG    Abort loop
From callback <function Signal.start.<locals>.cb_wrapper at 0x7fabcdda86a8>:
Traceback (most recent call last):
  File "/home/veegee/projects/python/guv/pyuv_cffi/__init__.py", line 264, in cb_wrapper
    def cb_wrapper(uv_signal_t, signum):
  File "/home/veegee/local/lib/python3.4/site-packages/gunicorn/workers/base.py", line 149, in handle_quit
    sys.exit(0)
SystemExit: 0
From callback <function Prepare.start.<locals>.cb_wrapper at 0x7fabc93deae8>:
Traceback (most recent call last):
  File "/home/veegee/projects/python/guv/pyuv_cffi/__init__.py", line 193, in cb_wrapper
    def cb_wrapper(prepare_h):
  File "/home/veegee/local/lib/python3.4/site-packages/gunicorn/workers/base.py", line 149, in handle_quit
    sys.exit(0)
SystemExit: 0
From callback <function Timer.start.<locals>.cb_wrapper at 0x7fabc93deea0>:
Traceback (most recent call last):
  File "/home/veegee/projects/python/guv/pyuv_cffi/__init__.py", line 235, in cb_wrapper
    def cb_wrapper(timer_h):
  File "/home/veegee/local/lib/python3.4/site-packages/gunicorn/workers/base.py", line 149, in handle_quit
    sys.exit(0)
SystemExit: 0

wrk reports that requests are served correctly (and quite fast!) with no errors. This has probably got something to do with how we're dealing with terminating pyuv_cffi handles.

Update:

When using pyuv (not pyuv_cffi) by doing import pyuv as pyuv_cffi from the pyuv_cffi.py hub, this error goes away. This must mean that it's definitely something wrong with the pyuv_cffi implementation.

The gunicorn worker is not as fast as it could be

It could easily be made twice as fast, and with a little more work, possibly 4 times as fast.

Currently, the gunicorn AsyncWorker base is being used. To make it twice as fast, it can be updated to behave like the included guv WSGI server (guv.wsgi).

To further improve performance, both the guv WSGI server and gunicorn worker need to remove their dependency on socket.makefile and work with the socket directly (or so it seems).

module 'ssl' has no attribute 'PROTOCOL_SSLv3' ]

gunicorn test:hello  -b 0.0.0.0:8060 -w 2 -k guv.GuvWorker

Error: class uri 'guv.GuvWorker' invalid or not found:

[Traceback (most recent call last):
  File "/home/project0/venv/lib/python3.5/site-packages/gunicorn/util.py", line 142, in load_class
    mod = import_module('.'.join(components))
  File "/home/project0/venv/lib/python3.5/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 986, in _gcd_import
  File "<frozen importlib._bootstrap>", line 969, in _find_and_load
  File "<frozen importlib._bootstrap>", line 958, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 673, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 665, in exec_module
  File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed
  File "/home/project0/venv/lib/python3.5/site-packages/guv/__init__.py", line 14, in <module>
    from .server import serve, listen, connect, StopServe, wrap_ssl
  File "/home/project0/venv/lib/python3.5/site-packages/guv/server.py", line 6, in <module>
    from .green import socket, ssl
  File "/home/project0/venv/lib/python3.5/site-packages/guv/green/ssl.py", line 5, in <module>
    from . import _ssl33 as _ssl3
  File "/home/project0/venv/lib/python3.5/site-packages/guv/green/_ssl33.py", line 26, in <module>
    PROTOCOL_SSLv3 = ssl_orig.PROTOCOL_SSLv3
AttributeError: module 'ssl' has no attribute 'PROTOCOL_SSLv3'
]

ubuntu 16.04 python 3.5.2

Installation complexity rolls over

I spent about 2 hours trying to make libuv 1.x under Ubuntu and gave up. I think that end-user of a such module does not even know that C/C++ appear in this world (joking of course) but they really could lost in such routines like configuring and making C/C++ projects. But for me GUV looks like very interesting and the only one alternative of terrible and slow supporting Gevent. So is it possible to make installation more user friendly. For example to bundle together all requirements. Or prepare docker image for instance (with separate environments of cpython and pypy). It would help people to envolve project faster.

Problem with socket monkeypatch

Hi, I'm trying to port locust (http://github.com/locustio/locust) from python2 + gevent to python3 + guv, but I have a problem with the socket mockey patch.

I don't know how to solve it, but when i run the app, the IOClosed exceptions reach a socket library exceptions handler, and crash.

Without the sockets in the monkey patch everything works (in sync mode).

Here is the trackeback

Traceback (most recent call last):
File "/usr/lib64/python3.4/socket.py", line 374, in readinto
    return self._sock.recv_into(b)
File "/home/jespino/Projects/Terceros/guv/guv/greenio.py", line 200, in recv_into
    timeout_exc=s_timeout("timed out"))
File "/home/jespino/Projects/Terceros/guv/guv/greenio.py", line 54, in _trampoline
    raise IOClosed()
guv.exceptions.IOClosed
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/jespino/Projects/Terceros/guv/guv/hubs/pyuv_cffi.py", line 179, in poll_cb
    cb(*cb_args)
File "/home/jespino/Projects/Terceros/guv/guv/greenthread.py", line 151, in main
    result = function(*args, **kwargs)
File "/home/jespino/Projects/Github/locust/locust/runners.py", line 112, in start_locust
    locust().run()
File "/home/jespino/Projects/Github/locust/locust/core.py", line 104, in run
    self.task_set(self).run()
File "/home/jespino/Projects/Github/locust/locust/core.py", line 251, in run
    self.on_start()
File "/home/jespino/Projects/Github/taiga/taiga-extras/load-test/locustfile.py", line 16, in on_start
    response = self.client.get("/api/v1/tasks?project={}".format(self._projects[0].get('id')), headers=self._auth_headers)
File "/home/jespino/.virtualenvs/taiga-tribe/lib/python3.4/site-packages/requests-2.6.0-py3.4.egg/requests/sessions.py", line 476, in get
    return self.request('GET', url, **kwargs)
File "/home/jespino/Projects/Github/locust/locust/clients.py", line 113, in request
    response = self._send_request_safe_mode(method, url, **kwargs)
File "/home/jespino/Projects/Github/locust/locust/clients.py", line 157, in _send_request_safe_mode
    return requests.Session.request(self, method, url, **kwargs)
File "/home/jespino/.virtualenvs/taiga-tribe/lib/python3.4/site-packages/requests-2.6.0-py3.4.egg/requests/sessions.py", line 464, in request
    resp = self.send(prep, **send_kwargs)
File "/home/jespino/.virtualenvs/taiga-tribe/lib/python3.4/site-packages/requests-2.6.0-py3.4.egg/requests/sessions.py", line 613, in send
    r.content
File "/home/jespino/.virtualenvs/taiga-tribe/lib/python3.4/site-packages/requests-2.6.0-py3.4.egg/requests/models.py", line 733, in content
    self._content = bytes().join(self.iter_content(CONTENT_CHUNK_SIZE)) or bytes()
File "/home/jespino/.virtualenvs/taiga-tribe/lib/python3.4/site-packages/requests-2.6.0-py3.4.egg/requests/models.py", line 656, in generate
    for chunk in self.raw.stream(chunk_size, decode_content=True):
File "/home/jespino/.virtualenvs/taiga-tribe/lib/python3.4/site-packages/requests-2.6.0-py3.4.egg/requests/packages/urllib3/response.py", line 273, in stream
    data = self.read(amt=amt, decode_content=decode_content)
File "/home/jespino/.virtualenvs/taiga-tribe/lib/python3.4/site-packages/requests-2.6.0-py3.4.egg/requests/packages/urllib3/response.py", line 203, in read
    data = self._fp.read(amt)
File "/usr/lib64/python3.4/http/client.py", line 500, in read
    return super(HTTPResponse, self).read(amt)
File "/usr/lib64/python3.4/http/client.py", line 539, in readinto
    n = self.fp.readinto(b)
File "/usr/lib64/python3.4/socket.py", line 381, in readinto
    if e.args[0] in _blocking_errnos:
IndexError: tuple index out of range
Exception while removing listener: 'NoneType' object has no attribute 'ref'

Add pyzmq support

We need to add pyzmq support. Good starting points are as follows:

We can do a pretty quick port from the gevent code, since the structure is almost the same. The only difference is that guv uses trampoline() instead of the lower level, direct constructs that gevent is using; if anything, this makes it easier.

Replace `fileobject` with patched `io`

Python 3 doesn't use fileobject; everything happens through the io module. The read/write methods in those classes need to be patched to allow for system-wide cooperative files.

This shouldn't be a very difficult thing to do.

Problems running examples

I'm probably doing something silly, so sorry if this is just noise.

Enviroment

  • Ubuntu 14.04
  • Python 3.4.1-1+saucy1
  • libuv-dev 0.10.22-2
  • libffi-dev 3.1~rc1+r3.0.13-12

Steps Taken

  1. Cloned master
  2. New Python 3.4 virtualenv
  3. pip install -e .
  4. python examples/crawler.py

Problem

examples/pyuv_cffi/__pycache__/_cffi__x735818ex4ef4bffa.c: In function ‘_cffi_f_uv_has_ref’:
examples/pyuv_cffi/__pycache__/_cffi__x735818ex4ef4bffa.c:368:3: warning: implicit declaration of function ‘uv_has_ref’ [-Wimplicit-function-declaration]
   { result = uv_has_ref(x0); }
   ^
examples/pyuv_cffi/__pycache__/_cffi__x735818ex4ef4bffa.c: In function ‘_cffi_f_uv_idle_start’:
examples/pyuv_cffi/__pycache__/_cffi__x735818ex4ef4bffa.c:449:3: warning: passing argument 2 of ‘uv_idle_start’ from incompatible pointer type [enabled by default]
   { result = uv_idle_start(x0, x1); }
   ^
In file included from examples/pyuv_cffi/__pycache__/_cffi__x735818ex4ef4bffa.c:190:0:
/usr/include/uv.h:1167:15: note: expected ‘uv_idle_cb’ but argument is of type ‘void (*)(struct uv_idle_t *)’
 UV_EXTERN int uv_idle_start(uv_idle_t* idle, uv_idle_cb cb);
               ^
examples/pyuv_cffi/__pycache__/_cffi__x735818ex4ef4bffa.c: In function ‘_cffi_f_uv_loop_alive’:
examples/pyuv_cffi/__pycache__/_cffi__x735818ex4ef4bffa.c:557:3: warning: implicit declaration of function ‘uv_loop_alive’ [-Wimplicit-function-declaration]
   { result = uv_loop_alive(x0); }
   ^
examples/pyuv_cffi/__pycache__/_cffi__x735818ex4ef4bffa.c: In function ‘_cffi_f_uv_loop_init’:
examples/pyuv_cffi/__pycache__/_cffi__x735818ex4ef4bffa.c:584:3: warning: implicit declaration of function ‘uv_loop_init’ [-Wimplicit-function-declaration]
   { result = uv_loop_init(x0); }
   ^
examples/pyuv_cffi/__pycache__/_cffi__x735818ex4ef4bffa.c: In function ‘_cffi_f_uv_prepare_start’:
examples/pyuv_cffi/__pycache__/_cffi__x735818ex4ef4bffa.c:785:3: warning: passing argument 2 of ‘uv_prepare_start’ from incompatible pointer type [enabled by default]
   { result = uv_prepare_start(x0, x1); }
   ^
In file included from examples/pyuv_cffi/__pycache__/_cffi__x735818ex4ef4bffa.c:190:0:
/usr/include/uv.h:1129:15: note: expected ‘uv_prepare_cb’ but argument is of type ‘void (*)(struct uv_prepare_t *)’
 UV_EXTERN int uv_prepare_start(uv_prepare_t* prepare, uv_prepare_cb cb);
               ^
examples/pyuv_cffi/__pycache__/_cffi__x735818ex4ef4bffa.c: In function ‘_cffi_f_uv_timer_start’:
examples/pyuv_cffi/__pycache__/_cffi__x735818ex4ef4bffa.c:1200:3: warning: passing argument 2 of ‘uv_timer_start’ from incompatible pointer type [enabled by default]
   { result = uv_timer_start(x0, x1, x2, x3); }
   ^
In file included from examples/pyuv_cffi/__pycache__/_cffi__x735818ex4ef4bffa.c:190:0:
/usr/include/uv.h:1217:15: note: expected ‘uv_timer_cb’ but argument is of type ‘void (*)(struct uv_timer_t *)’
 UV_EXTERN int uv_timer_start(uv_timer_t* handle,
               ^
Traceback (most recent call last):
  File "/home/isaac2/.virtualenvs/guv/lib/python3.4/site-packages/cffi/vengine_cpy.py", line 146, in load_library
    self.verifier.modulefilename)
ImportError: /home/isaac2/programming/guv/examples/pyuv_cffi/__pycache__/_cffi__x735818ex4ef4bffa.cpython-34m-x86_64-linux-gnu.so: undefined symbol: uv_loop_init

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "examples/crawler.py", line 1, in <module>
    import guv
  File "/home/isaac2/programming/guv/examples/guv/__init__.py", line 14, in <module>
    import pyuv_cffi  # only to compile the shared library before monkey-patching
  File "/home/isaac2/programming/guv/examples/pyuv_cffi/__init__.py", line 25, in <module>
    libuv = ffi.verify(f.read(), libraries=['uv'])
  File "/home/isaac2/.virtualenvs/guv/lib/python3.4/site-packages/cffi/api.py", line 340, in verify
    lib = self.verifier.load_library()
  File "/home/isaac2/.virtualenvs/guv/lib/python3.4/site-packages/cffi/verifier.py", line 75, in load_library
    return self._load_library()
  File "/home/isaac2/.virtualenvs/guv/lib/python3.4/site-packages/cffi/verifier.py", line 151, in _load_library
    return self._vengine.load_library()
  File "/home/isaac2/.virtualenvs/guv/lib/python3.4/site-packages/cffi/vengine_cpy.py", line 149, in load_library
    raise ffiplatform.VerificationError(error)
cffi.ffiplatform.VerificationError: importing '/home/isaac2/programming/guv/examples/pyuv_cffi/__pycache__/_cffi__x735818ex4ef4bffa.cpython-34m-x86_64-linux-gnu.so': /home/isaac2/programming/guv/examples/pyuv_cffi/__pycache__/_cffi__x735818ex4ef4bffa.cpython-34m-x86_64-linux-gnu.so: undefined symbol: uv_loop_init

Perhaps I've installed the wrong dependencies? I had hiccups at first that I got past by installing libuv-dev and libffi-dev.

Enable travis

I'm willing to contribute a .travis.yml script if enable it and provide info on how you run the tests.

OSError Bad file descriptor when stress testing with Thor

I use the simple WebSocket Server as described in #27. When I stress test it with Thor, the workers quit with OSError Bad file descriptor:

[2015-07-11 16:25:43 +0200] [3256] [ERROR] Socket error processing request.
Traceback (most recent call last):
  File "/home/dm/myprojects/Parenchym/libguv/guv/support/gunicorn_worker.py", line 93, in handle
    reraise(exc_info[0], exc_info[1], exc_info[2])
  File "/home/dm/myprojects/Parenchym/libguv/guv/support/__init__.py", line 19, in reraise
    raise value
  File "/home/dm/myprojects/Parenchym/libguv/guv/support/gunicorn_worker.py", line 80, in handle
    self.handle_request(server_name, req, client_sock, addr)
  File "/home/dm/myprojects/Parenchym/libguv/guv/support/gunicorn_worker.py", line 134, in handle_request
    respiter = self.wsgi(environ, resp.start_response)
  File "/home/dm/myprojects/Parenchym/libguv/guv/websocket.py", line 139, in __call__
    self.handler(ws)
  File "/home/dm/myprojects/foo/sock/srv.py", line 43, in my_handler
    ws.send(resp)
  File "/home/dm/myprojects/Parenchym/libguv/guv/websocket.py", line 658, in send
    self._send(payload)
  File "/home/dm/myprojects/Parenchym/libguv/guv/websocket.py", line 651, in _send
    self.socket.sendall(frame)
  File "/home/dm/myprojects/Parenchym/libguv/guv/greenio.py", line 222, in sendall
    b_sent = self.send(mv, flags)
  File "/home/dm/myprojects/Parenchym/libguv/guv/greenio.py", line 204, in send
    return super().send(data, flags)
OSError: [Errno 9] Bad file descriptor

Commenting out the ws.send(), i.o.w. just reading from the socket, seems to work.

This is the Thor command:

$ ./node_modules/thor/bin/thor ws://localhost:7098 --amount 2 --buffer 10240

Update:
Could it be related to the issue (and fix) mentioned here?

Why use guv instead gevent?

Now that gevent supports Python 3, is there a reason for guv's existence? What major differences/advantages does guv have over gevent at this point?

Just looking for data points to make decisions.

WSGI websocket: issues getting sock

I'm have a simple WebsocketWSGI server:

@guv.websocket.WebSocketWSGI
def my_handler(ws):
    print('INSIDE HANDLER')
    while True:
        from_browser = ws.wait()
        print('>', from_browser)
        ws.send("from server:" + from_browser)

# Served by gunicorn with guv.GuvWorker
def main(global_config, **settings):
    lgg = logging.getLogger('root')
    lgg.info('start')
    return my_handler

# Standalone
def serve():
    init_standalone()
    lgg = logging.getLogger('root')
    lgg.info('start')
    host = RC.g('server.host')
    port = int(RC.g('server.port'))
    server_sock = guv.listen((host, port))
    guv.wsgi.serve(server_sock, my_handler)

if __name__ == '__main__':
    serve()

At some point WebSocketWSGI tries to obtain the socket from the WSGI environment. The original code did not work for multiple reasons:

  • Key guv.input does not exist
  • There's key wsgi.input
  • A method get_sock() does not exist at all, but guv's class Input has property sock
  • wsgi.input is set to different objects, depending on whether gunicorn or guv serves.

Please have a look at this diff. Between some debug prints, the two important changes are to force Input to always set the sock (here) and the method how to fetch the socket from environ (here).

I am not sure what the intended behaviour actually should be.

Add Cassandra support

Add Cassandra support for this driver: https://github.com/datastax/python-driver

The files that are interesting are as follows:

  • geventreactor.py This is the core event loop implementation using gevent. We'll need to provide a similar one using guv.
  • cluster.py This module checks if gevent is present and uses it as the DefaultConnection class.

What needs to be done:

  • Create a module like geventreactor.py which uses guv instead.

A pull request will needed to be submitted so they can integrate these changes into the driver. However, they don't seem to be very quick in resolving their issues or pull requests. So for now, we can add support for this driver into guv directly by monkey-patching cluster.py with our own substitute for DefaultConnection.

Installation problems

I install libuv, after downloading it to /tmp/libuv/.
I installed it, using the instructions in their readme.

Then I cloned guv, and on running installation, I get this:

pyuv_cffi/__pycache__/_cffi__x4295e1d1xf9520033.c:190:16: fatal error: uv.h: No such file or directory
 #include <uv.h>
                ^
compilation terminated.
CompileError: command 'x86_64-linux-gnu-gcc' failed with exit status 1

Am I missing something here? First of all, libuv has python files written in python 2.x
So I had to run its installer in my global python environment, with the python 2.x installation.

Then I entered my virtualenv, and ran the guv's installation script. And above error is what I got.

I even tried to install guv after deactivating my virtualenv, hoping maybe it's a virtualenv thing. Still the same error.

But please note that, I need to install it in a python3 virtualenv.

setup.py install does not install pyuv_cffi binary

When installing guv system-wide and trying to import it afterwards as a user I get the following error message: PermissionError: [Errno 13] Permission denied: '/usr/lib/python3.4/site-packages/pyuv_cffi/__pycache__/_cffi__x9bdc36e8xf9520033.c'
The .c file and also the .so file is generated upon setup.py install, they are not copied to the install destination though. Thus, cffi tries to build it but lacks permission to write to the system directory.

guv.green.ssl is not usable for Python 3.3.3

Python 3.3.3 (default, Nov 27 2013, 17:12:35) 
[GCC 4.8.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import guv.green.ssl
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.3/dist-packages/guv/__init__.py", line 14, in <module>
    from .server import serve, listen, connect, StopServe, wrap_ssl
  File "/usr/local/lib/python3.3/dist-packages/guv/server.py", line 6, in <module>
    from .green import socket, ssl
  File "/usr/local/lib/python3.3/dist-packages/guv/green/ssl.py", line 5, in <module>
    from . import _ssl33 as _ssl3
  File "/usr/local/lib/python3.3/dist-packages/guv/green/_ssl33.py", line 31, in <module>
    SOL_SOCKET = ssl_orig.SOL_SOCKET
AttributeError: 'module' object has no attribute 'SOL_SOCKET'

guv version is 0.35.2

Finish pyuv_cffi implemention

Currently, most libuv handles are implemented. The remaining tasks are as follows:

  • Implement the remaining handle types, and implement those methods which currently raise NotImplementedError.
  • Write tests (see the pyuv project for interesting test cases). This includes checking for memory leaks and proper garbage collection. I already did preliminary memory profiling and the current code has no memory leaks on CPython and pypy3, so it's at a good starting point already.
  • Make sure it's API-compatible as much as possible with pyuv
  • Reorganize the package; currently, all classes are in __init__.py for the sake of simplicity, but this is not the right way to organize any multi-class package in python.

Add Sphinx docs

A preliminary set of docs need to be added, to help both end users and contributors understand the code. This must be registered on readthedocs as well.

Cassandra driver causes core dump if trampoline() is used in handle_write()

The Cassandra support module causes a core dump if we call trampoline() in handle_write(). This is not a bug since there shouldn't need to be a reason to explicitly call trampoline when the module is using green sockets.

This brings up two points:

  • Why do we need to call trampoline() in the first place? I based this driver off of the gevent driver in the Cassandra repository and it is doing things that seem redundant.
  • Why is this core dump happening at all?
  • The trampoline() call in handle_read() is working as expected, but this also brings up the question as to why it's there in the first place.

The specific error is as follows:

python: src/unix/core.c:823: uv__io_stop: Assertion `loop->watchers[w->fd] == w' failed.
[1] 9736 abort (core dumped) python cassandra_db.py

Name and protocol resolution fail on monkey-patched socket.

Python requests library cannot resolve names like localhost on guv patched socket.

import guv
guv.monkey_patch()
import requests
requests.get("http://127.0.0.1:8081/register")
requests.get("http://localhost:8081/register")

First request goes without problem, second fails with protocol error:

.../requests/adapters.py in send(self, request, stream, timeout, verify, cert, proxies)
    413 
    414         except (ProtocolError, socket.error) as err:
--> 415             raise ConnectionError(err, request=request)
    416 
    417         except MaxRetryError as e:

ConnectionError: ('Connection aborted.', gaierror(-9, 'Address family for hostname not supported'))

I see it rather bizzare, something either with requests module or monkey patched socket is realy wrong. On non patched socket localhost request gets sent. Maybe patched socket resolves localhost wrong? Or rather this is bug in requests module?

Bad performance

Tried stress load with wrk... and got:

wrk -t 1 -c 1000 -d 25s http://suite:5001/app2url --latency
Running 25s test @ http://suite:5001/app2url
  1 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.41s   431.98ms   1.89s    62.33%
    Req/Sec   449.79     40.89   480.00     95.14%
  Latency Distribution
     50%    1.47s
     75%    1.86s
     90%    1.88s
     99%    1.89s
  11104 requests in 25.01s, 2.19MB read
  Socket errors: connect 0, read 0, write 0, timeout 7815
Requests/sec:    444.03
Transfer/sec:     89.76KB

Why so ugly?

Implement PEP3156 event loop API

The PEP3156 event loop is the interface is Python's standard. The guv event loop should be restructured to implement this interface.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.