Comments (18)
Try to use the gevent aware db adapter in django
https://pypi.python.org/pypi/gevent-psycopg2
And probably apply monkey_patch() as early as possible.
see: https://github.com/zacharyvoase/gevent-psycopg2
import gevent_psycopg2; gevent_psycopg2.monkey_patch()
I'm also not sure if you are using the right WSGIHandler. Can you post your circus configuration?
from chaussette.
Hi
The cmd in my circus watcher is:
cmd = chaussette --backend gevent --fd $(circus.sockets.frontend) --pre-hook myproject.prehook.pre_hook myproject.wsgi.application
Full Circus config: http://pastebin.com/7cdkeSwV
The prehook is where I already use psycopg patch (https://bitbucket.org/dvarrazzo/psycogreen/)
def pre_hook(args):
from psycogreen.gevent import patch_psycopg
patch_psycopg()
and myproject.wsgi.application is the default Django wsgi application:
from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()
Regards,
from chaussette.
Thanks for the hook!
from chaussette.
Closing since it seems to be resolved
from chaussette.
shouldn't this be documented somewhere?
from chaussette.
I have the exact same problem with django 1.6.5, chaussette and circus, any fix ?
from chaussette.
The hook has improved the stability of circus and chaussette when I applied it but this error was still occurring sometimes. So, as I wanted to keep using circus for different reasons, I tried all the different backends. The only one which has really been fully stable is "waitress". Now, it's 7 months I'm using it in a my staging and production environment and I never had any issues. My only regret is about performance, "waitress" a quite good but "fastgevent" was really amazing! Maybe later I'll try again :-)
from chaussette.
But you still use waitress with a hook ?
from chaussette.
Reopening so we address the doc issue
from chaussette.
I'm using waitress without the hook.
from chaussette.
The hook indeed doesn't solve the thread exception problem. So far what I have reached is that the problem is related to how chaussette uses gevent monkey patch. Gevent monkey.patch_all should be called in the early stage before using patched modules, like socket, threading etc...
Socket is imported in server.py, while gevent monkey patching is done in the backend. When I modified server.py to patch_all first thing, I couldn't reproduce the issue (tested with django==1.6.5)
here is the diff
diff --git a/chaussette/server.py b/chaussette/server.py
index 9666181..64c8d0d 100644
--- a/chaussette/server.py
+++ b/chaussette/server.py
@@ -1,3 +1,6 @@
+from gevent import monkey
+monkey.patch_all()
+
import sys
import os
import argparse
Although this solves the issue, but it is not a generic solution. It is more like a gevent specific. So it needs to be implemented in a cleaner way.
It is also worth mentioning, that I faced Celery related instability issues when load testing my Django app. While load testing, some celery tasks "went missing" (using RabbitMQ & Postgresql as result backend). The above patch solved the instability problem as well.
I haven't tested the above solution heavily, but if someone can test it we can confirm the cause of the issue.
from chaussette.
Thanks for your tests.
Although this solves the issue, but it is not a generic solution. It is more like a gevent specific. So it needs to be implemented in a cleaner way.
maybe an option like --gevent-monkey-patch ?
from chaussette.
or by checking if --backend is gevent related backend.
from chaussette.
yes that too. But do we want to force the monkey patching everytime in that case ? I am not sure
from chaussette.
The monkey patch is already applied on gevent backends (_gevent, _fastgevent, _geventwebsocket), but it is applied late.
from chaussette.
ah right!
It's weird because we only read constants out of the socket module, and the patching seems to be applied before the socket creation: https://github.com/mozilla-services/chaussette/blob/master/chaussette/backend/_gevent.py#L32
could be interesting to follow with pdb the initialization process and see why we don't get the patched version. Maybe it's just a matter of moving that import line after the monkey patch : https://github.com/mozilla-services/chaussette/blob/master/chaussette/backend/_gevent.py#L4
from chaussette.
Did a quick test with _gevent backend monkey patched at the start of the file.
https://github.com/mozilla-services/chaussette/blob/master/chaussette/backend/_gevent.py#L1
Added the patching before importing any modules:
from gevent import monkey
monkey.patch_all()
import socket
from gevent.pywsgi import WSGIServer, WSGIHandler
# from gevent import monkey
from chaussette.util import create_socket
I couldn't reproduce the thread exception, and the env seemed to be stable. So I guess this can be the cleaner fix.
from chaussette.
Fixed by #62
from chaussette.
Related Issues (20)
- errors thrown by backends or chaussette enter infinite loop HOT 5
- Chaussette does not log requests HOT 1
- custom backend config HOT 4
- Add a -t --timeout option to let chaussette die if the request took more than TIMEOUT seconds. HOT 1
- Support graceful shutdown HOT 2
- SocketIO backend seems broken with latest version of gevent-socketio ('Server' object has no attribute 'config') HOT 7
- Chaussette vs. G-event+Redis (?!) HOT 6
- Need to add tornado in the docs
- waitress compat broken HOT 2
- File descriptors don't seemed to be closed after deletion HOT 3
- Cannot SIGTERM chaussette
- The --python-path argument doesn't do anything
- Non-ascii URL errors HOT 1
- Chaussette does not work anymore with waitress v1.0.0 HOT 4
- Raise the test coverage to 90%+
- Enable Bjoern in Python 3
- Chaussette under python3.6 can't discover gevent backend
- chaussette does not work with waitress > 0.9
- maintainership HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from chaussette.