If the Issue tracker is wrong place to discuss this, just let me know.
Preface:
I'm recently obsessed with Bottle and using it in every web project that allows me to use Python. I love its shocking simplicity and transparency, and the fact that each web request probably only goes through about a hundred lines of python code before getting to my response handler. Totally subjectively, I just love the way it "feels" and developing with it makes me all warm and fuzzy inside (as opposed to Twisted, which just rubs me the wrong way; or Tornado, which has perfectly reasonable syntax but just not the style I prefer).
Issue:
I have a few new projects that require a client "push" feature, aka comet, long polling, all that stuff. As you probably know, basically all of the possible ways to implement push in the pre-WebSockets era require a web response handler on the backend to block for an extended period of time in some form or another. As you also probably know, a stock WSGI environment will be largely ruined by this, since the entire process will be blocked on a single response handler.
I also use Redis with redis-py and potentially other blocking client libraries. Scaling for concurrent requests in a non-threaded environment becomes more complicated with any libraries that block on network I/O; it's not always sufficient to just run a whole bunch of Bottle processes. For example, some clients have already had their requests routed by the load balancer to a particular server instance, and if another client on that instance blocks, then the network packets will just sit in the queue for a while, causing slow response times and making poor use of the machine's CPU resources (since it could be doing something else while blocked on network I/O).
Right now I am serving the push feed in a separate Diesel app. This is not ideal, as I would prefer for the load balancer to distribute across a homogeneous pool of servers all running the same code for simplicity and maintainability reasons (and I don't want to have to monitor multiple process types if I can avoid it). I also don't seem to be able to flush the output buffer and keep going with Diesel's server, killing persistent-request feed options like MXHR, but that's possibly just something I'm doing wrong.
Proposal:
I propose light-weight support for asynchronous response handlers, possibly with an @async decorator. There seem to be two general camps about the syntax for asynchronous code in this kind of Python environment: callbacks and coroutines. Tornado uses callbacks, which are probably easier to understand but not as elegant and do not really support progressive output (like flushing the output buffer). Diesel uses coroutines, which can make some pretty elegant "pythonic" code and supports pausing repeatedly but can also be hard to wrap one's brain around and be tricky to implement cleanly on the backend. Note that there is also a project called Swirl that brings the coroutine style to Tornado.
Here is a possible syntax style for each option:
Callback
from bottle import route, async
import time
@route()
@async
def response():
def doStuff():
body = "I like sleep."
time.sleep(3)
body = body + " Alot."
time.sleep(3)
return body
def callback(): # Maybe with request & response args?
return "OK, I'm awake."
# Always return a 2-tuple of a worker callable and a callback callable
return (doStuff, callback)
Coroutine
from bottle import route, async
@route()
@async
def response():
yield "I like sleep."
time.sleep(3) # Or something like yield diesel.sleep(3)
yield " Alot."
time.sleep(3) # Or something like yield diesel.sleep(3)
yield "OK, I'm awake."
I'd vote for the coroutine option, but it's possible there are implementation issues with it I'm not thinking of.
The hard part
The really tricky part is that Bottle is not a WSGI server; it always uses another WSGI server (which is usually quite handy). This means that the main application loop is not handled by Bottle but the server (which may be using epoll or libevent or libev or eventlets or Stackless Python or ...).
Option 1: Do almost nothing in Bottle
It is possible that really minor modifications to Bottle would allow it to pass through generator return types to the underlying server and hope it supports it. However, builtin libs will still block the entire process (like time.sleep). Diesel and Cogen work around this by offering custom functions you have to call instead: yield diesel.sleep(3)
. Eventlet can actually modify/wrap the builtin packages to be asynchronous, which is pretty nifty. With the minor modification route, the @async decorator might not even be necessary, since Diesel will accept generators automatically. On a side note, Fapws3 claims to be asynchronous, but I have no idea what part of it is asynchronous. It does not seem to support any method of having an asynchronous response handler that I could find; I'll have to ask that community about it since I really like Fapws3 otherwise.
Option 2: Do everything in Bottle
It would be awesome if there were a magical way to handle it cleanly all inside of Bottle in a way that worked with all of the possible servers. Bottle would need to track async responses that had not finished, and probably execute async responses in a closure with local request/response object copies. However, I don't know if the underlying WSGI servers (or the rest of the WSGI) would handle having more than one request at a time; I'm not very familiar with WSGI yet. If they also have single global request/response objects like Bottle, they would almost certainly break.
If this approach is possible, Eventlet, Concurrence, and Cogen seem like the most likely candidates. Greenlet would probably do the Bottle part at least as efficiently, but it does not directly assist with things like networking.
The theory would be that Bottle would spawn two thread-like-things at the same time: one that just calls the WSGI server's main loop, and one for the pending response book-keeping. Whenever a synchronous response finishes, or an async response yields, the microthread's equivalent of switch() would be called (in the case of Concurrence or Eventlet), or yield (in the case of a Cogen coroutine).
Option 3: Require specific WSGI server[s]
I suspect that being asynchronous in Bottle simply will not work with most WSGI servers, and it will require using the WSGI server that comes with the particular async library. Fortunately, Concurrence, Eventlet, and Cogen all offer a WSGI server. There might need to be a global flag in Bottle like the debug flag that enables asynchronous support, and the async decorator would just do pass-through. In asynchronous mode, Bottle could throw an error if the given WSGI server callable is not in a list of known supported async servers.
My plan
If you don't hate the idea of supporting async requests in Bottle, I would love to make a temporary fork and do some or all of the implementation myself. I would attempt option 3, since it seems the easiest/safest, and I would first attempt with either Cogen or Eventlet. I'm somewhat torn between the two, but I'm leaning towards Cogen since I feel coroutines are more pythonic and elegant than explicit microthread switching (and it doesn't even require hacking the builtin packages like socket or time). Eventlet could make redis-py async automatically, but then it's not clear while looking at the code that a call like val = redis.get('mykey')
is going to yield. With Cogen you would have to manually yield like val = yield redis.get('mykey')
, but it's more clear what is happening.
If you've managed to continue reading all the way through this, I'm going to start a fork and play around with it in the hopes that it could get merged into Bottle master. I will probably attempt to first use Cogen, using this guide.
What are your thoughts? Do have any interest in Bottle supporting async requests, especially via coroutines? If so, how do you feel about an @async decorator vs. automatically detecting via something like inspect.isgenerator()? Also, once async works, would flushing the output buffer be possible for persistent requests?