what about timeouts? about asyncio-redis HOT 5 CLOSED

Actually, I'm not really sure. We should implement auto-reconnect and raise NotConnected in this case. From the tulip mailing list, I understood that it should be the transport that implements the reconnect, and that the transport should call connection_lost or connection_made.

Ideas on how to implement such a transport are very welcome.

from asyncio-redis.

I implemented timeout with decorator, added in Connection.getattr (but it is for each call of redis command - not very efficient)

def timeout_aware(self, cmd, protocol, timeout=0.1):

    @asyncio.coroutine
    @functools.wraps(cmd)
    def wrapper(*args, **kwargs):
        task = cmd(*args, **kwargs)
        result = None
        try:
            for _ in asyncio.as_completed([task], timeout=timeout):
                result = yield from _
        except asyncio.futures.TimeoutError:
            self.disconnect(protocol) # self is Connection instance
            raise
        return result
    return wrapper

in aiohttp.client package there is even simplier timeout realization:

yield from asyncio.wait_for(
    resp.start(proto, wrapper), timeout, loop=loop)

Without a timeout I had a sad situation: after shutting down redis-server, all redis calls are blocked for more than 2 minutes (actually, they had not finished at all).

I'm not sure about implementing timeouts at transport level. With a timeout to each socket operation, you will not be able to listen redis pub/sub (it is long-term operation), and transport can't distinguish GET and BLPOP (which has it's own read timeout at server side).
Reconnects are for protocol too: for pub-sub mode client should resend 'subscribe' commands after reopening socket connections.

I think, for redis timeouts should be at protocol level rather than transport level. Simpliest way for protocol is to became "timeouted" after a timeout, and Connection pool will be able to filter out and reconnect this protocol if necessary.

from asyncio-redis.

Thanks, but I'm not sure yet about this approach.

I'm almost convinced that timeouts should be implemented in the transport, not in the protocol. TCP has a retransmission timeout, but when it's unable to deliver a packet after several retries, it should be the transport, that notifies the protocol that the connection died. On the other hand, if the protocol expects an answer, but the transport doesn't receive anything, the transport is probably unaware of any problems, so in that case it could be the protocol, notifying the transport.

Anyway, I'm not sure, and just asked on the tulip mailing list.

from asyncio-redis.

Hi again!
As I see, a NotConnected exception is now raised after server shut down :)
I wrote a reconnecting connection pool with only RedisProtocol method wrapping and some helper functions to remove failed transport-protocol pair from pool (and do the reconnect)
https://gist.github.com/tumb1er/8024500
Glad if it will help you :)

from asyncio-redis.

Thank you tumb1er, that's nice. I made a little different approach, but it seems to work now. However, I'm still trying to improve everything.

from asyncio-redis.