[Tracking] roll into botocore about aiobotocore HOT 9 OPEN

We need to start thinking how to patch botocore so we can plugging our code using public api, for instance first easy change is to update https://github.com/boto/botocore/blob/develop/botocore/session.py#L108-L109 Session object to accept ClientCreator, so in aiobotocore we just pass AioClientCreator without coping screen of code.

from aiobotocore.

I do not object, just do not see clean way how to accomplish this. Could be tricky because botocore also supports python2.7 and aiobotocore uses yield from heavily.

from aiobotocore.

perhaps making the aio portion available for python3.x only? I think they do a similar thing with pyzmq: http://pyzmq.readthedocs.org/en/latest/api/zmq.asyncio.html

from aiobotocore.

I see this is being tracked in boto/botocore#458

from aiobotocore.

the best way to handle this issue if client could be somehow abstracted from query/url/headers builder from IO. Nice approach also used here https://github.com/cablehead/python-consul

from aiobotocore.

@jettify an option for abstraction of IO from any code: https://gist.github.com/graingert/ca6cdd5d9ae2e18ca917b4594ac8a633

from aiobotocore.

perhaps we can have a class by class goal of rolling into botocore to make it bite-sized chunks of work. But I think before that we need a high-level strategy defined of how we want this to integrate into botocore to create a straw-man we can work towards and get botocore dev approval on the design.

from aiobotocore.

another option is switching to the c++ library: https://github.com/aws/aws-sdk-cpp this would greatly increase performance as well, mainly due to signing.

from aiobotocore.

The general pattern in botocore is to register components or to hook up callbacks to an event-emitter (which itself could be a component). See

• https://boto3.amazonaws.com/v1/documentation/api/latest/guide/events.html#boto3-specific-events
• the boto3 example shows how to replace the BaseClient with a substitute parent class, which could be AioBaseClient
• it also shows how to add custom methods on a client; aiobotocore could add new async methods that could be used side-by-side with existing "synchronous" methods rather than replacing those methods (if the botocore API is preserved and extended with new async methods, it might have a better chance of inclusion in botocore)
• the boto3 event documentation is better than botocore event documentation, but essentially the same event system is used in botocore and boto3
• moto uses the event system to apply mocks as before-send callbacks

In that context, aiobotocore might create and register components for:

• custom event emitter that uses coroutines that can be awaited
• custom client creator and client that can await event callbacks
• custom client event handlers/coroutines that can be awaited

It’s not clear whether this would be any different, in the end, than what aiobotocore is already. This might be an alternative pattern to explore in aiobotocore as a POC, but it’s not clear whether it would work and it’s got little chance, if any, of ever getting merged into botocore (lots of useful PRs are stale on botocore).

from aiobotocore.