async support :) about deta-python HOT 15 CLOSED

hi @DanielChuDC , huge thanks to @leits we have an alpha version of the sdk with async methods for Base : https://docs.deta.sh/docs/base/async_sdk

we do not have async methods for Drive yet, if you want to work on it.

from deta-python.

Looks like a plan 👍

from deta-python.

@leits
Thanks for the offer, your help would be appreciated! 😊
We need to decide on the API first. I guess the it will stay mostly the same, the only difference could be the class name...

from deta import Base  # sync base direct import
from deta import AsyncBase  # async base direct import
from deta import Deta

db = Deta().Base("db")   # sync base
adb = Deta().AsyncBase("db")  # async base

db.get("1")
await adb.get("1")

Another way (which simplifies the API a bit) is to use a modifier method which will use the async methods:

db = Base("db")  # sync
adb = Base("db").async()  # async

I kinda prefer the latter but not sure how pythonic it is. I would like to discuss this with the team first. (we take API design seriously)

What kind of API would you like to see?

And btw, we try to not use external dependencies for our SDKs.

from deta-python.

Yeah I would like to help to contribute! May I know for the drive async, can I use the same style from the async base code base?

great 🎉 , yes please use the async base code as guide.

Or there is any specific requirements for drive async?

no nothing specific, but please keep the method signatures consistent with the current sync Drive methods. If you feel like something can/should be improved (API wise), we can make the improvements in a later version.

from deta-python.

Hey @harshitsinghai77 Please take on the drive async. Thank you!

from deta-python.

+1 here

In the tutorials, there are many FastAPI examples with asynchronous views but synchronous queries in Deta Base 😅

from deta-python.

Hi, @abdelhai!
I can help you with async support. Do you have some ideas? Or I can just come up with some solution in a pull request?

from deta-python.

Got you!

First of all, a second way is far away from python best practices because Explicit is better than implicit (from Zen of Python).

So there are two main ways in the python community for async support:

The first one is the way that you propose. Add explicit async client class in current client lib.
For example elasticsearch-py with Elasticsearch and AsyncElasticsearch:
https://elasticsearch-py.readthedocs.io/en/7.x/async.html

The second way is to create a separate async client. Like for all database clients - asyncpg, aiomysql, aiosqlite, motor (Mongo client), etc.
And also aiobotocore as an async client to AWS (sync client - botocore).
https://github.com/aio-libs

And there are two reasons for this.

1. There is no support for asynchronous HTTP requests in python built-in. And a recommendation from the python core dev team is to use lightweight aiohttp lib supported by them.
2. The principle of operation of asynchronous functions can be very different from synchronous, so under the hood of the library begins a terrible mess, if it supports both methods (just loot into elasticsearch-py sources).

But also, there is a third way, very clever but not popular for clients.
Write fully async client and make sync wrapper that will run async functions under-the-hood.
It's the way fastapi works.
I've already done this on a project from my work.

I can make such a client if you don't mind.

from deta-python.

@leits
Thank you for the clarification, now I have a better understanding of the industry practices.

I think we will likely keep everything in one SDK where's the "async" functionalities are optional/opt-in. You can get them by installing deta[async]/deta[all] and by importing the async service clients AsyncBase, etc.

You could start with 3rd party package where you could experiment with your ideas. I'm interested in seeing what you come up with. We could merge into our SDK if it passes our internal "rules".

We do some weird stuff with the current implementation for managing connections which we want to keep for now, so not sure how this will translate to your implementation.

from deta-python.

Hi May I know any update on this? Would like to help if can.

from deta-python.

Yeah I would like to help to contribute! May I know for the drive async, can I use the same style from the async base code base?

from deta-python.

Or there is any specific requirements for drive async?

from deta-python.

Hey, any updates on this?

I would like to work on async methods for Drive if it's not already been worked on.

@DanielChuDC are you working on this, or should I take this?

from deta-python.

Hey @harshitsinghai77 Please take on the drive async. Thank you!

cc: @aavshr

from deta-python.

hey @harshitsinghai77 feel free to propose a pr for drive async. ideally you open a new issue with the spec/sdk design.
lmk if you need any help! cheers!

from deta-python.