dolamroth / starlette-web Goto Github PK

This would require to switch to asynchronous template rendering, substituting default starlette's TemplateResponse, since they don't favor async template rendering https://www.starlette.io/templates/#asynchronous-template-rendering

Based on discussion agronholm/smtpproto#35

Some ideas may be fetched from:

• https://pypi.org/project/apispec-starlette/
• ...

Targets:

• add introspection for path parameters
• inherit from Maschmallow.Schema for custom registering schema
• additional parameters for Schema, such as description
• possibility to add additional schemas "method_name_schema", such as "post_request_schema"
• process scopes for security schemas (or just stick to permission classes ?)
• cache OpenApiView (with #5)
• etc

• Linux AIO https://github.com/mosquito/aiofile
• io_uring https://github.com/tree4096/uring_file
• SUDDENLY WINDOWS mhammond/pywin32#430

In this case, module __file__ is None

• Default installed apps do not match default libraries, installed without extra dependencies (comment out default installed apps)
• Default asgi.py in project root must setdefault STARLETTE_SETTINGS_MODULE before call to get_app
• If there is import error in installed apps. app is not getting imported, resulting in incorrect error message. Maybe import modules in initialize method?
• Alembic errors on migrate/makemigrations are incomprehencible. Maybe add makemigrations/migrate management commands?

Starlette itself is anyio-compatible. However, underlying implementations of low-level libraries are mostly not.

• With current scheme, SQLAlchemy is main core part, which is not anyio-compatible.
• Anyio-compatible websockets are available, if using hypercorn (beta, as of right now) + anysocks/asyncwebsockets. However, so far these are less reliable/feature-complete than asyncio implementation.
• There is no DB engines (at least for Postgres) with anyio, but there are separate implementations for asyncio and trio
  • pg-purepy exists, but it is not feature-complete
  • trio-pg is just wrapper around asyng pg
• Same for redis / mqtt / rmq
• There is anyio implementation for SMTP https://pypi.org/project/smtpproto/

Desireable contrib libraries to have around for production

• MongoDB driver, apparently easy to impement via https://github.com/mongodb/motor
• Elasticsearch driver, should be easy to implement with httpx
• aioftp/aiosftp => anyio - seems moderately difficult
• aiortc => anyio - seems very difficult
• aiozmq => anyio - seems very difficult, also see: python-trio/trio#116
• aiobotocore for S3 file storage - pending issue aio-libs/aiobotocore#749

All pros and cons must be carefully decided.

Requirements:

• Tortoise ORM is out of beta
• Tortoise ORM fully supports subqueries, F and Q expressions as Django does
• Tortoise ORM has no breaking bugs or issues (at least, not monkey-patch'able)
• tortoise/tortoise-orm#1341
• FastAPI must not require redis fastapi-admin/fastapi-admin#81

Additional requirements:

• Tortoise ORM fully supports anyio tortoise/tortoise-orm#1119
• FileField support is included tortoise/tortoise-orm#100

FastAPI-admin seems to be more feature complete, than starlette_admin, but has a hard requirement for Tortoise ORM.

Additional point to consider is that Tortoise ORM basically autocommit all-the-way-through, similar to Django ORM.

• Pros: simplicity (i.e., managing files with SQLAlchemy in a non-autocommit mode is a daunting task, to say at least https://github.com/jowilf/sqlalchemy-file), no need to pass session in every query, easier processing of triggers
• Cons: worse performance, worse data isolation

Will probably require to move from marshmallow to pydantic, but needs consideration. Pydantic is widely reported to be faster and more feature complete, but marshmallow is said to be more customizable. However, it was pointed out, that Pydantic reliance on Python typing features may be a limitation in certain tasks.

It appears, this library has no support for composite primary keys, which potentially may be a problem.

Must be rstrip instead of strip (as of now, strips initial slash "/" on Linux)

strip/strip -> rstrip/lstrip

• https://github.com/florimondmanca/asgi-caches

• https://github.com/eerimoq/argparse_addons

def argparse_uuid_type(arg_value: str) -> uuid.UUID:
    try:
        return uuid.UUID(arg_value)
    except (ValueError, TypeError):
        raise argparse.ArgumentTypeError('Invalid UUID value')


def argparse_date_type(arg_value: str) -> datetime.date:
    try:
        return datetime.datetime.strptime(arg_value, "%Y-%m-%d").date()
    except ValueError:
        raise argparse.ArgumentTypeError('Invalid date value')

https://marshmallow-sqlalchemy.readthedocs.io/en/latest/recipes.html#transient-object-creation
https://stackoverflow.com/questions/68212540/how-to-use-marshmallow-sqlalchemy-with-async-code

https://pypi.org/project/starlette-graphene3/

https://docs.djangoproject.com/en/4.2/topics/db/multi-db/

While at it, also move from initializing app with session maker in each command, but provide a global application object.

Redo settings.DATABASE -> settings.DATABASES and provide global variable connections, in the same manner as Django

ConnectionHandler is a wrapper around project sessionmaker(s). This will be a step for #21
https://docs.sqlalchemy.org/en/20/orm/extensions/asyncio.html

# using example from "Custom Vertical Partitioning"

import random

from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.ext.asyncio import create_async_engine
from sqlalchemy.ext.asyncio import async_sessionmaker
from sqlalchemy.orm import Session

# construct async engines w/ async drivers
engines = {
    'leader':create_async_engine("sqlite+aiosqlite:///leader.db"),
    'other':create_async_engine("sqlite+aiosqlite:///other.db"),
    'follower1':create_async_engine("sqlite+aiosqlite:///follower1.db"),
    'follower2':create_async_engine("sqlite+aiosqlite:///follower2.db"),
}

class RoutingSession(Session):
    def get_bind(self, mapper=None, clause=None, **kw):
        # within get_bind(), return sync engines
        if mapper and issubclass(mapper.class_, MyOtherClass):
            return engines['other'].sync_engine
        elif self._flushing or isinstance(clause, (Update, Delete)):
            return engines['leader'].sync_engine
        else:
            return engines[
                random.choice(['follower1','follower2'])
            ].sync_engine

# apply to AsyncSession using sync_session_class
AsyncSessionMaker = async_sessionmaker(
    sync_session_class=RoutingSession
)

Validation for constance has sense if it is used in admin panel, and has not much sense otherwise.

This issue requires the following task to be performed before:

• adapt admin panel into framework
• make custom admin page for constance
• allow custom widgets for admin-panel and various data types

Validation should be done by making base class ConstanceType, which defines .clear() method, and widget for admin-panel.

Targets for 0.2:

• Filesystem storage
• SFTPStorage (with aiosftp)
• S3 Storage (with aioboto3/aiobotocore)

• Readthedocs
• Sphinx

• array_json, row_json (subqueryjson, subqueryjsonagg)
• truncate
• bulk methods / on conflict
• ts vectors https://amitosh.medium.com/full-text-search-fts-with-postgresql-and-sqlalchemy-edc436330a0c
• gin/gist indices examples
• exclusion constraints

Consider, which of these may be useful to include/link in the framework

• Result wrapper for TaskGroup https://pypi.org/project/aioresult/
• https://github.com/tiangolo/asyncer/blob/main/asyncer/_main.py

Just add hardcoded warnings, the duplicates are already forbidden