neoteroi / blacksheep Goto Github PK

View Code? Open in Web Editor NEW

1.7K 29.0 72.0 2.24 MB

Fast ASGI web framework for Python

Home Page: https://www.neoteroi.dev/blacksheep/

License: MIT License

Makefile 0.14% Python 93.79% HTML 0.34% JavaScript 0.01% CSS 0.11% Cython 5.45% Jinja 0.16%

asyncio server http asgi blacksheep python web framework http-server

blacksheep's People

Contributors

Stargazers

Watchers

Forkers

hzlmn xrick keshapps dutradda themushrr00m turall duzhanyuan perfmjs ryefccd belingud hefnymah luafan2 asuradoll vinayakkorti skivis 5l1v3r1 bjpjg kun1996 hymanzhan gitter-badger newday1313 cdayz yassineelbouchaibi informatica92 cuongnguyen2190 joffreybvn jordemort rafalohaki q0w adriangb jxcorra zammalhabe yxlwfds nlann klavionik roland-ruedenauer janusznowak nico-vromans casualuser onlyone0001 phenobarbital rirex openssl-sg-insights carlosm27 gh0st-work jack-fireworkhq antipooh la5tway airiuwu bitnom thatsuseful tridro tyzhnenko rophxi arpitjain799 godd0t thomafred pythonsamp thearchitector xiahui1986 group4life joshua-auchincloss bymoye 9hundred100five1 habibzadeh randomneo ranjititio daudln xzregg stasyao mmangione rudy-th

blacksheep's Issues

GraphQL Support

BlackSheep looks promising but anyone tried to implement Graphene for GraphQL?

Improve bad request response for invalid dataclass

If a JSON input for a dataclass is missing a required property, currently the response doesn't specify the class that is missing a property.

Bad Request: missing 1 required parameter: 'foo'

Improve by returning the information about the class that caused the exception.

Inconsistent `request.form()` return values for multipart vs urlencode

I think the urlencode return value is the best presentation. So, even if the content-type is multipart, request.form() should also return dictionary.

Change to wiki text.

Something worth changing on the wiki text:

"Christie is recommending Python community to adopt a standard API he designed, ASGI"

The ASGI spec is authored and maintained by Andrew Godwin. I'm just a supporter of its adoption. 😄

request.query values as is not list data type

Is your feature request related to a problem?

background
insert or update database from cli must convert first
e.g.

def dict_values(raw_ori):
    processed_raw = {}
    for i in raw_ori.items():
        if len(i[1] ) > 1:
            processed_raw[i[0] ] = i[1]
        else:
            for j in i[1]:
                processed_raw[i[0] ] = j
    return processed_raw

@app.route('/api/post/{tablename}', methods = ['POST'] )
def post(request, tablename):
    if request.query:
        values = dict_values(request.query)
    row = db[tablename].validate_and_insert(**values)
    db.commit()
    return response.json(row.as_dict() )

request.query
curl -X POST -i "http://localhost:8000/api/post/instrument?name=name3&strings=3"

result without calling dict_values() function, the value data type list, will return an error when insert or update database

request.query = {'name' : ['name3'], 'strings' : ['3'] }

expected result, solve by calling dict_values() function

request.query = {'name' : 'name3', 'strings' : '3'}

Describe the solution you'd like

insert or update database from cli as is data (values not list data type) from request.query
e.g.

@app.route('/api/post/{tablename}', methods = ['POST'] )
def post(request, tablename):
    if request.query:
        values = request.query
    row = db[tablename].validate_and_insert(**values)
    db.commit()
    return response.json(row.as_dict() )

Additional context

question
is it possible to have request.query values as is not list data type ?
so that can insert or update database without calling dict_values() function

thanks

Reduce serve files verbosity

Consider removing ServeFilesOptions class.

Jinja2 has an async mode

Unless I'm mistaken, you could make use of the Jinja2 async mode, (an example in Quart).

Wrong logic in write_small_response

write_small_response lacks check for request.content not None

Request Cookies are not read properly

Fix error in reading cookies; and add tests for this scenario. Add a dedicated class for Cookies, instead of using dict. Make them more user-friendly.

How to bind path to api endpoint?

Hey, I'm trying to bind dynamic path to api endpoint in BlackSheep just like we do in flask and quart. But it's not working.

Example:

In flask and quart it can be done like this:

url: xyz.com/get/file/dynamic/path/to/file

@api.route('/get/file/<path:filepath>', methods=['GET'])
def get_file(filepath):
       .....

How to do this in BlackSheep?

Consolidate essentials dependency

Consolidate this dependency:

ERROR: After October 2020 you may experience errors when installing or updating packages. This is because pip will change the way that it resolves dependency conflicts.

We recommend you use --use-feature=2020-resolver to test your packages with the new resolver before it becomes the default.

essentials-openapi 0.0.9 requires essentials==1.1.3, but you'll have essentials 1.1.4 which is incompatible.

Include encoding in Content-Type response header

Include encoding in Content-Type response header.

Example for views rendered with Jinja2:

Header(b'Content-Type', b'text/html; charset=utf-8')

Correct error happening with latest pip-20.3.1

ERROR: Cannot install blacksheep and blacksheep==0.2.8 because these package versions have conflicting dependencies.

The conflict is caused by:
    blacksheep 0.2.8 depends on essentials==1.1.4
    essentials-openapi 0.0.9 depends on essentials==1.1.3
    blacksheep 0.2.8 depends on essentials==1.1.4
    essentials-openapi 0.0.2 depends on essentials==1.1.3

The Controller class lacks the method to return files

os.stat offers better performance than aiofiles.os.stat?

When the number of concurrent connections is limited, reading files using open without a thread-pool offers better performance than using thread-pools (like in aiofiles). However, when the number of concurrent connections is higher, for example 1000; a thread pool shows its benefit.

The same doesn't hold true for os.stat, that offers much better performance than aiofiles.os.stat even with 1000 concurrent connections.

Add more binders

For request HTTP method
For request URL
FromText, FromBytes, and FromFiles

Improve the API to set response cookies

BadRequestFormat exception produces HTTP 500 response

BadRequestFormat should produce an HTTP 400 response.

Internal Server Error.
While handling request: POST /something

Exception type:
BadRequestFormat
Exception message:
Declared Content-Type is application/json but the content cannot be parsed as JSON.

Uvicorn setup change

Hi @RobertoPrevato,

I can see you ping uvicorn version in requirements.txt.
https://github.com/RobertoPrevato/BlackSheep/blob/a0bacc8c28f7a6158b73e21f8096bb73949c2406/requirements.txt#L46

The setup of this dependency changes for 0.12, see https://github.com/encode/uvicorn/blob/master/CHANGELOG.md#0120---2020-09-28

Regards,

ASGI Support

Congratulations, this looks to be a really cool project 👍 .

I'd like to encourage you to adopt ASGI. (I see you discussed this in #7, but I wanted to add my voice of encouragement).

request.json() causes exception when the request doesn't have any body

await request.json() causes exception when the request doesn't have any body.

await request.json()

 File "blacksheep/messages.pyx", line 231, in json
AttributeError: 'NoneType' object has no attribute 'decode'

Consider requiring an authenticated user in default authorization policy

Consider changing the following to request an authenticated user by default:

        if strategy.default_policy is None:
            # by default, a default policy is configured with no requirements,
            # meaning that request handlers allow anonymous users, unless
            # specified otherwise
            # this can be modified, by adding a requirement to the default
            # policy
            strategy.default_policy = Policy("default")

This is to make @auth more useful and less confusing when the user doesn't specify any authorization rule.

Upgrade to httptools 0.1.1

Add workflow to test, update branch policies

Currently the build happens in Azure DevOps.

Add a workflow to do CI build and validation for PR in GitHub.

Documentation improvements

Correct sentences in https://www.neoteroi.dev/blacksheep/binders/:

Rephrase to repeat "name":

An example of implicit binding is when a request handler parameter is read from the request URL's route parameters because its name matches the one of a route parameter:

A dot is missing in the following sentence:

input is obtained reading the request payload, parsing it as JSON, and creating an instance of CreateCatInput from it if an exception occurs while trying to parse the request payload or when instantiating the CreateCatInput, the framework produces automatically a 400 Bad Request response for the client.

Segmentation fault at PySet_Discard

Thread 1 "python" received signal SIGSEGV, Segmentation fault.
PySet_Discard (set=set@entry=0x0, key=key@entry=0x7fffeee1a2a8) at Objects/setobject.c:2331
2331        if (!PySet_Check(set)) {

More details:

#0  PySet_Discard (set=set@entry=0x0, key=key@entry=0x7fffeee1a2a8) at Objects/setobject.c:2331
#1  0x00007fffef8ca077 in __Pyx_PySet_Discard (key=0x7fffeee1a2a8, set=0x0) at blacksheep/connection.c:9544
#2  __pyx_f_10blacksheep_10connection_16ServerConnection_dispose (__pyx_v_self=0x7fffeee1a2a8) at blacksheep/connection.c:3873
#3  0x00007fffef8bd36e in __pyx_pf_10blacksheep_10connection_16ServerConnection_14connection_lost (__pyx_v_exc=<optimized out>, 
    __pyx_v_self=<optimized out>) at blacksheep/connection.c:3574
#4  __pyx_pw_10blacksheep_10connection_16ServerConnection_15connection_lost (__pyx_v_self=<optimized out>, __pyx_v_exc=<optimized out>)
    at blacksheep/connection.c:3561
#5  0x00007fffeff31693 in __Pyx_PyObject_CallMethO (arg=0x555555a48dc0 <_Py_NoneStruct>, func=<optimized out>) at uvloop/loop.c:161546
#6  __Pyx_PyObject_CallOneArg (func=<optimized out>, arg=0x555555a48dc0 <_Py_NoneStruct>) at uvloop/loop.c:30505
#7  0x00007ffff0046257 in __pyx_f_6uvloop_4loop_15UVBaseTransport__call_connection_lost (__pyx_v_self=0x555555e7c2b8, 
    __pyx_v_exc=0x555555a48dc0 <_Py_NoneStruct>) at uvloop/loop.c:83709
#8  0x00007ffff006bd59 in __pyx_f_6uvloop_4loop_6Handle__run (__pyx_v_self=__pyx_v_self@entry=0x7fffef669048) at uvloop/loop.c:57826

investigate and resolve.

Cookie parsing not RFC 6265 compliant (enough, at least.)

SUMMARY

parse_cookie (at blacksheep/cookies.pyx) does not currently split cookie values correctly so parsing cookies with an equals sign in the value fails. I don't know what direction you intend to take cookie parsing in in the future, but limiting to one split should at least allow the unpack to complete for now.

ISSUE TYPE

Bug Report

COMPONENT NAME

blacksheep/cookies.pyx

OS / ENVIRONMENT

N/A

STEPS TO REPRODUCE

Google seems to reliably give me a cookie with an equals sign in it:

import asyncio

import blacksheep.client

async def amain():
    async with blacksheep.client.ClientSession(follow_redirects=False) as session:
        response = await session.get("https://www.google.com/")
        for i in response.headers:
            print(i)
        print(await response.text())

if __name__ == "__main__":
    asyncio.run(amain())

EXPECTED RESULTS

Assuming I can index the headers collection (I was trying out blacksheep when I hit this), I should see a list of header names, and the body text of the response?

ACTUAL RESULTS

A ValueError is thrown as unpacking fails.

Improve built-in synch logging

in the built-in integration with sync logging,
consider (TBD):

use a dedicated file for each process
create a file on disk only if sync logging is used
when logging exceptions (built-in error handling), include the id of the process that was handling the request

Remove aiofiles

aiofiles is only used to do read operations on files. This library is not actively maintained. Remove.

CORS policy

Hi,

How can we use the CORS policy in the system. Will you make an addition for this? If you don't have an add-on plan for now, can we do it with middleware support?

Unable to get any identifying information from a request

Trying to get some kind of identifying information from a blacksheep.request object.

Poking around in the source I found request.identity which is used for auth, but this is NoneType when trying to use it in a route.

Basically just some way to get the IP of the request is what I'm looking for.

Update the README to point to the new website

Remove links to the Wiki in GitHub, update the README and link to the new website
https://www.neoteroi.dev/blacksheep/

Setting option to the server

Hi @RobertoPrevato,

Following https://github.com/RobertoPrevato/BlackSheep/wiki/Automatic-reload, I use

from blacksheep.server import Application, ServerOptions

to set host and port for the server

but on 0.1.4 I have

ImportError: cannot import name 'ServerOptions' from 'blacksheep.server' (/usr/local/lib64/python3.7/site-packages/blacksheep/server/__init__.py)

Regards,

OpenAPIHandler not using json_spec_path and yaml_spec_path in UI

OpenAPIHandler not using json_spec_path and yaml_spec_path in UI.

It must use the right paths in the generated HTML.

Provide wheels

Change the release pipeline to build wheels and include them in the distribution package.

Free ports at application stop with WLS (Windows Linux Subsystem)

It looks like ports are not freed properly when the application stops when using WLS. This does not happen consistently, and also this error happens:

/blacksheep/server/application.py", line 372, in spawn_server
    s.shutdown(SHUT_RDWR)
OSError: [Errno 107] Transport endpoint is not connected

blacksheep can't show debug when error occured

blacksheep can't show debug when error occured (either embed in python file or execute the uvicorn server in terminal)
first think related with uvicorn, but test with another framework the same command or python uvicorn.run() can show debug on terminal when error occured (e.g. starlette, fastapi)
e.g.
server.py

import uvicorn
from music import *
from music.settings import app
if __name__ == "__main__":
    uvicorn.run("server:app", host = "0.0.0.0", port = 8000, log_level = "debug", reload = True)

test with shell after comment the if condition to execute uvicorn.run()

source activate python3_test
cd ~/Downloads/blacksheep
uvicorn server:app --host 0.0.0.0 --port 8000 --log-level debug --reload

result on terminal
INFO: 127.0.0.1:52791 - "PUT /api/put/instrument/2 HTTP/1.1" 500 Internal Server Error

expected result
have an error traceback in detail (debug level)

thanks

Use ThreadPoolExecutor to serve files

Code to serve files dynamically (here) must use a ThreadPoolExecutor to not block the event loop.

Review also the built in support for logging (asynchronous should be the preferred way anyway), and the code to serve files statically (kept in memory).

Use aiofiles.

Trusted Host Middleware

Hi, @RobertoPrevato

By looking at a few examples, I prepared middleware for the trusted host. There was no problem with my checks. Is the middleware I prepared suitable for your standards? I will try to add www redirect to it. I am trying to understand the your redirect response function.

from typing import Sequence, Callable, Awaitable

from blacksheep import Request, Response
from blacksheep.server.responses import text

ENFORCE_DOMAIN_WILDCARD = "Domain wildcard patterns must be like '*.example.com'."


class TrustedHostMiddleware:
    def __init__(
            self,
            allowed_hosts: Sequence[str] = None,
    ) -> None:
        if allowed_hosts is None:
            allowed_hosts = ["*"]

        for pattern in allowed_hosts:
            assert "*" not in pattern[1:], ENFORCE_DOMAIN_WILDCARD
            if pattern.startswith("*") and pattern != "*":
                assert pattern.startswith("*."), ENFORCE_DOMAIN_WILDCARD

        self.allowed_hosts = list(allowed_hosts)
        self.allow_any = "*" in allowed_hosts

    async def __call__(self,
                       request: Request,
                       handler: Callable[[Request], Awaitable[Response]]
                       ) -> Response:
        host = request.headers.get_single(b"host").decode("utf-8").split(":")[0]
        is_valid_host = False
        for pattern in self.allowed_hosts:
            if host == pattern or (
                    pattern.startswith("*") and host.endswith(pattern[1:])
            ):
                is_valid_host = True
                break
        if is_valid_host:
            response = await handler(request)
        else:
            response = text("Invalid host header", status=400)
        return response

Move on_start callback before handlers normalization

To improve the user's experience and goodness of dependency injection, move on_start callback before handlers normalization. This way, services that need asynchronous initialization (e.g. a db context / pool for PostgreSQL) can be automatically injected in request handlers.

Python3.8 Beta deprecation warning one

To fix:

/blacksheep/client/session.py:264: DeprecationWarning: The loop argument is deprecated 
since Python 3.8, and scheduled for removal in Python 3.10.
    return await asyncio.wait_for(pool.get_connection(),

/blacksheep/blacksheep/client/session.py:314: DeprecationWarning: The loop argument is
 deprecated since Python 3.8, and scheduled for removal in Python 3.10.
    return await asyncio.wait_for(connection.send(request),

Client example code doesn't work, results in ImportError, and then TypeError

When running the client like this:

import asyncio
from blacksheep.client import ClientSession

async def client_example():
    async with ClientSession() as client:
        response = await client.get('https://docs.python.org/3/')
        text = await response.text()
        print(text)

loop = asyncio.get_event_loop()
loop.run_until_complete(client_example())

There is an ImportError:

  File "/venv/webapp/lib/python3.7/site-packages/blacksheep/server/application.py", line 22, in <module>
    from blacksheep.middlewares import get_middlewares_chain
ImportError: cannot import name 'get_middlewares_chain' from 'blacksheep.middlewares' (/home/david/venv/api/lib/python3.7/site-packages/blacksheep/middlewares.py)

I don't understand why Python is giving this error, since the server example code runs fine, and I see the code for get_middlewares_chain in middlewares.py actually exists. But even trying to import it in an empty python console gets the same error.

I temporarily copied the code from get_middlewares_chain into application.py to try again, and there's no import error this time, but now there is a TypeError:

Traceback (most recent call last):
  File "/venv/webapp/lib/python3.7/site-packages/blacksheep/client/pool.py", line 58, in get_connection
    return self._get_connection()
  File "/venv/webapp/lib/python3.7/site-packages/blacksheep/client/pool.py", line 41, in _get_connection
    connection = self._idle_connections.get_nowait()  # type: ClientConnection
  File "/usr/lib/python3.7/asyncio/queues.py", line 182, in get_nowait
    raise QueueEmpty
asyncio.queues.QueueEmpty

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/mnt/data/src/ochous-api/asdf/await.py", line 13, in <module>
    loop.run_until_complete(client_example())
  File "uvloop/loop.pyx", line 1451, in uvloop.loop.Loop.run_until_complete
  File "/mnt/data/src/ochous-api/asdf/await.py", line 7, in client_example
    response = await client.get('https://docs.python.org/3/')
  File "/venv/webapp/lib/python3.7/site-packages/blacksheep/client/session.py", line 324, in get
    None))
  File "/venv/webapp/lib/python3.7/site-packages/blacksheep/client/session.py", line 281, in send
    return await self._handler(request)
  File "/venv/webapp/lib/python3.7/site-packages/blacksheep/middlewares.py", line 6, in middleware_wrapper
    return await handler(request, next_handler)
  File "/venv/webapp/lib/python3.7/site-packages/blacksheep/client/cookies.py", line 285, in cookies_middleware
    response = await next_handler(request)
  File "/venv/webapp/lib/python3.7/site-packages/blacksheep/client/session.py", line 130, in root_handler
    return await self._send_core(request)
  File "/venv/webapp/lib/python3.7/site-packages/blacksheep/client/session.py", line 289, in _send_core
    response = await self._send_using_connection(request)
  File "/venv/webapp/lib/python3.7/site-packages/blacksheep/client/session.py", line 302, in _send_using_connection
    connection = await self.get_connection(request.url)
  File "/venv/webapp/lib/python3.7/site-packages/blacksheep/client/session.py", line 257, in get_connection
    loop=self.loop)
  File "/usr/lib/python3.7/asyncio/tasks.py", line 416, in wait_for
    return fut.result()
  File "/venv/webapp/lib/python3.7/site-packages/blacksheep/client/pool.py", line 60, in get_connection
    return await self.create_connection()
  File "/venv/webapp/lib/python3.7/site-packages/blacksheep/client/pool.py", line 68, in create_connection
    ssl=self.ssl)
  File "uvloop/loop.pyx", line 1782, in create_connection
  File "uvloop/sslproto.pyx", line 237, in uvloop.loop.SSLProtocol.__init__
TypeError: Expected unicode, got bytes

default example not work on Linux Ubyuntu.

I try to run on ubuntu

Successfully built blacksheep
Installing collected packages: httptools, cchardet, guardpost, rodi, aiofiles, essentials, blacksheep
Successfully installed aiofiles-0.4.0 blacksheep-0.2.2 cchardet-2.1.5 essentials-1.1.2 guardpost-0.0.5 httptools-0.1.1 rodi-1.0.8

, default example with error:

$ python3 start_001.py
  File "start_001.py", line 10
    return text(f'Hello, World! {datetime.utcnow().isoformat()}')
                                                               ^
SyntaxError: invalid syntax

@app.router.put(b'/:d')
async def example(a: FromQuery(List[str]),
                  b: FromServices(Dog),
                  c: FromJson(Cat),
                  d: FromRoute(),
                  e: FromHeader(name='X-Example')):

Standard syntax for generics would look like:

@app.router.put(b'/:d')
async def example(a: FromQuery[List[str]],
                  b: FromServices[Dog],

mypy marks this as an error

Design a solution to maintain the current features, but using standard syntax.

Example:


class FromBody(Generic[T]):

    def __init__(self, value: T):
        self._value = value

    @property
    def value(self) -> T:
        return self._value

html() response method not exposed for contollers

At the moment the Controller class exposes methods for all response types except html.

Example:

# We can do
return self.json({'Hello': 'World'})
return self.text('Hello, World')
return self.no_content()
# ...and so on

# But we can't do:
return self.html('...')

Not sure if this is intentional or not. :)

I'll submit a PR for this and you can have a look.

Adopt linters to improve code quality, maybe Black formatter

Flake8
mypy
Black

Maybe reference a benchmmark site ?

I found this site :
https://www.techempower.com/benchmarks/

with a clother look at Python only framework there:
https://www.techempower.com/benchmarks/#section=data-r18&hw=ph&test=fortune&l=zijzen-7

... where I found blacksheep

Enrich the API for OpenAPI Docs

support defining common responses to be shared across all operations
support defining ~~security and~~ servers settings without subclassing OpenAPIHandler

neoteroi / blacksheep Goto Github PK

blacksheep's People

Contributors

Stargazers

Watchers

Forkers

blacksheep's Issues

Is your feature request related to a problem?

Describe the solution you'd like

Additional context

More details:

SUMMARY

ISSUE TYPE

COMPONENT NAME

OS / ENVIRONMENT

STEPS TO REPRODUCE

EXPECTED RESULTS

ACTUAL RESULTS

Recommend Projects

Recommend Topics

Recommend Org