How to pass list in query parameters? about fastapi HOT 20 CLOSED

It is supported (and tested), but not documented yet, but you can just use standard Python types (a List) :)

from typing import List

from fastapi import FastAPI, Query

app = FastAPI()
@app.get("/items/")
def read_items(q: List[int] = Query(None)):
    return {"q": q}

from fastapi.

Is it possible to do:
http://localhost/item?num=1,2,3,4,5,6

instead of:
http://localhost/item?num=1&num=2&num=3&num=4&num=5&num=6?

from fastapi.

Is there a "proper" way to handle lists using the following format: /foo?items[]=1&items[]=2?

This format is used by the axios library which is well known in javascript.

Right now, I use the following trick:

@app.get("/foo")
async def foo(items: List[int] = Query(None, alias="items[]")):
    ...

The problem is that it only handles this syntax in this case and passing this won't work: /foo?items=1&items=2.

One could argue about the fact that the API decides what format to use I guess. But still, it would be cool to handle both syntax as they are pretty close.

from fastapi.

It is now properly documented 📝 🎉

For query parameters: https://fastapi.tiangolo.com/tutorial/query-params-str-validations/#query-parameter-list-multiple-values

And for duplicate headers: https://fastapi.tiangolo.com/tutorial/header-params/#duplicate-headers

from fastapi.

@neilpanchal You could do this by adding a custom dependency that expected a string-value query parameter, then splits the string on , and converts the result to ints. But obviously that's a little less ergonomic.

I think the main reason why this isn't easier is that FastAPI is basically just deferring to starlette for query parameter string extraction, then pydantic for parsing.

So starlette gives query parameter num="1,2,3,4,5,6", and pydantic doesn't know how to parse "1,2,3,4,5,6" into a list of ints.

Starlette does have built-in support for repeated values of the same query parameter, which is why the more verbose approach does work out of the box.

from fastapi.

here's an example dependency that I wrote to handle this sort of thing

I wanted to be able to handle normal query arg lists as well as these awful "string" lists that are wrapped in square brackets and sometimes contain double quotes and spaces that were somehow a part of our existing API.

def parse_list(names: List[str] = Query(None)) -> Optional[List]:
    """
    accepts strings formatted as lists with square brackets
    names can be in the format
    "[bob,jeff,greg]" or '["bob","jeff","greg"]'
    """
    def remove_prefix(text: str, prefix: str):
        return text[text.startswith(prefix) and len(prefix):]

    def remove_postfix(text: str, postfix: str):
        if text.endswith(postfix):
            text = text[:-len(postfix)]
        return text

    if names is None:
        return

    # we already have a list, we can return
    if len(names) > 1:
        return names

    # if we don't start with a "[" and end with "]" it's just a normal entry
    flat_names = names[0]
    if not flat_names.startswith("[") and not flat_names.endswith("]"):
        return names

    flat_names = remove_prefix(flat_names, "[")
    flat_names = remove_postfix(flat_names, "]")

    names_list = flat_names.split(",")
    names_list = [remove_prefix(n.strip(), "\"") for n in names_list]
    names_list = [remove_postfix(n.strip(), "\"") for n in names_list]

    return names_list


@app.get("/hello_list")
def hello_list(names: List[str] = Depends(parse_list)):
    """ list param method """

    if names is not None:
        return StreamingResponse((f"Hello {name}" for name in names))
    else:
        return {"message": "no names"}

from fastapi.

In case it helps anyone, I used a middleware to turn comma-delimited query parameter strings into repeated query parameters which work well with the out of the box/Pydantic list parsing. Pure-python in 3.7 or later (maybe lower?)

import fastapi

from starlette.types import ASGIApp, Scope, Receive, Send
from urllib.parse import parse_qs as parse_query_string
from urllib.parse import urlencode as encode_query_string

class QueryStringFlatteningMiddleware:
    def __init__(self, app: ASGIApp) -> None:
        self.app = app

    async def __call__(self, scope: Scope, receive: Receive, send: Send) -> None:
        query_string = scope.get("query_string", None).decode()
        if scope["type"] == "http" and query_string:
            parsed = parse_query_string(query_string)
            flattened = {}
            for name, values in parsed.items():
                all_values = []
                for value in values:
                    all_values.extend(value.split(","))

                flattened[name] = all_values

            # doseq: Turn lists into repeated parameters, which is better for FastAPI
            scope["query_string"] = encode_query_string(flattened, doseq=True).encode("utf-8")

            await self.app(scope, receive, send)
        else:
            await self.app(scope, receive, send)

app = fastapi.FastAPI()
app.add_middleware(QueryStringFlatteningMiddleware)

Converts ?state=COMPLETED,ENQUEUED -> ?state=COMPLETED&state=ENQUEUED

note: This code hasn't gone through any kind of hardening, it's hot off the presses so to speak.

from fastapi.

A simpler implementation of the middleware solution from @allenhumphreys. Works for me, use at your own risk.
Like the original, Converts ?state=COMPLETED,ENQUEUED to ?state=COMPLETED&state=ENQUEUED.

from urllib.parse import urlencode
from fastapi import Request

@app.middleware("http")
async def flatten_query_string_lists(request: Request, call_next):

    flattened = []
    for key, value in request.query_params.multi_items():
        flattened.extend((key, entry) for entry in value.split(','))

    request.scope["query_string"] = urlencode(flattened, doseq=True).encode("utf-8")

    return await call_next(request)

from fastapi.

Here is my solution (py3.10), I hope it will be useful for someone:

class TypeParametersMemoizer(type):
    _generics_cache = weakref.WeakValueDictionary()

    def __getitem__(cls, typeparams):

        # prevent duplication of generic types
        if typeparams in cls._generics_cache:
            return cls._generics_cache[typeparams]

        # middleware class for holding type parameters
        class TypeParamsWrapper(cls):
            __type_parameters__ = typeparams if isinstance(typeparams, tuple) else (typeparams,)

            @classmethod
            def _get_type_parameters(cls):
                return cls.__type_parameters__

        return types.GenericAlias(TypeParamsWrapper, typeparams)


class CommaSeparatedList(list, metaclass=TypeParametersMemoizer):
    @classmethod
    def __get_validators__(cls):
        yield cls.validate

    @classmethod
    def validate(cls, v: str | list[str]):
        if isinstance(v, str):
            v = v.split(",")
        else:
            v = list(itertools.chain.from_iterable((x.split(",") for x in v)))
        params = cls._get_type_parameters()
        return pydantic.parse_obj_as(list[params], list(map(str.strip, v)))

    @classmethod
    def _get_type_parameters(cls):
        raise NotImplementedError("should be overridden in metaclass")

Usage:

@app.get("/items")
def get_items(ids: CommaSeparatedList[int] = Query(...)):
     ...

pros:

• it's works (at least with py 3.10)
• openapi works as expected
• supports both formats ?param=FOO&param=BAR & ?param=FOO,BAR

cons:

• complex
• looks like black magic (may stop working after a new version of python)

from fastapi.

here's an example dependency that I wrote to handle this sort of thing

I wanted to be able to handle normal query arg lists as well as these awful "string" lists that are wrapped in square brackets and sometimes contain double quotes and spaces that were somehow a part of our existing API.

def parse_list(names: List[str] = Query(None)) -> Optional[List]:
    """
    accepts strings formatted as lists with square brackets
    names can be in the format
    "[bob,jeff,greg]" or '["bob","jeff","greg"]'
    """
    def remove_prefix(text: str, prefix: str):
        return text[text.startswith(prefix) and len(prefix):]

    def remove_postfix(text: str, postfix: str):
        if text.endswith(postfix):
            text = text[:-len(postfix)]
        return text

    if names is None:
        return

    # we already have a list, we can return
    if len(names) > 1:
        return names

    # if we don't start with a "[" and end with "]" it's just a normal entry
    flat_names = names[0]
    if not flat_names.startswith("[") and not flat_names.endswith("]"):
        return names

    flat_names = remove_prefix(flat_names, "[")
    flat_names = remove_postfix(flat_names, "]")

    names_list = flat_names.split(",")
    names_list = [remove_prefix(n.strip(), "\"") for n in names_list]
    names_list = [remove_postfix(n.strip(), "\"") for n in names_list]

    return names_list


@app.get("/hello_list")
def hello_list(names: List[str] = Depends(parse_list)):
    """ list param method """

    if names is not None:
        return StreamingResponse((f"Hello {name}" for name in names))
    else:
        return {"message": "no names"}

I've done a little modification to this in order to make the code reusable across different usecases. This is the code:

from typing import List, Optional, Type
from fastapi import Query

def parse_list(query: Query, class_type: Type):
    def inner_parse(elements: List[str] = query) -> Optional[List[class_type]]:
        def remove_prefix(text: str, prefix: str):
            return text[text.startswith(prefix) and len(prefix):]

        def remove_postfix(text: str, postfix: str):
            if text.endswith(postfix):
                text = text[:-len(postfix)]
            return text

        if query.default != Ellipsis and not elements:
            return query.default

        if len(elements) > 1:
            elements_list = elements
        else:
            flat_elements = elements[0]
            if flat_elements.startswith("["):
                flat_elements = remove_prefix(flat_elements, "[")

            if flat_elements.endswith("]"):
                flat_elements = remove_postfix(flat_elements, "]")

            if query.default != Ellipsis and not flat_elements:
                return query.default

            elements_list = flat_elements.split(",")
            elements_list = [remove_prefix(n.strip(), "\"") for n in elements_list]
            elements_list = [remove_postfix(n.strip(), "\"") for n in elements_list]

        errors = {}
        results = []
        for idx, el in enumerate(elements_list):
            try:
                results.append(class_type(el))
            except ValueError as e:
                errors[idx] = repr(e)
        if errors:
            raise Exception(f"Could not parse elements: {errors}")
        else:
            return results
    return inner_parse

The idea here is that I can call the parse_list function passing the Query element that I need, thus making the code more reusable.

For example, I can create an Enum like this:

class SimpleEnum(str, Enum):
    aaa = 'aaa'
    bbb = 'bbb'
    ccc = 'ccc'

and then use the parse_list function like this:

parsed_enums: List[SimpleEnum] = Depends(
    parse_list(query=Query([], alias="enums", description="Some description"), class_type=SimpleEnum)
)

This way, the Query object can be defined in the proper router.

Disclaimer: I did not spend time making this code perfectly robust, but it works just fine for what I needed. I hope this can help.

from fastapi.

Possible solution I'm using.

Handler: https://github.com/dotX12/pyfa-converter/blob/master/examples/main.py#L50
Model: https://github.com/dotX12/pyfa-converter/blob/master/examples/models.py#L51

Result:

from fastapi.

@GabrieleMazzola I liked your function! But to make it work (on FastAPI 0.78) I had to set the alias parameter of the Query that I pass to parse_list have the same name as the query parameter, parsed_enums in the example you showed

parsed_enums: List[SimpleEnum] = Depends(
    parse_list(query=Query([], alias="parsed_enums", description="Some description"), class_type=SimpleEnum)
)

Super useful anyway to not have to declare a new dependency per parameter, thanks!

from fastapi.

I guess this is solved now, so I'll close the issue. But feel free to add more comments or create new issues.

from fastapi.

Hey @falkben (or any one else in the thread),

This is a great solution and works great.

But I had to refactor my method signature from:

@app.get("/hello_list")
def hello_list(names: List[str] = Query(None, description="List of names to greet")):
    """ list param method """

to

@app.get("/hello_list")
def hello_list(names: List[str] = Depends(parse_list)):
    """ list param method """

and I lost the nice documentation provided by the 'description' field.

Any idea how I could combine Query with Depends? Should I open another issue for greater visibility?

from fastapi.

You should be able to move the description into the Depends method. So in the example, the depends method signature would look like this:

def parse_list(names: List[str] = Query(None, description="List of names to greet")) -> Optional[List]:

from fastapi.

Hmmm, that is interesting, but that locks you into using the same description to all query params that you might want to check.

But wait, now that I am looking at it, this looks like it is tied to one query param name only, and the same 'parse_list` method cannot be reused for another param....? :(

Seems like, I'll have to take a closer look...

[EDITED]: to say that your suggestion is indeed working, for one param that appears as a list, but I cannot add a second one.

from fastapi.

In case it helps anyone, I used a middleware to turn comma-delimited query parameter strings into repeated query parameters which work well with the out of the box/Pydantic list parsing. Pure-python in 3.7 or later (maybe lower?)

import fastapi

from starlette.types import ASGIApp, Scope, Receive, Send
from urllib.parse import parse_qs as parse_query_string
from urllib.parse import urlencode as encode_query_string

class QueryStringFlatteningMiddleware:
    def __init__(self, app: ASGIApp) -> None:
        self.app = app

    async def __call__(self, scope: Scope, receive: Receive, send: Send) -> None:
        query_string = scope.get("query_string", None).decode()
        if scope["type"] == "http" and query_string:
            parsed = parse_query_string(query_string)
            flattened = {}
            for name, values in parsed.items():
                all_values = []
                for value in values:
                    all_values.extend(value.split(","))

                flattened[name] = all_values

            # doseq: Turn lists into repeated parameters, which is better for FastAPI
            scope["query_string"] = encode_query_string(flattened, doseq=True).encode("utf-8")

            await self.app(scope, receive, send)
        else:
            await self.app(scope, receive, send)

app = fastapi.FastAPI()
app.add_middleware(QueryStringFlatteningMiddleware)

Converts ?state=COMPLETED,ENQUEUED -> ?state=COMPLETED&state=ENQUEUED

note: This code hasn't gone through any kind of hardening, it's hot off the presses so to speak.

Thanks!
Would be useful if FastAPI had this as default.

from fastapi.

@avylove Thanks for the good work. But It could be a problem cause splitting all strings. If I get Query parameter like "Hello, World". This is just long string not List parameter, but the code will split the sentence.

from fastapi.

what about using a query param with a separated string by a comma?
for example:
/foo?id=x,y,z
expected behavior to get it in fast API like [x,y,z] currently we get it as a List with one string which is "x,y,z"

is there planning to support it? or is there any patch code for it?

from fastapi.

what about using a query param with a separated string by a comma? for example: /foo?id=x,y,z expected behavior to get it in fast API like [x,y,z] currently we get it as a List with one string which is "x,y,z"

is there planning to support it? or is there any patch code for it?

foo: List[str] = Query(…) is
/users?foo=a&foo=b&foo=c

from fastapi.