Code Monkey home page Code Monkey logo

webargs's People

Contributors

ch00k avatar damianheard avatar dependabot-preview[bot] avatar dependabot-support avatar dependabot[bot] avatar frol avatar hartror avatar hello-josh avatar hugovk avatar imhoffd avatar immerrr avatar isira-seneviratne avatar itayazolay avatar jmcarp avatar jvrsantacruz avatar lafrech avatar lee3164 avatar lefterisjp avatar marcellarius avatar nateyo avatar pre-commit-ci[bot] avatar pyup-bot avatar sirosen avatar sloria avatar stas avatar thiefmaster avatar timgates42 avatar ugultopu avatar venuatu avatar zaro avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

webargs's Issues

`use_args` does not use the parser's locations

If I specify custom locations in a parser:

parser = FlaskParser(locations=('data', 'query'))

@parser.location_handler('data')
def parse_data(req, name, arg):
    ...

Then decorate a method with use_args:

class WidgetList(MethodView):

    @parser.use_args(widget_args)
    def post(self, args):
    ...

Those custom locations are ignored and the default locations are used:

https://github.com/sloria/webargs/blob/f57c9b730d9b274edce85de38cdc39fdbdc2ecca/webargs/core.py#L445

This requires me to override locations in every use_args decorator:

    #                                v
    @parser.use_args(my_widget_args, locations=parser.locations)

Was this intentional? To me, it seems like line 445 in use_args should be:

locations = locations or self.locations

JSON payload unicode problem on flask

Sending non ASCII text returns : 400 Bad Request

Browser side code :

$.ajax({
    url: "/comments/a123",
    type: "post",
    data: JSON.stringify({'cm': 'é'}),
    contentType: "application/json",
})

Server side code:

comment = {
    'cm': Arg(str),
}

@app.route('/comments/<string:comment_id>', methods=['POST'])
@webargs.flaskparser.use_args(comment)
def comments(args, comment_id):
    comment_markdown = args['cm']

Parsed arguments in Python 3.x w/ Tornado are bytestrings instead of unicode

I ran into some unexpected behavior using this library with Tornado in Python 3.x. It's easy to see using the hello world example:

import tornado.ioloop
from tornado.web import RequestHandler
from webargs import Arg
from webargs.tornadoparser import use_args

class HelloHandler(RequestHandler):
    """A welcome page."""

    hello_args = {
        'name': Arg(str, default='Friend')
    }

    @use_args(hello_args)
    def get(self, args):
        response = {'message': 'Welcome, {}!'.format(args['name'])}
        self.write(response)


if __name__ == '__main__':
    app = tornado.web.Application([
        (r'/', HelloHandler),
    ], debug=True)
    app.listen(5001)
    tornado.ioloop.IOLoop.instance().start()

Querying localhost:5001?name=foo will result in the output:

{"message": "Welcome, b'foo'!"}

This is because Tornado's arguments are bytestrings in Python 3.x, and casting them to str causes them to be in the form b'<val>'.

There are a few ways to fix this behavior. The easiest, requiring no actual code change, is to just use tornado.escape.to_unicode:

from tornado.escape import to_unicode

hello_args = {
    'name': Arg(to_unicode, default='Friend')
}

This results in the output you'd expect, and is reasonable enough, though I believe this should be documented if it's the recommended way to use this library with Tornado in Python 3.x.

Alternatively, get_value in tornadoparser.py could be updated to use Tornado's get_argument methods, which automatically decode to unicode: http://tornado.readthedocs.org/en/latest/web.html#input. This would mean any custom converter method would be passed a regular (unicode) string instead of a bytestring, which may be preferred.

Upgrading from 0.5.1 to 0.6.0 : default is not set (example in description)

import webargs.flaskparser

PAGE = 'p'
page = {
    PAGE: Arg(int,
              default=1,
              validate=lambda p: p > 0,
              error=u"La page demandée n'existe pas",
              target='querystring'),
}

@post.route('/')
@webargs.flaskparser.use_args(page)
def post_index(args):
    page = args[PAGE]
    #in 0.6.0 page is None when it should be set to default (i.e. 1)
    #in 0.5.1 page is correctly set to 1

Checking for argument that is a list of 'int'

I was trying to define a nested Arg that allows one of the parameters to be a list of 'int'. While Arg(list) only checks if it is of type list, I was able to get the necessary validation to work by using multiple=true as Arg(int, multiple=true). This gives me a list of int and rejects other types like str.
However, looking at the relevant code, I am not sure if this is the way it's expected to be used. Could you clarify this? If this is an unexpected behavior or something that won't be maintained or guaranteed, is there another way to achieve this?

Pyramid parser use_kwargs throws exception when used

The following code using the pyramid parser throws an exception:

@parser.use_kwargs({'myvalue': Arg(int)})
def baz(request, myvalue):
    return {'myvalue': myvalue}

The exception:

        kwargs['as_kwargs'] = True
>       return self.use_args(*args, **kwargs)
E       TypeError: use_args() got an unexpected keyword argument 'as_kwargs'

Dates... ?

I'll preface this with what I would like to do:

args = {'start_date': Arg(datetime, convert=from_iso8601)}
parser.parse(args)
# {'start_date': datetime.datetime(2015, 9, 4, 22, 42, 31, 238157, tzinfo=<UTC>)}

So from_iso8601 is a input conversion function supplied by webargs. It'd be cool to have from_rfc3339 and from_format used like partial(from_format, 'd/%m/%y %H:%M') which uses strptime, and maybe from_rfc822.

I don't see this happening without extending Arg and overriding _validate because of this line, which assumes type conversion is as simple as "calling" the type, where, in reality, there may be several different methods to convert one type to another, such as a decimal to an integer, where rounding alone yields six conversion methods.

I will do a pull request, but I just wanted to know if there is something I am overlooking. I just started using webargs yesterday.

Behavior of "allow_missing=True" with "multiple=True"

Currently, when you define an Arg such as:

myargs = {'somearg': Arg(allow_missing=True, multiple=True)}

When somearg is not supplied, the parsed myargs is {'somearg': []}.

However, when you define an Arg with multiple=False:

myargs = {'somearg': Arg(allow_missing=True)}

When somearg is not supplied, the parsed myargs is {}.

I was surprised by this behavior as I assumed that when allow_missing=True and the argument is not supplied, that no parsed value would be returned for both the multiple=False and multiple=True cases.

Is this intentional? Or should no parsed value be returned in the case of Arg(allow_missing=True, multiple=True) instead of []?

Include failing argument name for validation errors

When a validation occurs I want to be able to return the name of the failing argument. At the moment, this is not possible since the validation error contains just a string. An example response:

{
    "message": "A validation error occured.",
    "params": {
        "foo": "Longer than 10"
    }
}

Is this something you feel is worth adding?

User-friendly error messages for type validation

Let's say I have a PUT method that must be given a JSON of this structure:

{ "imdb_data": { "id": 1234567 } }

My code then would look like this:

def validate_imdb_data_json(val):
    if not isinstance(val, dict):
        raise ValidationError('imdb_data is not a dict')
    # further validations of inner dict

imdb_data_json = {'imdb_data': Arg(dict, required=True, validate=validate_imdb_data_json)}

class MovieIMDBDataAPI(Resource):
    def __init__(self):
        super(MovieIMDBDataAPI, self).__init__()

    @use_args(imdb_data_json, targets=('json',))
    def put(self, args, movie_id):
        print args

The problem I have is with webargs' type coversion, namely this part of code. So if I provide this kind of JSON: { "imdb_data": 1 }, my custom validation function validate_imdb_data_json will not even get a chance to work because validation will fail before it gets called, on line 172. And I will result in a response like this:

HTTP/1.0 400 BAD REQUEST
Content-Type: application/json
Content-Length: 50
Access-Control-Allow-Origin: *
Access-Control-Allow-Headers: Content-Type
Server: Werkzeug/0.9.6 Python/2.7.8
Date: Sat, 29 Nov 2014 11:37:29 GMT

{
    "message": "'int' object is not iterable"
}

which is actually the result of an attempt to do dict(1) (btw this is again weird because the code catches ValueError but this kind of error is a TypeError, but this is a different conversation).
The only way I can think of to work around this is to not pass type_ at all. Then all the validation will be done by validate_imdb_data_json function. But that does not look right to me. Is there something I am missing?
This is somewhat similar to #18.

Flask Parser cannot parse json when Content-Type header contains more than just 'application/json'

In flaskparser.py:

def parse_json(self, req, name, arg):
        """Pull a json value from the request."""
        if req.content_type == 'application/json' and hasattr(req, 'json'):
            try:
                return core.get_value(req.json, name, arg.multiple)
            except AttributeError:
                return None
        else:
            return None

The 'Content-Type' header is checked to be exactly equal to 'application/json'.

This prevents parsing of other valid 'Content-Type' headers, such as
Content-Type:application/json;charset=UTF-8
which is what my application sends with its AJAX requests.

Could we change the code to check that the Content-Type contains instead of equals 'appliation/json' to handle these cases?

Nested Fields Usage

I've looked through the documentation and source code and now I just have to ask.

How do I format requests for endpoints that utilize nested fields? Can more documentation be written about nested fields than what exists currently?

v0.5.1 not installable via pip

I tried to upgrade to 0.5.1, but pip says that 0.5.0 is the latest even though pypi shows this: https://pypi.python.org/pypi/webargs/0.5.1:

(env)[~/Desktop/test_webargs] $ pip --version
pip 1.5.6
(env)[~/Desktop/test_webargs] $ pip search webargs
webargs                   - A utility library for parsing HTTP request arguments, with built-in support for popular
                            web frameworks, including Flask and Django.
(env)[~/Desktop/test_webargs] $ pip install webargs
Downloading/unpacking webargs
  Downloading webargs-0.5.0-py2.py3-none-any.whl
Installing collected packages: webargs
Successfully installed webargs
Cleaning up...
(env)[~/Desktop/test_webargs] $ pip search webargs
webargs                   - A utility library for parsing HTTP request arguments, with built-in support for popular
                            web frameworks, including Flask and Django.
  INSTALLED: 0.5.0
  LATEST:    0.5.1

However, 0.5.1 is missing from https://pypi.python.org/simple/webargs/ which is where (I believe) pip is checking:

screenshot from 2014-08-31 11 16 16

The substitute for 'use=lambda' in new API

Just putting this here for others who may be wondering what to do after use= (the callable that let you preprocess the value, see this in pre 0.16.0 code).

For 0.16.0 onwards, use of marshmallow is encouraged. Marshmallow brings a whole slew of serialization/deserialization features, which is exactly what you're doing when you take web requests from various frameworks and turn them into native python data types: deserializing.

use= used to let you change the value of a field while it was parsed. You can accomplish that and more now with marshmallow's post_load decorator:

from marshmallow import Schema, post_load
from webargs import fields

class ArgSchema(Schema):
    name = fields.Str(required=True, location='querystring')

    @post_load
    def uppercase_name(self, data):
        data['name'] = data['name'].upper()
        return data

See the documentation for how to use marshmallow schemas in webargs.

Since you now have access to all fields in the data argument of your post_load method, you can do cross preprocessing, meaning you can change the value of one field based upon the value of another.

Allow empty collections

When a fields.List is being parsed, it returns 'missing' if list is empty. Why not return empty list instead?

Default error handlers should return 422 status code to client?

Currently, error handlers return a 400 response. It may be more appropriate to return a 422, which is typically used for ValidationErrors.

From RFC 4918:

The 422 (Unprocessable Entity) status code means the server
understands the content type of the request entity (hence a
415(Unsupported Media Type) status code is inappropriate), and the
syntax of the request entity is correct (thus a 400 (Bad Request)
status code is inappropriate) but was unable to process the contained
instructions. For example, this error condition may occur if an XML
request body contains well-formed (i.e., syntactically correct), but
semantically erroneous, XML instructions.

url parameters

Hi,
I am new to Flask and I am very very new to webargs, I was wondering why you guys decided to treat segment parameters differently than other parameters (form, query, heads), view_args is part of request object and why not provide a view as location collection to parse the view_args and than before passing it back to the end point you can make sure it doest not exist kwargs.

Cheers

Add "bundle_errors" options to send multiple errors for one parse

Hi, I just started using webargs with the purpose to fully replace reqparse from Flask-RESTful in my application.
So far I am very happy with webargs nested arguments capabilities (which I could not properly work out with reqparse).

However, with reqparse you have a bundle_errors option, which when set to true, tries to validate all arguments and does not stop after the first error, but rather aggregates all errors and throws an exception which message field is a dictionary with all errors found per field.

This option is very useful to avoid many roundtrips between client and server when you have a complex form or web service.

I could not find an equivalent feature in webargs, does such a feature exist? If not, it would be very nice to have and that would put webargs to par (and even better) with reqparse.

Remove implicit error handling

mentioned in #21 : Currently, Parser will catch all generic Exceptions and pass them to the parser's error handler function.

This is not ideal. Non-webargs errors should be handled outside of webargs so that unexpected errors result in a proper 500 response rather than implicitly rendering them as 400s.

Pre-processing should occur before type conversion

Currently, the Arg#use callable is called after type conversion. I think it would be more useful if called prior to type conversion.

Example: self.type(self.use(...)) instead of self.use(self.type(...))

Bulk-type arguments support

I'm not sure whether this case can be nicely handled with webargs, but anyway let me know if there is already an elegant solution to the issue I have.

I'm implementing a PATCH method in my RESTful API following RFC 6902, so the arguments would look like this:

PATCH /users/123

[
    { "op": "replace", "path": "/email", "value": "[email protected]" }
]

I have naively tried to pass many=True to my marshmallow schema, but it didn't work. Reading the code I couldn't find any hint that this might be a supported case.

Just for the sake of completeness, here is what was my naive attempt:

@use_args(MySchema(many=True))
def get(self):
    ...

I use Flask-RESTplus if this matters at all.

default not working with nested args

Nested args aren't respecting the default keyword.

For example:

@use_args({
    "param1": Arg(str, required=True, target="json"),
    "param2": Arg(str, required=False, default='bar', target="json"),
})

Behaves as expected. When making a request with the payload{"param1": "foo"}, param2 is correctly added with the value "bar".

On the other hand, with this:

@use_args({
    "param": Arg({
        "sub_param1": Arg(str, required=True, target="json"),
        "sub_param2": Arg(str, required=False, default='bar', target="json"),
    }),
})

making a request {"param": {"sub_param1": "foo"} } does not add sub_param2.

Without looking to the code too much, my guess would be this has the same root cause as #39 and #40.

Embedded document (Dict) validation

hi Sloria,

Nice framework! Is it possible to validate embedded document (dict) within JSON?

dateadd_args = {
    'value': Arg(default=dt.datetime.utcnow, use=string_to_datetime),
    'data': {
             'param1': Arg(str, validate=validate_unit),
             'param2': Arg(str, validate=validate_unit),
             'parame3': Arg(str, validate=validate_unit)
    }
    'addend': Arg(int, required=True, validate=lambda val: val >= 0),
    'unit': Arg(str, validate=validate_unit)
}

is it possible?

thanks,
Dmitry

Source and target parameters are confusing

I was expecting the target argument to specify the location in the parsed set where the argument value would be stored (similar to stdlib argparse's or flask-restful's dest kwarg).
I also thought that source was the location in the request to look for the argument (see source arg from flask-restful).

Due to the pending deprecation of the reqparse module of Flask-Restful (flask-restful/flask-restful#335), I think swapping these two arguments would help migration (I've been referring people to this package to use instead of reqparse) and general ease of use.

Validating files in Flask is broken

Example code:

@app.route('/', methods=['post'])
@use_args({'file': Arg(FileStorage, required=True, target='files')})
def index(args):
    print(args.get('file'))          # <FileStorage: 'file' (None)>
    print(request.files.get('file')) # <FileStorage: 'readme.md' ('application/octet-stream')>

    return '', 200

Python 3.4
webargs==0.10.0

The problem is in core.py line 216 where we do conversion to type:
ret = self.type(ret)
One way to fix this is by making the conversion optional?

Marshmallow Schema construction should be handled by Parser

Hi @sloria. Loving the new marshmallow integration! Thanks so much.

I had a suggested tweak to the API I was hoping to run past you. Presently marshmallow schemas are used like:

# example A
@use_args(UserSchema())
def my_pryamid_view(request, args):
    pass

I suggest it makes sense to push the construction of the schema instance into the core.Parser becoming:

# example B
@use_args(UserSchema)
def my_pyramid_view(request, args):
    pass

This has a number of benefits:

  • The current approach requires API users to explicitly set strict=True either in the schema constructor or the schema Meta class. This could be ensured internally if core.Parser handles construction.
  • Since webargs presently requires schema instances be constructed at module scope and usage of the marshmallow context will persist between requests. Re-constructing the schema each requests ensures no state can persist.
  • Presently core.Parser is already responsible for schema construction with dictionary based webargs usage. The proposed change would bring the two usages more into line.

Of course example A over simplifies the construction of the marshmallow schemas since there are a number of arguments that could be supplied with marshmallow construction. What we probably would require is either:

# example C
@use_args(UserSchema, only=('name', 'email'))
def my_pyramid_view(request, args):
    pass

or

# example D
@use_args(lambda: UserSchema(only=('name', 'email'))
def my_pyramid_view(request, args):
    pass

or

# example E
@use_args(UserSchema, schema_args={'only': ('name', 'email')})
def my_pyramid_view(request, args):
    pass

I dislike C because it utilizes kargs which I think have limitations should the use_args decorator, for example, require additional arguments be passed to core.Parser in later versions of the API.

I favour E over D as it allows us to address the issue of setting strict=True during schema construction.

This would also be compatible with webargs's dictionary based usage. So with dictionaries E becomes:

# example F
user_dict = {
    'name': field.String(),
    'email': field.Email(),
    'password': field.String(),
}

@use_args(user_dict, schema_args={'only': ('name', 'email')})
def my_pyramid_view(request, args):
    pass

I'm happy to put a PR together should there be interest in this change.

Falcon support?

It seems that webargs could fit in nicely with Falcon's hooks:

class HelloResource:

    @falcon.before(use_args({'name': fields.Str()}))
    def on_get(self, req, resp):
        return 'Hello {}'.format(req.args['name'])

class HelloResource2:

    @use_kwargs({'name': fields.Str()})
    def on_get(self, req, resp, name):
        return 'Hello {}'.format(name)

Edit: Modify expected API.

Feature request: Arg(allow_none)

This feature would allow one to parse None values when they are explicitly sent in the request.

Currently, with allow_missing=True, sending a null value (e.g. over JSON) results in an empty parse.

Setting allow_missing=False, allows a null value through but does so even if the value wasn't actually sent in the request.

Something like allow_none=True would set None for the parameter but ONLY if it was included in the request.

My particular use case is in an API that allows partial updates to a database model. I need a way to send in None but only have it parsed if it's actually sent, e.g., request.data={'param': None} => parsed={'params': None} and request.data={} => parsed={}.

allow_missing=True not working with nested args on passing null

I'm trying to follow jsonapi's spec on updating relationships, so I made the following args

args = {
    'data': Arg({
        'type': Arg(str, required=True, target='json'),
        'id': Arg(int, required=True, target='json')
    }, required=False, allow_missing=True)
}

@private.route('/departments/<int:dept_id>/relationships/companies', methods=['PATCH'])
@use_args(args, locations=('json',))
def function(args, dept_id):
    ...

here I'm expecting that when I pass an object in data, it will try to validate it if it has the necessary fields (type and id) and when I pass null it will continue on and give me dict object of {data: None}, but only the former is working, if I try to pass null it gives me an 422 UNPROCESSABLE ENTITY error saying "Expected type 'object' for data, got 'null'"

as mentioned in #9 , with allow_missing=True values will only be excluded if they are not found on the request; otherwise if given value of null then it will be included in the parsed arguments.

Support delimited list parameters

Currently, webargs supports list parameters in query strings using repeated parameters: foo=bar&foo=baz. But it's also a common pattern to encode lists in query strings using delimited values: foo=bar,baz, foo=bar|baz, etc. This is described using the collectionFormat in swagger: https://github.com/swagger-api/swagger-spec/blob/master/versions/2.0.md#fixed-fields-8. It would be helpful to expose control over the formatting of list values to support both repeated and delimited parameters.

Here are a few thoughts about interfaces:

  • Use a subclass of List, like DelimitedList, that splits and joins on a delimiter on de/serializing.
  • Add an optional delimiter argument to List that has the same behavior as above.
  • Add an option to Schema.Meta that controls de/serialization of lists:
class ArgSchema(Schema):
    class Meta:
        list_delimiter = ','

Not sure if I like any of these. What do you think @sloria?

Keep track of decorated function names to assist with automatic API documentation

This is a feature request for webargs that would make it easier to automatically generate API documentation in a web app. The specific use case in mind is for Flask, but this could probably help with other types of apps.

The only code change necessary in webargs is to keep track of the wrapped function names and associate it with the arguments attached to that function, and have a method to extract that mapping so it can be used by the app.

For example:

from flask import Flask
import webargs
from webargs.flaskparser import use_kwargs

app = Flask(__name__)

@app.route("/signup")
@use_kwargs({
    "name": Arg(str),
    "password": Arg(str),
    "accept_agreement": Arg(bool),
})
def signup(name, password, accept_agreement):
    pass

@app.route("/api_doc")
def make_api_doc():
    # we should be able to extract the route and arg mapping from webargs here
    webargs_map = webargs.arg_map()

The argument map would be a data structure that contains attributes like the following:

{
    "endpoint": "signup",  # the python function name wrapped by @use_kwargs
    "kwargs": {
        "name": Arg(str),
        "password": Arg(str),
        "accept_agreement": Arg(bool),
    }
}

Using this data, a Flask app can correlate it with current_app.url_map (where Flask keeps a mapping between URL routes and function names) to generate automatic API documentation. For example, an endpoint could generate HTML docs listing out the Flask URL routes (like /signup) and all of the parameters it accepts, including their names and types.

This can probably be implemented in webargs.core.Parser in the use_args decorator's implementation; keep a mapping between the wrapped func.__name__ and its argmap and store it in a global variable and provide a method to get the current contents of that variable.

Use marshmallow under the hood

Proposal

Replace Arg objects with marshmallow fields. Use marshmallow for validation.

Why?

Webargs and marshmallow have redundant functionality for validating input data against a user-defined schema. Marshmallow does a better job of than webargs in a number of respects:

  • Nested validation is much more powerful in marshmallow. marshmallow's Nested field would solve #39 and #51.
  • Marshmallow does error bundling. Solves #58
  • Marshmallow handles null and missing inputs in a more predictable way.
  • Marshmallow has more consistent error messaging.
  • Marshmallow's validation is more stable and well-tested

Proposed API

from webargs import use_kwargs, fields


@use_kwargs({
    'title': fields.Str(required=True, location='json'),
    'description': fields.Str(required=False),
})
def myview(title, description):
    return 'title: {}, description: {}'.format(title, description)


# with nesting
@use_kwargs({
    'data': fields.Nested({
        'title': fields.Str(),
        'description': fields.Str(),
    })
})
def myview(data):
    return 'title: {}, description: {}'.format(**data)


# respects load_from
@use_kwargs({
    'x_custom_header': fields.Str(load_from='X-Custom-Header')
})
def myview(x_custom_header):
    return 'Header: {}'.format(x_custom_header)


# Passing in a Schema
from marshmallow import Schema

class PostSchema(Schema):
    title = fields.Str(required=True, location='json')
    description = fields.Str(required=False)
    id = fields.Int(dump_only=True)  # won't be included in parsed output

@use_kwargs(PostSchema)
def myview(title, description):
    return 'title: {}, description: {}'.format(title, description)

Doesn't respect defaults

As of 0.16, webargs doesn't respect default values for missing fields:

@use_kwargs({'foo': fields.Str(default='bar')}
def view(foo):
    ...

If foo isn't found in the request, it should default to "bar", but it actually defaults to marshmallow.missing. It looks like we used to check for missing in Parser.parse_arg and return the default value if available, so we could restore that behavior to fix. Does that seem like the right approach @sloria? I can give this a try, or use another approach. Or you can fix it for me 🎸

Tested with Flask.

Add a cache for storing processed request bodies

A short-lived cache could be used to store the results of processing request bodies to prevent repeated work while parsing multiple arguments.

One immediate use for this is to store parsed JSON body in the TornadoParser. Currently, the JSON-decoded body is stored on the json attribute of the parser. It would be better to have a more generalized means of storing temporary data.

h/t @jmcarp for the suggestion

Null value handling

Null values in webargs are currently not handled very well.

  • None must not be passed to the type function.
    Python's builtin type functions don't handle None nicely (e.g. str(None) == 'None')
  • There needs to be a way to reject null values where they are not valid.
  • required=True currently accepts null values, which can be a bit surprising.

I think the simplest solution is to skip type conversion for None values and rely on
the existing validation feature to implement a not-null check. e.g.

Arg(str, required=True, validate=lambda x: x is not None)

The documentation for the 'required' parameter should make it clear that None values are not
considered missing.

Thoughts?

allow_missing=False not working with nested args

Nested args currently do not respect the allow_missing flag.

Example:
Argmap: {"nested": Arg({"a": Arg(str), "b": Arg(str)})}
JSON request body: {}
Should return {"nested": {"a": null, "b": null}}
Instead returns {}

I was looking at the current nested args parsing logic, it seems to bypass most of the logic of dealing w/ allow_missing in Parser#parse.

Perhaps nested args can be implemented simply by calling Parser#parse with the nested argmap. Any thoughts?

Validate list values with `multiple=True`

For Arg objects with multiple=True, the validators are applied to each item in the list, but there doesn't seem to be a way to validate the overall list. Use cases: checking for duplicates, checking length of list is greater or less than some count.

Should webargs support this, or does this logic fit better elsewhere?

JSONDecodeError when using default locations and required=True argument

When using default setup with aiohttp and default locations, the server will throw exception "json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)" if the client doesn't send the required parameter and there is no json data in the http request.

I am not sure if that's a bug or expected behavior, but that's absolutely counter-intuitive, especially if you just started to use the library.

Error handling when using webargs together with Flask-RESTful

I'm having difficulties understanding how to deal with default webargs errors while using webargs together with flask-restful. I'm always getting this error when the arg validation fails:
TypeError: ValidationError(u"Required parameter 'imdb_data' not found.", status_code=400) is not JSON serializable
It's understandable that flask-restful expects a dictionary so it could serialize if to JSON. What I don't understand is how do I actually provide that dictionary.
I guess the solution proposed in #14 does not really fit here because it does not use flask-restful's error handling but flask's native. I would really appreciate an example of error handling for flask-restful.
This is how I use webargs:

imdb_data_json = {'imdb_data': Arg(dict, required=True)}

class MovieIMDBDataAPI(Resource):
    def __init__(self):
        super(MovieIMDBDataAPI, self).__init__()

    @use_args(imdb_data_json, targets=('json',))
        def put(self, args, movie_id):
            print args

If imdb_data validation fails I get the above exception.

support for webapp2 framework?

Hi, just wondering if you planned on supporting the webapp2 framework which is using standard webob requests objects. This would make this library usable on appengine with more frameworks and would be awesome.
Thanks!

`error` message for json payload not working

Hi all,

The error message on Arg() is not printing the error message if validation fails, but instead is printing only the default 400 message.

from flask import Flask
from flask import request
from webargs import Arg
from webargs.flaskparser import FlaskParser

parser = FlaskParser()

app = Flask(__name__)

tracks_args = {
    'event': Arg(str, target='querystring'),
    'origin': Arg(str, validate=lambda w: w in ['suggest'], target='json', error='Invalid choice for     origin')
}

@app.route('/tracks/<string:event>', methods=['post'])
def post(event):
    args = parser.parse(tracks_args, request)
    print args
    print event

    return "OK"


if __name__ == '__main__':
    app.run(debug=True, port=8001)

with a correct request like

{
    "suggest":"test"
}

and an incorrect one like
{
"suggestsadf":"test"
}
that should print the error message.
Thanks!

Nested args aren't filtered when extra parameters provided

When defining nested args, extra parameters aren't removed from the resulting parsed object.

For example, given the following args:

nested_args = {
    'level1': Arg({
        'level2': Arg()
    })
}

If the incoming parameters are something like:

incoming = {
    'level1': {
        'level2': 'ok',
        'extra': 'not ok'
    }
}

then the resulting parse is incoming, but I would expect it to be:

expected = {
    'level1': {
        'level2': 'ok'
    }
}

Replacing `Arg(dict)` with marshmallow?

In webargs < 0.16, I have arguments defined as Arg(dict, ...). How do I convert these to marshmallow fields? The changelog shows examples with strings and nested arguments, but I couldn't find an example with dict.

Compatibility of `multiple=True` with nested args

Correct me if I'm wrong, but it doesn't seem like nested args supports lists of dicts.

For example, using this:

args = {
    'collection': Arg({'foo': Arg(), 'bar': Arg()}, multiple=True)
}

to parse incoming data like this:

incoming = [
    {'foo': 1, 'bar': 2},
    {'foo': 3, 'bar': 4},
    {'foo': 5, 'bar': 6}
]

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.