As a follow up to an issue raised here, given that the Form constructor in 2.x now supports the data attributes, do we really need wtforms-json anymore? At the very least, should there be a notice on the README stating that this library is only required for 1.x? Have I missed anything at all?

If you're using forms for handling patch requests, you have to use validators.Optional() at the end of your validation chain for all fields.

That said, the test that validators.Optional() uses to determine if a field is blank is different than the test that patch_data uses. Entering white space for a field will match validators.Optional(), so validations will be skipped, but the resulting patch_dict will contain the field still.

For example:

from wtforms import Form, fields, validators

import wtforms_json

wtforms_json.init()

class PatchUserForm(Form):
    email = fields.TextField(
        'email',
        [validators.Email('Invalid email address.'), validators.Optional()]
    )

form = PatchUserForm.from_json({'email': ' '})
assert form.validate()
assert 'email' not in form.patch_data # will fail here.

The argument against this is that people should be able to set fields to null or blank via patch, but in that case wtforms-json could check the field's optional flag, yea?

For my use case, I'm using a custom validator in place of validators.Optional that doesn't consider ' ' falsy.

So when you validate your form like this:

form = ComposeForm.from_json(request.json)

form.errors should be a json string/dictionary right? Because now it says, that it's an object.

This is more of an enhancement request than a bug.

I like using patch_data to handle PATCH requests with a subset of fields from the full form, but when fields are defined with a Required() validator, then the form won't validate. Making fields optional works well for patching, but then you lose the required-field validations if the form is also used to process POST requests for creating new objects.

It'd be nice if there were a flag to from_json() (or some other way) to indicate that it's ok if fields are missing, for patching purposes.

I've looked around the docs and Googled a bit, and I can't figure out how to accommodate this situation, other than creating separate forms for PATCH and POST handling, which is less than ideal.

Getting this exception when JSON has a key which is not in the form. Previously these extra keys would be skipped without causing an exception.

Please change WTForms requirement to 1.0.3
It's the new version, your extension works great with it

Hi !

Could you perform a release including #37 ?

Thanks by advance !

Hay

I have been wondering if there is a way - using the format below - to get kwargs from the user, but I have not been able to find a solution. Does wtform(-json) allows this structure?

{
    "kwargs": {
        "k1": "v1",
        "k2": "v2"
    }
}

or is my only option the more "ugly"?

kwargs = FieldList(FormField(key_value_form))

{
    "kwargs": [
        {
            "key": "k1",
            "value": "v1"
        },
        {
            "key": "k2",
            "value": "v2"
        }
    ]
}

Please advice :)

from wtforms import Form, TextField, validators
import wtforms_json

wtforms_json.init()

class NetworkForm(Form):
  address = TextField('Address', [validators.IPAddress()])
  gateway = TextField('Gateway')
  netmask = TextField('Netmask')

network = dict(address = 123, gateway = u'gw', netmask = u'nm')

form = NetworkForm.from_json(network)

print 'form.validate()=', form.validate()

Gives the following output :

form.validate()=
Traceback (most recent call last):
  File "test.py", line 15, in <module>
    print 'form.validate()=', form.validate()
  File "/usr/local/lib/python2.7/dist-packages/wtforms/form.py", line 265, in validate
    return super(Form, self).validate(extra)
  File "/usr/local/lib/python2.7/dist-packages/wtforms/form.py", line 130, in validate
    if not field.validate(self, extra):
  File "/usr/local/lib/python2.7/dist-packages/wtforms/fields/core.py", line 176, in validate
    stop_validation = self._run_validation_chain(form, chain)
  File "/usr/local/lib/python2.7/dist-packages/wtforms/fields/core.py", line 196, in _run_validation_chain
    validator(form, self)
  File "/usr/local/lib/python2.7/dist-packages/wtforms/validators.py", line 295, in __call__
    valid = (self.ipv4 and self.check_ipv4(value)) or (self.ipv6 and self.check_ipv6(value))
  File "/usr/local/lib/python2.7/dist-packages/wtforms/validators.py", line 303, in check_ipv4
    parts = value.split('.')
AttributeError: 'int' object has no attribute 'split'

This seems to be a bug in the IPAddress validator, but I think more generally that WTForms is not ready to accept non-string values yet.

Am I doing something wrong?

class QueryDict(dict):
    def getlist(self, key):
        val = self[key]
        if not isinstance(val, list):
            val = [val]
        return val

class FooForm(Form):
    items = SelectMultipleField(
        choices=(
            (1, 'a'),
            (2, 'b'),
            (3, 'c'),
        ),
        coerce=int
    )

Before patching:

data = {'items': ['1', '3']}
form = FooForm(QueryDict(data))
assert form.validate()
assert form.items.data == [1, 3]

Everything as usual, no AssertionError.

After patching:

data = {'items': [1, 3]}
form = FooForm.from_json(data)
assert form.validate()
assert form.items.data == [1, 3]

FooForm validates, but the second assert raises an AssertionError becuase form.items.data returns an empty list.

Could you add a LICENSE file and/or license headers to the .py files (depending on your license choice)?

It'd also be great to see wtforms-json up on PyPi.

Thanks for building this!

I'm getting the error "flatten_json() missing 1 required positional argument: 'json'" when using it with one json argument like in the docs:

flatten_json(json)

Edit: It looks like the documentation should be updated, since now the code reads:

def flatten_json(
    form,
    json,
    parent_key='',
    separator='-',
    skip_unknown_keys=True
):

after applying patch, if try to use usual form aka Form(request.POST):

File "/....../wtforms_json.py", line 109, in process
if formdata[self.name] is None:
TypeError: 'WebobInputWrapper' object has no attribute 'getitem'

I have some JSON that looks like this {'images': [{'name': 'field-1', 'value': 'http://example.com/foo'}, {'name': 'field-2', 'value': 'http://example.com/bar'}, ...]}.

What I'd like to know is what is the best way to validate each of these name-value pairs as though they were fields? In particular, I need them to be optional and valid URLs.

Patch_data ignores empty arrays. This becomes issue when trying to remove all elements from specific model, like in the example, remove all items from order. I think patch_data should be in this situation {"items": []}, but it's actually just an empty dictionary.

$ python example.py 
wtforms version: 2.0.1
wtforms_json version: 0.2.10
form.data {'items': [{'name': 'Foo'}, {'name': 'Bar'}]}
form.patch_data {}
Traceback (most recent call last):
  File "example.py", line 67, in <module>
    assert(form.patch_data == {"items": []})
AssertionError

Here is a working example to reproduce the issue:

# -*- coding: utf-8 -*-
from sqlalchemy import Column, ForeignKey, Integer, Unicode
from sqlalchemy.orm import relationship
from sqlalchemy.ext.declarative import declarative_base
import wtforms
from wtforms import Form
from wtforms.fields import (
    FormField,
    StringField,
    FieldList,
)
import wtforms_json
wtforms_json.init()

Base = declarative_base()

import pprint
print "wtforms version:", wtforms.__version__
print "wtforms_json version:", wtforms_json.__version__


#1. Setup SQLAlchemy Models

class Item(Base):
    __tablename__ = 'item'
    id = Column(Integer, primary_key=True)
    order_id = Column(Integer, ForeignKey("order.id"))
    order = relationship("Order", backref="items")
    name = Column(Unicode())


class Order(Base):
    __tablename__ = 'order'
    id = Column(Integer, primary_key=True)


#2. Forms definition

class ItemForm(Form):
    name = StringField()


class OrderForm(Form):
    items = FieldList(FormField(ItemForm))


#3. Create existing instance and fill the form

instance = Order(items=[
    Item(name="Foo"),
    Item(name="Bar"),
])

json_data = {
    "items": []
}

form = OrderForm.from_json(json_data, obj=instance)

print u"form.data", pprint.pformat(form.data)
print u"form.patch_data", pprint.pformat(form.patch_data)


#4. patch_data should contain an empty list

assert(form.patch_data == {"items": []})

Is there anything that can be done to resolve this issue?

The 2 methods are like the same usage: return [self[key]] or, did I missed something?

I added wtform-json to my project to handle incoming json validation with the same code we use for post data handling. I found your project after looking for ways to prevent duplication of validation code.

The issue I found is that your replacement for BooleanField.process_formdata = boolean_process_formdata breaks WTForms in a way that it no longer handles false values correctly ('false', '') should stay falsy.

I'm giving you a pull-request that changes the behavior in a way that preserves the WTForm behavior, and also adds False (bool). It can be done by appending False to WTFroms.BooleanField's false_values = ('false', '')

Thanks for your work and let me know when it gets merged.

The specific issue is that if wtforms_json is used, the monkey patch does not allow fallthrough/bubble-up of different datatypes.

For example, if using flask-admin and a StringField is overridden on the form as a FileUploadField, the formdata processor will normally parse the provided data, save the file, and store the filename to the StringField. The monkey patch breaks this process since it transforms all input types into their text representation via six:

    if isinstance(self, TextField):
        if self.data is None:
            self.data = u''
        else:
            self.data = six.text_type(self.data)

This is obviously incorrect. I'm not sure about the motivations for this code, so can't say what the correct solution would be. I actually suspect that this block of code should be removed altogether, since it's certainly possible that the back-end should store no value if that's what's received - rather than a blank string. Or it should be handled as the model expects, rather than as overridden text.

I'm happy to submit a pull request, but wanted to first understand what the intentions were.

The following example:

from wtforms import Form
from wtforms.fields import IntegerField, TextField, BooleanField
import wtforms_json
wtforms_json.init()

class TestForm(Form):
    
    login = TextField()
    password = TextField()
    enabled = BooleanField()

data = {
    "login": "zorg",
    "password": 5,
    "enabled": True,
}

form = TestForm.from_json(data)

print "Raw data =", data
print "Form data =", form.data

print form.validate()

gives me this output:

Raw data = {'login': 'zorg', 'password': 5, 'enabled': True}
Form data = {'login': u'zorg', 'password': u'5', 'enabled': True}
True

Since 'password' is provided as an integer value instead of a string, I would have expected that 'from_json' would fail instead of translating 5 to '5'.

Is it the expected behavior? Or I am missing something... ;-)

Thanks!

The classmethod from_json has default meta=None, which fails at https://github.com/lepture/flask-wtf/blob/master/flask_wtf/form.py#L85 as that assumes meta is a dict if present:

File "/usr/local/lib/python2.7/dist-packages/wtforms_json/init.py", line 233, in from_json
**kwargs
File "/usr/local/lib/python2.7/dist-packages/wtforms/form.py", line 212, in call
return type.call(cls, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/flask_wtf/form.py", line 85, in init
kwargs.setdefault('meta', {}).setdefault('csrf', csrf_enabled)
AttributeError: 'NoneType' object has no attribute 'setdefault'

I'm having issues validating a required IntegerField when the JSON value passed is 0:

import wtforms_json
wtforms_json.init()

from wtforms import Form, IntegerField
from wtforms.validators import InputRequired

class MyForm(Form):
    a = IntegerField(validators=[InputRequired()])

# Works with > 0
form = MyForm.from_json({'a': 5})
assert form.validate()

# Does not work with value 0
form = MyForm.from_json({'a': 0})
assert form.validate()

# Works with string '0'
form = MyForm.from_json({'a': '0'})
assert form.validate()

Using wtforms 2.0.2

In WTForms last version False is included in the false_values.

I'm guessing this is related to #10

and the changeset 255e589#diff-ce9bd1bee68d7afc2df0b83ca98cd5cf

The skip_unknown_keys arg is never passed to flatten_json.
https://github.com/kvesteri/wtforms-json/blob/master/wtforms_json/__init__.py#L201

I've inherited a project with a vendored version of wtforms-json and changing from_json to look like this worked for me:

@classmethod
def from_json(
    cls, formdata=None, obj=None, prefix='', data=None, meta=None, **kwargs
):
    skip_unknown_keys = kwargs.get('skip_unknown_keys', True)
    form = cls(
        formdata=MultiDict(flatten_json(cls, formdata, skip_unknown_keys=skip_unknown_keys)) if formdata else None,
        obj=obj,
        prefix=prefix,
        data=data,
        meta=meta,
        **kwargs
    )
    return form

I've found some unexpected (to me) behaviour in WTForms-JSON where an empty string in an optional string field with a minimum length did not result in a validation error nor exclusion of the field from patch_data. I've written a small test script, here is the output I'm getting from it:

WTForms-JSON version 0.3.3

json {}
errors {}
form.data {'name': None, 'potato': None}
form.patch_data {}

json {'name': ''}
errors {}
form.data {'name': None, 'potato': None}
form.patch_data {'name': None}

json {'name': 'a'}
errors {'name': ['Field must be between 2 and 128 characters long.']}
form.data {'name': 'a', 'potato': None}
form.patch_data {'name': 'a'}

json {'name': 'ab'}
errors {}
form.data {'name': 'ab', 'potato': None}
form.patch_data {'name': 'ab'}

json {'potato': None}
errors {}
form.data {'name': None, 'potato': None}
form.patch_data {'potato': None}

json {'potato': ''}
errors {}
form.data {'name': None, 'potato': None}
form.patch_data {'potato': None}

json {'potato': 'minimum'}
errors {'potato': ['Invalid value, must be one of: maximum.']}
form.data {'name': None, 'potato': 'minimum'}
form.patch_data {'potato': 'minimum'}

json {'potato': 'maximum'}
errors {}
form.data {'name': None, 'potato': 'maximum'}
form.patch_data {'potato': 'maximum'}

Length validation doesn't happen with the second test case, but it also isn't removed from patch_data like in the first. Potato intuitively should only ever be "maximum" or not present, but both an empty string and None are accepted as None.

>> class F(Form):
..    g = BooleanField()
..    h = BooleanField()

>> f = F.from_json({'g': None, 'h': 'ooo'})
>> f.validate()
False
>> f.errors
{'h': [u'This field is required.']}

The error message should be {'h': [u'Not a valid integer value']}

I'm occasionally getting a RecursionError on an application of mine (reported by Sentry).

The recursion happens on line 189 of the wtforms_json/__init__.py file:

if call_original_func:
    func(self, formdata, data=data)

My code that is triggering the error is quite simple:

@tips_blueprint.route('/', methods=['POST', 'PUT'])
@oauth_provider.require_oauth()
def post_data():
    """Post new data."""
    data = request.get_json()
    form = TipForm.from_json(data)

The data being submitted as json is very simple and a flat key/value pair - no nesting or anything like that:

{
  'comp_code': 'randomcopcode',
  'game_id': 5891,
  'team_id': 4
}

The issue seems to be in the additional validation that happens in the monkey_patch_field_process function. The form itself is quite simple but does do some database checks for validation. However there are no loops in the form validation which means it is highly unlikely that it's an issue with the validation.

Annoyingly, it's extremely hard to pin down the exact cause of the issue. I can submit the same data multiple times and it will only trigger a recursion error every second or third time which I realise is very unhelpful. I just thought I'd post the issue here in case others were having the same problem or there was a known solution.

I'm using:

• Python 3.5.2
• Flask 0.12.2
• WTForms 2.1
• Flask-WTF 0.14.2
• WTForms-JSON 0.3.3

WTForms supports FormFields inside a FieldList:
http://wtforms.readthedocs.org/en/latest/fields.html#wtforms.fields.FieldList

class IMForm(Form):
    protocol = SelectField(choices=[('aim', 'AIM'), ('msn', 'MSN')])
    username = StringField()

class ContactForm(Form):
    first_name  = StringField()
    last_name   = StringField()
    im_accounts = FieldList(FormField(IMForm))

WTForms-JSON should either

• monkey patch FieldList instead of just FormField

or

• check the unbound_field in def patch_data when f is an instance of FieldList, and call patch_data on each item instead of calling f.data

The "First Example" part of the documentation is a little peculiar.

First there's a typo in

from wtforms.field import BooleanField, TextField

it should be (note the s)

from wtforms.fields import BooleanField, TextField

Then instantiating MyForm obviously fails since it hasn't been defined. By the look of the rest of the example it seems possible to validate models of multiple levels. Maybe something like this?

import wtforms_json                                                                
from wtforms import Form                                                           
from wtforms.fields import BooleanField, TextField, FormField                      

wtforms_json.init()                                                                

class LocationForm(Form):                                                          
    name = TextField()                                                             
    address = TextField()                                                          


class EventForm(Form):                                                             
    name = TextField()                                                             
    is_public = BooleanField()                                                     
    location = FormField(LocationForm)                                             


json = {                                                                           
    'name': 'First Event',                                                         
    'location': {'name': 'some location'},                                         
}                                                                                  

form = EventForm.from_json(json)

Eg., I have one=BooleanField(default=True) in my Form. When I pass data with no key 'one' to the form, I always get form.data['one']=False. Isn't it strange, is it?

Model:

class Match(db.Model, BourneMixin):

    player_id = db.Column(db.BigInteger, db.ForeignKey(User.id))
    team_id = db.Column(db.BigInteger, db.ForeignKey(Team.id))
    team = db.relationship('Team')


def attributes(self):
    return [
            'player_id as player',
            'team_id as team',
        ]

Form:

class MatchForm(ModelForm):
    class Meta:
        model = Match
    team = QuerySelectField(query_factory=get_teams)
    player = QuerySelectField(query_factory=get_users)

request.json -> {"team":24,"player":43}

Match.from_json(request.json) -> {"team":None,"player": None}

After the commit 20c124d , if you try to populate a form using an object from the model, the returned form has all the values set as None or empty.

post = Post.query.get(13)
form = PostForm(obj=post)