Proposal: option to purge unknown fields on validation about cerberus HOT 15 CLOSED

i would add a del self.document[field] as alternate action instead of throwing an error if a purge_unknown=True was provided (mind the schema-validation, allow and purge must not be True at the same time).

be aware that you have to fetch the result from Validator.document after validation.

from cerberus.

sorry, my mistake, the validator-configuration doesn't relate to schema-validation. and on a second thought, i think it's okay to allow any combination:

• allow_unknown - throw an error or not
• purge_unknown - always silent

from cerberus.

del self.document[field] operates on a copy of the original document so will not work :(

from cerberus.

yep, that's why you have to fetch the document-property from the used Validator-instance afterwards. Validator.validate itself only returns a boolean.

the copying of the document has been introduced just recently to provide a non-destructable mechanism for 'value-coerceing'.

from cerberus.

Exactly, the only obvious way I saw how to get to the document was to wrap Validator.validate, run the validation and then get the validated/coerced etc. document - afterwards as you say. Then loop through the unknown field errors together with the document and remove fields where necessary, including updating the errors dict.

from cerberus.

i don't get it. can you post some snippet of the code that uses the validator?

from cerberus.

Sure, you can see what I've done here https://gist.github.com/misja/d9b88ea849b9ab86af95

from cerberus.

yes, i saw that. i mean the code that actually calls a document-validation. what would hinder you to access the document-property in that context?

from cerberus.

The validated method doesn't really solve the issue. I've created a pull request which actually keeps a reference to and not a copy of the current document being validated. This then makes it possible to manipulate document fields, like removing unknown fields and renaming fields (both included in the pull request).

from cerberus.

why would the v.document not be helpful? the copying of the document was introduced in the context of document-manipulation. i'd like to understand why this is unsatifactory in a similar use-case.

the _validate-method seems more and more hackish to me where everyone puts his hack in order to get his use-case working, but it doesn't seem that there's a concept how it should operate. with your pr included, there would be 14 lines of code just to figure out what is to be validated. this seems unnecessary complicated and makes extending the code harder and harder in a straight-foward manner.

please also extend the documentation according to you changes. this also makes code-reviews much easier.

from cerberus.

I think my changes are very lightweight and don't change the essence much? The main issue is that _validate iterates over a document recursively, and the only available context being the root document during each iteration. So a validating method will have access to the root document v.document, but only receives a field and value, so how would it be able to tell which subdocument currently is being validated? My pull request keeps that subdocument reference to the root document via self.current so that one can make changes in-place instead of having to traverse v.document in search for the field.

I'll add some documention for the remove_unknown flag and rename validator.

from cerberus.

thanks for the elaboration. then i'd say that the current implementation of _validate is not suitable at all. (see also #111 on that.)

from cerberus.

Just did a test with one of the problems mentioned in #111, think that by passing self.current to _validate_required_fields fixes the issue, just updated the pull request.

from cerberus.

@misja do you still plan on submitting a PR for this feature?

from cerberus.

@nicolaiarocci no, will be holding it off for the moment

from cerberus.