Acceptable error threshold [Enhancement] about accelerated-data-lake HOT 1 CLOSED

Validation / data quality is a huge subject and there is no "one size fits all" approach. The accelerated data lake schema checking can be as strict or as forgiving as the user requires, but if the user's schema validation fails then the object is moved to the failed bucket. This is the correct approach as even a small percentage of input data having an invalid schema can have adverse later on when querying the data.

To test for low quality data, the data can be staged with schema validation that is as strict as the user requires, then Amazon's Deequ ran against the data.

from accelerated-data-lake.