Hi,
Nice project, thanks, i've used it for my evaluations.
I've noticed the huge gap between 2 libraries to all other libraries
- ts-quartet
- ts-json-validator
This huge gap is probably because of the way the project is running the tests.
The 2 libraries above use a different strategy than all others to create the validators.
While others mostly use predefined, hard-coded validator functions and through composition of them create a schema, the fastest 2 libraries will compile JS code at runtime (eval()
or new Function(...)
) to create discrete validation functions that do not call other functions internally (no composition) but instead have all the required validation code within the same function created specifically for the schema.
For example, Quartet:
For the following schema:
const checkData = v<Data>({
number: v.safeInteger,
negNumber: v.negative,
maxNumber: v.positive,
string: v.string,
longString: v.string,
boolean: v.boolean,
deeplyNested: {
foo: v.string,
num: v.number,
bool: v.boolean,
},
});
It will generate the following validator function:
function validator(value) {
if (value == null) return false
if (!Number.isSafeInteger(value.number)) return false
if (value.negNumber >= 0) return false
if (value.maxNumber <= 0) return false
if (typeof value.string !== 'string') return false
if (typeof value.longString !== 'string') return false
if (typeof value.boolean !== 'boolean') return false
if (value.deeplyNested == null) return false
if (typeof value.deeplyNested.foo !== 'string') return false
if (typeof value.deeplyNested.num !== 'number') return false
if (typeof value.deeplyNested.bool !== 'boolean') return false
return true
}
This has a deep impact on performance depending on how you run your code.
The benchmark code in this project will use 1 schema and iterate over it for a certain period of time. This is perfect for quartet
because of how V8 works.
The function becomes super hot, it quickly becomes inlined and additionally, if any internal function call exists within the validator it will get inlined as well!
In other libraries, this can not happen because so many functions are called, due to the composition, so most of them are cold and nothing get's inlined.
In real world scenarios, such a perfect order does not exists. For example, when handling incoming request, so many functions are called that by the time we reach the validator it is no longer hot!
And of course, we also need to factor in handling of multiple incoming requests.
The major advantage of the 2 libraries in question does not play along in real world scenarios and the results of the 2 are distorted in the benchmark.
I should also note the security risks of using runtime code evaluation. For a popular and heavily used library like ts-json-validator
(which is actuall ajv
) this is less of a concern. For a almost not used, not popular library like quartet
I will take caution.
I general sucj a huge gap does not make sense, otherwise everyone would have used these libraries entirely.
Thanks again!