Comments (5)
I added a quick invalid data test, and as expected see results below. myzod goes goes from being 2.7x faster than zod for safe parsing, but then drops to 2x faster than zod in the failure case. Again, I don't know if myzod is failing fast or not based on the test I wrote. Also, I did not actually call any function that do error reporting. I'm not familiar enough with myzod's behavior.
Running "parseSafe" suite...
myzod:
4 063 636 ops/s, ±0.39%
Running "parseSafeInvalidData" suite...
myzod:
88 576 ops/s, ±0.65%
Running "parseSafe" suite...
zod:
1 511 938 ops/s, ±0.69%
Running "parseSafeInvalidData" suite...
zod:
45 169 ops/s, ±2.45%
from typescript-runtime-type-benchmarks.
Sorry if I'm misinterpreting the code, but it appears that the benchmarks only measure validating successful validations.
That is correct!
If that's correct, then it should be noted that libraries like myzod will completely re-parse all of the data in order to collect errors.
Valid point!
How many libraries do you think behave this way?
Just as a general theme, we do try to stick to a "common denominator" across all validator libraries. It is understood that libraries will sometimes have different trade-offs in their designs to improve speed. But in the end, we do need to have some similarities across them to be able to bench.
Something that fails validation is certainly a very common scenario, I think, so probably worth adding it.
Ping @hoeck thoughts?
from typescript-runtime-type-benchmarks.
How many libraries do you think behave this way?
No idea, but adding this benchmark will certainly put a lens on libs that do.
The sketchy side of this is that by taking error-reporting shortcuts, it means that the validation of correct data is faster as a result, so it kind of manifests itself twice by also inflating the benchmark numbers for valid data.
I was just curious as to why myzod claimed such performance gains.
from typescript-runtime-type-benchmarks.
Also, there's a subtly in semantics of libraries - some libraries validate entire objects and do not abort when they encounter an error. They validate the entire object so the user can see all the errors in an object. I think there should be three different failure tests:
- Fail on any error
- Fail on all properties
- Fail on all properties AND call any method that gathers ALL the errors and reports them to the user. (For cases with 'or' clauses/unions only 1 error needs to be reported.)
In general, benchmarks can be dangerous, because library authors do not want to see their library at the bottom of benchmarks. Without covering the the full user experience, then it's tempting to only optimize the parts that are measured.
from typescript-runtime-type-benchmarks.
@akutruff Great findings, thanks for sharing. Certainly looks like something we shall add.
Would you be interested in contributing a PR?
from typescript-runtime-type-benchmarks.
Related Issues (20)
- Add `vality` HOT 2
- feat(package): `parse-dont-validate` HOT 2
- Failing build HOT 1
- How do I add a testcase (Request: Documentation on that) HOT 5
- Add `@Typia` HOT 3
- Suggest: add a table whether each type can be validated or not HOT 3
- Add `caketype`
- Add `@fp-ts/schema`
- Add `@gapstack/light-type` HOT 4
- Add `arktypeio/arktype` HOT 2
- Node 20 HOT 2
- Preview SVG image has been broken HOT 8
- Show operations/s as a number for each benchmark HOT 3
- Show weekly npm downloads for each package HOT 2
- Add `valibot`
- ParseSafe for Zod calls parse() not safeParse() HOT 1
- Categories for AOT, JIT and Dynamic Validation HOT 5
- I don't really quite understand the categories... HOT 2
- Add Effect-TS schema HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from typescript-runtime-type-benchmarks.