Performance Test Suite about csg.js HOT 13 CLOSED

This is very very important to have : we have discussed the notion of 'different backends' (ie different csg libraries, or ways to do csg operations) with @Spiritdude a few times, and without hard numbers of the current system, it will be hard to compare the different implementations.

from csg.js.

@mtippett this seems right up your alley :)
Would you be willing to contribute something for this ?
I intend to do partial rewrites of the internals of CSG.js soon, but without performance tests it is too dangerous (could introduce really bad regressions etc)

from csg.js.

I have some basic test suites for Vector2D coming...

from csg.js.

Thoughts on first tests...

• test CSG basic shapes, testing all minimum and maximum options
• test CAG basic shapes, testing all minimum and maximum options
• test CSG union, intersection, subtract using converging simple shapes

Note: CAG uses CSG for union, subtract, etc.

from csg.js.

@kaosat-dev Can you add some tests for Line2D and Line3D?

I'll try to complete the functional tests of CSG in the next few weeks.

from csg.js.

@z3dev Sure , I'll have more time starting this weekend , I'll add that to my back log :)

from csg.js.

bump

I'm back to my next cycle of CSG focused consideration, and I'm as per usual hitting some issues with performance and/or memory consumption.

I'll probably invest some time getting a better handle on modelling my usage and creating test cases around those.

Unless otherwise indicated, I'll look to create some perf tests based on my interpretation of the current tests.

Since benchmarking is always a fun sport, I'm going to look to avoid straight up benchmarking, but look to scalability (both perf and memory). Most likely I'll target tests that provide roughly linear increases in complexity, and look to report the memory or performance. In general, I have seen performance tracking O(n^2) or worse.

I tend do most of my csg stuff in vagrant, so I'll use likely use a standard vagrant file to restrict memory. This will result in a "push it till it breaks" for memory complexity (unless there is an easy way in Node to get process memory). For perf, it's a bit harder - we'll just look to see how fast certain operations are, but rely on the developer doing a before/after comparison on matching hardware.

@z3dev - did you end up getting any baseline perf tests setup for Vector2D?

from csg.js.

@mtippett Welcome back!

I had a first attempt at performance test in waiting. Please see the latest changes on the new-test branch. The test suite is found in the perf directory, and initiated by 'npm run benchmark'

I'd like to create a set of performance tests for each of the major shapes and operations. Too many other things in the pot right now...

And as you mentioned, some tracking of performance is required to make sure that variances are immediately highlighted. I hope to save some basic statistics from each of the test suites, and compare those with each subsequent run. (Yet to be completed but heading that direction. )

Let me know if you have suggestions. I think both @kaosat-dev and I are generally inexperienced in performance testing.

from csg.js.

from csg.js.

That's cool. I was wondering if something like openbenchmarking.org already existed.

Sadly, I develop on a MacMini, so not sure that would be worth the time. Although it might be very close to some lap tops. If the time of day can be scheduled then I might consider.

So, what does a good performance testing machine look like today? Are there are there any specific profiles required?

from csg.js.

Glad to have you back @mtippett :)
In addition to what @z3dev dev said, there is also some hefty restructuring going on to the whole of csg.js here

Very interesting 'capping' of memory in Vagrant ! And Phoronix seems interesting !
What kind of machine perfs are needed to run it ? I have a few old (like really almost dead stuff) .
Are you seeing more memory or cpu consumption ?

This might not be the fanciest, but I found this article recently about perf testing and I used some of the advice in there, to do simple perf testing to compare our CSG.sphere() perf with a custom sphere using only vectors as arrays

• hint: turns out it is pretty much the same, slightly faster for array variant , median'ed across 200 iterations except the array variant I find simpler and more readable)

I am digging through a lot of CSG/ boolean op white-papers , and reading up on more optimised data structures like half edges or triangle centric structures, and it seems that pretty much any of them would be a better fit than what we have now , as far as perfs go.
I still think js centric (thus CPU) approach is the wrong one in the long term, GPU should be leveraged (signed distance fields, or 3d textures to represent depth etc)
Still I am really itching to do some experimentation with different implementations :)

from csg.js.

could also be interesting : https://github.com/mafintosh/nanobench

from csg.js.

There’s an independent performance test suite, located here...

https://github.com/jscad-community/csg-performance

This is community owned so anyone can add or change.

from csg.js.