Searching Performance about defiant.js HOT 6 CLOSED

Hi @charbel14,
Thanks for using DefiantJS...I've been thinking of improving the performance with two different approaches and I think I've decided the way to go. Roughly speaking, Defiant prepares a JSON structure for search and then performs the "actual search". The preparation is what takes the longest time and does not need to be re-prepared, especially if there is no change to the JSON data.

The addition I will add, will look something like the code below and I think it's the best approach. It is "backwards" compatible. That is, it doesn't affect the way DefiantJS has been used so far and the new version of the lib can be replaced with the old, without any risk for breaking anything.

// JSON data
var data = {... json-data...};

// create a re-usable snapshot
var snapshot = Defiant.getSnapshot(data);

JSON.search(snapshot, '//leaf');  // this search is much faster
JSON.search(data, '//leaf');      // ...than this one

//...especially if the snapshot is re-used several times
JSON.search(snapshot, '//leaf');  

It would be great to get feedback on this...what do you think?
Notice that this pattern gives the implementing developer the responsibility to use updated snapshots of the JSON structure.

As for your second question; there is no difference if you have full path expression. Your JSON data might contain a structure that matches your expression multiple times - hence the search goes through the full structure regardless.

Therefore, IMHO, the best expression if what makes it easiest to read, when you re-visit the code you wrote today, in the future. The best code is the one that supplies the fastest overview.

from defiant.js.

Yes that makes sense. i.e. to compile before hand.. and I did expect that this is what is slowing it down.

By the way the search(data, path,true ) did not seem to work for me. I had an array of 800 or so entries and I tested something right at the index 800 and something at index 1, I expected that with true the 1 should be faster and 800 would be slower.

But did not get any result back for either

Thanks again.

Charbel

from defiant.js.

@charbel14,
I have applied the snapshot feature and my tests shows an improvement of more than 100 times. Check out the "json-search-4.htm" in the demo folder, on this branch:
https://github.com/hbi99/defiant.js/tree/snapshot-feature

I tested searching on a ~1.6MB large JSON file (also in demo folder) on Firefox.

Search without snapshot took: 561 ms
Search with snapshot took: 4 ms

I think this is an acceptable improvment :)

PS: I also fixed the JSON.search(data, path, true) feature...that is, returning the first occurrence of the query. Though, with the snapshot improvement, the difference might be 3ms now on a big JSON file. On smaller files, it might be no difference at all (at best).

from defiant.js.

I can confirm the above figures. I ran my test data and it works.

The first occurrence feature does not seem to be working .. but as you mentioned the timing difference makes it redundant almost.

Thanks for the quick turnaround.

When will this be pushed to main.?

from defiant.js.

Now :)

Thanks for testing it on your end as well

from defiant.js.

I had forgotten to grunt to project before merging it...sorry.
It's now done and updated

from defiant.js.