2 GB+ json parser support about cgltf HOT 5 CLOSED

Hi!

I definitely think that's a justified change. It sounds like you need to change jsmn as well? Do you think it would make sense to make a PR for that library first, integrate the results back into cgltf and then modify cgltf?

from cgltf.

There isn't a lot traction in jsmn. Last commit end of 2021. Plenty open MRs. So not sure if worth doing. But I can certainly do another fork at least

from cgltf.

FWIW a first step could be to upgrade byte indices to ptrdiff_t/size_t - this is likely to mostly impact jsmn and not a whole lot else. int i inputs to parsing functions refer to token locations in the stream - I don't know what the "normal" ratio is between bytes and tokens but I'd guess that most 2+GB JSON GLTFs have a much smaller number of tokens. A subsequent conversion could allow for >2B tokens as well but that could be done separately to reduce churn.

from cgltf.

That was indeed the minimal work I did on my end to get it working. Just thought it may be a bit unclean to commit to main. But would be less churn as you said, so fine with me.

from cgltf.

Alright, I'm also fine getting this change in without involving jsmn. Feel free to open a PR then.

from cgltf.