Comments (4)
Hello @EAirPeter,
Your analysis is correct. The compressed tracks and database data is a binary blob that can easily be relocated with memcpy.
For the decompression context, it may be easier to re-create it on demand on the stack whenever you need to decompress. That is what the Unreal Engine plugin does and what most engines do. Re-using the context will only save you its construction/initialization cost which is very cheap. If you wish to re-use it after relocation you could simply call initialize
again with your new compressed tracks/database pointers/references. You'll need to seek again as well before decompression but you probably do that right now anyway. The decompression context itself can also be relocated with memcpy if need be.
For the database context, things are a bit different. You can't initialize twice, but you can call reset
before you initialize it again. The database context allocates memory internally and as such we need to free it. You'll likely want to reset it before the memory is reclaimed in your scheme to prevent use after free. How you handle the database streamed data is up to you within the streamer object. You could make the stream in/out option a no-op if the data is already present or you could opt to truly stream in/out altogether.
Getting rid of some of these pointers is something I'd like to do. Storing offsets is more compact and just as fast on x64 (single instruction). That's doable for the decompression context with the exception of the pointers to the compressed tracks/database. Those must remain as proper pointers as it is illegal in C++ to compute an offset from two pointers not part of the same aggregate structure/blob. It may or may not work depending on the platform. On x64, memory is unified and as such it might work but that is not always the case.
For the database context, we could similarly store offsets for most things but two pointers will remain: the compressed database and the base of the dynamic allocation for metadata tracking. Possibly, we could add a function to allocate both the database context AND the metadata buffer together as one blob. This would allow us to remove the pointer to the metadata since we could use this
and an offset. However, this would force everyone to use that API which might not be desirable. I think this might be something we can handle through the database_settings
at compile time. The context could store a union of a pointer and offset along with a function that returns the pointer either computed from this
with the offset or returns the pointer. This would be a good middle ground I think.
For the database context, there is also the question of how we migrate the allocated metadata. If we had everything in one blob, then relocation would work well. We would simply need to fixup the compressed database pointer. But what about the information regarding what is streamed in/out? If you don't stream things out through the API to update the metadata before your relocation, the context will think things are still streamed in after relocation and the onus will be on you to make sure that is the case.
To summarize, I would like to propose the following changes to better address your use case:
- Introduce a
reset
function on thedecompression_context
to mirror the behavior in thedatabase_context
even if it isn't strictly required. This will make the API symmetrical and consistent. - Introduce a
relocate
function on both context objects to fixup the required pointers. - Convert as many things as possible into offsets to reduce the number of pointer fixup required.
- Expose a new API to allocate the
database_context
together with its metadata so that both can be relocated together in one blob and so that we may use offsets there as well.
How does that sound?
from acl.
After digging more into code, I found that those contexts actually store only the pointers to the tracks, database and bulk data themselves, not to any members of them. It turns out that I can achieve what I want simply by updaing the tracks/db
pointer and make sure this does not happen between seek
and decompress_track|tracks
.
from acl.
Sounds great. Currently, I achieve my goal through a similar relocate
function and it works smoothly. Thank you so much, looking forward to new versions.
from acl.
I merged a fix for this along the lines of what we discussed.
A reset
function has been added.
I renamed is_dirty
to is_bound_to
to clarify usage and meaning.
I added a relocated
function (not relocate
to clarify that it does not relocate anything) that does the pointer fixup.
After review, no further pointers were fixed to offsets. In the decompression context, aside from the compressed tracks and database pointers, the others are fixed during the call to seek
. I simply ensured that seek
must be called following relocation which seems entirely reasonable to me.
I have not added any functionality to relocate the context objects themselves. If this is needed, I'll tackle it later.
from acl.
Related Issues (20)
- Add support for per sub-track sample rounding mode
- Add support for mesh space animations
- Implement looping playback wrap auto-detection
- Deprecate redundant acl::sjson_raw_clip::bind_pose
- Segment balancing is not optimal when num samples is a multiple of the ideal num samples per segment
- Investigate what is the optimal segment length
- Add key frame importance track for database building
- Add track description flag to control whether missing values can be reconstructed through interpolation
- Add scalar track support in database streaming HOT 2
- header->num_tracks is always 0 HOT 1
- Investigate different segment range packing
- Implement individual clip streaming support
- Treat values as fixed point when converting to bit rates during quantization
- Implement a velocity/acceleration based error metric
- Use object space shell distance to detect constant and default sub-tracks instead of threshold HOT 1
- Investigate if we still need the second pass during the bit rate optimization HOT 5
- Improve keyframe stripping
- require bool curve support HOT 1
- Request for Mirror Animation Feature in ACL HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from acl.