Improve custom allocator support about lodepng HOT 11 OPEN

Hi,

I'll do it in an update soon. I've indeed seen similar allocators with void argument used elsewhere.

Out of curiosity, what do you need it for?

Thanks

from lodepng.

It does require a lot of changes to pass the user object through all code paths.

If lodepng is called only once at the time, maybe the object can be made global instead so your custom function can get it from there? Would that work for now?

from lodepng.

@golightlyb what is that void *user_arg good for? For what do you use it? I'm just curious.

from lodepng.

Be careful not to add too much bulk to LodePNG. It's beauty is that it's interface is simple and compact. If we understand what the user_arg is for, it will help decide?

I'm using LODEPNG_NO_COMPILE_ALLOCATORS to link into my garbage collection logic that overrides new & delete in C++. But the problem for me is new & delete can't realloc, so I've had to add padding to manage realloc requests. Is there any chance to disable realloc requests?

from lodepng.

It'd probably be through the state object, not an extra function argument, if I do it. Not sure when though.

But actually, indeed maybe it's not needed at all? The required state for allocations can be stored in the memory you manage. Is that right?

from lodepng.

Hi Lode (nice to meet you).

I'm not sure I understand your question about state objects.

I override new and delete to manage memory. When new fails to allocate, I call a special method that deletes memory (in my case, cached resources such as textures) until the allocation is successful. This guarantees that new will never fail, so I don't need to check every time I make a new object.

So, I use LODEPNG_NO_COMPILE_ALLOCATORS to use my custom new & delete operators so that lodepng never fails to allocate. Works very well. But to make lodepng_realloc() work, I have to store a size_t for every lodepng_malloc(), so I know how much memory to memcpy() into the new_size. It works, but it's a little messy.

This is how I'm storing the size_t:

void * lodepng_malloc(size_t size) { char *n = new char[size + sizeof(size_t)]; *(size_t*)n = size; n += sizeof(size_t); return n; }

No big problem, but without the need for realloc, this would be a lot cleaner for anyone trying to do what I'm doing in C++.

from lodepng.

realloc is quite convenient :)

What is the reason for not being able to use it?

AFAIK realloc is often able to run in constant time because if memory behind it was free anyway it can immediately use it. Do your allocators take this into account?

Do you use any C++ features like std::strings or std::vectors that also need to allocate more memory sometimes? How do you use those?

Thanks!

from lodepng.

What if malloc() or realloc() fail because there is no free memory? I have no way to catch the error and free memory for it to try again.

I could, instead of using 'new', use malloc() and realloc() and call my garbage collection if they fail, then retry, but these allocations are outside of my control if I am not using new & delete operators, which have no way to realloc, which is why in lodepng_realloc() I use new char[new_size], memcpy(), delete old ptr;

I'm not using std::strings or std::vectors. Will the memory they dynamically allocate get caught in my overridden 'new' operator, so I can implement garbage collection on them if they fail due to out of memory?

from lodepng.

LodePNG will return an error number 83 in case any allocation fails, if that helps :)

LodePNG only allocates primitive types, so C's malloc and free should do.

from lodepng.

Actually, that's a good idea. If LodePNG returns 83, I can release some memory and retry. That's even simpler than before, lol.

Okay, forget about dropping realloc. All is ok :)

from lodepng.

Just a quick suggestion. Any plans to make constants or #defines for the error codes?

from lodepng.