umm_info() may be called before heap is initialised, in which case umm_heap is NULL. Check is required.

Compiler build error! There was a comment somewhere in the commits that macros from dbglog.h are moved to umm_malloc.c. It is not the case. The folder c-helper-macros is emply in the original repo.

Building this library on Linux has a number of issues:

• Warnings in the DBGLOG macros due to %x expecting unsigned int but argument is void *
• Undefined reference to sqrtf() probably due to a missing -lm linker option under Linux

Fix these issues before moving on

size_t is not defined by default and depending on how you build a project that includes umm_malloc, you can run into issues when umm_malloc.h tries to use it before stddef.h gets included elsewhere.

Hello! I made a "banner print3r" robot according to your instructions from the Lego website. (https://www.lego.com/cdn/cs/set/assets/blt18fd6ecf0dc4d30f/BANNER_PRINT3R.pdf) Can you tell me where to get the PROGRAM for this printer? Thank you for the attention.

Hi,
Im developing a project that has a DDR2 memory that is used only to store a specific type of data, I would like to manage this data with your code but I couldn't find a way to have the block list save on the on chip memory (.bss) managing data pointers that is on a another specific memory region.

do you think that is possible ?

I somehow downloaded an older version dated 2014-04-02, but it's already deployed so not changing it.
However, I looked at the latest source and this issue doesn't seem to have been addressed.
umm_realloc() always hits the last case and calls malloc to move the chunk when the chunk to enlarge is at the end of the heap.
This happens because, in this case, umm_assimilate_up() does nothing.
I didn't put a lot of thought into it, but this seems to have fixed it for me. I made assimilate_up return true if it's main condition passes:

   if ( !umm_assimilate_up( c, umm_heap ) && // did assimilate up fail?
        !(UMM_NBLOCK(UMM_NBLOCK(c)) & UMM_BLOCKNO_MASK) // are we at the end of the heap?
      ) {
      // Assimilate up as much as needed or as much as possible.
      unsigned short int upBlocks = blocks;
      if ( UMM_NUMBLOCKS <= c+upBlocks+1 )
      	upBlocks = UMM_NUMBLOCKS-c-2; // not enough blocks available: grab as many as possible
      if ( upBlocks > 0 ) {
	      // Copied this logic from malloc.
	      UMM_NFREE(UMM_PFREE(c)) = c+upBlocks;
	      memcpy( &UMM_BLOCK(c+upBlocks), &UMM_BLOCK(UMM_NBLOCK(c)), sizeof(umm_block) );
	      UMM_NBLOCK(c) = c+upBlocks;
	      UMM_PBLOCK(c+upBlocks) = c;
	  }
   }

I'd appreciate any thoughts if someone has a better idea or sees a problem.
Thanks!

realloc calls both malloc and free. The code path for malloc is ok, but the code for free has complications. The call to free is within a critical section guarded by UMM_CRITICAL_ENTRY and UMM_CRITICAL_EXIT. The free function body has its own critical section, which means it is nested within the one in the realloc body.
For small(ish) systems, such as the ESP8266, the ENTRY and EXIT guards enable/disable interrupts. In such cases, the code after the EXIT guard in free() and the final EXIT guard in realloc won't be guarded as is expected:

realloc()
  //some code
  UMM_CRITICAL_ENTRY //disable interrupts
  //some critical code
  call free()
    //some code
    UMM_CRITICAL_ENTRY //disable interrupts again (does nothing)
    //some critical code
    UMM_CRITICAL_EXIT //enable interrupts (code point A)
    //more code
    return from free()
  //more critical code
  UMM_CRITICAL_EXIT //enable interrupts again (does nothing, code point B)
  //final code
  return from realloc()

The code between code points A and B is expected to be guarded because of the ENTRY and EXIT, but it is not.

It is possible to fix this within the ESP compiling environment with a lock class instance for ENTRY, because there umm is being compiled as .cpp. In that case, each class instance saves the previous interrupt state and restores it on destruction. However, that is not viable in other systems where umm is being built as .c.

I think it would be best to implement a fix here by removing the nested critical sections. The solution would be to refactor free() into two functions: an internal unguarded core function, and a wrapper that calls the core function between ENTRY and EXIT guards. Then, the free() call in realloc() would be replaced with the unguarded function.

Edit: the code path for malloc is also not ok. While implementing #16 I realized that malloc is also called from within the critical section, so I refactored that as well.

I have been working with a very old branch of umm_malloc that is in the Arduino ESP8266 project. I am in the process of adapting your current version into the project. I have a few thoughts I will share over time. For now, I want to focus on a change I noticed that might have been intentional or an oversight. A little background is needed. In the old version, I saw the comment:

"Bill Dittman provided excellent suggestions, including macros to support using these functions in critical sections, and for optimizing realloc() further by checking to see if the previous block was free and could be used for the new block size. This can help to reduce heap fragmentation significantly."

Of interest here is the "reduce heap fragmentation". While the top-level comment mentioned this, the code where it happens did not identify that it was doing it. The logic flow was to always try and assimilate up, then look to see if an assimilate down was possible. By always assimilating down when possible it resulted in locally deframenting the allocation. This process also applied to allocations that were being shrunk. This, of course, comes at the price of a memory copy.

After some searching, I found where the enhancement was removed: 13008fa (around line 531 of umm_malloc.c left side). The additional code cleanup changed the logic to one of reducing copy on realloc.

I can see where some would prefer to reduce memory copy and others would prefer the defragmentation. IMHO for the Arduino ESP8266 with very limited heap space, the local defragmentation when possible would be prefered.

The current umm_malloc test framework does not have any speed tests. The recent addition of inline vs one-time fragmentation metric is an example of the need for performance testing of the algorithms used in umm_malloc.
The challenge is that the test conditions will vary depending on the hardware used, and umm_malloc is used on a variety of platforms, from ARM Cortex M[0..7] to ESP8266 to Intel processors (for fast local heaps)
Initially I will write the test case for both 32 and 64 bit MinGW compilers running on my Dell 7150 Pro - a small (and older) 2 in 1 tablet.
In the long run I would like to be able to run the test suite on a variety of devices, so the test may eventually have an input for the platform the test is running on and platform specific pass/fail criteria.

As per ansi spec, realloc is supposed to return null and leave the original pointer in tact in the case of failure if size > 0 and the original pointer was not null.

The code in this repo violates this condition in the OOM case by calling free on the original pointer regardless of whether the call to malloc suceeds

hi.

I'm have problems on cc3220sf with umm_malloc.

umm_malloc.c have some code -

  /* init heap pointer and size, and memset it to 0 */
  umm_heap = (umm_block *)UMM_MALLOC_CFG__HEAP_ADDR;
  umm_numblocks = (UMM_MALLOC_CFG__HEAP_SIZE / sizeof(umm_block));
  memset(umm_heap, 0x00, UMM_MALLOC_CFG__HEAP_SIZE); 

If i'm printf (&umm_heap) - i'm get address from data section , eg - 0x20004939
if i'm printf (UMM_MALLOC_CFG__HEAP_ADDR) - i'm get address from heap, eg - 0x20005be3

From main program i'm make test -

char *test = umm_malloc(20);
printf(&test);
out = 0x20004d25

This is right?

And some time from logs i'm see that umm try free ptr with addr in data or bss sections, eg - free(0x20004de8) - and doo nothing becase this ptr null - and not allocating with umm_malloc.

Where problem? or i'm wrong?

Currently there is no good way to set up different configurations of the umm_malloc library for testing. By using the ceedling options: capability to support multiple option files, we can more easily test other cofigurations, such as BEST_FIT vs FIRST_FIT

Hi

Dunno if this is a bug or feature, but umm_malloc_cfg.h that is committed into your library conflicts with a file created by the user. This way users have to modify your umm_malloc_cfg.h file to change settings or remove it and provide their own somewhere else. And then if the users (like myself) are using your library as a git submodule, git reports modifications to the umm_memory directory which cannot be pushed unless they fork the umm_malloc. I think that only umm_malloc_cfg_example.h should exists in the repo. Please let me know if I'm missing something.

Currently umm_malloc has the Unity (TDD for Embedded) framework as a submodule. This is OK but we end up not being able to take advantage of some of the features like mocking, lightweight exception handling and a test oriented build system.

Remove the Unity submodule, and update this repo to have the full ceedling environment.

i use the Ozone to debug the hardfault , and i found the reason : above variant is too big which cause the arrary Cross-border access to a outoff memory space.
i use best fit algorithm, UMM_BLOCK_BODY_SIZE is (16) and i always alloc 34byte, what reason could cause these number too big ? please help me.

Make the on-the-fly metric calculation a compileable option, the metric is useful even without a fill-blown umm_info() implementation that can dump the heap structure.

You may find some interest in esp8266/Arduino#5090 .

This could be useful to test the efficiency of realloc().

This is a low priority task as I need to settle on a code formatting tool and then configure it to meet a reasonable code standard - probably somewhere between the Linux and LLVM standards :-)

In some cases, it is not clear if a function is really part of the source. For example the fragmentation metric can be calculated on-the-fly or on-demand. In the on-the-fly case, there are calls like this:

umm_fragmentation_metric_init();
umm_fragmentation_metric_add(c);

When on-the-fly calculation is disabled these are simply 3defined as empty stings and have no effect on the generated code.

Unfortunately they look like normal functions, so to give the reader a hint that they are in fact macros without resorting to inline #ifdef guards that make the code messy, we will mae all code that can be #defined out into upper case macros - like this:

#ifdef UMM_METRICS
#define UMM_FRAGMENTATION_METRIC_ADD(c) umm_fragmentation_metric_add(c)
#else
#define UMM_FRAGMENTATION_METRIC_ADD(c)
#endif // UMM_METRICS

According to the documentation, max heap size = block_size * uint15 max, so for 8-byte blocks (by default), it's:

8 * 32767 = 2621424 B

This limitation is not mentioned in the "configuration" section of the README. I would consider this information important enough to mention there, but even more important would be adding a runtime check to verify if the numbers align.

I would consider adding an assert inside umm_init_heap() to check if requested size of the heap is within limits.

@rhempel what do you think about it? I can submit a PR if you agree it makes sense.

Trying to understand how the initialization works. In https://github.com/rhempel/umm_malloc/blame/master/README.md#L214 you mention that

The current block pointer will be at the end of the free list, and we know we're at the end of the list because the nf index is 0, like this:

In the example just above, https://github.com/rhempel/umm_malloc/blame/master/README.md#L175 the nf index is 1 though:

   BEFORE                             AFTER

               nf   pf                            nf   pf
   +----+----+----+----+              +----+----+----+----+
0  |  0 |  0 |  0 |  0 |           0  |  1 |  0 |  1 |  0 |
   +----+----+----+----+              +----+----+----+----+
   +----+----+----+----+
1  |  0 |  0 |  0 |  0 |
   +----+----+----+----+

Is #175 potentially wrong or am i missing something?

I read the config file, and there i found that define. But I do not find anything in the src - so I do not understand how that should replace the normal malloc.

many thanks
cheers
mathias

Hi,

I'm banging my head against a brick wall trying to get the DBGLOG stuff to work ... everything is fine if I put some null defines in to replace it, but it's not part of the repo and doesn't seem to be a submodule???

Previous issues raised suggest it's a submodule and has been documented ... but I've tried a --recurse-submodules clone, and I've also tried a submodule init and submodule update ... but there's nothing! And there's no references anywhere I can find??

What am I missing?

This may be benign and only an issue during stress testing where the poison area around an allocated block is being tested. Still, it needs investigation and root causing.

I understand this won't generally be an issue for the intended use of this library, but it did cause me some trouble and think it's worth fixing. The reason it's an issue for me is that (for my application) large allocations must fail when requested of umm before the system moves on to another memory manager.

The issue is that when size_t is 32bit, umm_blocks() will overflow for larger block sizes:
uint16_t umm_blocks(size_t size)

So with my umm configuration, if I request 8MB from umm (which, obviously it can't and shouldn't serve), it will succeed and return a ptr to some unknown chunk size.

The simple solution for me was to make umm_blocks() return zero when size is out of range.

I replaced this line:
return 2 + size / (UMM_BLOCKSIZE);
With this:
size /= (UMM_BLOCKSIZE);
if (size > 0xFFFF-2) {
return 0;
}
return 2 + size;

Then test for zero as an error condition in alloc cores:
blocks = umm_blocks(size);
if (blocks == 0)
return (void *)NULL;

From @mattico

I made a few changes to umm_malloc to help with integration:

1. Runtime initialization. The standard Rust Embedded linker scripts don't put the heap at a fixed location, so I made umm_init take heap_addr and size. Not sure what a good general solution would be that preserves the ability to statically define the address/size.

2. Support extern functions for critical section entry/exit. Rust projects may have arbitrary locking mechanisms, and rather than require users to translate those into C to #define I use extern functions for critical section entry and exit. Being unable to inline those is not ideal, but I couldn't think of a better solution. I do intend to provide an assembly solution for cortex-m that just enables/disables interrupts which should work for 80% of cases.

Some additional ideas that could help:

3. Rust's GlobalAlloc API takes a size and alignment. I'm currently just asserting that the requested alignment is less than or equal to 8, the size of umm_block. In practice this works fine, requiring a larger alignment than that is pretty rare. Ideally umm_malloc would provide a memalign like API, though.

4. I do want to expose umm_malloc's diagnostics APIs to Rust users easily. I have to do more investigation to see what is required, though.

Hello rhempel,

Thanks for the umm_malloc library. Could you please add version numbers for it?

Thanks

Currently the stress testing mechanism is built into the poison block testing. This needs an awkward data structure to manage calling the correct version of the allocation functions. Even worse, the stress test results are slightly different for 8 byte blocks and higher block sizes due to the extra memory that must be allocated to handle the poison areas.
Ideally, we need to separate the stress testing from poison testing, and also update the poison data area size to correctly allocate extra memory depending on the block size to that the number of blocks allocated during poison testing remains the same regardless of the block size.

Hi
with the "#define UMM_MALLOC_CFG_HEAP_SIZE" I can set the heap size to a user-defined size. However, the heap size is limited to 256kB due to the maximum number of blocks. Is it possible to increase the heap to 1MB or more? Where do I have to make changes to achieve this?
I have read that it is possible to add more data bytes to the body of the memory block at the expense of free block size overhead.
Where can I add more data bytes to the body of the memory block?

Compiling the test suite under mingw or Cygwin produces different results during random allocations, even with the same seed number.

We believe this is due to the fact that each run-time library uses a slightly different random number generator algorithm.

To work around this problem, we will include in the test suite a random number generator that is platform independent.

https://en.wikipedia.org/wiki/Linear_congruential_generator

In the current source, created about 10 years ago, we used standard C types such as unsigned short int to force the size of struct elements and other values,

This is NOT portable to 64 bit architectures so we need to use explicitly sized values where possible:

• uint8_t
• uint16_t
• uint32_t

VS2017RC: there are 17 warnings for x86 and 25 warnings for x64

Hi guys,

I noticed a couple of things in the master branch:

• umm_info -> Compiler error -> line 165/171. Variable umm_metrics does not exist (I think it should be ummHeapInfo)
• umm_integrity_check -> use of printf instead of DBGLOG_INFO functions
• umm_poison -> get_unpoisoned() wrong type of variable: uint8_t c must be uint16_t c

Best regards,
Douwe

In some use scenarios, there may not be linker supplied values to tell us where the heap should be or how big it is. In this case we will need an API call umm_init_heap() that lets us pass in the start address and size manually. The old umm_init() API is still there, it just calls umm_init_heap() with the linker supplied values.
While we are at it, we can let the user choose whether they want a check for an initialized heap before any heap operation, and if so, to allow this condition to be handled as needed.
Finally, we clean up the override mechanism for umm_malloc to allow compile time -D flags to take priority over a user supplied config file. If none of these are present then the system default umm_malloc_cfg.h file takes precedence.
To implement this successfully we will need the following incremental changes:

• Add a way to either auto-initialize a heap or call an error trap if any allocations are called on an uninitialized heap
• Implement the concept of one or more heap control blocks to allow multiple heaps
• Improve the umm_malloc compile time configuration system
• Implement heap control block initialization for integration into Rust

Currently we do not have any code coverage metrics - and that's about to change.
Add support for gcov tasks in project.yml
In general we want to run each configuration separately when doing coverage testing, I think,