Cancellation has a memory leak about fluture HOT 8 CLOSED

Terminology

• Main Future: The Future on which methods are called
• Other Future: The Future returned from the f given to mainFuture.chain(f)
• To clear: Cancel or dispose resources of a Future through usage of the clear function returned by Future().fork().
• Branch: The history of all Futures behind a Future. A branch splits when ap, race, chain or any function that combines multiple Futures into one somehow is executed.
• Infinite branch: Currently, chain, chainRej and parallel are capable of generating infinite recursion, causing some of their branches to grow indefinitely.
• Tree: All branches from all splits starting from the root.

Problem

After some further delving, I realized the memory leak only exists because I insist on the other Future in chain to be cleared when the main Future is cleared. This allows for the cancellation of the entire tree underlying a Future. The only time this is a problem is when we're dealing with infinite branches, (or branches with >~ 100.000 nodes because the cancel function will exceed the maximum callstack size), because the clear function will grow infinitely large.

Solutions

1. A rigorous solution would be to ignore the other Future in chains entirely in our clear function. This would mean that only Futures on the "main" branch of a chain would be cleared, and cancellation of Futures on the "other" branch wouldn't be possible. This is bad news for recursive programs, because the recursion is always on the "other" branch, causing the program to continue running after the main resources have been disposed, probably leading to an eventual crash.
2. Another solution I've tried is to introduce a specialized chain, which unlike the regular chain does not include the clear function of the other Future into its own clear function. This new chain, probably called recurse, could then be used to created infinite recursive processes which are still cancellable. I like this solution, but I'm not sure if it's acceptable to burden users with the possibility of memory leaks that have to be avoided caused by a feature I'm adding.
3. A combination of 1 and 2: Ignore the other Future in the regular chain, but add a specialized chain which does clear the other Future when it is cleared. My problem with this is that this is the function you would want to use in 99% of cases.

from fluture.

I went for solution 2.

from fluture.

Just for my better understanding: what happens when user cancels a future after completion? Is underlying computation cancel callback executed in this case?

from fluture.

In most cases; yes. Though sometimes (specifically in the case of chain) one of the two is not called after its computation completes due to a memory optimization I did.

from fluture.

In most cases; yes.

Ok, this is different from fun-task. In fun-task if a computation completed it cannot be canceled (since it already completed). I understand in Fluture cancelation have a bit different meaning. Just wanted to point it out and clear for myself. Not saying which meaning is better :)

Though sometimes (specifically in the case of chain) one of the two is not called after its computation completes due to a memory optimization I did.

That's interesting though. Wouldn't you want the guarantee that cancelation always propagates to all sources (event after completion)? Also I thought that you did it in Future#recur() not in Future#chain() as pointed out in https://github.com/Avaq/Fluture/releases/tag/2.0.0

from fluture.

Or is it just unfortunate implementation detail? I.e. users are not supposed to cancel after completion and use this "feature".

from fluture.

The cancel function is expected to be idempotent and a noop after completion (like clearTimeout is). Though you have a point. I'll probably patch it so that user-land cancel functions are never called after the computation they were created in has called a continuation. This means that "In most cases; yes" will change to "no, not for user-created Futures, only internal Futures are trusted to contain safe cancel functions".

I thought that you did it in Future#recur() not in Future#chain()

Both contain different optimizations. Future#recur() only keeps the cancel function of the parent Future, whereas Future#chain() only keeps the cancel function of the child Future (purely in order to save on memory consumption).

from fluture.

I'll probably patch it so that user-land cancel functions are never called after the computation they were created in has called a continuation.

Cool! Seems like a logical step after you've made Future.Guard() the default behavior.

from fluture.