slice is not lazy now about imlazy HOT 8 CLOSED

Haha! Don't worry. I'm learning so much with these talks. Currently I'm removing
iterarray dependency of some parts of iterum library and the performance is increasing. Then, maybe I will keep iterarray for some parts that really is useful (combinatorial methods, for example) but I'll try to avoid to the extent of possible.

On the other hand, you could have better performance for array or string spliting the logic of slice:

function* betterSliceForArrays(start, end, array) {
    const init = Math.max(start, 0);
    const length = Math.min(array.length, end);
    for (let i = init; i < length; ++i) {
        yield array[i]
    }
}

from imlazy.

Hey, I went through a big debate by myself ages ago about this - good to get someone else looking at it too! My drop implementation was also eager https://github.com/benji6/imlazy/blob/v6.2.7/index.js#L105-L110 but rather inconsistently I seem to have changed that in the latest version 🤔

Basically if you eagerly drop values from iterable A to create iterable B, any iterable C you create from iterable B will necessarily be faster than if you had done no work in creating iterable B. Ultimately I was thinking to support a use-case where you could drop values from A to create B, then B to create C, then C to create D ad infinitum (perhaps with a setInterval or within animation frames) without any perf hit at all. If you do this lazily the time taken to extract any value from each created iterable increases linearly to infinity and it's impossible to support that use-case. If you intend to ever use the iterable created with slice or drop then I can see on disadvantage to being eager...

I hope I explained this OK? Do you have any thoughts on it? I'm wondering if I should make my drop implementation eager again or make the slice lazy 🤔

from imlazy.

OK, I understand you.

My approach about this use-case but keeping slice lazy is iterarray.

I have an object that has 4 fields and slice method:

• cache
• start
• end
• iterator

When this object calls slice method, it creates a new object with the same cache reference and start & end fields changed (It doesn't make traversing work).

When the user wants to traverse one of the sliced elements at first time, the cost could be big, because at first time the cache is empty and I need to call iterator field to get the data and fill to the cache. However, the cache is shared by all the sliced objects and traversing the next sliced objects does not have much cost.

Even, in the case of array or string iterables, I can put directly this data to cache field and I don't have any big cost on the first traversing of sliced object.

However I understand that if you don't want to handle cached data, you need other solutions to keep performance code without that.

Thanks for the explanation!

from imlazy.

The thing with your caching solution though is that it ends up consuming infinite space in the scenario I described above - you will eventually run out of memory if you keep pulling from an infinite iterable right? Unless you are using a more sophisticated and less naive form of caching? I should probably take a look at your code 😅

from imlazy.

My weak hipothesis is that you need infinite time to pull infinite data. But it's a weak hipothesis, because the memory will obviously run out before than time. 😆

from imlazy.

Yeah it will crash at some point! Another problem with caching is that not all iterables will be cachable:

const randoms = {[Symbol.iterable]: function * () { while (true) yield Math.random() }}
const times = {[Symbol.iterable]: function * () { while (true) yield Date.now() }}

If you cache either of those you'll cause loads of problems to the user that won't be obvious. I guess there's a reason why caching is one of the 2 hardest problems in computing!

I'm going to close this issue because I think the eagerness of slice has been addressed. Thanks for taking a look :)

from imlazy.

Hi @benji6,

On the other hand, I've seen that you are also doing the same approach with iterToGenFactory than me with iterarray. I guess that all of functions that use iterToGenFactory are caching the data. Then, they will have the same problem of pulling infinite memory for potential infinite iterations.

from imlazy.

Very good point! It was so long ago I wrote this code I kind of forgot how it all works 😅

I did this because it was the only way I could think to make drop operations eager. The problem is I can create an iterator from an iterable and call next on it until I've dropped the required number of values. Then I need to return an iterable that can be used multiple times but the iterator can only be used once so I cached it... So I guess I've been a touch hypocritical here! 😅

I'm thinking that it may not be possible to eagerly drop without caching so... I might make the trade-off by removing caching and losing the eagerness of drop, dropWhile, slice and tail. I guess it will be easier code too... Again thanks so much for reading through my code @xgbuils !

from imlazy.