File uploads are slow, increasing chunk size can help about meshcentral HOT 3 CLOSED

ok this is intresting.
just did my own experiment and indeed increasing those values does speed up the upload!
in theory we can also increase the file download too! so ill look into that
i found setting the value to 65565 is better than 0x100000
because 0x100000 is 1,048,576, and you have to remember this is on the handlebar files which is client side!
so the bigger the buffer the more memory the client side would need in there browser!
but yes indeed just increasing from 16384 to 65565 does make a difference so nice catch!

from meshcentral.

Thanks Simon, that is a significant improvement!

I will say, 1,048,576 is still only 1MB, and by my testing meshcentral itself already takes up ~200MB on the user's device. Since the buffer gets discarded immediately after use and cleaned up by the garbage collector shortly afterward, I don't think that .5% increase during an upload operation should be too much of a burden, and it is much faster. I tried uploading a 10GB file with the 1MB block size and didn't see any noticeable jump in memory usage over the old 16k.

Admittedly, though, that's hardly an exhaustive test, and there could be edge cases I'm missing, hence the deference, just something to think on. Super happy to just have the 65k, though!

from meshcentral.

For my testing, i used my pi zero w (on wifi too) and uploaded a 58mb file. Originally, it took about 10 minutes, but after changing it to the 65565, it only took 3 minutes! Which is a big improvement!

from meshcentral.