Proposal: Support for TransferList API about useworker HOT 9 CLOSED

Hey @alewin! That's a great suggestion! I really like it, it solves most of the complexity and design of the API. Are you thinking to add this as a default or as an opt in with the new field in the options?

I'm just thinking because sometimes you don't want the optimization because the array on the main thread gets completely empty when transferred, which in some cases can or not be useful.

An example of this can be when you want to render something based on the current state (a typed array) and then pass that array's buffer to the worker to calculate something but without getting rid of the array on the main thread because is still useful for the current task.

With this in mind, I think a nice approach can be to, either:

• Add as a default optimization but with an option to deactivate
• Add is as a opt-in in the options object

Note: In general, one can implement a workaround for the example I mentioned, but still, I think it will be nice to give more options to developers :)

from useworker.

Yes, I agree, its better to have one more option and let user decide 👌

const [addNumberWorker] = useWorker(addNumber, { transferList: false })

from useworker.

Hey @alewin! Sorry couldn’t answer before, I found myself quite busy these days 😅

I see your concern. And I think we then have some considerations to make, in the sense that, we can still go ahead with your suggestion (I like it more than op3). But if the user wants some objects to get transferred, they should pass them as arguments. For example:

someWorkerFunction(demo, demo.buffer1, demo.buffer2)

We then check all the arguments and transfer the ones that are transferable. I think it is a bit more cleaner than having to rely on arguments order as op3 suggests, because if we transfer based on argument type, one can for example:

someWorkerFunction(demo.buffer1, demo, demo.buffer2)

It's going to work still, with the added benefit that they'll need to write the variable just once. For example if we implement op3, they'll need to write the same variable variable twice:

// If we rely on the last argument being the transferList,
// and they want to use an array buffer in the function as an argument
// They'll need to pass it as an argument and also in the transferList
someWorkerFunction(demo, demo.buffer1, [demo.buffer1])

And as we can see above they get the transfer and argument for free if we go with your suggestion.

Another argument can be if users do not want those extra arguments on their functions. However, this will be easy to work around, because they can just pass the function they want to useWorker with the first arguments being what they actually want in the function and then ignore the others that they pass to get transferred.

// On call
someWorkerFunction(demo, demo.buffer1, demo.buffer2)

// They ignore the other arguments they passed just to get them transferred
const someFunction = (demo) => doSomething(demo) 

In a way, we move the need to rely on argument order to the user side. They can implement their functions and pass arguments the way they want, having in mind how useWorker works. And of course if they have a big array of things to transfer and do not want to write them as arguments, arguments destructuring is available:

const transferList = [demo.buffer1, demo.buffer2]
someWorkerFunction(demo, ...transferList)

It just feels more natural to me, "Ah! So every argument that has a transfer type gets transferred. I can play with this...". On the other hand having a magic last argument feels weird.

And lastly, if we go with this, we will need a pretty good documentation. Making clear that:

Every argument passed to a function returned by useWorker will be optimized for zero-copy (transferred) but only if it's passed explicitly, just like one will do if using postMessage.
To opt-out this optimization, one can turn it off with transferList: false on the options object.

I think both have some tradeoffs, for example, one can argue that op3 is more explicit and thus better.

At the end, they are also a lot of edge cases to consider that we will only start to see when we we actually ship the feature haha 😅

What do you think?

from useworker.

Thanks for the explanation 👏, I have never used Transferable before.

I was looking at a similar library: greenlet ( very clean approach), trying to understand how they managed this case.

Greenlet checks if the variable is a "Transferable"

const isTransferable = val = > ( val instanceof ArrayBuffer || val instanceof MessagePort || val instanceof ImageBitmap );

In this way, the library takes care of the buffer, without having to specify an additional parameter.

What do you think?

from useworker.

Awesome! Then I think we are ready to move forward, great discussion 🎉

Are you good with starting the implementation? 😄

from useworker.

Yes, It was helpful! 😅I'll take care of it!

from useworker.

Great, looking forward to it 👌

from useworker.

While I was implementing this feature, I had another doubt ...
If we want to check if what is passed to the web worker is a Transferable, it would be difficult to analyze all the parameters, or simply an object with nested Transferables.

Example:

 worker.current.postMessage(demo, [demo.p1.buffer1, demo.buffer2]) // pseudo-code implementation

where demo is:

const demo = {
        p1: {
          buffer1: [1, 2, 3], // ArrayBuffer
          p3: [
            {
              p4: {
                buffer2: [1, 2, 4], // ArrayBuffer
              },
            },
          ],
        },
        buffer3: [1, 2, 3], // ArrayBuffer
      }

Also I cant enable the transferList only for buffer2..

Your Option 3 should solve these problems..

from useworker.

Hey @alewin! Sorry couldn’t answer before, I found myself quite busy these days

No problem, unfortunately I'm busy these weeks too..

At the end, they are also a lot of edge cases to consider that we will only start to see when we we actually ship the feature haha 😅

Yes. .to consider all possible edge cases we need to ship this feature 😅

Im working here: https://github.com/alewin/useWorker/tree/feature/ISSUE-47-transferable

from useworker.