vegetableman / betamax Goto Github PK

The support for WebP is easy to do but the resulting animation had visual artefacts. So, support for it had to dropped. Find a workaround.

The current packing is meant to run in a slow, brute force manner and considering we are dealing with large @2x files, it slows down generation of the packed image tremendously.

Dig deeper on packing algorithms to optimise this process while retaining the compact size of the final result.

Differing size of two identical frames out of Mediarecorder blob due to the video encoding process. It's amazing how compression works on the fly.

Frame captured around 2.644 seconds (size: 236KB):

Frame captured at 8.562 seconds (size: 183KB):

Compression is highly recommended on the packed image and I do it through external tools.

Reference for implementation:
https://github.com/antelle/wasm-image-compressor
https://github.com/photopea/UPNG.js

Differences in color contrast out of grabFrame of ImageCapture. More info on it: https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/renderer/modules/imagecapture/README.md
Browser: Chrome v119.0.6045.159
OS: Mac Monterey 12.5.1

pre recording

post recording

MediaRecorder is the right option for recording frames but due to compression applied by video codecs during the recording, even visually identical frames will differ in this data and end up being marked as dissimilar and packed separately — increasing the size of the packed image. Refer #2.

There are three ways to go about it:

• Fix the diffing so that it takes in to account visual similarity.
• Add a mechanism to detect visually similar matches during packing similar to matchTemplate.
• Find a workaround so that we don't end up with dissimilar images in the first place through mediarecorder.

Pyodide has been incredibly helpful and this is super low on the list of things to do, simply because it's too complex. But the matter of fact is, python is slow and the latest version of pyodide is 3-5 times slower than python. So, we quickly run in to limitations with this environment.

Figma is a great example of the possibilities of running native code through webassembly.
https://www.figma.com/blog/webassembly-cut-figmas-load-time-by-3x/

We can go with C, Rust, Zig. whichever is most compatible with the scientific stack.