zulko / gix Goto Github PK

Motivation:

Users may want to apply filters like "blur" or "black and white" to one asset of they project. This ticket is not to implement these effects already but more to put the basic architecture in place, possibly with one easy demo effect like black-and-white

Specs

• The Asset menu has a new tab "Filters". Where users can add filters using a "New filter" button.
• A dropdown menu enables to choose the filter type. Filter parameters will depend on the filter type.
• The architecture makes it easy to add new filters (in the gix-renderer, and in the online editor).
• This comes delivered with one demo effect like black-and-white.

Technical notes

• Filters will very probably be implemented at the FrameServer level
• JS libraries for image effects exist and can be an inspiration

Blocked by (on simultaneous to): #33

Specs:

• In the Asset Filters, users can define a black-and-white filter
• It's fast!
• Very optional: users have the choice between several black-and-white algorithms

Technical details

• Seems easy to implement using Jimp

When displaying a GIX on Firefox, the first loop is blinky (typically, frame / black screen / new frame / black screen / etc.), as every frame from a GIF or video source appears with a lag. This is due to a known SVG rendering issue (see this discussion), and I never found a workaround. A radical solution could be to switch from SVG-based rendering to Canvas rendering.

Motivation

Right now users can only tweak the speed at which a Video or GIFs will play. Operations like freezing on one frame, playing backwards, or progressively slowing down, are not possible.

Specs

• Users can check "Use an arbitrary timeline" in the editor (linked to a new usesCustomTimeline in the Asset schema). This disables the "speed factor" and "time crop" elements of the forms, and instead creates a form similar made of two columns Project Time and Asset Time, which enables the users to say "this time in the project corresponds to that time in the asset".
• The gix-renderer is updated so that the frame returned for that asset will take into account the custom timeline when it is defined.

Motivation

Gix enables you to generate GIFs, but these are generally unoptimized and heavy. Would be nice if we offered the possibility to optimize the GIFs, so they are more easily sharable.

Specs

• Ideally the optimization step would come after the GIF rendering. Like, the user is presented with a GIF that is said to weight 6Mo, and there is a new button that says "Optimize GIF size" that the user can click on.
• If the implementation relies on a heavy framework, this should only be downloaded when the user uses the feature for the first time.

Technical details

There might not be any existing framework for that 😬 but possible paths include:

• Lobbying for this project to create a webasm browser build of gifsicle (looks like they did most of the heavy lifting already).
• Looking into ffmpeg-for-browser (not sure it supports GIFs already. But desktop ffmpeg creates lighter gifs).

Motivation

If seems that browsers cache Gix aggressively. When a new version is pushed online, the user's browser may keep serving them an older version until they do a full reload (Ctrl+F5).

Specs

• Gix gets updated in user browsers after a new release is pushed online.

Motivation

in Gix, frames are composed and rendered using the SVG format. This is a very practical format:

• It provides features for free (rotate, edge colors, transparency, font support via canvg, etc.)
• SVG components are clickable and draggable, in particular with the Vue framework

But there are inconvenients:

• SVG rendering means that each frame of the original asset must be converted to JPG, inserted in a SVG <image>, then it gets extracted by the SVG engine's JPG reader, and finally displayed. It should be much faster to just draw from a canvas to another.
• SVG rendering may or may not be slower than canvas for bitmap images (unclear).

Note:

This is a lot of work and might not even be a good idea. But huge if it works.

Specs

• Frame rendering for the gix-renderer uses only canvas operations instead of SVG+Canvg
• All GIF features (stroke line, outline, transparency, etc.) are conserved.
• Optional: the GixPlayer also uses canvas operation. Interactivity with that canvas is ensured either using Fabric, or by overlaying transparent SVG elements on top of the canvas.

Motivation:

Some GIFs, videos and images can have a green (or black or white) background. Users may want to make that background transparent so they can overlay the asset over another one.

Blocked by: #33

Specs

• In the Asset Filters (not yet implemented as of writing), users can define a new filter where they define the color to make transparent.
• There may be a need for another parameter defining the sharpness of the filter to apply.

Technical notes

• There doesn't seem to be any library providing the effect, but this may be doable with something like Jimp
• Unclear how much this would slow down previews if done in real time. An alternative (at least for gifs and videos with few frames) could be to cache the green-screen frames so they only get computed once.

Motivation:

Right now the handles to rotate and resize are home-made and don't look incredible. Existing frameworks could make them better.

Specs:

• As before, the resize/rotate handles only appear when hovering an element
• Drag/Drop/Resize now works with the new framework.
• The asset crop preview (which also uses a drag-and-resize widget) works well with the new framework.

Technical details

• JS frameworks: moveable has a vue binding
• Vue frameworks in particular: vue-drag-rotate-resize
• There may be more!

Motivation:

This would enable users to embed GIXes in their own apps and websites, make other kinds of editors without reinventing the wheel, etc.

Specs

• Users can write import {GixPlayer} from 'GixPlayer' in their project.
• There is an easy way to bump the version and upload.

Technical Notes:

This could maybe happen without creating a new repo, just by creating a library in-place where the GixPlayer lives.

Blocked by (on simultaneous to): #33

Specs:

• In the Asset Filters, users can define a blur filter
• It's fast enough! (some blurs are fast)
• Users have options such as blur background
• Brownie points for extra features like selective gaussian blur if possible

Technical details

• Seems easy to implement using Jimp

Motivation:

It looks like Gix works OK on mobile devices, up to the stage where you render the final GIF, where it just does nothing. This is very probably due to the fact that GIF generation uses Web Workers, and these are disabled in mobile navigators.

Specs

• GIF generation works on mobiles
• Any advantage of web workers, such as nice speed of rendering, are preserved on non-mobile platforms.

Motivation

If Gix introduces a breaking schema change, this will break all the existing Gix projects out there. We need a way to ensure the schemas auto-update.

Specs

• gix-renderer has a new function that will convert any project data to the latest version.
• Any project loaded to the editor undergoes this migration to the latest schema automatically.

Technical details

• Here is a good schema migration pattern explained.

Motivation:

Gix is mostly made ugly by a lack of skills. Any improvement is welcome. This is a standing issue for reference, that probably won't be closed before a long time.

Specs:

• Gix is less ugly (maybe provide screenshots).
• The solution doesn't use a super-heavy framework like Elements, so Gix will continue to load fast.

Motivation:

Gix-Renderer does not depend on Vue and could therefore be its own library, which would allow the following:

• Other projects, built with any platform will be able to generate GIXes
• The GIF editor repo would really just focus on GIF editing
• Better, separate version management of the renderer and the editor.
• The gix-renderer repo would be a good place to write GIF generaiton tests and avoid feature regressions.

Note:

• Not urgent until there is a precise use case for it.
• We should probably move Gix to an organization of its own when this happens, to keep all repos at the same place.

Specs:

• gix-renderer is in its own repo.
• There is a Github pipeline for bumping versions and releasing.
• There is an easy recipe to develop the renderer and the editor at the same time.