hunkim98 / dotting Goto Github PK

This issue is closely related to issue #77. If we want users to manipulate canvases, then we should add listeners for canvas info. The information that should be included in the canvas info listener should be the panzoom values, top left corner screen position and so on.

Currently there are only two brush modes: Dot, Eraser.

In many art editors there is a paint bucket brush mode that allows neighboring colors to be colored at once.

It might be great if dotting had a pain bucket brush mode.

This feature can greatly help later on when other brush modes are needed for dotting.

Two things must be considered for this pain bucket brush mode

• If the current pixel that is clicked has empty color(color: ""), then all the neighboring empty color pixels should be colored with the paint bucket color
• If the current pixel that is clicked already has a color (ex: color #ff0000), then all the neighboring #ff0000 color pixels should be colored with the paint bucket color

After all pixels have been colored, a data-change event should be emitted.

One could use a recursive method when implementing the paint bucket tool!

Dotting seems to lag a little when there are many pixels to render.

It seems that the nested for loop that is run when render() is called is the main culprit of the lag.

I believe optimization will be achievable if we separate the Dotting component into many layers, thus Separation of Concerns

First, we could have a grid layer. This layer will show the grids of the pixels.

Instead of running a nested for loop for this grid layer, we could simply change the grid when there is a change in data, and use canvas transform api to better the performance.

Below that grid layer, we can have an interaction layer. This layer will be used for the pixels the user is interacting with her mouse. We can render the this.strokedPixels just by running a for loop in this.strokedPixels and render them in the �interaction layer. Once the user moves her mouse outside of the canvas element or has triggered a mouseup, then this layer will be empty and the changes will be applied to the layer below, the data layer.

Below the interaction layer, we can have the data layer. This layer is the layer that will show the pixel data. This is the only part where we will need to run a nested for loop. However, the nested for loop will be only run once when the data has changed. If there are no changes in the data and the user is simply panning and zooming, we can use the canvas transform api to better the performance.

Below the data layer, we can have a background layer. This layer will show the background of Dotting. This will be the easiest layer to implement.

Having four layers, the grid layer, the interaction layer, the data layer, and the background layer will be enough to optimize the performance. But for user customization we can also add a custom layer, the layer at the bottom, where users can manipulate the canvas. This can be a feature that users would want because some users would like to place an image behind the pixel grid canvas and draw it.

Also, having these multiple layers can lessen the complexity of our canvas logic since it will separate each feature into smaller parts. This will enhance the readability of our current code!

• Create multiplayer able interaction layer
• Create multiple layers in one class

Currently the extend button ui is like below.

I find this ui unpleasing. Plus, I found that most people did not notice that the buttons were meant for extending the pixel canvas. Thus, I came up with a new ui.

I came up with a new ui that has only lines. I am planning to implement this feature. If you have any feedback on the ui, please feel free to comment!

The current design sets the (0,0) to be the middle of the pixel grid. This can make things complicated when we implement multiplayer for Dotting. For instance, the Selected Area should change every time someone changes the dimension of the pixel grids. Plus, it is difficult for a user to associate the world position and the pixel grid with the current design. I will work on this issue!

In the process of painting, the 'DOT' part works normally, but the 'paint bucket' process does not work well as the video below.

Currently if both fingers are in the canvas grid, either of them is ignored and interpreted as one finger. Thus, pinch zoom is disabled.

We could add a feature where we detect the number of fingers that touches the screen and activate pan zoom when the number is over 2.

Currently the canvas background is white, and thus users cannot distinguish the background and white brush

We need to change the canvas background to another style such as a checkered background like photoshop

Currently the canvas is default to have a black border style.

One could make users decide how to style to canvas

Currently the initial pan zoom is the same for all inputted initial data. Some user would like to have a 256 x 256 grid canvas. It would be great to adjust the initial pan zoom value to fit the grid canvas inside the canvas element initially.

Also, our current min and max scale limits the user in interacting with the canvas. For instance, a user would like to view the art from a faraway view. We should change the min and max scale range to fix this issue.

Currently, there exists a minor problem where the hovered pixel appears or other extension buttons are hovered while the user is dragging an extension button due to the change of the pixel canvas size. It would be great if we could prevent the hovering animation occur when the user is dragging the pixel canvas to a specific direction.

For pixel modification to be easy, I think we need to allow users to change the size of the pixels inside the selected area when extended. An example is given below (brought from Aesprite)

This will help users expand their pixel assets easily.

Currently, there is a callback for setting one layer's data. However there are no functions to set layers. This is a necessary feature if we would like to initialize the canvas with the saved layer information

Currently, users can use useBrush hook to set brush mode or set brush color.

Why don't we just simply create brushMode and brushColor props in DottingProps?

This will allow users not have to use useBrush hook if they simply want to just change brush color or brush mode.

Currently there are no functions that allow users to update the brush pattern. Users can only color one pixel by pixel. A feature that allows users to change the brush pattern is necessary.

Dotting is meant to be used by many people who are interested in pixel canvas. To gather users, we should show dotting to as many developers as we can.

yorkie-js-sdk is an opensource library that I have contributed last year. To briefly give you some information on it, Yorkie is an opensource library that allows you to create a multiplayer editor (ex. Figma). It basically uses CRDT to manage concurrent operations.

If anyone is interested in contributing to yorkie, please give me a notice beforehand since this contribution will need some guidance.

In the PR #56, I have updated the interaction layer to mask the data layer when the user shrinks the pixel canvas. However, this only works for the backgroundMode "color". The implementation of masking when backgroundMode is "checkerBoard" is quite more complicated that masking when backgroundMode is set to "color", but it is worth a try.

Currently our editor has matured greatly. I can proudly say that we have accomplished the core features for creating an artwork through our editor. Now our concern is to allow users to freely export their work to their preferred file format. For instance, a user could prefer to first do some work in Dotting and continue their work in Adobe Illustrator or Aseprite. For such functions, one should think about format. How do we ease the implementation of exporting dotting data into other formats?

I currently have no clue in how to implement this template. I welcome anyone who wants to delve into this issue.

Currently we have many features provided, however, there are few examples where users can refer and customize dotting to create something of their own. Thus, it would be great if we could create some sample tutorials for people to follow for creating their own editor. Plus, the tutorial does not necessarily need to focus on editor, but can expand its topic to any other interesting subjects!

Users are only able to extend the grid if their cursor is exactly inside one of the extension buttons.

This will not come in handy when the user has zoomed out or use fingers to extend the grid.

We could enhance this by making extension activation area larger when the canvas is zoomed out a lot

While creating a grid canvas with more than 500 rows and 500 columns, I found there was a lag when the user pans the artwork. This happens because every time the user pans the canvas, all the canvas layers are rendered all together. I believe there is no need for unnecessary renders when a user pans through the canvas.

I came up with an idea to instead use css to remedy this issue. We could make the panning happen in the parent div element while the canvas element is not rendered. We could simply mask the area that is out of the window.

This approach requires a whole new change to our current code structure since the panning will happen outside of the canvas and the zoom happens in the canvas.

However, it will better the performance of our editor to accommodate examples that require more than 1 million grid squares.

Currently, there is no feature that allows users to set the data of the pixel data. This is necessary since users will want to set their data manually through code.

I have had some interviews with some designers on dotting-genai. It seems that many people want the layer feature in the editor. People want multiple layers on the canvas so that they can control each layer differently.

I previously thought that implementing layers should be done by the programmer who uses this dotting library. However, after much thought, I found out that it is very difficult for a programmer to implement a layer feature by importing dotting.

I believe we should implement this layer feature in the dotting library. This is a difficult task, but I believe it is worth a try!

Currently there is no action set for undoing and redoing actions.

To implement this, one should define actions that happen in the Canvas and track them.

Check the link below to get a glimpse on how to implement undo/redo

Reference: https://liveblocks.io/blog/how-to-build-undo-redo-in-a-multiplayer-environment

Currently there is no listener for stroke updates that happen in the interaction canvas. This listener will come in handy in a multiplayer environment where users want to track the real-time edit event happening the other person's canvas. To do this one should create a new listener for stroke updates in the interaction canvas.

This feature is necessary for the issue in collab-dotting.

Refer to: lerrybe/collab-dotting#5

There is a need to emit an event when selected area changes. We should implement to allow users to delete the colored pixels of a selected area.

It seems that there is a demand for custom brush such as bigger brush size, spray, or pattern. Since Dotting is a pixel editor. I believe providing users the ability to create their own brush size is necessary. One could allow users to style the brushes by accepting a 2d-array such as below

const patternBrush = [
  [ 0, 1, 0,
    1, 1, 1,
    0, 1, 0]
]

The above brush will allow users to draw on the pixel grid with a +. We should also consider making the interaction canvas to show the brush pattern as an indicator when the user hovers her mouse on top of the pixel grid.

Currently, extension can happen in one way, either vertically or horizontally. It would be great if we allow users to extend the size diagonally.

There is a lag when the user has finished changing the grid size when the grid is extended by more than 100 rows or columns. We should find a way to remedy this issue.

Currently, the user cannot know which brush they selected.

It would be great if we could change the mouse cursor based on the selected brush mode.

Plus, since people could have their own brush modes, we could create a function that allows people to change the mouse cursors to the icon of their choice

Currently, there is no feature that allows users to set the indices. This would be necessary since users will want to set indices dynamically through code.

Currently the setPanZoom seems to cause bugs to the extension method. This bug especially happens when the grids are extended to a certain threshold that the canvas cannot cover. the setPanZoom function is implemented as below:

setPanZoom({
    offset,
    scale,
    baseColumnCount,
    baseRowCount,
  }: Partial<PanZoom> & { baseColumnCount?: number; baseRowCount?: number }) {
    if (scale) {
      this.panZoom.scale = scale;
    }
    if (offset) {
      const correctedOffset = { ...offset };
      const columnCount = baseColumnCount
        ? baseColumnCount
        : this.dataLayer.getColumnCount();
      const rowCount = baseRowCount
        ? baseRowCount
        : this.dataLayer.getRowCount();

      // rowCount * this.gridSquareLength * this.panZoom.scale < this.height
      // Offset changes when grid is bigger than canvas
      const isGridRowsBiggerThanCanvas =
        rowCount * this.gridSquareLength * this.panZoom.scale > this.height;
      const isGridColumnsBiggerThanCanvas =
        columnCount * this.gridSquareLength * this.panZoom.scale > this.width;

      const minXPosition = isGridColumnsBiggerThanCanvas
        ? -(columnCount * this.gridSquareLength * this.panZoom.scale) -
          (this.width / 2) * this.panZoom.scale
        : (-this.width / 2) * this.panZoom.scale;

      const minYPosition = isGridRowsBiggerThanCanvas
        ? -rowCount * this.gridSquareLength * this.panZoom.scale -
          (this.height / 2) * this.panZoom.scale
        : (-this.height / 2) * this.panZoom.scale;

      const maxXPosition = isGridColumnsBiggerThanCanvas
        ? this.width - (this.width / 2) * this.panZoom.scale
        : this.width -
          columnCount * this.gridSquareLength * this.panZoom.scale -
          (this.width / 2) * this.panZoom.scale;
      const maxYPosition = isGridRowsBiggerThanCanvas
        ? this.height - (this.height / 2) * this.panZoom.scale
        : this.height -
          rowCount * this.gridSquareLength * this.panZoom.scale -
          (this.height / 2) * this.panZoom.scale;
      if (correctedOffset.x < minXPosition) {
        correctedOffset.x = minXPosition;
      }
      if (correctedOffset.y < minYPosition) {
        correctedOffset.y = minYPosition;
      }
      if (correctedOffset.x > maxXPosition) {
        correctedOffset.x = maxXPosition;
      }
      if (correctedOffset.y > maxYPosition) {
        correctedOffset.y = maxYPosition;
      }
      this.panZoom.offset = correctedOffset;
    }

    // relay updated information to layers
    this.relayPanZoomToOtherLayers();
    // we must render all when panzoom changes!
    this.renderAll();
  }

If you see the code, you can see that due to the minXPosition, maxXPosition, minYPosition, maxYPosition, the canvas offset is modified when user extends the canvas to a certain level. I presume this is caused by setPanZoom being called when mouse is moved. We could fix this by disabling setPanZoom when extension is happening. (We do not necessarily need to offset the canvas to a certain point when user is extending!)

Also, it seems that there is a problem in extension not following the mouse well enough. A deeper look in to the problem is necessary.

This is a clip showing the bug:

Currently there are no test configured for Dotting.

We need tests to make sure that updates do not hamper original code.

It would be nice to also have a husky setup for checking tests before committing

I am currently creating an iPad app using dotting. I found that there is a need to overlay the webview with a native view for Apple Pencil touch events. To do that, I must have to call mouseDown mouseUp mouseMove manually through events.

I show you my current progress

As you can see, there is a need to distinguish Apple pencil touch and finger touch. This is why I need to expose the mouseDown mouseUp mouseMove functions

Some users requested that they would like to export a pixel art as svg format. We could add options for downloading an image in a svg format.

Currently there are not methods for manipulating the panzoom values (scale, offset value)
It seems that we could add methods for manipulating them since some users would need to set it.
I also believe that we should change the method of panzoom to a more flexible one since the current version has its min and max panzoom value fixed.

It seems that calling undo after select area is extended is error prone. It seems that some pixels are not correctly erased when undo is called. There seems to be a slight bug in the recordAction for select area extend.

Some users might want to draw something on top of the canvas or behind the canvas. For instance, there might be some use cases where people would want to enable pixel drawing on top of a photo.

To enable such features, we should add additional canvas layers to our editor

Currently if you see the grid size change part, you can see that setPanZoom is called every time.

https://github.com/hunkim98/dotting/blob/6c38fbd7c103467679a139d6ea4246b77340a3a8/src/components/Canvas/Editor.tsx#L523C5-L579

 private extendInteractionGrid(direction: ButtonDirection) {
   const interactionLayer = this.interactionLayer;
   interactionLayer.extendCapturedData(direction);
   // this.renderInteractionLayer();
   const interactionCapturedData = interactionLayer.getCapturedData()!;
   const baseRowCount = getRowCountFromData(interactionCapturedData);
   const baseColumnCount = getColumnCountFromData(interactionCapturedData);
   if (direction === ButtonDirection.TOP) {
     // ⬇️ From here
     this.setPanZoom({
       offset: {
         x: this.panZoom.offset.x,
         y:
           this.panZoom.offset.y -
           (this.gridSquareLength / 2) * this.panZoom.scale,
       },
       baseRowCount,
       baseColumnCount,
     });
    // ⬆️ To here
     this.extensionPoint.lastMousePos.y -= this.gridSquareLength / 2;
  //..
 }
}

Setting panzoom everytime causes all canvases to render because all canvases rely on the panzoom.

However changing panzoom when user changes the grid size seems quite unnecessary. Why do we need to change the panZoom when we change the grid size?

The reason such nuisance occurred is because I poorly designed the grid extension mechanism. I believe I designed it this way because I have considered users extending the grid even when their mouse has left the canvas.

Maybe a deeper analysis on this issue is necessary. Maybe we could eliminate setPanZoom method for grid size changes?

Currently, the BrushTool.DOT is designed to draw pixels on the current mouse position when mouse is down. This causes problems when the user drags the mouse too fast. For instance see the below example.

The browser cannot track every mouse position precisely. To fix this issue, we need a new algorithm for drawing the pixels.

What we should do can be seen in the image below.

Finding the intersections is not easy and is a waste of time. Instead, you could use a line drawing algorithm based on the cosine value of the neighboring points.

That algorithm is called Bresenham's line algorithm

For Korean materials that explain Bresenham's line algorithm, one could take a look at the below materials.

• 이득우의 게임수학 chapter.12
• 42seoul 브레젠험 직선 알고리즘

To implement this, one could use this.mouseDownWorldPos and this.mouseMoveWorldPos and some additional array to keep track of the neighboring points.

Currently, when user changes the brush mode, different logic is called based on the brush mode when a user interacts with the canvas. This style can be cumbersome to manage when we have various drawing mechanisms.

For ease of creating other brush modes, @lerrybe has suggested that we could use a strategy pattern for changing brush modes.

A strategy pattern is a pattern where the context(In Dotting, it is class Canvas) does not need to know the specifics of a specific algorithm but instead chages its strategy(i.e. strategy) and simply calls on of the method inside the strategy. More Information

To implement this, one should create a generic interface for all brush strategy, and create classes for each brush mode based on the interface.

After the appropriate classes are created, one could create a new variable named brushWorker(or any name you would like) inside the Canvas class and change the brushWorker instance to another instance whenever a user changes the brush mode.

For your information, I leave you some useful links you could reference when you implement this

[Changing brush mode by changing the tool to another tool instance]

• https://github.com/yorkie-team/toonie/blob/79f9a7c6033fceb5d23bb335d35bf8e66ec8361f/src/components/Editor/DrawingBoard/Canvas/Board.ts#L275-L305

[A generic interface for all tools and the list of the tools]

• https://github.com/yorkie-team/toonie/tree/79f9a7c6033fceb5d23bb335d35bf8e66ec8361f/src/components/Editor/DrawingBoard/Canvas/Worker

Currently the user can set brush color with useBrush hook.

However, since setBrushColor in Canvas.tsx can be called after the canvas element is mounted, I have added an initBrushColor prop to handle initially passed brush colors.

The problem is that this adds confusion to users. Maybe we could integrate setting brush color and initBrushColor prop into one?

This issue is closely related to issue #17

Currently, the DataChangeHandler delta does not consider select area tool operations. We should fix the tool to emit its modified pars to DataChangeHandler

Currently, if the user deletes a rowIndex or a columnIndex that does not exist inside the current grid indices, an error is raised. We should fix delteGridIndices to just ignore the deletions of nonexistent row indices or column indices

Currently the MIN_SCALE and MAX_SCALE is set to default numbers and users are not able to change them.

It would be great if we allow users to set their preferred max and min scale

I found that undo does not work correctly when extension happens diagonally. This is because the current code does not take into consideration of sideways extension.

Most pixel art editors have a selection tool. Currently we can manipulate the pixel colors only with brushes. It would be better to allow users to select multiple pixels with a selection box.

An example of a selection tool would be like below.

Currently the grid extension happens to lag a little when the canvas is zoomed out.

The algorithm of the grid extension needs to be enahced.

Recently, during the PR conversation in #79, @Lee-Si-Yoon has suggested that we create an importable utilities file where we store external functions that do not particularly pertain to using Dotting as an editor. An example of such external function would be downloadImage. Instead of putting that downloadImage as a innate function inside Dotting that should be imported through connecting a hook, we could change this to allow users to simple import that downloadImage function and call it on DottingData.

Currently it seems that downloadImage is the only external function we need to care about.

This issue has to do with refactoring rather than implementing a logic. If there is anyone who would like to try this, please feel free to contact me.

Currently DataChangeListener only emits the changed whole data. This entails great difficulties for multiplayer environments since we need to emit one player's modified parts not the whole data for optimization. This is also a necessary feature for lerrybe/collab-dotting#7.