This is a large ticket and will likely need to be split into smaller ones.

• Create a concept of virtual keys so that the application can map some native keyboard events into uikit virtual keys that the UI can react on.
• Create a concept of "input focus", i.e. which widget has the input focus for receiving key events.
  • focus indication
• React on some uikit virtual keys and perform tasks such as "move focus to next widget", "move focus to prev widget".
  • focus order (also called tab order) of widgets

The project can have references to multiple opaque blobs of data that are defined outside of the game studio project. This could be text, cvs, json, etc.

There needs to be some way for the game to be able to access this data.

• Loading the data files/buffers/handles
• Ability to interpret/access the actual content in some way. Not all possible game data formats can be supported out of the box. A plugin system could be useful here but that's beyond the scope. However for JSON based data we can provide a Lua API around data/reader/writer

This is a large ticket and will likely need to be split.

A shader graph editor would be a new feature in the editor based on the idea of being to able to visually define material shaders by building graphs out of pre-defined material nodes. The evaluation of the graph would be used to build the actual shader could that would be stored and then later on used to create the shader program.

Enhance and refactor the stencil masking, add a specific option for kind of stencil masking operation to apply.

For example in the demos/breakout there are simple "spot lights" that are for visual appearance only. It'd be nice if this type of thing could really be used to compute light effects. For example in the same demo the individual tiles could simulate being lit or being in the shadow by seeing how much light falls upon them. This would mean having to rasterize the "spot lights" into some buffer that can be used to rasterize the game tiles.

Currently the physics sub system will traverse the scene's render graph and upon finding an entity node with rigid body will then synchronize the state from the b2Body to the rigid body. This might not be the most efficient way of doing this.

Related issue #36

Figure out a way to visualize physics joints. Attach a drawable shape?

Handle errors happening during WASM game init and context creation. Propagate the error information to the UI and display to the user.

This is distinct from the "packaged game data" (see #47) since this is data that is generated by the game when it runs. This is data such as high scores, save games etc.

It's likely possible to share some data access details between this and #47

Useful for things such as volume setting

https://developer.nvidia.com/content/alpha-blending-pre-or-not-pre
https://shawnhargreaves.com/blog/texture-filtering-alpha-cutouts.html
https://shawnhargreaves.com/blog/premultiplied-alpha.html

Add versioning and version checking in various contents such as materials, entities scenes etc. in order to make future expansion easier.

Figure out a way to elide a memcpy from the buffer coming from the audio graph and into the buffer pointer in the audio FillBuffer callback when possible.

Currently the particle simulation runs in its own simulation space with only bounds for the simulation. This however means that the particles do not interact with the content of the game world, i.e. the simulation proceeds regardless the particle would in fact fall upon or touch some game space object. It'd be nice to be able to realize something like obstacles. This would require mapping of the game object that fall within the particle engines simulation boundary into the simulation space and then interfacing with the actual particle simulation.

IT should be be possible to do a brute force solution for this type of problem using the GPU by using for example depth testing (since we're not using z) or stencil testing and doing a read back to see if a particle has collided with something.

Add mouse and keyboard grabbing when specified by the project / workspace settings.

Mouse grabbing is important for a game such as demos/breakout for example since in a windowed mode the mouse going outside of the window the application would no longer receive mouse events and game play would suffer. Grabbing the mouse would help here (until a global grab is taken by some other application of course...)

This needs to be implemented in wdk also.
ensisoft/wdk#21
ensisoft/wdk#22

Create an entity with a node that has an offset relative to the entity origin.
Add a rigid body item to the node.
Place the entity in the scene and apply a rotation to the entity in the scene.
The rigid body transformation is wrong.

Add interaction entity node component that can be used to interact with through some means of native input such as mouse/keyboard/touch events.

This would provide a higher level of interaction capability to the application so that it would not have to rely on low level keyboard/mouse/touch events. The API towards the game would something like "interaction component X was clicked" or "interaction component X was dragged"

"Oops, there was an error. :(
See JavaScript console for details."

Console:

Failed to load resource: the server responded with a status of 404 (Not Found)
game.html:203 wasm streaming compile failed: TypeError: Failed to execute 'compile' on 'WebAssembly': Incorrect response MIME type. Expected 'application/wasm'.
printErr @ game.html:203
game.html:203 falling back to ArrayBuffer instantiation
printErr @ game.html:203
GameEngine.js:1 Filesystem sync. Error= null
GameEngine.js:1 [0.000000] Info: main.cpp:202 "Ensisoft Gamestudio 2D"

GameEngine.js:1 [0.000000] Info: main.cpp:203 "Gamestudio Copyright (c) 2010-2021 Sami Vaisanen"

GameEngine.js:1 [0.000000] Info: main.cpp:204 "http://www.ensisoft.com"

GameEngine.js:1 [0.000000] Info: main.cpp:205 "http://github.com/ensisoft/gamestudio"

GameEngine.js:1 Uncaught TypeError: Cannot read properties of undefined (reading 'getParameter')
    at emscriptenWebGLGet (GameEngine.js:1:165224)
    at _emscripten_glGetIntegerv (GameEngine.js:1:167101)
    at invoke_vii (GameEngine.js:1:234115)
    at 02e158da:0x8208ff
    at invoke_iii (GameEngine.js:1:233951)
    at 02e158da:0x8b9e91
    at invoke_vii (GameEngine.js:1:234115)
    at 02e158da:0x43a6fc
    at tick (GameEngine.js:1:198147)

OS: macOS 10.15.4
Browser: Chrome

Use the KomikaTitle-wide.ttf font and write for example "Go". The G is cut off.

See Jos Stam real time fluid dynamics for games

http://www.dgp.toronto.edu/people/stam/reality/index.html

When the scene contains no (semi-)transparent objects it should be possible to reduce overdraw by inversing the scene panting order so that the nearest layer is drawn first and the lowest (furthest away) layer is painted last. Combined with depth buffering this should take advantage of (early) z testing to discard fragments that would not be visible.

The steps.

• enable depth buffering in the render target
• enable depth testing when drawing a scene without transparent objects
• generate a depth value for the primitive's vertices based on the primitive's layer
• profit

This is not the same as ray casting in the physics subsystem API which only considers physics bodies. This will work in the scene and consider entities (and entity nodes).

Pulseaudio by default stops and pauses the stream if it thinks the write operation didn't provide enough data.
When the stream is paused the write callback is no longer called. This means that the audio source has no chance of providing more
data.

I guess there are 2 options to try to fix this.

• use the pa_buffer_attr to try to turn this feature off
• resume the stream conditionally (uncorck) again

Add support for HDPI display scaling.

What this means.

• rendering resolution and render target size are no longer the same as the display size.

• the display window can be scaled either manually or by the underlying platform.

• the content rendered by the engine needs to be displayed with some scaling factor/mechanism

  • render to a offscreen render target and then rescale and resample in order to display in the window
  • maintain aspect ratio or stretch

• need engine config flags to define the setting for hdpi

• need runtime game accessible way to access the hdpi display scale factor

• need runtime mechanism for letting the game to adjust the hdpi display scale factor

Also applies to fullscreen not just windowed mode.

Similar to entity animation/action track but at the scene level.

Complicated, so postponing now.

Investigate how to support and build an acceleration structure in the scene for resolving and finding entities on some action such as a ray cast query or bounding box query.

This is likely to need some static space partitioning structure with some flags to identify objects that can be optimized (i.e. things that will not move and can have pre-computed world positions/AABBs etc.)

Investigate how to limit the number of physics bodies in simulation. Currently the engine will create physics bodies for all entities that have nodes with rigid bodies regardless of whether they're currently actually contributing to the scene and its game play. For example simulating a physics such as a moving platform body that isn't affecting the user and isn't yet visible in the screen is wasted work.

Related to #37

Currently all of the scene's contents are loaded at once. For large scenes this can be wasteful and load too much data at once. Instead it'd be nice to be able to load the scene in chunks and shuffle the content in and out of memory as needed while the game play advances.

Add (optional) automatic entity killing when outside scene boundaries.
There are 2 ways to kill entities that fall outside of game play (imagine for example a falling body that falls through the scene and no longer contributes to the game play). Either the game makes sure that the entity is killed when some criteria is met such as falling to a distance of X meters from the "baseline" in game. But this can be tedious and error prone depending on the exact use case.

Alternative would be to have the game (scene) define a "hard" boundary and any entity completely outside these bounds would then be automatically killed.

Obvious reasons for this killing are

• performance, get rid of entities that no longer contribute to the game play
• stability of the game in for example physics simulation, at some point the floats and the physics sim would blow up potentially creating unexpected events in the game.

Current main objectives

• instanced rendering
• feedback buffers (relevant for particle sim)
• MSAA FBO

In the web environment it's possible for the game to suffer an "context lost" event. This might happen when for example when the browser decides to clean up the underlying graphics context when the user tabs out of the current browser tab that is running the game.

A recovery would require:

• realize that the rendering context was lost
• recreate the rendering context
• recreate the graphics device and context state, reupload all the required data, vertex buffers, textures, programs

If a scaling factor that is other than (1,0f, 1.0f) is applied on an entity instance creation the entity nodes are not placed correctly.

Currently the Lua based game code is handed raw pointers to entities. This works as long as the game doesn't try to do any kind of "caching" of these pointers. This can be a little bit problematic in the sense that any access to an entity / game object outside of the designated callback that handed the object could result in use after delete bug.

So far the an idea has been to use intrusive reference counting implemented in the entities. The scene and the Lua game code would be the most likely parties participating in this scheme (the engine subsystems are expected to respect lifetimes properly).
The unclear question though is which mechanism to use in order to make the ref counting implicit in the Lua integration so that the Lua based game code doesn't need to do it explicitly (which is no better than the current implementation!)