Various data structures are created while loading a glTF model. These need to be cleaned up when no longer needed or if one of the jobs fails.

Created during parse step (GLTF_Parse_Job):

• Object.Data.JSON of type JSON_Value_Access
• Object.Data of type GLTF_Data_Access

Created during processing step (GLTF_Process_Buffers_Job):

• Object.Data.Buffers (I).Buffer of type Byte_Array_Access

• Execute GPU jobs on render thread via job graph system
• Be able to create and sort draw calls and other commands
• Submit drawcalls by calling OpenGL procedures in render thread at beginning of next frame

If an executor in package Orka.Jobs.Executors has processed a job and this job has only 1 dependent job (that is, Job.Dependent /= Null_Job and Jobs.Empty), then process this job on the same executor instead of enqueuing it by calling Queue.Enqueue. This should make the job graph system a bit more efficient because it should avoid transferring data to another CPU core.

• HiZ occlusion culling

Support animations and vertex skinning features of glTF:

Basic structure

• Animations

Vertex skinning

• Skins

• Vectors
• Matrices
• Quaternions

Add support for UBO's, SSBO's, atomic counters, compute shader.

Add retrieving offset and size via tex level parameter (see texture_buffer_range extension).
Add Type_Invariant checks to texture tagged types.

Changing the binding point of an SSBO or UBO is likely not going to be supported. The binding point can be explicitly set in the shader with a layout qualifier.

See Mesamatrix for more extensions that need bindings.

OpenGL 4.6

• ARB_gl_spirv
• ARB_indirect_parameters
• ARB_pipeline_statistics_query
• ARB_polygon_offset_clamp
• ARB_texture_filter_anisotropic
• ARB_transform_feedback_overflow_query
• ARB_spirv_extensions

OpenGL 4.5

• ARB_clip_control
• ARB_conditional_render_inverted
• ARB_direct_state_access
• ARB_get_texture_sub_image
• ARB_texture_barrier

OpenGL 4.4

• ARB_buffer_storage
• ARB_clear_texture
• ARB_multi_bind
• ARB_query_buffer_object
• ARB_texture_mirror_clamp_to_edge

OpenGL 4.3

• ARB_clear_buffer_object
• ARB_compute_shader
• ARB_copy_image
• KHR_debug
• ARB_framebuffer_no_attachments
• ARB_internalformat_query2
• ARB_invalidate_subdata
• ARB_multi_draw_indirect
• ARB_program_interface_query
• ARB_shader_storage_buffer_object (use Get_Program_Resource for introspection)
• ARB_texture_buffer_range
• ARB_texture_storage_multisample
• ARB_texture_view
• ARB_vertex_attrib_binding

OpenGL 4.2

• ARB_base_instance
• ARB_map_buffer_alignment
• ARB_internalformat_query
• ARB_shader_atomic_counters (use Get_Program_Resource for introspection)
• ARB_shader_image_load_store
• ARB_texture_storage

OpenGL 4.1

• ARB_separate_shader_objects
• ARB_viewport_array

OpenGL 4.0

• ARB_sample_shading
• ARB_shader_subroutine
• ARB_tessellation_shader
• ARB_texture_buffer_object_rgb32
• ARB_texture_cube_map_array

OpenGL 3.3

• ARB_instanced_arrays

OpenGL 3.2

• ARB_draw_elements_base_vertex
• ARB_seamless_cube_map
• ARB_depth_clamp
• ARB_sync

OpenGL 3.1

• ARB_draw_instanced
• ARB_copy_buffer
• ARB_texture_buffer_object
• ARB_uniform_buffer_object (use Get_Program_Resource for introspection)

OpenGL 3.0

• ARB_map_buffer_range

OpenGL 1.4

• EXT_multi_draw_arrays

Procedure Load_And_Compile in Orka.Rendering.Programs.Modules directly reads files using GL.Files.Load_Shader_Source_From_File. Instead use the resource system. This will make it easier to enable automatic live reloading of resources in the future.

Add procedures Read_Data in package Orka.Rendering.Buffers.Mapped.

When creating a logger, it should be possible to specify the minimum severity of log messages that the logger must log.

Add a new parameter Level : Severity := Debug to the constructors of Orka.Loggers.Terminal and Orka.Loggers.Location.

Build a frame graph to allow defining arbitrary rendering pipelines. Manage resource creating/deletion/binding/barriers.

• http://ourmachinery.com/post/high-level-rendering-using-render-graphs/
• https://www.slideshare.net/DICEStudio/framegraph-extensible-rendering-architecture-in-frostbite

Tasks:

• Create an API to build a graph
• Add ability to cull the graph
• Create FBOs, attachments, and texture binds
• Present a resource using default FB for last render pass, blit, or render to default FB
• Clear and invalidate framebuffer attachments
• After culling we may want to sort the render passes (the Passes vector) that have a Side_Effect or References > 0 so that we don't iterate over unused passes in procedure Render.

Improve the API of tagged type Framebuffer in package Orka.Rendering.Framebuffers a bit:

• Attaching and detaching textures
• Clearing and invalidating attachments

If a texture contains multiple mipmap levels, only the first level is written to a KTX file.

Support rendering features of glTF:

Rendering

• Techniques
• Programs
• Shaders

Extensions

• KHR_material_common
• FRAUNHOFER_materials_pbr

Add a Orka.Resources.Textures.KTX package that is able to load .ktx files. See the KTX File Format Specification.

• Perform pass reordering
• Add ability to "move" a resource to another resource (see the slides from DICE)
• Insert memory and texture barriers where needed
• Maybe support non-texture resources like buffers (SSBOs)

See the links in #13 for more information.

Package Orka.Inputs specifies an interface Pointer_Input to provide a pointer on the screen. Currently GLFW_Window.Process_Input in package Orka.Windows.GLFW modifies the object which implements Pointer_Input using the position and scroll offset of the mouse.

Package Orka.Windows.GLFW should be modified so that the pointer can be controlled via a joystick as well. Clients may want to be able to control whether the mouse, a joystick, or both control the pointer. They should also be able to change this at run-time. Note that joysticks can (dis)appear during run-time.

I used GNAT 2017 GPL, clean sync on latest of both json-ada and orka : checkout master.

make build, along the way you should cross the following:

• json-ada is not recognized as a package (I did 'make install' for json-ada and I find these libs under /usr/gnat/lib. I do not know what is wrong)

• orka-futures-slots.ads:56:21: internal call cannot appear in precondition of protected operation

• orka-loops.adb:21:06: file "orka-buffer_fences.ads" not found

• orka-loops.ads:63:41: internal call cannot appear in precondition of protected operation

Once fixed finally passed with many warnings though.

make examples:

• orka_test-test_2_two_triangles.adb:20:06: file "orka-buffers.ads" not found ... compilation of orka_test-test_1_triangle.adb failed ...

Once everything changed for Orka.Rendering.xyz most example builds. The execution of those binaries all leads to this:

Execution terminated by unhandled exception
raised PROGRAM_ERROR : gl-objects-renderbuffers.adb:144 access before elaboration
Call stack traceback locations:
0x7f8fab10ed8d 0x7f8fab1145a7 0x40a2de 0x4089e4 0x7f8fa9a07b95 0x408a18 0xfffffffffffffffe

In the light of those errors I propose you amend the Readme to let know the project is still in a WIP phase and might not build. Thx.

p.s: see pull request with all the quick fixes to make it build.

Support rendering debug axes (x = red, y = green, z = blue).

Let clients listen for keyboard and joystick events.

• Joysticks
• Keyboard

Mesa drivers currently return Byte or Unsigned_Byte for integer internal formats, even if the component size is 16 or 32 bits (depending on the Internal_Format (in GL.Pixels) that is used). For float internal formats, Float is returned.

Functions Read_Pixels_type, Texture_Type, and Get_Texture_type in GL.Pixels.Queries never return Short, Unsigned_Short, Int, Unsigned_Int, or Half_Float.

See https://gitlab.freedesktop.org/mesa/mesa/blob/master/src/mesa/main/formatquery.c#L609-617

Appropriate data type is required in procedure Write_Texture in package Orka.Resources.Textures.KTX.

Compute the data type ourselves instead of calling Get_Texture_Type in procedure Write_Texture.

Currently culling and binding buffers is done for each instance. This should be done once. Refactor Orka.Resources.Models to improve the efficiency.

Also make sure each instance has a Behavior_Ptr so that each instance can have an arbitrary implementation of Behavior and is able to override Fixed_Update for example. Right now each instance is of the type Model_Instance, which means the user does not have a way to provide their own implementations of Fixed_Update, Update, etc. for physics and/or animations.

In a separate repository on Github:

• Web application using Material UI
• Render frame graph with VX (based on D3)
  • Be able to select a resource to be presented
  • Add/remove/modify resources and render passes
  • Clicking on a resource will show a side bar containing a video stream of the resource
  • Communicate with engine over a websocket connection
• Render a list of used resources over time (row index is the resource used, column index is the render pass that reads/writes to the resource)
  • Should visualize opportunities for reusing resources if they have same description and do not overlap in time

Add packages Orka.Windows.SDL and Orka.Inputs.SDL using lucretia/sdlada's bindings.

Automatic resolve multisampled resources if the resource was written as a FB attachment by blitting the framebuffer. If the resource was written as an image, then so automatic resolving should occur.

For example:

P1 -> R1 -> P2

R1 can be resolved by adding an extra pass that does nothing but a blit operation from the FB of P1 to P3:

P1 -> R1 [-> P3 -> R2] -> P2

The extra render pass should be able to blit mulitple resources (1 color and/or 1 depth and/or 1 stencil). The extra resource (R2) must have the same format.

Improve the KTX viewer so that it can display textures other than those of kind Texture_2D:

• Texture_1D
• Texture_2D_Array as a grid of Texture_2D
• Texture_3D as a cube (volumetric rendering)
• Texture_Cube_Map as a cube and inverted cube (360 deg background)

glTF has several problems because it was designed for WebGL instead of OpenGL 4.x:

• Requires PNG and JPG support (needs to be decoded and do not use GPU compression) but does not support KTX textures
• Requires parsing a JSON file, which slows down loading models
• Each mesh in a model's scene may use a different vertex format. Although in practice the format of the attributes is the same for all meshes, their index type (uint16 or uint32) may differ
• No SPDX license identifier for meshes

Create a binary storage format:

• Based on document container files using the dcf-ada library (already used for locations in #35)
• Require all files stored in the container to use the "store" compression format
  • Vertex data may be compressed with Draco
• Optimized for OpenGL 4.3 or later and Vulkan
• Any software supporting this format must ignore any unknown files in the container
• TODO Add that assets can be assumed/required to be on SSDs? (No slow storage)
• TODO Support cameras, lights and speakers?
• Textures must be stored in one or more KTX files in the container
• TODO Specify PBR requirements for materials
  • TODO Specify transparency, refraction, and clear coat requirements
• Vertex attributes must not be interleaved
  • TODO Allow quantization parameters?
• Vertex buffers and the index buffer are stored in separate files in the container
• Must contain an index buffer
• Contains a file describing the vertex format
  • Index type must be unsigned short or unsigned int
  • A vertex format has a single specific index type
• Contains a file describing the bounding boxes of meshes
• Contains a file with an array of draw commands (GL.Types.Indirect.Elements_Indirect_Command_Array)
  • TODO Multiple arrays for different pipeline state? (Opaque and transparent meshes) (in that case there may be multiple vertex formats as well)
• Contains a file describing the scene tree
  • TODO Specify how the tree must be stored (a single array of the levels or multiple levels)
• Contains a file with copyright and SPDX license identifiers of the meshes
  • TODO Contain matrices (or euler or quaternions) and/or visibility flags?
  • TODO Support multiple LoDs?
• TODO Support skinning and animation?

Support automatic reloading of resources (shaders, glTF models, KTX textures) if one of the files used to construct a resource has changed.

If reloading fails (e.g. exception is raised in a job), the current resource should remain active.

• Buffers
• Programs
• Textures
• Framebuffers

Debug rendering:

• Bounding boxes
• Coordinate axes
• Lines
• Spheres

Add a loader that can load .gltf files that follow the glTF 2.0 specification. Use json-ada for parsing JSON.

• Update loader from 1.0.1 to 2.0 specification

Basic structure

• Scenes
• Nodes
• Cameras

3D objects

• Meshes
• Textures
• Images
• Samplers

Data layout

• Buffers
  • Inline data as base64 encoded string instead of an URI
  • External .bin files
• Buffer views
• Accessors
• Binary glTF

Rendering

• Materials

Procedure Write_Texture in Orka.Resources.Textures.KTX currently uses a hardcoded format RGBA and data type Float. Implement GL extension ARB_internalformat_query2 so that these two variables can be inferred from the Texture object.

Function View_Matrix of type Look_At_Camera in Orka.Cameras and type Model in Orka.Resources.Models use a rotation matrix to convert positions from structural frame to OpenGL.

This rotation matrix should not be hard-coded, but configurable.

Add a package Orka.Resources.Locations.Archives which implements the Location interface.

Only support ISO/IEC 21320-1:2015 .zip archives.

Use Zip-Ada. Port it to Git and clean it up by removing all the prohibited functionality.

• Create dcf-ada library
• Improve API of new library
• Add package Orka.Resources.Locations.Archives and update README.md

Load_Texture in Orka.Resources.Textures.KTX should be converted to a Jobs.GPU_Job.

A Loaders.Loader tagged type should be added which implements a Load procedure to create and enqueue the GPU job. The old empty Load at the end of the KTX package can be removed.

See https://github.com/onox/orka/blob/master/src/orka/implementation/orka-resources-models-gltf.adb#L355-L397 and https://github.com/onox/orka/blob/master/src/orka/interface/orka-resources-models-gltf.ads#L31-L35 for an example.

Use https://squidfunk.github.io/mkdocs-material/

Vulkan only supports the following texture border colors:

• transparent black
• opaque black
• opaque white

OpenGL supports arbitrary colors via procedure Set_Border_Color in package GL.Objects.Textures.

See slide 20 of zink_fosdem19.pdf.

Currently messages can be printed from any task via Orka.Logging.Insert_Message or from the rendering task via GL.Debug.Insert_Message (which uses KHR_debug).

• One way of formatting and printing messages (unify types like Source and Severity)
• Support asynchronous behavior with a PO
• Support multiple outputs:
  • Screen
  • Log files
  • Others like WebSocket

Add a scene graph to compute the world transform matrices of nodes and to perform view frustum culling on the CPU.

• Update world transform matrices
• Add unit tests

• Spatial partitioning data structure
• Instance culling with bounding spheres / box on CPU
• LoD

Automatic resource management, including physical resource aliasing of virtual resources that do not overlap.

Get_Data and Get_Compressed_Data in GL.Objects.Textures assume no PBO is bound and operate synchronously. Refactor these functions so they can operate (only) asynchronously (requires binding a buffer to a PBO and setting a fence).

• Refactor Get_Data and Get_Compressed_Data (bind a buffer as a PBO + set a fence)
• Add support for downloading data asynchronously in a Orka.Rendering.Textures (use job graph system)
• Find out how to obtain a file descriptor from an EGL/GL sync object.

After the fence has been retired, we can either:

1. Copy the data with Get_Data in Orka.Rendering.Buffers
2. Map the buffer, copy to an array, unmap the buffer
3. Make sure a specific region of a PMB has been bound as a PBO (useful for video streaming)

The job graph system might need to be modified in order to postpone the execution of the remaining jobs of a job graph until a fence has been retired. Currently we either need to wait on a fence in a GPU job or insert a new GPU job between the current GPU job and the next job in a job graph. However, this new GPU job might immediately get executed by the thread which processes the GPU jobs. Currently there's no way to tell the system to wait until the next frame.

https://on-demand.gputechconf.com/gtc/2012/presentations/S0356-GTC2012-Texture-Transfers.pdf

GLFW 3.1 functions:

• glfwGetWindowFrameSize
• glfwPostEmptyEvent
• glfwSetDropCallback

GLFW 3.2 functions:

• glfwGetKeyName
• glfwSetWindowAspectRatio
• glfwSetWindowSizeLimits
• glfwSetJoystickCallback
• glfwSetWindowMonitor
• glfwWaitEventsTimeout

GLFW 3.3 functions: (https://www.glfw.org/docs/3.3/news.html#news_33)

• glfwGetKeyScancode
• glfwGetJoystickHats
• glfwGetJoystickGUID
• glfwGetMonitorUserPointer
• glfwSetMonitorUserPointer
• glfwJoysstickIsGamepad
• glfwUpdateGamepadMappings
• glfwGetGamepadName
• glfwGetGamepadState
• glfwRequestWindowAttention
• glfwSetWindowMaximizeCallback
• glfwFocusWindow
• glfwGetMonitorContentScale
• glfwGetMonitorWorkarea
• glfwGetWindowContentScale
• glfwGetWindowOpacity
• glfwInitHint
• glfwMaximizeWindow
• glfwSetWindowAttrib
• glfwSetWindowContentScaleCallback
• glfwSetWindowOpacity

May be binded in the future:

GLFW 3.1 functions:

• glfwCreateCursor
• glfwCreateStandardCursor
• glfwDestroyCursor
• glfwSetCursor

Not binded:

GLFW 1.0 functions:

• glfwGetTime (not needed in Ada)

GLFW 2.2 functions:

• glfwSetTime (not needed in Ada)

GLFW 3.1 functions:

• glfwSetCharModsCallback (deprecated)

GLFW 3.2 functions:

• glfwCreateWindowSurface (Vulkan)
• glfwGetRequiredInstanceExtensions (Vulkan)
• glfwGetInstanceProcAddress (Vulkan)
• glfwGetPhysicalDevicePresentationSupport (Vulkan)
• glfwVulkanSupported (Vulkan)
• glfwGetTimerValue (not needed in Ada)
• glfwGetTimerFrequency (not needed in Ada)
• glfwSetWindowIcon (has no use on Wayland)

GLFW 3.3 functions:

• glfwGetJoystickUserPointer (not needed in our case)
• glfwSetJoystickUserPointer (not needed in our case)
• glfwGetError (not useful; already using glfwSetErrorCallback)
• glfwGetProcAddress (3.3 supports EGL_KHR_get_all_proc_addresses)
• glfwWindowHintString (only used on X11 and macOS)

Along the way you should cross the following:

• orka_test-test_6_gltf.adb:195:28: prefix of "Access" attribute must be aliased

• orka-loops.ads:59:41: internal call cannot appear in precondition of protected o peration

• orka-simd-sse4_1-doubles-math.ads:26:7: error: '.builtin_ia32_roundpd' nee ds isa option -m32 -msse4.1

• orka-atomics.ads:24:13: error: inlining failed in call to always_inline 'Orka.Atomics.Decrement': function body not available. (note: Did not find proper fix yet.)

I let go even though it looks interesting.