Code Monkey home page Code Monkey logo

openxr-tutorials's Introduction

OpenXR-Tutorial

This OpenXR-Tutorial repository contains both the text and code samples for the OpenXR-Tutorial website (https://openxr-tutorial.com/).

image image

Table of Contents

Chapters

1. Introduction

This chapter describes the goals of OpenXR and gives an overview of the core concepts used in the API. It describes how to set up your project as well.

2. OpenXR Setup

This chapter explains how to set up an XrInstance and XrSession. It also discusses how to poll events in OpenXR.

3. Graphics

XrSwapchain construction is explained in this chapter along with the XrEnvironmentBlendMode and the XrReferenceSpaceType. Finally, it explains how to set up a render loop and draw some simple geometry.

4. Interactions

This chapter focuses solely on explaining and demonstrating both the interaction profile and action systems that OpenXR uses. These systems provide a way for developers and users to bind inputs to actions allowing greater cross-vendor compatibility of an application.

5. Extensions

This chapter discusses a few extensions for Hand tracking and Composition Layer Depth. Extensions allow greater functionality to be added to an application provided that the hardware and runtime support them.

6. Next Steps

This chapter contains a collection of topics to act as a jumping-off point to explore further the diverse and ever-growing world of XR development. Of note are the sections on Multiview rendering, OpenXR API Layers and Multithreaded rendering.

Website

https://openxr-tutorial.com/

image

Setup

Prerequisites: git

Clone the repo with submodules using the following command:

git clone https://github.com/KhronosGroup/OpenXR-Tutorials.git
cd OpenXR-Tutorials

Follow build instructions for your platform below.

Build

Supported Platforms

Building the Tutorial Website

The tutorial website is built with Sphinx, a Python-based documentation build tool. To build the website, set the CMake variable XR_TUTORIAL_BUILD_DOCUMENTATION to TRUE. Set the CMake variable BUILD_ALL_EXTENSIONS to FALSE if you only intend to build the documentation and might be missing requirements for graphics APIs. Install the following Python modules:

pip install sphinx==6.1.3
pip install breathe
pip install myst-parser
pip install sphinxcontrib-jquery
pip install sphinx-copybutton
pip install sphinxcontrib-mermaid
pip install reuse

Now build the project Sphinx from the command line or your IDE.

Contributors

The original text of the OpenXR Tutorial is by Roderick Kennedy and Andrew Richards of Simul Software Ltd. The design of the site is by Calland Creative Ltd. The site is overseen by the Khronos OpenXR Working Group. Thanks to all volunteers who tested the site through its development.

Legal Notice

Website

© 2023, The Khronos Group. Creative Commons International 4.0 BY Attribution. Code is licensed as Apache 2.0.

"OpenXR" and the OpenXR logo are trademarks owned by The Khronos Group Inc. and are registered as a trademark in China, the European Union, Japan and the United Kingdom. Vulkan and the Vulkan logo are registered trademarks of the Khronos Group Inc.

OpenGL® and the oval logo are trademarks or registered trademarks of Hewlett Packard Enterprise in the United States and/or other countries worldwide. OpenGL is a registered trademark and the OpenGL ES logo is a trademark of Hewlett Packard Enterprise used by permission by Khronos.

Microsoft DirectX, Microsoft DirectX11, and Microsoft DirectX12 are trademarks owned by Microsoft.

The OpenXR Tutorial is owned and operated by The Khronos Group Inc. Khronos Legal Notices, Privacy Policy, and Code of Conduct apply.

Repository

See LICENSE.

Related resources

Khronos OpenXR Main page

OpenXR 1.0 Specification

OpenXR 1.0 Manual Pages

openxr-tutorials's People

Contributors

andrewrichards-code avatar dnovillo avatar pfontain avatar rbessems avatar rpavlik avatar rvkennedy avatar simul-build avatar utzcoz avatar xelarse avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

openxr-tutorials's Issues

Choice of linear algebra library

We need a solution for linear algebra. There's a file "xr_linear.h" in the OpenXR SDK which we could use.
We can copy it directly into the project, that obviously separates it from its intended source, plus it's PDX-License-Identifier: Apache-2.0, and copyright Khronos and "Oculus VR LLC" i.e. Meta.
We could add a new minimal header-only lib, or we could use something like glm.

Why Actions and explain sub action paths.

If I were to write it I'd begin with a preamble which would frame the action system. i.e. why does it exist? What problems is it trying to solve? Why does it need to be so abstract and weird? Why can't I just access controller hardware directly etc...? The questions that developers may have going into this weird and unconventional system if they have NEVER seen it before. I think its crucial for developer understanding to get them into the right head space and framing for this and for that we need to give context.

There is also no mention of subaction paths, which is a huge point of confusion from developers. I believe that should also be given some context (motivation and history), and potentially an example as to why it would be used.

I'd prefer that the examples of actions given were not just controller button actions, but rather game / app actions like "teleport" "interact" "shoot" "jump" etc...

OS Tabs -> Color / BW Icons.

OS Selector tabs -> Color Icon when selected, Black&White when not selected ( do keep tab highlighted functioning currently)

Local space

  1. The space section is missing nuances of re-localization that might occur.

  2. https://github.com/KhronosGroup/OpenXR-Tutorials/blob/main/tutorial/4-actions.rst?plain=1#L14

Spec defines local space as:

The LOCAL reference space establishes a world-locked origin, gravity-aligned to exclude pitch and roll, with +Y up, +X to the right, and -Z forward. This space locks in both its initial position and orientation, which the runtime may define to be either the initial position at application launch or some other calibrated zero position.

The tutorial says:

the XR hardware’s “local” space - either the headset’s starting position or some other world-locked origin.

We should probably clarify this a bit.

Make OpenXR library a fetch dependency instead

Instead of describing how to download the OpenXR SDK manually and set various paths in CMake should we use this automatic fetched dependency?

set(BUILD_ALL_EXTENSIONS ON CACHE INTERNAL "Build loader and layers with all extensions")
set(BUILD_TESTS OFF CACHE INTERNAL "Build tests")
set(BUILD_CONFORMANCE_TESTS OFF CACHE INTERNAL "Build conformance tests")
set(BUILD_API_LAYERS ${USE_OPENXR_LAYERS} CACHE INTERNAL "Use OpenXR layers")
FetchContent_Declare(
        oxr
        URL_HASH MD5=390455e9395a92d6b730afb8f23fac8e
        URL openxr_source_sdk,https://github.com/KhronosGroup/OpenXR-SDK-Source/archive/refs/tags/release-1.0.27.zip)```

Then the application can use this simply via:

target_link_libraries(openxrtutorialch2 PUBLIC
        openxr_loader

EDIT: Doing this will remove a lot of explanation code from the tutorial on how to build and install the OpenXR SDK which is just distracting from the purpose.

Move actions before interaction profiles

Move actions before interaction profiles. From a game developer perspective actions are the hero, understanding actions is the abstraction your game uses without caring what's behind it, thats the important bit.

Define good actions, build your game based on those actions.

Than interaction profiles are just a means to map those actions to controllers you're using.

Interaction profiles are just how you customize exposing those actions on specific controllers
getting your actions (and sub-action paths) and action sets right is critical

Tabs and Spaces

It appears the source code is mixing tabs and spaces.

indentation = 4 spaces ... Not tabs.

Questionable sentence about multiple XrInstance in Concepts

when initializing XR support in your app. If the Runtime supports it, you might have more than one Instance
at a time, if more than one XR device is in use.

If the Runtime supports it, you might have more than one Instance at a time, if more than one XR device is in use.

See also

https://registry.khronos.org/OpenXR/specs/1.0/loader.html#functional-flow

The loader supports a single XrInstance at a time in order to avoid tracking handle values and their relationship to the LoaderInstance. Every XR function call is assumed to be for the single XrInstance that has been created. This enables the loader to work with future extensions and handle types without change.

Code blocks renders in front of top navigation

Hi, while going through the tutorial, I did found that the code examples are rendered on top of the top menu, rest of the content seems to go behind the top menu.

I don't know what the design that was decided, is the top meny supposed to be sticky or is it supposed to scroll with the text? Just wanted to put this here for discussion.

Tested on current Edge.

image

Android missing KHR loader init

On Android the Loader requires the JVM pointers. For example:

#ifdef XR_USE_PLATFORM_ANDROID
bool init_android(struct android_app *state) {
  PFN_xrInitializeLoaderKHR xrInitializeLoaderKHR = nullptr;
  xrGetInstanceProcAddr(
      nullptr, "xrInitializeLoaderKHR",
      reinterpret_cast<PFN_xrVoidFunction *>(&xrInitializeLoaderKHR));
  if (xrInitializeLoaderKHR == nullptr) {
    ALOGE("Unable to find xrInitializeLoaderKHR");
    return false;
  }

  XrLoaderInitInfoAndroidKHR init_loader{
      .type = XR_TYPE_LOADER_INIT_INFO_ANDROID_KHR,
      .next = nullptr,
      .applicationVM = state->activity->vm,
      .applicationContext = state->activity->clazz};

  XR_CHECK_RET(xrInitializeLoaderKHR(
      reinterpret_cast<XrLoaderInitInfoBaseHeaderKHR *>(&init_loader)));

  return true;
}
#endif

Missing OpenXR intent category and broker permission

https://github.com/KhronosGroup/OpenXR-Tutorials/blob/simul-pitch/Chapter2.1_Android/app/src/main/AndroidManifest.xml#L22

      <intent-filter>
        <action android:name="android.intent.action.MAIN" />
        <category android:name="android.intent.category.DEFAULT" />
        <category android:name="android.intent.category.LAUNCHER" />
        <category android:name="org.khronos.openxr.intent.category.IMMERSIVE_HMD" />
      </intent-filter>

and broker permission

<uses-permission android:name="org.khronos.openxr.permission.OPENXR_SYSTEM" />

Comments -- need more of them

Given that this is a tutorial, I think it's important that there be an abundance of comments in the code explaining (perhaps in high detail) exactly what is going on.

In particular I think each and every function should have a comment that describes what that function/method does, how it's used, and probably even why it's needed.

Plus I'd like to see more comments throughout the code in general.

Debug output abstraction

Currently debug output is being written to std::cout. On android that goes nowhere special. Please add a small class that abstracts debug output so it is redirected to the appropriate location ( e.g. logcat on Android / std::cout elsewhere )

Blend Mode Selection

Chapter 3 needs an update to the way environment blend mode is selected. This should be a negotiation between the what the application can support and what the runtime supports. See pull request #24

Page footer updates

Text: (c) 2023, Khronos Group. CC-BY 4.0 International
Code: (c) 2023, Khronos Group. Apache License 2.

W.I.P. disclaimer

Can we add a work in progress disclaimer just in case google finds the site.

Explain Why Interaction Profiles Exists

It seems to be somewhat more focused on using the interaction profiles than explaining the why...

It would be good to have something in there describing why interaction profiles exist / why this pattern is used.

Logos for DirectX 11 and DirectX 12.

For the graphics API selection tabs, we have logos relating to their respective APIs. Khronos APIs are well-supported, but at present we don't have a good solution for Microsoft's DirectX APIs.
We are already in discussion with Microsoft for the usage of the Windows 10 logo, but we should also ask what would be best to use for the DirectX APIs.

Chapter 2 Structure - Project Setup

As Chapter 2 stand currently, the first half of the text is just setting up the project/build files and getting a debuggable application, which feels disconnected from the second half where we set up the XrInstance and XrSession and get going on OpenXR code and ideas.

Should we move Project Setup to its own separate chapter or add it as Chapter 1.4?
I feel it would be clearer for reading the text and for a user to follow.

Android Vulkan header VK_MAKE_API_VERSION not defined

For Android, we currently using this version of the Android NDK.

ndkVersion '23.1.7779620'

Unfortunately, VK_MAKE_API_VERSION is not defined in this version of the Vulkan headers. As a workaround, I have defined VK_MAKE_API_VERSION to wrap over VK_MAKE_VERSION.

#if defined(__ANDROID__) && !defined(VK_API_MAKE_VERSION)
#define VK_MAKE_API_VERSION(variant, major, minor, patch) VK_MAKE_VERSION(major, minor, patch)
#endif

Is this workaround suitable or bad practice?
I haven't checked, but do later versions of the NDK have newer Vulkan headers that have the VK_MAKE_API_VERSION macro?
If so, do we want to upgrade our NDK version for the tutorial?

Unable to build on Linux

I meant to make a formal submission, but just ran out of time. There are some issues with building the tutorial on Linux, and even more troublesome issues with running it -- at least for me. My apologizes for bunching this all in one issue, but it does all have to do with getting Chapter 2 to build and run on Linux.

First some issues with header files:

  • For starters, I think it's cleaner to include the standard C++ headers in the source file that requires them rather than all bunched into HelperFunctions.h, so I'd move that block to "main.cpp".
  • That said, there is actually a missing include of that does need to go in "HelperFunctions.h" for the use of "stdcmp()" -- I guess the MS-Windows compiler must be more lenient.
  • Also in "HelperFunctions.h", there is no need to include <unordered_map>.
  • The final issue with includes is that "main.cpp" also needs to include for the declaration of "unique_ptr".

The only other issue with compiling is:

  • The type "DebugOutput" is not declared anywhere in the Linux build, so I removed the line at the top of the "OpenXRTutorial_Main()" function. After that the build completed.

Unfortunately, when I then run "./OpenXRTutorialChapter2" it outputs the title of the tutorial, and then crashes on the line:
const std::vector<int64_t> &supportSwapchainFormats = GetSupportedSwapchainFormats();
I'm not sure whether this is the cause of the error, but in gdb, I was given this information:
(gdb) print supportSwapchainFormats
$4 = std::vector of length -525420, capacity -525420 = {6620760227972, 962689011016, -8895937474100832024,
-5437172960282474460, -4068641623515267036, -4519086758524671672, 6826918609914, -18360, [,,,]

A negative vector length seems like it could be a problem, but I don't know whether this is a cause or a symptom.

I tried a few more experiments:

  • Changing the example version from 2.3 to 2.1:
    #define XR_DOCS_CHAPTER_VERSION XR_DOCS_CHAPTER_2_1
  • and I tried forcing a graphics API setting -- which worked on the version from 07/25/23:
    #define XR_USE_GRAPHICS_API_OPENGL
    (also tried "VULKAN")

And one final thought -- last week I was able to get version Chapter 2.1 to build and run, and connect to the Monado OpenXR runtime, so that's a slight regression, but I'm happen to take a step back if it ultimately means we get where we want.

Difficulty getting the most basic OpenXR example compiled

Okay, I must be doing something wrong, because I can't get the simplest tutorial code to build.

In my first attempt, I was following along in section 2.1, but it's not always clear what file code snippets are supposed to go in -- for example, there's code that the instructions indicate go into the files "OpenXRHelper.h" and "HelperFunctions.h", but then a short while later it's stated that only three files should have been created by then "CMakeLists.txt" "Chapter2/CMakeLists.txt" and "Chapter2/main.cpp". And after running CMake (in my case "ccmake"), I try a build, but there are all sorts of things missing. Also, the code for "HelperFunctions.h" comes immediately after I'm told to open a new file called "main.cpp" -- this is confusing.

At one point I thought I read that at the end of each chapter I might be able to download the completed code files, but I don't see that now, so maybe I was imagining it. So instead I want to the github page, and started grabbing code from the "Chapter 2" section, but it seems like the "main.cpp" there is missing some header inclusions -- I'm getting a ton of errors. But also, it seems I need to download/copy several files to do a build, which seems to be overly complicated for the first Chapter with code. So perhaps we can start with something a lot simpler.

If we could have a tar/zip file with just the files/code needed for Chapter 2, I think that would be immensely helpful.

I have other issues too, but I'll save some of them for later, but one of them is comments -- there needs to be a lot more comments in the code.

Two issues with Linux build (09/18/23)

Hey all,

I did a rebuild today on the Linux branch of the github repo, and I encountered two issues.

  1. There's still a hard-coded directory for Roderick in the "Chapter2/main.cpp" code -- the logic implies that it should be overridden, but that doesn't seem to happen, and so since I don't have that path, it errors out.

  2. When building Chapter4, I get this error:
    [ 66%] Building CXX object CMakeFiles/OpenXRTutorialChapter4.dir/home/wrs1/Apps/OpenXR/Tutorial/SimulTut/OpenXR-Tutorials_git20230918/Common/OpenXRDebugUtils.cpp.o
    [ 68%] Linking CXX executable OpenXRTutorialChapter4
    GLSL ../Shaders/VertexShader.glsl
    make[2]: Vulkan_GLSLANG_VALIDATOR_EXECUTABLE-NOTFOUND: Command not found
    make[2]: *** [CMakeFiles/OpenXRTutorialChapter4.dir/build.make:219: OpenXRTutorialChapter4] Error 127
    make[2]: *** Deleting file 'OpenXRTutorialChapter4'
    make[1]: *** [CMakeFiles/Makefile2:190: CMakeFiles/OpenXRTutorialChapter4.dir/all] Error 2

Discuss aim vs grip pose.

It might also be good to have some discussion of the difference between aim and grip pose ; why they both exist; what the different purposes are, etc.

Session lost is not handled

If the session goes away from underneath an application it should exit gracefully, currently it fails in an infinite loop.

XR_ERROR_SESSION_LOST in xrWaitFrame: Session is lost
ERROR: OPENXR: -17(XR_ERROR_SESSION_LOST) Failed to wait for XR Frame.
Breakpoint here to debug.
XR_ERROR_SESSION_LOST in xrBeginFrame: Session is lost
ERROR: OPENXR: -17(XR_ERROR_SESSION_LOST) Failed to begin the XR Frame.
Breakpoint here to debug.
XR_ERROR_SESSION_LOST in xrEndFrame: Session is lost
ERROR: OPENXR: -17(XR_ERROR_SESSION_LOST) Failed to end the XR Frame.
Breakpoint here to debug.

you can recreate this by running against monado and closing monado before the application.

Code Style guides

I propose we use the following code style guide:

.clang-format

---
BasedOnStyle:  Google
Language:        Cpp
AllowShortFunctionsOnASingleLine: None
Standard: Cpp11
TabWidth: '4'
UseTab: Never
...
  void OnPreRender(const XrFrameState &state) override {
    UpdateGaze(state.predictedDisplayTime);
    XrVector3f wind{-0.001f, 0.0f, 0.0f};
    for (auto &bubble : bubbles_) {
      if (bubble->Update(state.predictedDisplayTime, wind)) {
        missed_++;
      }
    }
  }


    for (auto &bubble : bubbles_) {
      if (bubble->TestHitRay(head_pos, gaze_dir)) {
        score_++;
        auto snd = sound_node_->GetComponent<Components::Sound>();
        if (snd) {
          sound_node_->SetTranslation(gaze_target);
          snd->Play();
        } else {
          ALOGE("Hmm no sound component");
        }
      }
    }

Android build system setup and manifest

I left a number of comments on 7738eb6

My overall concern is that there is a lot of prose being spent on things to remove and add, when:

  • Google frequently changes the Android gradle plugin so it's hard to keep up with the contents of the template (stuff to remove)
  • Several of the changes actually add back deprecated Android gradle plugin usage
  • Only a few details are actually important here for the details of using OpenXR in native code on Android: the other parts are just "normal native code on android" things or "stuff you don't actually want to do but that's what we did to match our example"

Basically in nearly every release we have to adjust stuff in our Gradle files to work with the latest Android Studio changes. It's preferable, at least to me, if the text sticks to just those things that are OpenXR-specific (which haven't had to change in quite a while) so updates can just be to the build, not to the text as well.

cc @rbessems

Simple example in section 2.1 not connecting to OpenXR runtime

So I've been working through the OpenXR tutorial in as small of steps as I can, as though (and in fact) a naive developer for OpenXR, and I'm only part-way through section 2.1, and have it building, but when running it doesn't connect to my OpenXR runtime (Monado), running on Linux.

I've had to make some changes here and there to the code in order to get it to compile, so it's possible my changes were not correct -- but the original code seems to have some mistakes in it. One mistake in particular is that there seems to be inconsistent usage of "XrInstance" vs "xrInstance" vs. "instance" and I think even "xr_instance". I think many uses of "XrInstance" should be just "instance" so that is the change I made.

If anyone wants to look at my changes, I put a tar file of my version at:

Any place where I made changes to what's in the online tutorial are commented with "WRS".

NDK Version

NDK 4.2.2 is pretty old and I believe throwing warnings in newer Android Studio versions. Let's get some more recent in here.

Class member variable naming

Member variables currently have no special distinction. I propose we follow hello_xr's pattern and name class member variables

m_snakeCase

Graphics Detection

The current method of graphics api selection is fairly static and will fail to run if the underlying device doesn't support whatever was hard coded even though the code supports alternate API's.

Ideally the code has an ordered set of graphics API's and selects the first from that list that the underlying runtime supports.

Chapter 2 build fails on Linux looking for "hDC" field

I did a test this evening from the current github clone on my Linux system, and encountered an error building Chapter 2 where:
error: ‘struct ksGpuWindow’ has no member named ‘hDC’
And I found that in "gfxwrapper_opengl.h" this field only exists for MS-Windows, so the correct value needs to be selected for the Linux build.

Bill

SVG Icons: feedback

I've had an icon made as a test, this is to represent an XR hand controller for Chapter 4.

HandsetIcon

You can see it at a more usual size in-place at Chapter 4

Please post any feedback on general design, colour (this one is grey but most will use pastels) etc.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.