Tracking Lost Service is not behaving correctly about mixedrealitytoolkit-unity HOT 5 CLOSED

@edgar-rodrigo-santos , thank you for the response. I apologize for the lack of documentation here. The Tracking Lost Service is actually the only MRTK2 component that relies on the AR Session. 😮 There's a gap in my history as to why this change was made initially

• #8282

However, the WorldManager APIs were removed in Unity 2021, so this the why these is a need for AR Session. Unfortunately, I think it was an oversight that the Tracking Lost Service documentation wasn't updated.

There is some additional documentation on setting up AR Foundation and AR Session with MRTK2, but this experimental documentation was for iOS and Android. https://learn.microsoft.com/en-us/windows/mixed-reality/mrtk-unity/mrtk2/supported-devices/using-ar-foundation?view=mrtkunity-2022-05

I apologize for the poor documentation here. Again, I recommend you investigate using MRTK3. Moving from MRTK2 to MRTK3 is large effort, so it might not be feasible. However, if this is a new application, I strongly recommend MRTK3.

Given you found a working solution, I'll be closing this issue. If you have further problems, please reach out, or open a new issue.

from mixedrealitytoolkit-unity.

@edgar-rodrigo-santos , are you using OpenXR? If so, OpenXR origin anchoring works a little differently than the older WMR strategies (i.e. what the Shell uses).

If you are using OpenXR, to correctly handle tracking lost, you need to put the application's tracking mode into an "Unbounded Reference Space". The easiest way to do this in MRTK2 is to add the EyeLevelSceneOrigin component to your MRTK2 rig's XR origin component.

Please try this, and reply back.

‼️ MRTK2 support is limited, and we recommend developers start looking at MRTK3 (mixedrealitytoolkit.org). If this is a new app that you are working on, we highly recommend using MRTK3 instead.

from mixedrealitytoolkit-unity.

@AMollis Thank you for your reply.
Yes, I'm using OpenXR.
I just tried adding the EyeLevelSceneOrigin and the behavior seems to be even worse.

If I cover up the sensors the tracking state doesn't change at all (even if I cover the sensors for a long time).
When I uncover them, it goes to limited for a while.
I also tried going into a completely dark room and the tracking state didn't change either.

from mixedrealitytoolkit-unity.

@edgar-rodrigo-santos I assume you are using AR Foundation 4.2, along with AR Subsystems 4.2?

@edgar-rodrigo-santos I couldn't reproduce the issue you are having, even after clearing my HL2's maps. However, I saw a slightly different problem. Tracking would never become completely lost, just limited. The service doesn't show the tracking lost message when in a limited state, see LostTrackingService::Update:

public override void Update()
{
    using (UpdatePerfMarker.Auto())
    {
        XRSessionSubsystem sessionSubsystem = SessionSubsystem;
        if (sessionSubsystem == null)
        {
            return;
        }

        if (sessionSubsystem.trackingState == lastTrackingState && sessionSubsystem.notTrackingReason == lastNotTrackingReason)
        {
            return;
        }

        base.Update();

        // This combination of states is from the Windows XR Plugin docs, describing the combination when positional tracking is inhibited.
        if (sessionSubsystem.trackingState == UnityEngine.XR.ARSubsystems.TrackingState.None && sessionSubsystem.notTrackingReason == NotTrackingReason.Relocalizing)
        {
            SetTrackingLost(true);
        }
        else
        {
            SetTrackingLost(false);
        }

        lastTrackingState = sessionSubsystem.trackingState;
        lastNotTrackingReason = sessionSubsystem.notTrackingReason;
    }
}

Mainly this line:

sessionSubsystem.trackingState == UnityEngine.XR.ARSubsystems.TrackingState.None

The service only indicates a loss in tracking when the state gets to 'none', which I had a hard time achieving. I had to change this line to the following, so I could see the lost tracking message:

(sessionSubsystem.trackingState == UnityEngine.XR.ARSubsystems.TrackingState.None || sessionSubsystem.trackingState == UnityEngine.XR.ARSubsystems.TrackingState.Limited) 

After the device reobtained tracking, the "lost tracking" message disappeared.

I still suspect that your issue is because of a missing or misconfigured EyeLevelSceneOrigin component. Here's my sample XRRig setup, that worked for me. Perhaps compare this with what you have, please notice the EyeLevelSceneOrigin on the root XRRig game object:

Another thing to check, ensure you see this message in your Unity log. If you don't see this, then your app doesn't have a XRInputSubsystem running or you are missing the EyeLevelSceneOrigin:

EyeLevelSceneOrigin: TrySetTrackingOriginMode to Unbounded

Also make sure you don't see this immediately after the first message:

EyeLevelSceneOrigin: Failed to set tracking origin to Unbounded

from mixedrealitytoolkit-unity.

@AMollis Thank you so much for the detailed reply.

After putting everything together like you described, it started behaving much better.

Even though we have ARFoundation in the project, we never had to use the ARSession and ARSessionOrigin components (neither did most if not all mrtk samples, at least in mrtk2).

We will do further testing to confirm nothing else was affected by these changes.

Thank you.

from mixedrealitytoolkit-unity.