[BUG] MRTK 3 Hand Interaction Example: Quest 3 passthrough not working

[einstyle]

Describe the bug

I tried the Hand Interaction Example in MRTK3 with Meta Quest 3 and the passthrough doesn't work. I see the skybox in background. I tried to change the A value of the camera Background and it just switch from skybox to solid black.

Is there a specific setting to change?

Expected behavior

I want to see my environment and the virtual elements (holograms) in overlay.

Your setup (please complete the following information)

• Unity Version 2022.2
• MRTK 3

Target platform (please complete the following information)

• Meta Quest 3
• OpenXR

[BenediktTobias]

For passthrough on Meta Quest 3 you need need the Unity OpenXR Meta package (Meta XR All-in-One SDK / Oculus Integration work too but can cause some limitations / incompabilities with certain MRTK features).

[einstyle]

Hi @BenediktTobias,

Thanks for the reply. I already installed that package and added the right settings in the Project settings, however it seems the Passthrough doesn't work. I just opened one Example from MRTK3 (HandInteractables), I created a build for Quest 3, and what I see are the virtual objects with either the skybox in background or a black background (depending if the camera background is set to A=max or A=0)

Any help?

[BenediktTobias]

Based on your second screenshot, it appears that the Unity OpenXR Meta package is not properly installed. Can you please double check in the Package Manager if Meta OpenXR Feature* is listed there?
Afterwards check the camera settings according to the project setup of Unity OpenXR Meta, primarily the clear flags (solid color, A=0), and enable the required OpenXR Feature Groups (AR Foundation: Meta Quest Features, Meta Quest Support)**.

* Based on your version (2022.2) the newest supported version (0.1.2) was still called Meta OpenXR Feature instead of Unity OpenXR Meta as it is in newer versions (1.0.0+)

** In newer versions of Unity OpenXR Meta this is split into separate features

[shaynie]

@whebertML has gotten this working before, he's going to test and give some guidance.

[whebertML]

@einstyle, here are the steps I've taken to get Passthrough to work with MRTK3.

As @BenediktTobias mentions, there may be issues with the meta all-in-one package, so for simplicity I'll use the Unity OpenXR Meta, com.unity.xr.meta-openxr, package - latest version 1.0.1. The minimum Unity version for that is 2022.3, as mentioned on the package Requirements.

For this example, I'm using Unity 2022.3.27f1.

Install the com.unity.xr.meta-openxr package, version 1.0.1:

OpenXR config:

MRTK Profile Settings:

Last but not least, in your scene, add a "AR Camera Manager" script to your main camera (see below). Also, MRTK3 uses a "Camera Settings Manager" to validate camera settings depending on the type of display, whether opaque display (like Quest 3), or transparent displays (HoloLens 2, Magic Leap 2, etc.). Camera Settings Manager will override the camera settings itself during start, so you must change the opaque display settings to Solid Color, and make sure color is 0 in all channels, including alpha (see below):

[QuweQuwe]

I had the same phenomenon and tried many things.

Unity 2022.3.38f1
Create a new project with Universal3D
Add com.unity.xr.meta-openxr in Package Manager
Add MRTK3 in Tool

Changed OpenXR settings as follows

In the original scene
Removed Maincamera
Added the following to the scene
MRTK XR Rig
Cube (for confirmation during pass-through)

Add ARCameraManager to the MRTK XR Rig main camera
Set Alpha of SolidColor in OpaqueDisplay of CameraSettingManger to 0

Now when I ran the patch, I did not get the pass-through display.
Disabling CameraSettingManger resulted in pass-through display.

I'm trying to confirm if there is something wrong with the MRTK3 settings.
I placed XR Origin (AR) and disabled MRTK XR Rig and it showed pass-through.

[jonashein]

@einstyle @QuweQuwe Since it has not been mentioned here yet, did you follow the scene setup instructions for the Unity OpenXR Meta package?

I ran into the same issue at first, but after the described adjustments to the Universal Render Pipeline it works fine. I used Unity 2022.3.43f1, the Universal3D template, Unity OpenXR Meta version 1.0.1.