Bring your Unity VR app to a fully immersive space

Highlight

Unity brings its full XR ecosystem to visionOS, letting developers port VR projects using familiar workflows. The core path: choose the visionOS build target, enable the XR Plug-in, use URP for foveated rendering and single-pass instanced rendering, and adapt controller input to hand gestures with XR Interaction Toolkit or Unity Hands Package.

Core Content

Two paths to fully immersive experiences

(00:33) Unity supports two immersive experience types on visionOS:

• Fully immersive: Replaces the user’s surroundings, transporting them to another world. Uses Compositor Services and Metal rendering
• Mixed immersive: Blends content with real-world passthrough. Coexists with other apps in Shared Space

This session focuses on the fully immersive path. See “Create immersive Unity apps” for the other path.

Build and run workflow

(02:25) Unity provides full build support for visionOS:

1. Select the visionOS build target
2. Enable the XR Plug-in
3. Recompile native plugins (.mm source files need no changes)
4. Generate an Xcode project from Unity
5. Build and run on device or simulator in Xcode

This workflow matches iOS, Mac, and Apple TV targets that developers already know.

Key graphics pipeline configuration

Universal Render Pipeline (URP)

(03:29) URP is recommended because it supports visionOS-exclusive foveated rendering.

Foveated rendering concentrates more pixel density in the center of the lens and reduces detail in the periphery. Human eyes are more sensitive to the center region, so this technique improves visual quality without increasing rendering load.

With URP, static foveated rendering is automatically applied across the entire pipeline, compatible with post-processing, camera stacking, HDR, and more. Custom render passes can use Unity 2022’s new APIs to leverage this technique.

Single-Pass Instanced Rendering

(04:40) Single-pass instanced rendering now supports the Metal API and is enabled by default. The engine submits draw calls once for both eyes, reducing rendering pipeline overhead in culling, shadows, and more, lowering CPU load for stereo rendering.

If your app already correctly uses single-pass instanced rendering on other VR platforms, shader macros ensure it works on visionOS too.

Depth buffer

(05:14) The system compositor uses the depth buffer for reprojection. Every pixel must write a correct depth value; areas missing depth information display incorrect colors.

Skyboxes are typically infinitely far from the user and write depth 0 with reverse Z, requiring modification to display correctly on device. Unity has fixed depth writing in all built-in shaders, but custom effects (custom skyboxes, water effects, transparency effects) must ensure every pixel has a depth value.

Interaction systems: from controllers to hands

visionOS interaction relies on hands and eyes. Unity provides three layers of tools for adaptation:

XR Interaction Toolkit (XRI)

(06:09) XRI is a high-level interaction system that abstracts input types, letting interaction code work across platforms. It supports 3D and UI objects, responding to hover, grab, select, and other common interactions.

Core components:

• Interactable: Objects in the scene that receive input. Three built-in types: Simple (receives interaction events), Grab (follows the interactor when selected), Teleport (defines teleport areas/points)
• Interactor: How to interact with Interactables. Direct Interactor (touch to select), Ray Interactor (long-range ray selection, curved/straight), Socket Interactor (defines areas that accept objects), Poke Interactor (direction-filtered poke interaction), Gaze Interactor (gaze interaction, automatically enlarges colliders)
• Interaction Manager: Coordinates interaction state between Interactors and Interactables
• XR Controller: Passes input actions to Interactors

Unity Input System

(11:35) Use system gesture input directly. Pinch gestures are passed as values containing position and rotation. Gaze direction and pinch gestures are delivered in the same frame.

Unity Hands Package

(12:12) Provides low-level hand joint data with consistent cross-platform access. Use it to detect gestures (thumbs up, pointing index finger) or drive custom hand meshes.

Detailed Content

Detecting whether the index finger is extended

(12:46)

static bool IsIndexExtended(XRHand hand)
{
    if (!(hand.GetJoint(XRHandJointID.Wrist).TryGetPose(out var wristPose) &&
          hand.GetJoint(XRHandJointID.IndexTip).TryGetPose(out var tipPose) &&
          hand.GetJoint(XRHandJointID.IndexIntermediate).TryGetPose(out var intermediatePose)))
    {
        return false;
    }

    var wristToTip = tipPose.position - wristPose.position;
    var wristToIntermediate = intermediatePose.position - wristPose.position;
    return wristToTip.sqrMagnitude > wristToIntermediate.sqrMagnitude;
}

Key points:

• XRHandJointID.Wrist, IndexTip, IndexIntermediate get three key joints
• TryGetPose gets joint pose; returns false when joint data is invalid
• Calculate vectors from wrist to fingertip and wrist to intermediate joint
• Compare squared distances: if fingertip distance is greater than intermediate joint distance, the index finger is extended
• Extend this logic to other fingers for basic gesture detection
• Call in OnHandUpdate events, passing left or right hand

XRI Interaction Manager configuration

The Interaction Manager is the intermediary between Interactors and Interactables:

// Usually one Interaction Manager handles all interactions
// You can also configure multiple Managers for different scenes/menus

// Example: separate Managers for gameplay and menu
public InteractionManager gameplayManager;
public InteractionManager menuManager;

void EnterGameplay() {
    gameplayManager.enabled = true;
    menuManager.enabled = false;
}

void EnterMenu() {
    gameplayManager.enabled = false;
    menuManager.enabled = true;
}

Key points:

• A single Interaction Manager is usually sufficient
• Multiple complementary Managers can activate/deactivate specific interaction sets
• Example: gameplay scene and menu use different Interactable sets

Custom hand meshes

RecRoom uses raw hand joint data to drive stylized hand models:

// Get all joint positions
foreach (XRHandJointID jointID in Enum.GetValues(typeof(XRHandJointID)))
{
    if (hand.GetJoint(jointID).TryGetPose(out var pose))
    {
        // Map joint positions to custom bones
        customHandMesh.SetJointPosition(jointID, pose.position);
        customHandMesh.SetJointRotation(jointID, pose.rotation);
    }
}

Key points:

• Unity Hands Package provides consistent cross-platform joint data
• Can drive any custom hand model
• Unifies hand visual style with overall game style
• Can also display other players’ hand models to enhance immersion

Depth buffer checklist

Effect type	Depth write requirement	Notes
Opaque objects	Must write	Unity built-in shaders fixed
Skybox	Needs modification	Infinite distance object, reverse Z writes 0
Transparent/water	Ensure values exist	Custom effects need checking
Particle systems	Check settings	Some particles may skip depth writes
Post-processing	No impact	Runs after depth testing

Core Takeaways

1. Port existing Unity VR games to visionOS

• What to do: Bring Unity games from SteamVR or Oculus to visionOS
• Why it’s worth it: Workflow is nearly unchanged: select target, enable XR Plug-in, generate Xcode project. URP automatically gets foveated rendering; single-pass instanced rendering reduces CPU overhead
• How to start: Upgrade to Unity 2022, switch to URP, check custom shader depth writes, replace controller-specific code with XRI, iterate testing on the simulator

2. Design new game mechanics for gesture interaction

• What to do: Design a VR puzzle or action game natively supporting gestures
• Why it’s worth it: visionOS hand tracking is accurate with low latency. XRI’s Poke Interactor and Grab Interactor make natural gestures direct game input. No controllers means lower barrier to entry
• How to start: Use XRI’s Grab Interactor for object grabbing, Poke Interactor for button presses, Unity Hands Package to detect specific gestures triggering special abilities

3. Build immersive virtual training simulators

• What to do: Develop VR simulation apps for medical, industrial, or safety training
• Why it’s worth it: Fully immersive space lets users focus on training content. Hand tracking makes operating virtual tools feel close to real. Foveated rendering ensures key details stay clear
• How to start: Create an Immersive Space set to Full Immersive style, use XRI’s Socket Interactor for tool docking, Ray Interactor for long-range selection

4. Create social VR spaces

• What to do: Build a multiplayer virtual gathering or collaboration space
• Why it’s worth it: RecRoom has proven this path works. Unity networking can be reused; XRI’s interaction system abstracts input types so cross-platform players can interact together
• How to start: Reuse existing Unity multiplayer networking code, display player hands with custom hand meshes, use Gaze Interactor to make gaze selection easier to hit

Related Sessions

• Create immersive Unity apps — Immersive Unity app development in the visionOS Shared Space
• Build great games for spatial computing — Overview of spatial computing game development techniques
• Discover Metal for immersive apps — Direct Metal rendering for fully immersive visionOS experiences