This skill covers building VR and XR applications using Unity, the most widely-used engine for XR development. It addresses Unity's XR Interaction Toolkit, OpenXR integration, performance profiling, rendering pipelines, and platform-specific deployment for Meta Quest, Apple Vision Pro, and PC VR headsets.

## Key Points

1. Create project with "3D (URP)" template
2. Install packages via Package Manager:
3. Configure XR Plugin Management:
4. Configure Rendering:
5. Configure Quality Settings:
6. Scene Setup:
1. On XR Origin:
2. On Controller:
3. In Scene:
1. Profile on target device (not in Editor)
2. Use Development Build + Autoconnect Profiler
3. Check GPU section (often the bottleneck in VR)

Unity VR/XR Development

Purpose

This skill covers building VR and XR applications using Unity, the most widely-used engine for XR development. It addresses Unity's XR Interaction Toolkit, OpenXR integration, performance profiling, rendering pipelines, and platform-specific deployment for Meta Quest, Apple Vision Pro, and PC VR headsets.

Unity XR Architecture

Package Ecosystem

Unity XR Stack:
┌─────────────────────────────────────────────┐
│ Application Code (your scripts)             │
├─────────────────────────────────────────────┤
│ XR Interaction Toolkit (XRI)                │
│ ├── Interactors (ray, direct, poke, gaze)   │
│ ├── Interactables (grab, simple, teleport)  │
│ ├── Locomotion (teleport, snap turn, move)  │
│ └── UI interaction (tracked device input)   │
├─────────────────────────────────────────────┤
│ XR Hands / XR Controller subsystems         │
├─────────────────────────────────────────────┤
│ Unity OpenXR Plugin                         │
│ ├── Meta Quest Feature Group                │
│ ├── Microsoft Mixed Reality Feature Group   │
│ └── Other platform feature groups           │
├─────────────────────────────────────────────┤
│ XR Management (loader, lifecycle)           │
└─────────────────────────────────────────────┘

Project Setup Checklist

New Unity XR Project Setup:
1. Create project with "3D (URP)" template
   └── URP is recommended; Built-in RP has limited XR features

2. Install packages via Package Manager:
   □ XR Plugin Management
   □ OpenXR Plugin
   □ XR Interaction Toolkit
   □ XR Hands (for hand tracking)
   □ Unity OpenXR Meta (for Quest)
   □ Shader Graph (comes with URP)

3. Configure XR Plugin Management:
   □ Project Settings → XR Plug-in Management
   □ Enable OpenXR for target platforms
   □ Add Interaction Profiles:
     ├── Meta Quest Touch Pro Controller
     ├── Oculus Touch Controller
     └── HTC Vive Controller (if needed)

4. Configure Rendering:
   □ URP Asset: Single Pass Instanced rendering
   □ Anti-aliasing: MSAA 4x
   □ HDR: Off for mobile, optional for PC
   □ Shadow resolution: 1024 mobile, 2048+ PC

5. Configure Quality Settings:
   □ Create separate quality levels for Quest / PC
   □ VSync: Don't Sync (XR runtime handles vsync)
   □ Target frame rate: 72/90/120 per platform

6. Scene Setup:
   □ Add XR Origin (Action-based)
   □ Configure Input Action Manager
   □ Set tracking origin mode (Floor or Device)
   □ Add locomotion system

XR Origin and Camera Setup

XR Origin Hierarchy:
XR Origin (XR Origin component)
├── Camera Offset (height offset for tracking origin)
│   ├── Main Camera (tracked pose driver, camera)
│   ├── Left Controller (action-based controller)
│   │   ├── Controller Model (auto-loaded or custom)
│   │   ├── Ray Interactor (far interaction)
│   │   ├── Direct Interactor (near interaction)
│   │   └── Poke Interactor (UI touch)
│   ├── Right Controller (same structure)
│   ├── Left Hand (XR Hand, hand tracking)
│   └── Right Hand (XR Hand, hand tracking)
├── Locomotion System
│   ├── Teleportation Provider
│   ├── Snap Turn Provider
│   └── Continuous Move Provider
└── Interaction Manager (XR Interaction Manager)

Input System

Action-Based Input

Unity's new Input System with XR actions:

using UnityEngine;
using UnityEngine.InputSystem;
using UnityEngine.XR.Interaction.Toolkit;

public class VRInputExample : MonoBehaviour
{
    [SerializeField] private InputActionReference triggerAction;
    [SerializeField] private InputActionReference gripAction;
    [SerializeField] private InputActionReference thumbstickAction;
    [SerializeField] private InputActionReference primaryButtonAction;

    private void OnEnable()
    {
        triggerAction.action.performed += OnTriggerPressed;
        triggerAction.action.canceled += OnTriggerReleased;
        gripAction.action.performed += OnGripPressed;
        primaryButtonAction.action.performed += OnPrimaryButton;
    }

    private void OnDisable()
    {
        triggerAction.action.performed -= OnTriggerPressed;
        triggerAction.action.canceled -= OnTriggerReleased;
        gripAction.action.performed -= OnGripPressed;
        primaryButtonAction.action.performed -= OnPrimaryButton;
    }

    private void Update()
    {
        // Continuous reading for thumbstick
        Vector2 thumbstick = thumbstickAction.action.ReadValue<Vector2>();
        if (thumbstick.magnitude > 0.1f)
        {
            // Handle movement
        }
    }

    private void OnTriggerPressed(InputAction.CallbackContext ctx)
    {
        float value = ctx.ReadValue<float>(); // 0.0 - 1.0
    }

    private void OnTriggerReleased(InputAction.CallbackContext ctx) { }
    private void OnGripPressed(InputAction.CallbackContext ctx) { }
    private void OnPrimaryButton(InputAction.CallbackContext ctx) { }
}

Hand Tracking in Unity

using UnityEngine;
using UnityEngine.XR.Hands;

public class HandTrackingExample : MonoBehaviour
{
    private XRHandSubsystem handSubsystem;

    void Start()
    {
        var subsystems = new List<XRHandSubsystem>();
        SubsystemManager.GetSubsystems(subsystems);
        if (subsystems.Count > 0)
            handSubsystem = subsystems[0];
    }

    void Update()
    {
        if (handSubsystem == null) return;

        XRHand leftHand = handSubsystem.leftHand;
        if (leftHand.isTracked)
        {
            // Get specific joint
            XRHandJoint indexTip = leftHand.GetJoint(XRHandJointID.IndexTip);
            if (indexTip.TryGetPose(out Pose pose))
            {
                Vector3 position = pose.position;
                Quaternion rotation = pose.rotation;
                // Use joint data
            }

            // Pinch detection
            XRHandJoint thumbTip = leftHand.GetJoint(XRHandJointID.ThumbTip);
            if (thumbTip.TryGetPose(out Pose thumbPose) &&
                indexTip.TryGetPose(out Pose indexPose))
            {
                float pinchDistance = Vector3.Distance(
                    thumbPose.position, indexPose.position);
                bool isPinching = pinchDistance < 0.02f;
            }
        }
    }
}

XR Interaction Toolkit Patterns

Grab Interaction

using UnityEngine;
using UnityEngine.XR.Interaction.Toolkit;

// Attach to any object you want to be grabbable
[RequireComponent(typeof(Rigidbody))]
public class GrabbableObject : XRGrabInteractable
{
    [SerializeField] private AudioClip grabSound;
    [SerializeField] private AudioClip releaseSound;

    protected override void OnSelectEntered(SelectEnterEventArgs args)
    {
        base.OnSelectEntered(args);

        // Haptic feedback on grab
        if (args.interactorObject is XRBaseControllerInteractor controller)
        {
            controller.SendHapticImpulse(0.5f, 0.1f);
        }

        AudioSource.PlayClipAtPoint(grabSound, transform.position);
    }

    protected override void OnSelectExited(SelectExitEventArgs args)
    {
        base.OnSelectExited(args);
        AudioSource.PlayClipAtPoint(releaseSound, transform.position);
    }
}

// Inspector configuration:
// Movement Type: Velocity Tracking (physics-based, most natural)
//   or Instantaneous (snappy, no physics)
//   or Kinematic (balanced)
// Throw On Detach: true
// Attach Transform: set grab point (or leave null for center)

Teleportation Setup

Teleportation System Components:

1. On XR Origin:
   └── Teleportation Provider (manages teleport execution)

2. On Controller:
   └── XR Ray Interactor (configured for teleport)
       ├── Line Type: Projectile Curve
       ├── Valid Color: Green
       ├── Invalid Color: Red
       └── Interaction Layer: "Teleport"

3. In Scene:
   ├── Teleportation Area (walkable surfaces)
   │   ├── Collider matching floor geometry
   │   ├── Interaction Layer: "Teleport"
   │   └── Teleport Anchor (optional snap points)
   └── Teleportation Anchor (specific positions)
       ├── Transform defines landing position + rotation
       └── Visual indicator (ring, pad)

Socket Interaction

// Socket: a slot where specific objects can be placed
// Example: putting a battery into a device
public class BatterySocket : XRSocketInteractor
{
    [SerializeField] private string requiredTag = "Battery";

    public override bool CanSelect(IXRSelectInteractable interactable)
    {
        // Only accept objects tagged as "Battery"
        return base.CanSelect(interactable) &&
               interactable.transform.CompareTag(requiredTag);
    }

    protected override void OnSelectEntered(SelectEnterEventArgs args)
    {
        base.OnSelectEntered(args);
        // Battery inserted — trigger game logic
        GetComponentInParent<Device>().OnBatteryInserted();
    }
}

Rendering Pipeline Configuration

URP for VR

URP Settings for VR:
┌─────────────────────────┬──────────┬──────────┐
│ Setting                 │ Quest    │ PC VR    │
├─────────────────────────┼──────────┼──────────┤
│ Render Scale            │ 0.8-1.0  │ 1.0-1.5  │
│ MSAA                    │ 4x       │ 4x       │
│ HDR                     │ Off      │ Optional │
│ Depth Texture           │ Off*     │ On       │
│ Opaque Texture          │ Off      │ Optional │
│ Main Light Shadows      │ On       │ On       │
│ Shadow Resolution       │ 512-1024 │ 2048     │
│ Shadow Cascades         │ 1        │ 2-4      │
│ Additional Lights       │ 0-2      │ 4-8      │
│ Reflection Probes       │ Few      │ Many     │
│ Post Processing         │ Minimal  │ Full     │
│ Rendering Path          │ Forward  │ Forward+ │
│ Stereo Rendering        │ Single Pass Instanced │
└─────────────────────────┴──────────┴──────────┘
*Enable if needed for specific effects

Shader Considerations

// VR-compatible shader checklist:
// □ Supports Single Pass Instanced rendering
// □ Uses unity_StereoEyeIndex for per-eye effects
// □ No screen-space effects that break in stereo
// □ Efficient for target GPU

// Single Pass Instanced compatibility in custom shaders:
// Include this in vertex shader:
// UNITY_SETUP_INSTANCE_ID(input);
// UNITY_INITIALIZE_OUTPUT(v2f, output);
// UNITY_INITIALIZE_VERTEX_OUTPUT_STEREO(output);

// In fragment shader:
// UNITY_SETUP_STEREO_EYE_INDEX_POST_VERTEX(input);

Performance Profiling

Unity Profiler for VR

Key Profiling Metrics:
├── CPU Main Thread: < 11ms (90 Hz) or < 8.3ms (120 Hz)
├── CPU Render Thread: < 11ms
├── GPU: < 11ms
├── Draw Calls: < 100 (Quest), < 500 (PC)
├── Triangles: < 100K (Quest), < 500K (PC)
├── SetPass Calls: < 50 (Quest), < 200 (PC)
└── Memory: < 2GB (Quest), < 6GB (PC)

Profiling Workflow:
1. Profile on target device (not in Editor)
2. Use Development Build + Autoconnect Profiler
3. Check GPU section (often the bottleneck in VR)
4. Look for spikes, not just averages
5. Profile with content density matching final product

Common Performance Issues:
├── Shader compilation stutter → Pre-warm shaders
├── GC allocation spikes → Eliminate per-frame allocations
├── Physics too complex → Simplify colliders, reduce rigidbodies
├── Too many real-time lights → Bake lighting, limit dynamic lights
├── Overdraw from transparency → Minimize transparent surfaces
└── Expensive post-processing → Disable or simplify for mobile

Meta Quest-Specific Profiling

OVR Metrics Tool:
- Overlay showing real-time performance metrics
- CPU/GPU utilization, frame timing, thermal state
- Enable via Meta Quest Developer Hub

Key Quest Metrics:
├── CPU Level: 0-4 (auto-adjusts clock speed)
├── GPU Level: 0-5
├── FPS: Must be consistently at target
├── App CPU Time: < frame budget
├── App GPU Time: < frame budget
├── Compositor Time: Ideally < 1ms
└── Thermal: Watch for thermal throttling

OVRPlugin Performance Hints:
// Suggest performance level to runtime
OVRManager.suggestedCpuPerfLevel = OVRManager.ProcessorPerformanceLevel.SustainedLow;
OVRManager.suggestedGpuPerfLevel = OVRManager.ProcessorPerformanceLevel.SustainedLow;

Platform Deployment

Quest Build Settings

Quest Build Configuration:
├── Platform: Android
├── Texture Compression: ASTC
├── Minimum API Level: 29 (Android 10)
├── Target API Level: 32+
├── Scripting Backend: IL2CPP
├── Target Architecture: ARM64
├── Managed Stripping Level: High
├── Color Space: Linear
├── Graphics API: Vulkan (preferred) or OpenGL ES 3.0
└── Install Location: Auto

Manifest Requirements (AndroidManifest.xml):
<uses-feature android:name="android.hardware.vr.headtracking"
              android:required="true" />
<category android:name="com.oculus.intent.category.VR" />

Quest-Specific Settings:
├── Target devices: Quest 2, Quest 3, Quest Pro
├── Refresh rate: 72/90/120 Hz options
├── Foveated rendering: Enable (significant GPU savings)
├── Application SpaceWarp: Enable for complex scenes
└── Scene API: Enable for mixed reality

PC VR Build Settings

PC VR Configuration:
├── Platform: Windows (Standalone)
├── Graphics API: DirectX 12 (or 11)
├── Color Space: Linear
├── Scripting Backend: IL2CPP (for performance)
├── OpenXR runtime: SteamVR or Oculus (auto-detected)
└── Quality settings: Allow user adjustment

SteamVR considerations:
├── Include SteamVR Input action manifest
├── Test with different resolution scales
├── Support varied GPU capabilities
└── Binding UI for custom controller mapping

Common Patterns

Object Highlighting

public class HighlightOnHover : MonoBehaviour
{
    [SerializeField] private Material highlightMaterial;
    private Material originalMaterial;
    private MeshRenderer meshRenderer;
    private XRBaseInteractable interactable;

    void Awake()
    {
        meshRenderer = GetComponent<MeshRenderer>();
        originalMaterial = meshRenderer.material;
        interactable = GetComponent<XRBaseInteractable>();

        interactable.hoverEntered.AddListener(OnHoverEnter);
        interactable.hoverExited.AddListener(OnHoverExit);
    }

    void OnHoverEnter(HoverEnterEventArgs args)
    {
        meshRenderer.material = highlightMaterial;
        // Haptic pulse
        if (args.interactorObject is XRBaseControllerInteractor ctrl)
            ctrl.SendHapticImpulse(0.2f, 0.05f);
    }

    void OnHoverExit(HoverExitEventArgs args)
    {
        meshRenderer.material = originalMaterial;
    }
}

VR UI Canvas Setup

World-Space Canvas for VR:
├── Canvas
│   ├── Render Mode: World Space
│   ├── Scale: 0.001 (1 Unity unit per 1000 canvas pixels)
│   ├── Dynamic Pixels Per Unit: 1
│   ├── Add: Tracked Device Graphic Raycaster
│   └── Remove: Graphic Raycaster (2D, not for VR)
├── Event System
│   ├── Add: XR UI Input Module
│   └── Remove: Standalone Input Module
└── On controllers:
    └── XR Ray Interactor with "UI" interaction layer

Canvas Distance: 1-2m from player
Canvas Size: ~1m wide for comfortable reading
Font Size: 24+ for readability at VR distances

When to Apply This Skill

Use this skill when:

• Setting up a new Unity VR/XR project
• Implementing VR interactions (grab, teleport, UI)
• Configuring rendering for VR performance targets
• Deploying to Meta Quest or PC VR platforms
• Debugging VR-specific issues in Unity
• Integrating hand tracking in Unity XR projects