This skill covers VR and XR development with Unreal Engine 5, leveraging its advanced rendering features (Nanite, Lumen, Virtual Shadow Maps) while meeting VR performance constraints. It addresses project configuration, interaction framework setup, optimization strategies specific to UE5's architecture, and deployment to major VR platforms.

## Key Points

1. Create Project:
2. Project Settings:
3. Quality Scalability:
- Motion Controller Component: Automatically tracks controller/hand
- Widget Interaction: Enables interaction with UMG UI in world space
- Enhanced Input: UE5's action-based input system
1. Line Trace Setup:
2. Valid Landing Check:
3. Execute Teleport:
4. Snap Turn:
1. □ Reduce overdraw (minimize transparent materials)
2. □ Enable foveated rendering

Unreal Engine VR/XR Development

Purpose

This skill covers VR and XR development with Unreal Engine 5, leveraging its advanced rendering features (Nanite, Lumen, Virtual Shadow Maps) while meeting VR performance constraints. It addresses project configuration, interaction framework setup, optimization strategies specific to UE5's architecture, and deployment to major VR platforms.

Project Setup

Template and Configuration

UE5 VR Project Setup:

1. Create Project:
   ├── Template: VR Template (includes pre-built VR pawn and interactions)
   │   OR
   ├── Template: Blank → manually configure for VR
   └── Target platform: Mobile (Quest) or Desktop (PC VR)

2. Project Settings:
   ├── Platforms → XR:
   │   ├── Enable OpenXR Plugin
   │   ├── Enable OpenXR Hand Tracking
   │   └── Add Interaction Profiles (Meta Quest Touch, etc.)
   ├── Engine → Rendering:
   │   ├── Forward Shading: ON (mobile VR) or OFF (PC VR with deferred)
   │   ├── Mobile Multi-View: ON (stereo rendering)
   │   ├── Instanced Stereo: ON (PC VR)
   │   ├── Mobile HDR: OFF (Quest)
   │   ├── Anti-Aliasing: MSAA 4x (forward), TAA (deferred)
   │   └── VR Mode: Check "Start in VR"
   ├── Engine → General Settings:
   │   ├── Framerate: Uncapped (XR runtime handles vsync)
   │   └── Use Fixed Frame Rate: OFF
   └── Plugins:
       ├── OpenXR
       ├── OpenXR Hand Tracking
       ├── Meta XR (for Quest features)
       └── SteamVR (for PC VR via SteamVR)

3. Quality Scalability:
   ├── Create VR-specific scalability profiles
   ├── Disable expensive defaults (screen-space reflections, etc.)
   └── Set up platform-specific configs (Quest vs PC)

VR Pawn Architecture

VR Pawn Blueprint Structure:
BP_VRPawn (Character or Pawn)
├── VROrigin (Scene Component — tracking origin)
│   ├── Camera (Camera Component — auto-tracked to HMD)
│   ├── MotionController_Left (Motion Controller Component)
│   │   ├── HandMesh_Left (Skeletal Mesh — controller or hand model)
│   │   ├── GrabSphere_Left (Sphere Collision — grab detection)
│   │   ├── WidgetInteraction_Left (Widget Interaction — UI)
│   │   └── LaserBeam_Left (Static Mesh — interaction ray visual)
│   └── MotionController_Right (mirrored structure)
├── Capsule Component (collision, optional)
└── Character Movement Component (if using Character base)

Key Components:
- Motion Controller Component: Automatically tracks controller/hand
- Widget Interaction: Enables interaction with UMG UI in world space
- Enhanced Input: UE5's action-based input system

Enhanced Input for VR

Input Mapping

// VR Input Action definitions
// Create these as Data Assets:

// IA_Grab (Boolean)
// IA_Trigger (Axis1D)
// IA_Thumbstick (Axis2D)
// IA_PrimaryButton (Boolean)
// IA_Menu (Boolean)

// Input Mapping Context — binds actions to hardware
// IMC_VRDefault:
//   IA_Grab:
//     Left Hand:  OpenXR Left Grip
//     Right Hand: OpenXR Right Grip
//   IA_Trigger:
//     Left Hand:  OpenXR Left Trigger
//     Right Hand: OpenXR Right Trigger
//   IA_Thumbstick:
//     Left Hand:  OpenXR Left Thumbstick
//     Right Hand: OpenXR Right Thumbstick

// C++ VR Input handling
void AVRPawn::SetupPlayerInputComponent(UInputComponent* PlayerInputComponent)
{
    Super::SetupPlayerInputComponent(PlayerInputComponent);

    UEnhancedInputComponent* EIC = Cast<UEnhancedInputComponent>(PlayerInputComponent);
    if (EIC)
    {
        // Grab actions
        EIC->BindAction(GrabAction_Left, ETriggerEvent::Started, this,
            &AVRPawn::OnGrabLeft);
        EIC->BindAction(GrabAction_Left, ETriggerEvent::Completed, this,
            &AVRPawn::OnReleaseLeft);

        // Trigger (analog)
        EIC->BindAction(TriggerAction_Right, ETriggerEvent::Triggered, this,
            &AVRPawn::OnTriggerRight);

        // Thumbstick locomotion
        EIC->BindAction(ThumbstickAction_Left, ETriggerEvent::Triggered, this,
            &AVRPawn::OnMoveInput);
    }
}

void AVRPawn::OnGrabLeft(const FInputActionValue& Value)
{
    TryGrabNearestObject(EControllerHand::Left);
}

void AVRPawn::OnMoveInput(const FInputActionValue& Value)
{
    FVector2D Input = Value.Get<FVector2D>();
    FVector Forward = Camera->GetForwardVector();
    Forward.Z = 0; Forward.Normalize();
    FVector Right = Camera->GetRightVector();
    Right.Z = 0; Right.Normalize();

    AddMovementInput(Forward * Input.Y + Right * Input.X);
}

Interaction System

Grab System

// VR Grab Component
UCLASS()
class UVRGrabComponent : public UActorComponent
{
    GENERATED_BODY()

public:
    void TryGrab(UMotionControllerComponent* Controller)
    {
        // Overlap check
        TArray<FOverlapResult> Overlaps;
        FCollisionShape Sphere = FCollisionShape::MakeSphere(GrabRadius);

        bool bHit = GetWorld()->OverlapMultiByChannel(
            Overlaps,
            Controller->GetComponentLocation(),
            FQuat::Identity,
            ECC_PhysicsBody,
            Sphere
        );

        if (bHit)
        {
            // Find closest grabbable
            AActor* Nearest = FindNearestGrabbable(Overlaps);
            if (Nearest)
            {
                // Attach to controller
                UPrimitiveComponent* GrabComp = Nearest->FindComponentByClass<UPrimitiveComponent>();
                GrabComp->SetSimulatePhysics(false);
                Nearest->AttachToComponent(Controller,
                    FAttachmentTransformRules::KeepWorldTransform);

                HeldActor = Nearest;

                // Haptic feedback
                Controller->GetPlayerController()->PlayHapticEffect(
                    GrabHaptic, Controller->GetTrackingSource());
            }
        }
    }

    void Release(UMotionControllerComponent* Controller)
    {
        if (HeldActor)
        {
            HeldActor->DetachFromActor(FDetachmentTransformRules::KeepWorldTransform);
            UPrimitiveComponent* Comp = HeldActor->FindComponentByClass<UPrimitiveComponent>();
            Comp->SetSimulatePhysics(true);

            // Apply throw velocity
            Comp->SetPhysicsLinearVelocity(Controller->GetComponentVelocity());

            HeldActor = nullptr;
        }
    }

private:
    UPROPERTY() AActor* HeldActor = nullptr;
    float GrabRadius = 10.0f;
    UPROPERTY() UHapticFeedbackEffect_Base* GrabHaptic;
};

Teleportation

Teleportation System (Blueprint approach):

1. Line Trace Setup:
   ├── Start: Controller location
   ├── Direction: Controller forward
   ├── Type: Projectile path (parabolic arc)
   ├── Collision: NavMesh query at landing point
   └── Visual: Niagara particle beam or spline mesh

2. Valid Landing Check:
   ├── Project point to NavMesh
   ├── Check slope angle (< 30 degrees)
   ├── Check ceiling clearance (> player height)
   └── Show green/red indicator at landing

3. Execute Teleport:
   ├── Fade screen to black (0.1s)
   ├── Move VR Origin to target location
   ├── Maintain relative head position within play space
   └── Fade back in (0.1s)

4. Snap Turn:
   ├── Thumbstick right/left past threshold (0.7)
   ├── Rotate VR Origin by 30-45 degrees
   ├── Brief screen fade (optional, reduces discomfort)
   └── Cooldown to prevent rapid repeated turns

UE5 Rendering for VR

Nanite in VR

Nanite for VR — Considerations:
├── Supported: PC VR (requires modern GPU)
├── NOT supported: Quest standalone (no Nanite on mobile)
├── Benefits:
│   ├── Automatic LOD — no manual LOD chains needed
│   ├── Massive triangle counts without draw call cost
│   └── Virtual geometry streaming
├── VR Challenges:
│   ├── Higher base GPU cost than traditional meshes
│   ├── No MSAA support (requires TSR or FXAA)
│   ├── Stereo rendering doubles the work
│   └── Must profile actual VR frame times
└── Recommendation:
    ├── Use Nanite for environment geometry on PC VR
    ├── Use traditional meshes for mobile VR
    └── Test performance with both eyes rendering

Lumen in VR

Lumen for VR — Considerations:
├── Hardware Ray Tracing Lumen: Too expensive for VR currently
├── Software Lumen: Marginal for high-end PC VR
├── Alternative for VR:
│   ├── Baked lighting (Lightmass or GPU Lightmass)
│   ├── Light probes for dynamic objects
│   ├── Simple real-time lights (1-2 dynamic)
│   └── SSAO for contact shadows (cheap, effective)
└── For Quest:
    ├── Fully baked lighting only
    ├── Light probes for avatars
    ├── No real-time shadows (or 1 directional cascade)
    └── Lightmap resolution: 32-64 for most surfaces

Fixed Foveated Rendering

Foveated Rendering (Quest):
├── Center: Full resolution
├── Inner ring: 50% resolution
├── Outer ring: 25% resolution
├── Savings: 30-50% GPU fill rate

// Enable in C++:
#include "OculusXRFunctionLibrary.h"
UOculusXRFunctionLibrary::SetFoveatedRenderingLevel(
    EOculusXRFoveatedRenderingLevel::HighTop);

Eye-Tracked Foveated Rendering (Quest Pro, PSVR2):
├── Foveal region follows gaze direction
├── Higher quality savings (40-60%)
├── Requires eye tracking permission
└── Fallback to fixed foveation if tracking lost

Performance Optimization

Profiling Tools

UE5 VR Profiling Toolkit:
├── Unreal Insights: Detailed CPU/GPU trace
│   └── Launch with: -trace=default,gpu -statnamedevents
├── stat unit: Frame time breakdown (CPU game, CPU draw, GPU)
├── stat GPU: Per-pass GPU timing
├── RenderDoc: GPU capture for shader debugging
├── Meta Quest Developer Hub:
│   ├── OVR Metrics Tool overlay
│   ├── GPU profiler
│   └── Thermal monitoring
└── stat RHI: Draw calls, triangles, texture memory

Console Commands for VR:
  stat unit                    — Frame timing
  stat gpu                     — GPU pass breakdown
  r.ScreenPercentage 80        — Reduce render resolution
  r.MobileContentScaleFactor   — Quest render scale
  ProfileGPU                   — One-frame GPU profile
  stat scenerendering          — Scene render stats

Optimization Checklist

UE5 VR Optimization Priority List:

GPU-Bound (most common in VR):
1. □ Reduce overdraw (minimize transparent materials)
2. □ Enable foveated rendering
3. □ Lower shadow quality/resolution
4. □ Reduce post-processing (remove DOF, motion blur, SSR)
5. □ Use LODs aggressively (auto-generate with Nanite or manual)
6. □ Merge static meshes (fewer draw calls)
7. □ Use instanced static meshes for repeated geometry
8. □ Simplify materials (fewer texture samples, simpler math)
9. □ Reduce reflection capture density
10. □ Lower render resolution (r.ScreenPercentage)

CPU-Bound:
1. □ Reduce tick frequency on non-critical actors
2. □ Use async traces instead of sync
3. □ Limit physics bodies in scene
4. □ Use LOD for skeletal mesh bone count
5. □ Cull invisible actors (set cull distance)
6. □ Batch AI/logic updates across frames
7. □ Use HLOD for level streaming
8. □ Reduce Blueprint tick usage (use timers instead)

Memory (Quest-specific):
1. □ Texture streaming with aggressive pool size
2. □ ASTC compression on all textures
3. □ Audio compression (Vorbis for music, ADPCM for SFX)
4. □ Level streaming to limit loaded content
5. □ Skeletal mesh LOD with reduced bone counts

Quest Deployment

Android Build Setup

Quest Build Configuration:
├── Platform: Android
├── Build Target: Android (ASTC)
├── Architectures: arm64 only
├── Minimum SDK: 29
├── Target SDK: 32
├── Package Game: ON
├── Cook everything: Evaluate per-project

Required Plugins:
├── OpenXR
├── Meta XR (provides Quest-specific features)
├── OpenXR Hand Tracking
└── Android Runtime Settings configured for Quest

AndroidManifest additions:
- VR intent filter
- Hand tracking permission (if used)
- Scene API permission (if mixed reality)
- Passthrough permission (if mixed reality)

Build Process:
1. Package for Android (ASTC)
2. Output: .apk file
3. Install via ADB or Meta Quest Developer Hub:
   adb install -r YourGame.apk
4. Or upload to Meta Quest Developer Dashboard for distribution

Common Patterns

Distance Grab

Distance Grab (force-pull objects from afar):
1. Ray cast from controller
2. If ray hits grabbable object:
   a. Highlight object (outline post-process)
   b. Show "grab" icon on controller
3. On grip press:
   a. Object flies toward hand (lerp over 0.2-0.3s)
   b. Haptic ramp as object approaches
   c. Object snaps to hand, enters grabbed state
4. Physics: Disable on grabbed object during flight

Alternative: Force grip (point and object comes to you)
Alternative: Tractor beam (hold trigger, object moves with ray)

VR Menu System

Wrist Menu Pattern:
1. Detect "look at wrist" gesture:
   ├── Controller palm facing camera
   ├── AND camera looking at controller (dot product check)
   └── Threshold: palm-to-camera angle < 40 degrees
2. Spawn menu widget attached to wrist
3. Interact with opposite hand (ray or poke)
4. Menu follows wrist position
5. Dismiss when wrist turns away

World-Anchored Menu:
1. Spawn at controller position on menu button press
2. Menu stays in world space (doesn't follow controller)
3. Interact via ray or direct touch
4. Dismiss on button press or distance threshold

When to Apply This Skill

Use this skill when:

• Starting a VR project in Unreal Engine 5
• Configuring UE5 rendering features for VR performance
• Implementing VR interaction systems in UE5
• Optimizing an existing UE5 project for Quest deployment
• Deciding between UE5 rendering features (Nanite/Lumen) for VR
• Setting up the Enhanced Input system for VR controllers