Mixed Reality (MR) blends virtual content with the physical environment so they coexist and interact. This skill covers UX design principles specific to MR, where digital objects must respect real-world surfaces, lighting, and spatial relationships while remaining intuitive and useful. It bridges the gap between traditional UX design and spatial computing.

## Key Points

1. Surface Attachment:
2. Physical Behavior:
3. Scale Consistency:
4. Persistent Placement:
1. Subtle glow or outline change
2. Gentle animation (pulse, bob)
3. Color shift (neutral → highlighted)
4. Spatial audio cue from object direction
5. Object moves slightly toward user
6. Notification badge appears
7. Full animation + sound + haptic
1. First Launch:

Mixed Reality UX Design

Purpose

Mixed Reality (MR) blends virtual content with the physical environment so they coexist and interact. This skill covers UX design principles specific to MR, where digital objects must respect real-world surfaces, lighting, and spatial relationships while remaining intuitive and useful. It bridges the gap between traditional UX design and spatial computing.

MR Design Spectrum

The Virtuality Continuum

Reality-Virtuality Continuum:
Real         │ Augmented  │ Mixed      │ Augmented  │ Virtual
Environment  │ Reality    │ Reality    │ Virtuality │ Environment
─────────────┼────────────┼────────────┼────────────┼────────────
Physical     │ Digital    │ Digital &  │ Virtual    │ Fully
world only   │ overlays   │ physical   │ world with │ virtual
             │ on real    │ interact   │ real       │ world
             │ world      │ mutually   │ elements   │
             │            │            │            │
Phone AR     │            │ Quest MR   │            │ Quest VR
             │ HoloLens   │ Vision Pro │            │

Key MR UX distinction:
In MR, virtual objects don't just overlay — they INTERACT with
the real world. A virtual ball bounces off a real table.
A virtual character hides behind a real couch.

Design Modes

MR Application Modes:
┌──────────────────────────────────────────────────┐
│ Mode 1: Enhancement                              │
│ Real world is primary, digital enhances it       │
│ Example: Virtual instructions on a real machine  │
│ Design: Minimal, contextual, on-demand           │
├──────────────────────────────────────────────────┤
│ Mode 2: Coexistence                              │
│ Real and virtual share equal importance          │
│ Example: Virtual pet exploring your real room    │
│ Design: Virtual content respects real layout     │
├──────────────────────────────────────────────────┤
│ Mode 3: Transformation                           │
│ Virtual transforms the real space                │
│ Example: Room becomes a forest, furniture becomes│
│ rocks, floor becomes grass                       │
│ Design: Real geometry drives virtual appearance  │
├──────────────────────────────────────────────────┤
│ Mode 4: Portal                                   │
│ Windows/doorways into virtual worlds within real │
│ Example: A virtual portal on a wall reveals      │
│ another world behind it                          │
│ Design: Clear boundary between real and virtual  │
└──────────────────────────────────────────────────┘

Spatial UX Principles for MR

Grounding Virtual Objects

Making Virtual Objects Feel Real:

1. Surface Attachment:
   ├── Objects rest ON real surfaces (not floating)
   ├── Snap to detected planes with slight settling animation
   ├── Cast shadows onto real surfaces
   └── Show contact shadow/ground plane at object base

2. Physical Behavior:
   ├── Respect gravity (objects fall to real floor)
   ├── Collide with real walls and furniture
   ├── Occlude behind real objects (hide what's behind the couch)
   └── React to real lighting conditions

3. Scale Consistency:
   ├── Virtual objects sized correctly relative to real objects
   ├── A virtual coffee mug should be coffee-mug sized
   ├── Use real objects as reference (next to a real table)
   └── Avoid impossible scales (breaks immersion)

4. Persistent Placement:
   ├── Objects stay where placed between sessions
   ├── Anchored to room coordinates (not device coordinates)
   ├── Handle room changes gracefully (furniture moved)
   └── Allow manual repositioning always

Transition Design

Transitioning Between Real and Virtual:

Immersion Slider:
├── 100% Real: Full passthrough, no virtual content
├── 75% Real: Subtle virtual overlays, information, UI
├── 50% Mixed: Virtual objects integrated with real space
├── 25% Real: Virtual dominates, real world partially visible
└── 0% Real: Full VR immersion

Transition Techniques:
├── Fade: Smoothly blend passthrough opacity (simplest)
├── Dissolve: Pixel-level transition (cinematic)
├── Portal: Walk through a boundary into virtual (spatial)
├── Iris: Circular reveal expanding from a point (focused)
└── Room: Virtual walls grow from real walls (architectural)

Transition Timing:
├── Quick mode switch: 0.3-0.5 seconds
├── Immersive transition: 1-3 seconds
├── Story-driven transition: 3-10 seconds (cinematic)
└── NEVER instant (disorienting, especially in MR)

User Control:
├── User must initiate transitions (never auto-switch immersion)
├── Provide preview of destination mode
├── Allow abort during transition
├── Physical "panic button" (e.g., double-tap headset to exit)
└── Guardian/boundary system active during all modes

Spatial Hierarchy and Attention

Attention Design in MR:
┌─────────────────────────────────────────────────┐
│ The real world competes with virtual content     │
│ for user attention. Design accordingly.          │
│                                                  │
│ Attention Priority Stack:                        │
│ 1. Safety (real-world hazards always visible)   │
│ 2. Primary task (what user is trying to do)     │
│ 3. Contextual info (relevant to current action) │
│ 4. Ambient info (time, notifications)           │
│ 5. Decorative (visual enhancement)              │
│                                                  │
│ Rules:                                           │
│ - Lower priority NEVER obscures higher priority │
│ - Real-world hazards override ALL virtual content│
│ - User's current task gets most visual weight   │
│ - Use peripheral placement for low-priority info│
│ - Animate to draw attention only when needed    │
└─────────────────────────────────────────────────┘

Attention-Grabbing Techniques (from subtle to strong):
1. Subtle glow or outline change
2. Gentle animation (pulse, bob)
3. Color shift (neutral → highlighted)
4. Spatial audio cue from object direction
5. Object moves slightly toward user
6. Notification badge appears
7. Full animation + sound + haptic

Room Understanding UX

Adapting to Physical Spaces

Room-Aware Design:

Room Scanning UX:
1. First Launch:
   ├── Guide user through room scan
   ├── Show progress (% of room mapped)
   ├── Highlight unmapped areas
   ├── Celebrate completion
   └── Save room model for future sessions

2. Content Placement Rules:
   ├── Never place content where user can't reach it
   ├── Respect furniture (don't place inside couch)
   ├── Use walls for panels, tables for objects, floor for characters
   ├── Account for room size (small room = fewer/smaller elements)
   └── Handle room changes (new furniture detected)

3. Adaptive Layout:
   Small Room (< 3m x 3m):
   ├── Fewer virtual objects
   ├── Closer placement distances
   ├── Vertical stacking instead of horizontal spread
   └── Minimize standing movement required

   Large Room (> 5m x 5m):
   ├── Spread content across space
   ├── Create zones (work area, entertainment area)
   ├── Use waypoints for distant interaction
   └── Allow room-scale exploration

   Unusual Shapes:
   ├── Detect room boundaries
   ├── Place content within accessible bounds
   ├── Avoid corners and narrow passages
   └── Gracefully handle missing walls (open plan)

Furniture-Aware Interactions

Leveraging Detected Furniture:
┌─────────────────┬──────────────────────────────┐
│ Furniture       │ Virtual Content Opportunity   │
├─────────────────┼──────────────────────────────┤
│ Table/Desk      │ Board games, virtual screens, │
│                 │ workspace tools, miniatures   │
├─────────────────┼──────────────────────────────┤
│ Couch/Chair     │ Virtual companion sits beside,│
│                 │ entertainment center nearby   │
├─────────────────┼──────────────────────────────┤
│ Wall            │ Virtual paintings, shelves,    │
│                 │ windows to other worlds,      │
│                 │ information panels            │
├─────────────────┼──────────────────────────────┤
│ Floor           │ Characters walk on it, games  │
│                 │ spread across it, portals     │
├─────────────────┼──────────────────────────────┤
│ Window          │ Overlay virtual view, weather │
│                 │ effects, information display  │
├─────────────────┼──────────────────────────────┤
│ Door            │ Portal to virtual room,       │
│                 │ character entry point         │
└─────────────────┴──────────────────────────────┘

MR Interaction Patterns

Passthrough-Aware Interaction

Interacting with Both Real and Virtual:

Pattern: Virtual Tool on Real Object
1. User picks up virtual tool (e.g., magnifying glass)
2. Holds tool over real object
3. Virtual overlay appears on real object (zoom, x-ray, info)
4. Put down tool to dismiss overlay

Pattern: Real Object as Virtual Input
1. Recognize real object (e.g., book, card, toy)
2. Real object triggers virtual effect
3. Moving real object updates virtual content
4. Example: Place real book on table → virtual pop-up book emerges

Pattern: Virtual Character in Real Space
1. Virtual character aware of room layout
2. Walks on real floor, sits on real furniture
3. Navigates around real obstacles
4. Acknowledges real objects in dialogue
5. Hides behind real furniture (peek-a-boo)

Pattern: Reality Modification
1. Select real surface (wall, table)
2. Apply virtual texture/effect
3. Real geometry drives virtual appearance
4. Example: Point at wall → wall appears to turn to glass/water/forest

Safety in MR

MR Safety Design:

Physical Safety:
├── Guardian/boundary system always active
├── Real obstacles remain visible (don't fully occlude hazards)
├── Pets and children detection (warn user)
├── Passthrough quality check (warn if cameras degraded)
├── Motion alerts (moving objects in periphery)
└── Emergency passthrough (instant full real-world view)

Safety Visualization:
├── Guardian boundary: Visible when approaching room edges
├── Detected obstacles: Highlighted when user approaches
├── Floor level changes: Stairs/edges highlighted
├── People detection: Silhouette of real people through virtual
└── Exit signs: Always visible, never occluded by virtual content

Implementation:
// Fade virtual content near boundaries
float boundaryDistance = GetDistanceToBoundary(userPosition);
float fadeStart = 1.0f; // meters
float fadeEnd = 0.3f;   // meters
float alpha = Mathf.InverseLerp(fadeEnd, fadeStart, boundaryDistance);
virtualContent.SetAlpha(alpha);
// Near boundary: virtual fades, real world shows through

MR-Specific UI Components

Spatial Menus

Menu Patterns for MR:
├── Palm Menu:
│   ├── Flip non-dominant hand palm up
│   ├── Menu appears hovering above palm
│   ├── Other hand taps/pinches menu items
│   └── Dismiss by turning palm down
│
├── Wrist Menu:
│   ├── Look at wrist (like checking a watch)
│   ├── Quick-access actions visible
│   ├── Expand for full menu
│   └── Natural, low-fatigue interaction
│
├── Context Menu:
│   ├── Long-press or pinch-hold on object
│   ├── Radial menu appears around object
│   ├── Slide to option and release
│   └── Actions specific to selected object
│
└── Anchored Panel:
    ├── Pin a UI panel to a real-world surface
    ├── Panel stays on wall/table when user walks away
    ├── Interact when nearby, auto-minimize when far
    └── Multiple panels for multi-tasking

Mixed Reality Notifications

Notification Design for MR:
├── Priority Levels:
│   ├── Critical: Edge glow + spatial audio + haptic
│   │   └── Example: Battery low, boundary breached
│   ├── Important: Notification badge + audio
│   │   └── Example: Message from friend, meeting starting
│   ├── Informational: Subtle visual indicator
│   │   └── Example: Download complete, daily summary
│   └── Ambient: Environmental change only
│       └── Example: Weather update, time passing

├── Placement:
│   ├── Never center of view (blocks real world)
│   ├── Peripheral: Lower-left or lower-right
│   ├── Anchored: Near relevant object/app
│   └── Sound first: Audio cue, visual appears on glance

├── Duration:
│   ├── Critical: Persists until acknowledged
│   ├── Important: 5-10 seconds, then minimizes
│   ├── Informational: 3-5 seconds, then fades
│   └── Ambient: Continuous but extremely subtle

Testing MR Experiences

MR UX Testing Protocol:
├── Environment Diversity:
│   □ Small room (apartment)
│   □ Large room (open office)
│   □ Bright lighting (near windows)
│   □ Dim lighting (evening)
│   □ Cluttered room (lots of furniture)
│   □ Empty room (minimal surfaces)
│   □ Outdoors (if applicable)
│
├── User Testing Focus Areas:
│   □ Can users distinguish virtual from real? (when intended)
│   □ Do virtual objects feel "grounded" in the room?
│   □ Is text readable in all lighting conditions?
│   □ Do users feel safe moving around?
│   □ Are transitions between real/virtual comfortable?
│   □ Can users find and dismiss virtual content?
│   □ Does the experience work in differently shaped rooms?
│
├── Edge Cases:
│   □ Room scan incomplete (missing walls/floor)
│   □ Very bright or very dark environments
│   □ Moving objects in the room during use
│   □ Other people walking through the space
│   □ Pets in the room
│   □ Mirrors and glass surfaces (confuse tracking)
│   □ Featureless walls (limited tracking features)
│
└── Performance:
    □ Consistent frame rate with passthrough active
    □ Tracking stability over extended sessions
    □ Battery consumption acceptable
    □ Thermal comfort (device heat)

When to Apply This Skill

Use this skill when:

• Designing applications that blend virtual and real-world content
• Building mixed reality experiences for Quest or Vision Pro
• Creating room-aware applications that adapt to physical spaces
• Designing interaction patterns that span real and virtual objects
• Ensuring safety in mixed reality environments
• Testing MR experiences across diverse physical environments