This skill covers designing augmented reality overlays that blend digital content with the real world. Unlike VR (which replaces reality), AR must coexist with the physical environment, creating unique design challenges around readability, placement, occlusion, and context-awareness. This skill addresses both mobile AR (phone/tablet) and head-worn AR (smart glasses, headsets).

## Key Points

1. Detect context change
2. Fade out current overlay (200ms)
3. Brief pause (100ms) — avoid visual chaos
4. Fade in new overlay (300ms)
5. Provide anchor point (where does new content appear?)
6. Allow manual dismiss if context detection was wrong
1. Less is more — Show only relevant data, not everything available
2. Progressive disclosure — Summary first, details on focus/tap
3. Consistent placement — Same type of data always in same relative position
4. Color coding — Use consistent, accessible color scales
5. Update frequency — Match data refresh to actual change rate
- Designing AR information overlays for mobile or headset

AR Overlay Design

Purpose

This skill covers designing augmented reality overlays that blend digital content with the real world. Unlike VR (which replaces reality), AR must coexist with the physical environment, creating unique design challenges around readability, placement, occlusion, and context-awareness. This skill addresses both mobile AR (phone/tablet) and head-worn AR (smart glasses, headsets).

AR Display Constraints

Device Categories

AR Device Spectrum:
┌─────────────────────┬────────────┬───────────┬────────────┐
│ Device Type         │ FOV        │ Opacity   │ Input      │
├─────────────────────┼────────────┼───────────┼────────────┤
│ Phone/Tablet AR     │ Camera FOV │ Full      │ Touch      │
│ (ARKit/ARCore)      │ (~60°)     │ (screen)  │ screen     │
├─────────────────────┼────────────┼───────────┼────────────┤
│ Smart Glasses       │ 25-50°     │ Additive  │ Voice,     │
│ (lightweight)       │ (small)    │ (no black)│ tap, gaze  │
├─────────────────────┼────────────┼───────────┼────────────┤
│ AR Headsets         │ 50-110°    │ Additive/ │ Hands,     │
│ (Quest, Vision Pro)│            │ Video     │ eyes, voice│
└─────────────────────┴────────────┴───────────┴────────────┘

Key Constraints:
├── Additive displays cannot show true black
│   └── Dark content appears transparent
│   └── Design with light/bright colors on dark backgrounds
├── Limited FOV means content gets "clipped" at edges
│   └── Users must turn head to see all content
│   └── Place critical info in center of FOV
├── Outdoor readability varies dramatically
│   └── Bright sunlight washes out AR displays
│   └── Use high contrast, bold elements
└── Battery life limits always-on AR
    └── Optimize rendering aggressively
    └── Use sensor fusion efficiently

Readability in Mixed Environments

Readability Strategies:
┌─────────────────────────────────────────────┐
│ Problem: Text on varying backgrounds        │
│                                             │
│ Solution 1: Billboard with backing          │
│ ┌───────────────────────┐                   │
│ │ ██ Information text ██│ ← Semi-transparent│
│ └───────────────────────┘   dark backing    │
│                                             │
│ Solution 2: Text with outline/shadow        │
│  Information text  ← White text with black  │
│                      outline (2-3px)        │
│                                             │
│ Solution 3: Adaptive contrast               │
│  Sample pixels behind text                  │
│  If background light → dark text            │
│  If background dark → light text            │
│                                             │
│ Solution 4: Depth-based dimming             │
│  Darken real-world area behind overlay      │
│  (video passthrough only, not optical)      │
└─────────────────────────────────────────────┘

Minimum Specifications:
├── Text size: 24px minimum at arm's length (phone)
│             1mm/m minimum (head-worn, see spatial computing)
├── Contrast ratio: 4.5:1 minimum (WCAG AA)
├── Line weight: 2px minimum for lines/icons
├── Color: Never rely on color alone (accessibility)
└── Animation: Minimal motion for persistent overlays

Overlay Types

World-Locked Overlays

Content anchored to real-world positions:

World-Locked Use Cases:
├── Navigation arrows on the ground
├── Building labels floating above structures
├── Product information next to store shelves
├── Measurement overlays on real objects
├── Virtual furniture placement
└── Maintenance instructions attached to equipment

Implementation Considerations:
├── Anchor to detected planes or surfaces
├── Update position smoothly as tracking refines
├── Handle tracking loss gracefully (fade, don't jump)
├── Billboarding: Text/UI always faces user
├── Distance scaling: Maintain readable size regardless of distance
│   ├── Option A: Fixed world size (natural, but tiny when far)
│   ├── Option B: Fixed screen size (always readable, feels like HUD)
│   └── Option C: Hybrid (world size with minimum screen size)
└── Occlusion: Content behind real objects should be hidden

Distance Scaling Formula:
// Maintain readable angular size
float minAngularSize = 2.0f; // degrees
float distance = Vector3.Distance(camera, overlay);
float worldSize = distance * Mathf.Tan(minAngularSize * Mathf.Deg2Rad);
overlay.transform.localScale = Vector3.one * Mathf.Max(worldSize, minWorldSize);

Screen-Locked Overlays (HUD)

Content fixed to the screen/view:

HUD Design for AR:
┌─────────────────────────────────────────────┐
│ ┌──────┐                        ┌────────┐ │
│ │Status│                        │ Mini   │ │
│ │ Bar  │                        │  Map   │ │
│ └──────┘                        └────────┘ │
│                                             │
│           ← Keep center clear →             │
│              (real world view)               │
│                                             │
│                                             │
│                                             │
│ ┌──────────────────────────────────────────┐│
│ │ Navigation: Turn right in 50m     →     ││
│ └──────────────────────────────────────────┘│
└─────────────────────────────────────────────┘

HUD Rules:
├── Minimize screen coverage (< 15% of view)
├── Keep center of view clear (the real world is the content)
├── Place info at edges and corners
├── Use transparency aggressively (60-80% transparent backgrounds)
├── Animate in/out based on relevance
├── Never lock to head position in VR/AR headsets (causes nausea)
│   └── Instead: Use tag-along (lazy follow with damping)
└── Provide dismiss/minimize for all HUD elements

Context-Aware Overlays

Content that appears based on what the user is looking at or doing:

Context Triggers:
├── Gaze-based: Content appears when user looks at a target
│   ├── Image recognition (product, poster, QR code)
│   ├── Object detection (furniture, appliance, vehicle)
│   ├── Face detection (name tag in meetings)
│   └── Text recognition (translation, accessibility)
├── Location-based: Content appears at GPS coordinates
│   ├── Navigation waypoints
│   ├── Points of interest
│   ├── Geofenced experiences
│   └── Indoor positioning (BLE beacons, Wi-Fi)
├── Time-based: Content changes with time
│   ├── Event schedules
│   ├── Day/night modes
│   └── Time-limited experiences
└── Activity-based: Content adapts to user action
    ├── Cooking: Step-by-step instructions
    ├── Exercise: Form guidance overlays
    ├── Repair: Part identification and instructions
    └── Shopping: Price comparison, reviews

Context Transition Design:
1. Detect context change
2. Fade out current overlay (200ms)
3. Brief pause (100ms) — avoid visual chaos
4. Fade in new overlay (300ms)
5. Provide anchor point (where does new content appear?)
6. Allow manual dismiss if context detection was wrong

AR Navigation Overlays

Wayfinding Design

AR Navigation Patterns:
┌─────────────────────────────────────────────┐
│ Pattern 1: Ground-Level Path                │
│ ┌───────────────────────────────┐           │
│ │  ════════╗                    │           │
│ │          ║    ← Colored line  │           │
│ │          ╚════════            │ on floor  │
│ └───────────────────────────────┘           │
│ Best for: Indoor navigation                 │
│ Anchor: Floor plane                         │
│                                             │
│ Pattern 2: Floating Waypoints               │
│ ┌───────────────────────────────┐           │
│ │        ◇──→ ◇──→ ◇           │           │
│ │  Floating markers at turns    │           │
│ └───────────────────────────────┘           │
│ Best for: Open outdoor spaces               │
│                                             │
│ Pattern 3: Destination Beacon               │
│ ┌───────────────────────────────┐           │
│ │    ↑ Pillar of light at       │           │
│ │    │ destination, visible     │           │
│ │    │ through walls            │           │
│ └───────────────────────────────┘           │
│ Best for: Direct "go here" guidance         │
│                                             │
│ Pattern 4: Contextual Arrows                │
│ ┌───────────────────────────────┐           │
│ │  ← Turn left at this corner  │           │
│ │    Arrow appears only at      │           │
│ │    decision points            │           │
│ └───────────────────────────────┘           │
│ Best for: Complex indoor environments       │
└─────────────────────────────────────────────┘

Data Visualization Overlays

Real-World Data Augmentation

Overlay Data on Physical Objects:

Architecture/Construction:
├── BIM model overlay on building (show hidden pipes, wiring)
├── Measurement display (wall dimensions, room area)
├── Material identification (highlight structural elements)
└── Progress tracking (completed vs. planned visualization)

Industrial/Maintenance:
├── Equipment status indicators (green/yellow/red halos)
├── Temperature/pressure readings floating near sensors
├── Step-by-step repair instructions anchored to parts
├── Safety zone visualization (danger areas highlighted)
└── IoT sensor data mapped to physical locations

Retail:
├── Price tags with comparison data
├── Nutritional information on food products
├── Customer reviews floating near products
├── Stock availability indicators
└── Personalized recommendations

Design Principles for Data Overlays:
1. Less is more — Show only relevant data, not everything available
2. Progressive disclosure — Summary first, details on focus/tap
3. Consistent placement — Same type of data always in same relative position
4. Color coding — Use consistent, accessible color scales
5. Update frequency — Match data refresh to actual change rate
   (temperature: every 10s, stock price: real-time)

Mobile AR Implementation

ARKit/ARCore Overlay Patterns

// iOS ARKit — Adding overlays to detected surfaces
func renderer(_ renderer: SCNSceneRenderer,
              didAdd node: SCNNode,
              for anchor: ARAnchor) {

    if let planeAnchor = anchor as? ARPlaneAnchor {
        // Create overlay on detected surface
        let overlay = createInfoOverlay(
            text: "Surface detected",
            width: planeAnchor.extent.x,
            position: SCNVector3(
                planeAnchor.center.x,
                0.01,  // Slight offset above surface
                planeAnchor.center.z
            )
        )
        node.addChildNode(overlay)
    }

    if let imageAnchor = anchor as? ARImageAnchor {
        // Product recognized — show AR overlay
        let productInfo = lookupProduct(imageAnchor.referenceImage.name)
        let infoCard = createProductCard(productInfo)
        infoCard.position = SCNVector3(0, 0.1, 0) // Above image
        node.addChildNode(infoCard)
    }
}

// Billboard constraint — overlay always faces camera
func createBillboardOverlay(text: String) -> SCNNode {
    let node = SCNNode()
    let billboard = SCNBillboardConstraint()
    billboard.freeAxes = .Y  // Only rotate around Y axis
    node.constraints = [billboard]
    // ... add text geometry
    return node
}

// Android ARCore — Overlay on detected plane
override fun onUpdate(frameTime: FrameTime) {
    val frame = arFragment.arSceneView.arFrame ?: return

    // Hit test from screen center
    val hits = frame.hitTest(screenCenterX, screenCenterY)
    for (hit in hits) {
        val trackable = hit.trackable
        if (trackable is Plane &&
            trackable.isPoseInPolygon(hit.hitPose)) {

            // Place overlay at hit point
            val anchor = hit.createAnchor()
            val anchorNode = AnchorNode(anchor)
            anchorNode.setParent(arFragment.arSceneView.scene)

            val infoNode = createInfoOverlay()
            infoNode.setParent(anchorNode)
            break
        }
    }
}

Performance and Battery

AR Overlay Performance Guidelines:
├── Rendering budget: < 16ms per frame (60 fps phone AR)
├── Minimize overdraw from transparent overlay backgrounds
├── Use SDF (Signed Distance Field) text rendering for quality + performance
├── Cache text textures — don't regenerate every frame
├── Limit simultaneous overlays (5-10 max visible)
├── Use LOD: Simplify distant overlays
├── Batch overlay draw calls where possible
└── Prefer 2D overlays (quads) over 3D geometry for info display

Battery Conservation:
├── Reduce camera resolution when possible
├── Lower tracking frequency when user is stationary
├── Pause AR processing when app is backgrounded
├── Use dark mode / minimal brightness for OLED screens
├── Debounce plane detection after surfaces are found
├── Limit continuous ML inference (image recognition)
└── Provide "pause AR" mode for reading captured info

Accessibility

AR Overlay Accessibility:
├── Text alternatives: Screen reader support for all overlay text
├── Voice control: Navigate overlays by voice command
├── Haptic feedback: Vibrate when overlay appears/is selected
├── High contrast mode: Override transparency with solid backgrounds
├── Text size: User-adjustable overlay text size
├── Color blind modes: Don't rely on red/green distinctions
├── Audio descriptions: Spoken description of visual overlays
├── Reduced motion: Minimize animations, no auto-moving overlays
└── Physical limitations:
    ├── Support one-handed interaction
    ├── Don't require precise targeting
    └── Allow longer dwell times for gaze-based selection

When to Apply This Skill

Use this skill when:

• Designing AR information overlays for mobile or headset
• Building navigation or wayfinding AR experiences
• Creating context-aware AR content that responds to the environment
• Implementing data visualization in augmented reality
• Optimizing AR overlays for readability and performance
• Ensuring AR overlay accessibility compliance