You are an expert in real-time collaborative editing systems, including CRDTs and Operational Transform.

## Key Points

- Requires a central server to determine operation order
- Used by Google Docs (internally)
- Complex to implement correctly — the transformation functions have subtle edge cases
- Efficient in terms of memory; does not grow with document history
- **Yjs** — high-performance CRDT library for JavaScript
- **Automerge** — JSON-like CRDT with rich data type support
- **Diamond Types** — Rust-based, very fast sequence CRDT
- No central server required (peer-to-peer capable)
- Deterministic merge — any order of applying operations yields the same result
- Higher memory overhead due to metadata per character
- Simpler correctness guarantees than OT
- **Choose CRDTs over OT for new projects** — CRDTs are easier to reason about, do not require a central transform server, and have mature libraries (Yjs, Automerge).

Collaborative Editing — WebSockets & Real-Time

You are an expert in real-time collaborative editing systems, including CRDTs and Operational Transform.

Overview

Real-time collaborative editing allows multiple users to simultaneously modify a shared document with changes appearing instantly for all participants. The core challenge is conflict resolution: when two users edit the same region at the same time, the system must merge their changes deterministically without data loss. The two dominant approaches are Operational Transform (OT) and Conflict-free Replicated Data Types (CRDTs).

Core Concepts

Operational Transform (OT)

OT works by transforming operations against each other so that concurrent edits produce the same final state regardless of the order they are applied. Each operation (insert, delete) is transformed against all concurrent operations before being applied locally.

Characteristics:

• Requires a central server to determine operation order
• Used by Google Docs (internally)
• Complex to implement correctly — the transformation functions have subtle edge cases
• Efficient in terms of memory; does not grow with document history

CRDTs (Conflict-free Replicated Data Types)

CRDTs are data structures designed so that concurrent updates can always be merged without conflicts. For text editing, sequence CRDTs assign each character a unique, ordered identifier that determines its position.

Popular CRDT algorithms for text:

• Yjs — high-performance CRDT library for JavaScript
• Automerge — JSON-like CRDT with rich data type support
• Diamond Types — Rust-based, very fast sequence CRDT

Characteristics:

• No central server required (peer-to-peer capable)
• Deterministic merge — any order of applying operations yields the same result
• Higher memory overhead due to metadata per character
• Simpler correctness guarantees than OT

Awareness

Beyond document state, collaborative editing requires awareness — cursor positions, selections, and user identity. This is a separate channel from document operations and is ephemeral (not persisted).

Implementation Patterns

Yjs with WebSocket (Full Stack)

// Server: y-websocket provider
import { WebSocketServer } from 'ws';
import { setupWSConnection } from 'y-websocket/bin/utils';

const wss = new WebSocketServer({ port: 1234 });

wss.on('connection', (ws, req) => {
  // The room name is the document ID, extracted from the URL path
  setupWSConnection(ws, req);
});

// Client: Yjs + y-websocket + TipTap (or any binding)
import * as Y from 'yjs';
import { WebsocketProvider } from 'y-websocket';

// Create a Yjs document
const ydoc = new Y.Doc();

// Connect to the WebSocket server; room name = document ID
const provider = new WebsocketProvider('wss://collab.example.com', 'doc-123', ydoc);

// Awareness (cursors, selections, user info)
const awareness = provider.awareness;
awareness.setLocalStateField('user', {
  name: 'Alice',
  color: '#ff5733',
});

// Get a shared text type
const ytext = ydoc.getText('content');

// Listen for changes
ytext.observe((event) => {
  console.log('Text changed:', ytext.toString());
  console.log('Delta:', event.delta);
});

// Make a local edit
ytext.insert(0, 'Hello, world!');

Yjs with TipTap (Rich Text Editor)

import { Editor } from '@tiptap/core';
import StarterKit from '@tiptap/starter-kit';
import Collaboration from '@tiptap/extension-collaboration';
import CollaborationCursor from '@tiptap/extension-collaboration-cursor';
import * as Y from 'yjs';
import { WebsocketProvider } from 'y-websocket';

const ydoc = new Y.Doc();
const provider = new WebsocketProvider('wss://collab.example.com', 'doc-456', ydoc);

const editor = new Editor({
  element: document.querySelector('#editor'),
  extensions: [
    StarterKit.configure({ history: false }), // Disable built-in history; Yjs handles undo
    Collaboration.configure({ document: ydoc }),
    CollaborationCursor.configure({
      provider,
      user: { name: 'Alice', color: '#ff5733' },
    }),
  ],
});

// Connection status
provider.on('status', ({ status }) => {
  console.log('Connection status:', status); // 'connected' | 'disconnected'
});

Server-Side Persistence with Yjs

import * as Y from 'yjs';
import { MongodbPersistence } from 'y-mongodb-provider';

// Persistent storage for Yjs documents
const persistence = new MongodbPersistence('mongodb://localhost:27017/collab', {
  collectionName: 'yjs-documents',
  flushSize: 100,       // Flush updates after 100 operations
  multipleCollections: true,
});

// Load document from persistence
async function loadDocument(docName) {
  const ydoc = new Y.Doc();
  const persistedState = await persistence.getYDoc(docName);
  Y.applyUpdate(ydoc, Y.encodeStateAsUpdate(persistedState));
  return ydoc;
}

// Store updates incrementally
async function storeUpdate(docName, update) {
  await persistence.storeUpdate(docName, update);
}

// Compact stored updates periodically
async function compactDocument(docName) {
  await persistence.flushDocument(docName);
}

Custom CRDT Sync Protocol

// Server: custom sync with Yjs encoding
import * as Y from 'yjs';
import * as syncProtocol from 'y-protocols/sync';
import * as encoding from 'lib0/encoding';
import * as decoding from 'lib0/decoding';

const docs = new Map();

function getDoc(docName) {
  if (!docs.has(docName)) {
    const doc = new Y.Doc();
    doc.on('update', (update, origin) => {
      // Broadcast to all other clients in this document
      broadcastUpdate(docName, update, origin);
    });
    docs.set(docName, doc);
  }
  return docs.get(docName);
}

wss.on('connection', (ws, req) => {
  const docName = req.url.slice(1); // e.g., /doc-123
  const doc = getDoc(docName);

  ws.on('message', (data) => {
    const decoder = decoding.createDecoder(new Uint8Array(data));
    const messageType = decoding.readVarUint(decoder);

    switch (messageType) {
      case 0: { // sync message
        const encoder = encoding.createEncoder();
        encoding.writeVarUint(encoder, 0);
        syncProtocol.readSyncMessage(decoder, encoder, doc, ws);
        if (encoding.length(encoder) > 1) {
          ws.send(encoding.toUint8Array(encoder));
        }
        break;
      }
      case 1: { // awareness
        // Forward awareness updates to other clients
        broadcastAwareness(docName, data, ws);
        break;
      }
    }
  });

  // Send initial sync step 1
  const encoder = encoding.createEncoder();
  encoding.writeVarUint(encoder, 0);
  syncProtocol.writeSyncStep1(encoder, doc);
  ws.send(encoding.toUint8Array(encoder));
});

Automerge Example

import * as Automerge from '@automerge/automerge';
import { Repo } from '@automerge/automerge-repo';
import { BrowserWebSocketClientAdapter } from '@automerge/automerge-repo-network-websocket';

// Initialize the repo with a WebSocket sync server
const repo = new Repo({
  network: [new BrowserWebSocketClientAdapter('wss://sync.example.com')],
});

// Create a new document
const handle = repo.create();
handle.change((doc) => {
  doc.text = new Automerge.Text();
  doc.text.insertAt(0, ...'Hello, world!'.split(''));
});

// Make concurrent edits
handle.change((doc) => {
  doc.text.insertAt(7, ...'beautiful '.split(''));
});

// Listen for remote changes
handle.on('change', ({ doc }) => {
  console.log('Document updated:', doc.text.toString());
});

Undo/Redo with CRDTs

import * as Y from 'yjs';

const ydoc = new Y.Doc();
const ytext = ydoc.getText('content');

// Create an undo manager scoped to specific shared types
const undoManager = new Y.UndoManager(ytext, {
  trackedOrigins: new Set(['local']), // Only track local changes
  captureTimeout: 500, // Group changes within 500ms into one undo step
});

// Make changes with origin tracking
ydoc.transact(() => {
  ytext.insert(0, 'Hello');
}, 'local');

// Undo/redo
undoManager.undo(); // Reverts the insert
undoManager.redo(); // Re-applies it

// Listen for stack changes
undoManager.on('stack-item-added', (event) => {
  console.log('Undo stack size:', undoManager.undoStack.length);
});

Best Practices

• Choose CRDTs over OT for new projects — CRDTs are easier to reason about, do not require a central transform server, and have mature libraries (Yjs, Automerge).
• Separate document state from awareness — cursor positions and selections are ephemeral and high-frequency. Do not persist them or merge them with document operations.
• Persist documents as compacted snapshots — store incremental updates for efficiency, but periodically compact them into a single snapshot to avoid unbounded growth.
• Use binary encoding — Yjs and Automerge use efficient binary encodings for sync. Do not serialize to JSON for transport.
• Scope undo to the local user — the undo manager should only revert the current user's changes, not changes made by collaborators.
• Show collaborator cursors and selections — visual presence is essential for the collaborative experience. Use awareness protocols to broadcast cursor state.

Common Pitfalls

• Implementing OT from scratch — the transformation function for even simple text operations has non-obvious edge cases (the "TP2 puzzle"). Use a well-tested library instead.
• Not handling offline and reconnection — CRDTs naturally support this (apply buffered changes on reconnect), but you must implement the sync protocol correctly to avoid resending the entire document.
• Ignoring document size growth — CRDT metadata (tombstones, character IDs) accumulates. Without periodic garbage collection or compaction, documents grow without bound.
• Blocking the main thread with large documents — CRDT operations on very large documents can be CPU-intensive. Consider using a Web Worker for Yjs updates.
• Missing conflict indicators — while CRDTs resolve conflicts deterministically, the merge result may not be what either user intended. For structured data (tables, code), consider showing conflict markers to the user.
• Not testing concurrent scenarios — collaborative editing bugs only appear under concurrency. Use property-based testing with randomized operations to verify convergence.

Core Philosophy

Collaborative editing is fundamentally a distributed systems problem disguised as a UI feature. The core challenge is not displaying text — it is ensuring that two users who make concurrent edits to the same document converge to the same state without data loss, regardless of network latency, ordering, or partial failures. Choose CRDTs over Operational Transform for new projects: they are mathematically simpler, do not require a central coordination server, and have mature, battle-tested libraries.

Separate document state from awareness. Document operations (insert, delete, format) must be persisted, ordered, and replayed correctly. Awareness data (cursor position, selection, user identity) is ephemeral, high-frequency, and should never be persisted. Mixing these two concerns — storing cursor positions in the document or routing document operations through awareness channels — creates both performance and correctness problems.

Offline support is not an edge case in collaborative editing — it is the normal state of mobile users, users on unreliable networks, and users who close their laptop mid-edit. CRDTs naturally support offline operation: buffer local changes, reconnect when possible, and merge seamlessly. Design your sync protocol to handle reconnection with delta sync rather than full document retransmission.

Anti-Patterns

• Implementing OT from scratch — the transformation functions for even simple text operations have subtle edge cases (the "TP2 puzzle") that are extremely difficult to get right; use a well-tested library like Yjs or Automerge instead.

• Serializing CRDT documents as JSON for transport — Yjs and Automerge use efficient binary encodings for synchronization; converting to JSON adds overhead, increases bandwidth, and may lose type information.

• Not scoping undo to the local user — a global undo that reverts any user's last change is confusing and destructive; the undo manager should only revert changes made by the current user.

• Ignoring document size growth over time — CRDT metadata (tombstones, character IDs) accumulates with every edit; without periodic compaction or garbage collection, document state grows without bound.

• Running CRDT operations on the main thread for large documents — applying updates to documents with millions of operations can be CPU-intensive; consider offloading Yjs processing to a Web Worker to keep the UI responsive.