Database

Covers testing AI agent behavior end-to-end: trajectory evaluation, tool-call sequence validation, multi-step correctness verification, stuck-loop detection, cost regression testing, and timeout handling. Triggers: "test my AI agent", "agent trajectory evaluation", "tool call testing", "multi-step agent testing", "agent stuck detection", "agent cost regression", "validate agent behavior".

Covers implementing CI/CD pipelines for AI applications: running LLM evals in GitHub Actions, gating deployments on eval scores, monitoring prompt and model drift, versioning prompts alongside code, cost tracking, and canary deployments for AI features. Triggers: "CI for AI", "run evals in GitHub Actions", "gate deployment on eval score", "prompt drift detection", "version prompts in CI", "AI deployment pipeline", "LLM CI/CD".

Covers popular LLM evaluation frameworks and how to use them: Braintrust, Promptfoo, RAGAS, DeepEval, LangSmith, and custom eval harnesses. Includes setup, configuration, writing eval cases, CI integration, and choosing the right framework for your use case. Triggers: "eval framework", "Braintrust setup", "Promptfoo config", "RAGAS evaluation", "DeepEval", "LangSmith evals", "custom eval harness", "which eval tool should I use".

Covers using LLMs to evaluate other LLM outputs: rubric design, pairwise comparison, reference-based and reference-free grading, calibration techniques, inter-rater reliability measurement, and cost-efficient judging strategies. Triggers: "LLM as judge", "use GPT to evaluate outputs", "AI grading AI", "rubric for LLM evaluation", "pairwise comparison", "LLM evaluator", "auto-grade LLM responses".

Covers the foundations of evaluating LLM-powered applications: why evaluation matters, the taxonomy of metric types (exact match, semantic similarity, LLM-as-judge), building and curating eval datasets, establishing baselines, detecting regressions, and designing eval pipelines that scale from prototyping through production. Triggers: "evaluate my LLM app", "set up evals", "how do I measure LLM quality", "create an eval pipeline", "LLM metrics", "eval dataset".

Covers testing and hardening prompts for LLM applications: prompt regression testing, A/B testing prompt variants, temperature sensitivity analysis, edge case libraries, prompt versioning strategies, and golden test sets. Triggers: "test my prompt", "prompt regression", "A/B test prompts", "prompt versioning", "temperature sensitivity", "golden test set for prompts", "prompt quality assurance".

Covers red-teaming AI applications for safety and robustness: adversarial prompt testing, jailbreak resistance evaluation, PII leakage detection, hallucination measurement, bias detection, safety benchmarks, and building automated red-team pipelines. Triggers: "red team my AI", "adversarial testing for LLMs", "jailbreak testing", "PII leakage test", "hallucination detection", "AI bias testing", "safety benchmark", "AI security testing".

Covers testing and validating structured outputs from LLMs: JSON mode validation, schema conformance with Zod and JSON Schema, handling partial and malformed outputs, retry strategies with exponential backoff, and building type-safe LLM response pipelines. Triggers: "validate LLM JSON output", "test structured output", "JSON schema validation for AI", "type-safe LLM responses", "handle malformed LLM output", "Zod validation for AI".

An adversarial find → dedup → verify → fix loop that audits a codebase or PR for REAL bugs — correctness, security, concurrency, resource leaks, API misuse — at high precision instead of a wall of false positives. Each round fans out perspective-DIVERSE finders (one lens each, because identical finders just agree on the same obvious bug), dedups fresh findings against ALL previously-seen items (not just the confirmed ones, or rejected findings reappear forever and it never converges), then puts every fresh finding through N independent skeptics each prompted to REFUTE with default-to-refuted (a verifier told to "confirm" rubber-stamps plausible-but-wrong claims). Only findings that survive a majority of skeptics AND carry a written repro get fixed; fixes must pass the project's existing test/typecheck gate. Loops until K consecutive rounds surface nothing new (loop-until-dry, not a fixed count — one dry round can be luck). Use when you need to adversarially hunt bugs in a codebase or PR with high precision, run an automated security/correctness audit, or converge a "is this actually broken?" review instead of dumping speculative nitpicks.

A cursor → batch → checkpoint → verify → resume loop for running a transformation over a LARGE dataset — a schema backfill, a re-index, a reprocessing job, millions of rows — that is too big for one pass and MUST survive interruption. Each iteration reads the next un-processed batch from a DURABLE cursor, processes it IDEMPOTENTLY (upsert by key, never blind insert, so a retried/crashed batch can't double-count), persists the new checkpoint, then GATES the batch on counts + invariants before advancing — a failing batch halts the loop and does NOT move the cursor. On restart it resumes from the last checkpoint, never from zero. It paces itself with backpressure so it doesn't melt the source DB, runs a read-only DRY-RUN/shadow first, verifies-as-it-goes to catch a bad transform at batch 3 not after 10M rows, and finishes with a full source-vs-destination RECONCILIATION (count + checksum) — because cursor-reached-end ≠ all-rows-correct. Use when you must backfill, migrate, re-index, or reprocess a large table/collection and need a job that resumes cleanly after a crash and is provably complete, not merely "stopped erroring".

An eval → improve-one-thing → re-eval hill-climbing loop for developing an LLM feature (prompt, pipeline, or agent) where "better" must be MEASURED, not vibed. You freeze a labeled eval set, score the current system per-component (accuracy, format, safety, latency — never one blended number), let an agent propose ONE attributable change (a prompt edit, a few-shot example, a tool, a guardrail), re-run the SAME evals, and KEEP the change only if the aggregate score holds and no category regresses past a threshold — otherwise revert. It guards against overfitting with a held-out test split you never optimize against, validates the LLM-as-judge against human labels so its bar doesn't drift, and stops on an honest plateau (or expands the eval set to surface new failures). Use when you're iterating on a prompt/agent/RAG pipeline and need to hill-climb a quality score with a regression gate instead of shipping changes that "look better."

A scout → pipeline → gate-each → residue-loop pattern for a large MECHANICAL change across a codebase — an API rename, a framework/version upgrade, a library swap, an import rewrite — that is too big for one context and too risky for one giant diff. You SCOUT first (discover the full work-list of call-sites inline via ripgrep/AST), then pipeline() each site independently (transform → gate the single file → on failure drop it to a residue list), then re-run the loop on the RESIDUE until it is empty. Uses a deterministic AST codemod (jscodeshift/ts-morph/comby) for the 95% and reserves an agent for the bespoke long-tail call-sites a codemod can't safely handle. Gate is per-item: the file typechecks/builds AND a runtime/test check passes for the changed surface (compiles ≠ correct), with a whole-repo gate before commit. Use when you must apply the same change to hundreds of call-sites and want convergence (residue → 0) instead of one unreviewable 400-file diff.

A characterize → green → tiny-refactor → green loop for restructuring code WITHOUT changing behavior — extract a function, rename, decompose a god-class, modernize a pattern — the dangerous kind where "it still compiles" is nowhere near enough proof. FIRST it establishes a green characterization-test net: if coverage of the target module is thin, the agent WRITES characterization tests that pin the CURRENT observable behavior — bugs and all — so "preserved" becomes verifiable instead of asserted. Then it loops in TINY steps: snapshot → make ONE structure-preserving change → run the FULL suite → green ⇒ commit the step; red ⇒ REVERT immediately and take a smaller step (never fix-forward inside a half-refactored state). The characterization tests are frozen — editing a test to make it pass is a behavior change masquerading as a refactor, and is forbidden. Loops until the target structure is reached with the suite green at EVERY step. Use when you must safely restructure, extract, rename, decompose, or modernize existing code under a test net without altering what it does.

A gather → synthesize → critique-gaps → fill loop that builds a comprehensive, fully-cited answer or document from many sources, where any single pass always misses something. Each round fans out MULTI-MODAL searchers that each attack a DIFFERENT angle — by-entity, by-time-window, by-source-type, by-counter-argument — each blind to the others (N identical searches find ONE thing N times; N different angles find N things). It then synthesizes into a claim→source structure, and a COMPLETENESS-CRITIC agent — the ENGINE of the loop, not a rubber stamp — asks what is MISSING (a modality not searched, a claim unverified against read text, a key source unread, a steel-manned counter-argument absent, a contradiction left unresolved) and returns a structured gap list that becomes the NEXT round's targeted queries. The gate: every claim must cite a source that was actually retrieved and read — each citation is verified to resolve to gathered text (no hallucinated cites) — and the critic must sign off that no material gap remains. Sources dedup by URL/DOI; one focused expansion (the critic's top gap) per round, so progress is attributable. Loops until the critic returns "no material gaps" twice or the token budget is hit. Use when you need a deep, multi-source, fact-checked research report and want it to converge on completeness instead of stopping at the first plausible-looking draft.

A screenshot → critique → improve-one-thing → test loop that systematically develops an admin panel (or any auth-gated, multi-page UI) page by page, in parallel rounds, with a hard typecheck/lint gate and honest stopping criteria. Covers a headless Playwright screenshot harness that mints a real admin session via the Firebase Admin SDK (no auth-bypass code), a fan-out improvement pass where N agents each own a batch of pages, a non-negotiable tsc/eslint gate that makes parallel autonomy safe, an "improvement ladder" that deepens each round, and convergence by skip-rate so you stop instead of degrading into busywork. Use when you have a working-but-undeveloped multi-page UI and want to compound many small, tested improvements without breaking it.

A red→green agentic loop for implementing or repairing code against an existing test suite. One iteration runs the test command, parses the FIRST failure, hands an agent only that failure plus the relevant source, takes the MINIMAL diff, and re-runs the WHOLE suite. The test command's exit code is the only oracle — green or it didn't happen. Includes a tamper guard (reject if the agent shrank or weakened the tests to go green), no-progress detection (same failure signature twice = stuck, escalate), flaky-test quarantine (a failure that passes on bare re-run is flaky, not a bug), environment-vs-test triage (a setup failure loops forever if treated as a code failure), and fan-out across independent failing modules. Use when you have a failing test suite or a broken build and want an agent to drive it to green safely — without it cheating by deleting assertions, looping on a flake, or "fixing" snapshots by regenerating them.

Teaches effective AI pair programming techniques for tools like Claude Code, Cursor, and Copilot. Covers when to lead versus follow the AI, providing persistent context through CLAUDE.md and .cursorrules files, breaking complex tasks into AI-manageable pieces, using git strategically with frequent commits as checkpoints, and recognizing when the AI is stuck in a loop. Use when working alongside AI coding tools in a collaborative development workflow.

Teaches how to debug code generated by AI tools, covering the unique failure modes of AI-generated code including hallucinated APIs, version mismatches, circular logic, and phantom dependencies. Explains how to read error messages back to the AI effectively, provide minimal reproductions, diagnose when the AI is giving bad fixes, and use systematic debugging approaches on codebases you did not write by hand. Use when AI-generated code is not working and you need to find and fix the issue.

Covers the unique challenges of maintaining codebases built primarily through AI code generation. Addresses inconsistent patterns across AI-generated files, refactoring AI sprawl, establishing coding conventions after the code already exists, documentation strategies for AI-built projects, and managing the specific forms of technical debt that AI tools create. Use when a vibe-coded project needs ongoing maintenance or has grown unwieldy.

Guides the complete journey from an idea to a working application using AI code generation tools. Covers writing effective app specifications, choosing the right tool for the job (Claude Code, Cursor, Bolt, v0, Lovable, Replit Agent), the spec-first approach, iterating on generated code without losing coherence, and managing scope creep during AI-assisted development. Use when someone wants to build an app from scratch using vibe coding.

Teaches how to review, audit, and evaluate AI-generated code effectively. Covers common AI code smells like over-engineering, dead code, wrong abstractions, and hallucinated APIs. Includes security review checklists, dependency auditing, performance review techniques, and strategies for catching the subtle bugs that AI confidently introduces. Use when reviewing code produced by any AI coding tool.

Guides the transition from a vibe-coded prototype to a production-grade application. Covers identifying when the project has outgrown pure vibe coding, refactoring AI-generated code for production reliability, adding tests retroactively to an untested codebase, introducing CI/CD pipelines, establishing code ownership and review processes, and building the engineering practices needed to sustain a growing application. Use when a vibe-coded project is succeeding and needs to become a real product.

Covers architecture decisions optimized for AI-assisted development. Teaches how to choose frameworks and structures that AI tools work well with, why monolith-first is the right default for vibe coding, how to organize files so AI can navigate them, which abstraction patterns help versus hinder AI code generation, and how to keep complexity within the bounds of what AI can reason about. Use when making technology and architecture choices for a vibe-coded project.

Teaches the foundations of vibe coding — the 2025-2026 paradigm of building software primarily through AI prompting. Covers what vibe coding actually is, the core prompting loop, when it works well (prototyping, MVPs, CRUD apps, internal tools) versus when it fails (distributed systems, real-time, safety-critical), how to manage context windows effectively, and when to drop out of the AI loop and take manual control. Use when someone is new to vibe coding or wants to improve their fundamentals.

Cloudflare Durable Objects for stateful edge computing, covering constructor patterns, storage API, WebSocket support, alarm handlers, consistency guarantees, and use cases like rate limiting, collaboration, and game state.

Cloudflare Workers AI for running inference at the edge, covering supported models, text generation, embeddings, image generation, speech-to-text, AI bindings, and streaming responses.

Cloudflare D1 serverless SQLite database for Workers, covering schema management, migrations, queries, prepared statements, batch operations, local development, replication, backups, and performance optimization.

Cloudflare Workers runtime fundamentals including V8 isolates, wrangler CLI, project setup, local development, deployment, environment variables, secrets, and compatibility dates.

Cloudflare Workers KV namespace for globally distributed key-value storage, including read/write patterns, caching strategies, TTL, list operations, metadata, bulk operations, and the eventual consistency model.

Production patterns for Cloudflare Workers including queue consumers, cron triggers, email workers, browser rendering, Hyperdrive database connection pooling, Vectorize vector search, and the analytics engine.

Cloudflare R2 object storage with S3-compatible API, covering bucket operations, multipart uploads, presigned URLs, public buckets, lifecycle rules, event notifications, and cost optimization compared to S3.

Request routing in Cloudflare Workers including URL pattern matching, path parameters, middleware patterns, error handling, CORS configuration, custom domains, route priorities, and Workers for Platforms.

Teaches Conflict-free Replicated Data Types (CRDTs), the mathematical foundation for local-first sync. Covers how CRDTs guarantee eventual consistency without coordination, the difference between state-based and operation-based CRDTs, and practical implementations of G-Counter, PN-Counter, LWW-Register, OR-Set, G-Set, and RGA (Replicated Growable Array). Includes causal ordering, vector clocks, and guidance on choosing the right CRDT for your data model.

Teaches ElectricSQL, a Postgres-backed local-first sync framework. Covers the Electric architecture where Postgres is the source of truth and data syncs to local SQLite databases on client devices via shape-based partial replication. Includes shape definitions, live queries, offline-first patterns, conflict resolution with rich CRDTs, integration with React and Expo (React Native), deployment patterns, and migration strategies.

Teaches IndexedDB patterns for local-first web applications, using Dexie.js as the primary wrapper library. Covers schema design and versioning, creating indexes for efficient queries, transaction patterns, performance optimization (bulk operations, pagination, lazy loading), migration strategies for schema evolution, storage quota management, data export and import, and integration patterns with sync engines and reactive frameworks.

Teaches authentication and authorization patterns for local-first applications that must work offline. Covers offline-capable auth with cached tokens, permission sync and local enforcement, encrypted local storage for sensitive data, key management with device-bound keys, device authorization and revocation, multi-device identity linking, end-to-end encryption for synced data, and secure patterns for handling auth in disconnected environments.

Teaches the local-first software paradigm where applications store data on the user's device, work fully offline, and sync to peers or servers when connectivity is available. Covers the spectrum from cloud-first to offline-first to local-first, core benefits (instant UX, offline capability, data ownership, privacy), key challenges (conflict resolution, sync complexity, storage limits), architectural patterns, and decision frameworks for when local-first is the right choice.

Teaches how to design and build a sync engine for local-first applications. Covers the operation log as the foundation, conflict resolution strategies (last-write-wins, operational transform, CRDTs), server reconciliation patterns, partial sync for large datasets, bandwidth optimization techniques, version vectors and causal consistency, clock synchronization, and practical implementation patterns with code examples.

Teaches building local-first collaborative applications with Yjs, the most widely adopted CRDT library for JavaScript. Covers the Y.Doc document model, shared types (Y.Map, Y.Array, Y.Text, Y.XmlFragment), the awareness protocol for presence and cursors, persistence and sync providers (WebSocket, WebRTC, IndexedDB), integrating with editors like ProseMirror/TipTap/CodeMirror/Monaco, undo/redo management, and performance optimization patterns.

Teaches Zero (by Rocicorp), the successor to Replicache, a sync engine for building local-first web applications with instant UI, optimistic mutations, and server-side authority. Covers the Zero architecture (client cache, sync engine, server), defining queries and mutators, the reactivity model, server-side authorization and permissions, optimistic updates with automatic rollback, deployment patterns, and migration from Replicache.

Rust commands with the invoke pattern, argument passing, return types, async commands, error handling, state management, and type safety between Rust and TypeScript in Tauri 2.0.

Distributing Tauri applications including installers for MSI, DMG, AppImage, and deb, auto-update with the built-in updater, code signing for Windows and macOS, CI/CD builds, and cross-compilation.

Frontend integration with Tauri 2.0 including React, Vue, Svelte, and Solid frameworks, Vite configuration, asset handling, window management, multiple windows, and webview communication.

Tauri 2.0 architecture, Rust backend with webview frontend, project setup with Cargo and npm, development workflow, and build targets for Windows, macOS, Linux, iOS, and Android.

Tauri 2.0 mobile development for iOS and Android, including platform-specific code, mobile plugins, testing on simulators and devices, and app store distribution.

Common Tauri 2.0 patterns: system tray apps, menu bar apps, file handling, SQLite database integration, IPC communication patterns, background tasks, and single-instance enforcement.

Tauri 2.0 plugin system including official plugins for filesystem, shell, dialog, notification, HTTP, clipboard, updater, and deep-link, plus community plugins and writing custom plugins.

Tauri 2.0 security model including capability-based permissions, allowlist configuration, Content Security Policy, IPC safety, sandboxing, code signing, auto-update security, and supply chain considerations.

Bun's built-in bundler: Bun.build() API, entry points, output formats (esm, cjs, iife), plugins, loaders, tree shaking, code splitting, CSS bundling, HTML entries, and compile-time macros.

Bun runtime overview: all-in-one JavaScript runtime, bundler, test runner, and package manager. Installation, project initialization, Node.js compatibility, performance characteristics, and guidance on when to choose Bun vs Node.

Building HTTP servers with Bun: Bun.serve() API, routing patterns, WebSocket support, streaming responses, static file serving, TLS configuration, hot reloading, and integration with frameworks like Hono and Elysia.

Migrating from Node.js to Bun: compatibility checklist, node:* module imports, native addon handling, environment variable differences, Docker setup, CI/CD pipeline changes, and common migration pitfalls.

Bun as a package manager: bun install, bun add, bun remove, the binary lockfile (bun.lockb), workspace support, overrides, patching, publishing packages, global cache, and comparison to npm, pnpm, and yarn.

Production patterns for Bun: Docker deployments, TypeScript configuration, shell scripting with Bun.$, monorepo setup, database access patterns, and deployment to Fly.io and Railway.

Bun-native runtime APIs including Bun.serve(), Bun.file(), Bun.write(), Bun.spawn(), Bun.sleep(), Bun.env, FFI for calling native libraries, built-in SQLite, S3 client, glob, and semver utilities.

Bun's built-in test runner: bun test command, describe/it/expect assertions, mocking and spies, snapshot testing, lifecycle hooks, DOM testing with happy-dom, code coverage, and watch mode.

Communicate with customers during an active incident — status page, email, in-app banners, social media. The engineering decisions are hard but separable from the communications decisions; this skill covers the latter. Use when an incident has customer impact and the question of what to tell them, when, becomes pressing.

Serve as the incident commander during an active production incident. The IC coordinates the response, tracks the status, communicates with stakeholders, and makes the calls. Distinct from the engineers investigating root cause. Use when an incident exceeds what one engineer can handle alone.

Write runbooks the on-call engineer at 03:00 AM can actually follow. Covers structure, decision points, escalation criteria, and the difference between procedural runbooks and diagnostic runbooks. Use when documenting any production system that can fail in ways the primary on-call may not have seen before.

Define a severity scale that triggers the right response without overreaction or underreaction. Covers SEV-1 through SEV-4, the customer-impact criteria, the response expectations per level, and how to handle escalation and de-escalation during an incident. Use when designing or reviewing your team's incident management process.