Market Data Engineering skills for AI agents

Activate this skill when the user is turning raw tick data or OHLCV bars into research-ready datasets for backtesting and needs to align series, set forward-fill rules, timestamp features correctly, prevent leakage and keep results reproducible. Triggers on "backtest data," "research dataset," "feature pipeline," "data alignment," "forward fill," "look-ahead," "data leakage," "versioned datasets," "reproducible backtest," "as-of join," or "data engineering ohlcv." Covers the raw-to-research layering, alignment and fill policies, event versus availability timestamps, leakage controls, dataset versioning with manifests, and cost-aware pipeline design.

Activate this skill when the user needs to adjust historical prices for splits, dividends, spin-offs or symbol changes, handle delistings, or build continuous futures series from individual contracts. Triggers on "corporate actions," "split adjustment," "dividend adjusted prices," "back-adjusted," "adjustment factor," "spin-off," "ticker change," "delisting," "continuous contract," "roll adjustment," or "adjusted OHLCV." Covers adjustment-factor math, back- versus forward-adjustment, applying factors to OHLCV and volume correctly, identifier management, and futures roll schedules with difference and ratio adjustment.

Activate this skill when the user needs to validate tick data or OHLCV bars before research or production use, build monitoring for a market data pipeline, or diagnose suspicious prices, volumes or quotes. Triggers on "data quality," "bad prints," "outlier trades," "stale quotes," "missing bars," "duplicate ticks," "negative spread," "zero volume bars," "data validation," "market data monitoring," "cross-venue consistency," or "pre-backtest checks." Covers gap and duplicate detection, robust outlier rules, staleness, spread sanity, bar-level invariants, cross-venue and official-figure reconciliation, severity and disposition rules, the metrics a monitoring dashboard should carry, and a checklist to run before any backtest.

Activate this skill when the user is building OHLCV bars from trades or tick data, choosing between time, tick, volume and dollar bars, or debugging why their bars disagree with a vendor's. Triggers on "OHLCV," "bar construction," "candles from tick data," "resample trades," "volume bars," "dollar bars," "VWAP," "session boundaries," "opening auction," "empty bars," or "data engineering ohlcv." Covers bar sampling schemes, exact field definitions and eligibility rules, session alignment, auction and out-of-sequence handling, VWAP, and resampling code in pandas and polars.

Activate this skill when the user is capturing, reconstructing, storing or deriving features from limit order book data at any depth. Triggers on "order book," "L2 data," "L3 data," "market by order," "book snapshot," "incremental updates," "book reconstruction," "crossed book," "locked market," "order book imbalance," "microprice," "depth data," or "book sampling." Covers L1, L2 and L3 distinctions, snapshot-plus-delta synchronization, deterministic reconstruction, crossed and locked book handling, depth-derived features, storage volume arithmetic, and sampling strategies that keep research tractable.

Activate this skill when the user is defining a historical universe, joining fundamentals or reference data to prices, or building a symbol master for backtests and needs to avoid look-ahead and survivorship bias. Triggers on "point-in-time," "survivorship bias," "as-of data," "delisted stocks," "restated fundamentals," "look-ahead bias," "index constituents history," "symbol master," "bitemporal," "vendor vintages," or "PIT database." Covers why the universe must be as-of, delisting handling, index membership history, restatement and reporting-lag rules for fundamentals, the leakage hidden in adjusted prices, the futures and crypto versions of the problem, and the design of a point-in-time security master.

Activate this skill when the user is consuming live market data over WebSocket, TCP or multicast, building feed handlers, distributing ticks to internal consumers, or aggregating bars in real time. Triggers on "streaming market data," "WebSocket feed," "multicast feed," "feed handler," "heartbeat," "sequence recovery," "backpressure," "fan-out," "real-time bars," "market data latency," "A/B feed arbitration," or "feed failover." Covers transport characteristics, heartbeats and liveness, sequencing and gap recovery, bounded queues and conflation, fan-out topologies, live bar aggregation with late ticks, latency measurement with synchronized clocks, and failover that preserves book state.

Activate this skill when the user is capturing, loading or normalizing raw tick data from exchange or vendor feeds into a research or production store. Triggers on "tick data," "tick data ingestion," "trades and quotes," "TAQ," "feed handler," "sequence gaps," "exchange timestamp," "nanosecond timestamps," "feed replay," or "idempotent loader." Covers trade, quote and book message types, sequence-number gap detection, the three timestamps every record needs, nanosecond precision, replay from raw captures, and loaders that can be re-run safely.

Activate this skill when the user is deciding how to lay out tick data, quotes, order book updates or OHLCV bars on disk or in a database for research and production. Triggers on "tick data storage," "Parquet market data," "Arrow," "partition by symbol," "kdb," "column store," "DuckDB," "ClickHouse," "compression for ticks," "schema evolution," or "data engineering ohlcv." Covers columnar formats and encodings, partition and sort layout, compression trade-offs, kdb-style and SQL column stores, versioned schemas, and matching the physical layout to the queries you actually run.

Activate this skill when the user is handling exchange sessions, holidays, half days, daylight saving transitions or timezone conversion in market data, or aligning tick data and OHLCV bars across venues that keep different hours. Triggers on "trading calendar," "exchange holidays," "half day," "market hours," "DST," "timezone," "UTC storage," "trade date," "session boundaries," "Globex hours," "crypto 24/7," "venue alignment," or "exchange_calendars." Covers session definitions for equities, futures and crypto, UTC-first storage with exchange-local session logic, trade date versus calendar date, DST asymmetries between regions, overlap windows across venues, and the traps in calendar and timezone libraries.