Technology & Engineering8 skills

Data Pipeline Services skills for AI agents

8 practitioner-grade data pipeline services skills, each a focused Markdown document your agent loads into context on demand. Search them from Claude Desktop, Cursor or any MCP client, or pull one with the CLI.

$ npx skilldb search "data pipeline services"

All 8 skills

Airbyte
Configure Airbyte open-source data integration with custom connectors, destinations, and CDC replication. Manage connections, sync modes, normalization, and deployment via the API and Terraform.
258 lines
Apache Airflow
Orchestrate data pipelines using Apache Airflow DAGs, operators, sensors, and XCom. Configure connections, pools, and task dependencies for reliable scheduled workflows.
191 lines
Apache Spark
Process large-scale data with Apache Spark using PySpark DataFrames, Spark SQL, and structured streaming. Optimize partitioning, joins, and serialization for distributed compute workloads.
220 lines
Bigquery
Build analytical pipelines on Google BigQuery using SQL, streaming inserts, and federated queries. Optimize costs with partitioning, clustering, materialized views, and BigQuery ML.
257 lines
Clickhouse
Build high-performance OLAP queries on ClickHouse using MergeTree engines, materialized views, and aggregations. Design schemas for real-time analytics with optimal partitioning, ordering keys, and data lifecycle management.
262 lines
DBT
Build and test data transformation pipelines using dbt models, macros, and incremental strategies. Define data contracts with schema tests, snapshots, and modular SQL for analytics engineering.
242 lines
Fivetran
Configure and manage Fivetran connectors for automated data ingestion into warehouses. Set up sync schedules, transformations, webhooks, and monitor connector health via the API.
246 lines
Snowflake
Build and optimize data pipelines on Snowflake using SQL, stages, streams, tasks, and Snowpipe. Configure warehouses, manage semi-structured data, and implement continuous ingestion patterns.
235 lines