Llm Engineering skills for AI agents

Triggers when users need help with LLM agent design, tool use, or multi-agent systems. Activate for questions about agent architectures (ReAct, plan-and-execute, reflexion), tool use and function calling, agent memory, multi-agent systems, agent evaluation, debugging, guardrails, safety for agents, agent frameworks (LangGraph, CrewAI, AutoGen), production agent deployment, and agent failure modes.

Triggers when users need help with LLM application design patterns and architectures. Activate for questions about classification with LLMs, extraction and structuring, summarization pipelines, code generation systems, chat application design, search augmented generation, document processing pipelines, multi-turn conversation management, LLM routing and model selection per query, and fallback strategies.

Triggers when users need help with LLM cost optimization, budgeting, or economic analysis. Activate for questions about token cost optimization, prompt compression, caching, model routing, API cost comparison across providers, self-hosted vs API tradeoffs, semantic caching, prompt deduplication, batch API usage, cost-per-task analysis, budget allocation across models, and cost monitoring and alerting.

Triggers when users need help with LLM evaluation, benchmarking, or assessment methodology. Activate for questions about MMLU, HumanEval, GSM8K, HellaSwag, ARC, TruthfulQA, MT-Bench, few-shot evaluation, zero-shot evaluation, chain-of-thought evaluation, LLM-as-judge, human evaluation protocols, contamination detection, evaluation harness setup, lm-eval-harness, and EleutherAI evaluation tools.

Triggers when users need help with LLM fine-tuning, adaptation, or specialization. Activate for questions about full fine-tuning, LoRA, QLoRA, prefix tuning, adapters, instruction tuning, dataset design, supervised fine-tuning pipelines, hyperparameter selection, catastrophic forgetting, domain adaptation, and continued pretraining vs fine-tuning decisions.

Triggers when users need help with LLM inference optimization, serving, or deployment performance. Activate for questions about KV cache management, continuous batching, speculative decoding, tensor parallelism, quantization (GPTQ, AWQ, GGUF, FP8), PagedAttention, vLLM, prefix caching, flash decoding, serving frameworks (TGI, TensorRT-LLM, SGLang), and throughput vs latency tradeoffs.

Triggers when users need help with LLM pretraining, data curation, or training infrastructure. Activate for questions about Common Crawl processing, deduplication, quality filtering, toxicity removal, tokenizer design (BPE, SentencePiece, Unigram), distributed training, checkpointing, scaling laws, Chinchilla, compute-optimal training, curriculum learning, data mixing ratios, training stability, loss spikes, and gradient norm monitoring.

Triggers when users need help with LLM safety, guardrails, or content moderation systems. Activate for questions about input/output guardrails, content filtering, PII detection and redaction, prompt injection defense, system prompt protection, input sanitization, jailbreak detection, toxicity classification, hallucination detection and mitigation, usage policies implementation, and safety evaluation benchmarks.

Triggers when users need help with advanced prompt engineering techniques for LLMs. Activate for questions about chain-of-thought prompting, few-shot example selection, system prompt design, structured output prompting (JSON mode, function calling), prompt chaining, self-consistency, tree-of-thought, meta-prompting, prompt optimization (DSPy, automatic prompt engineering), and adversarial prompt testing.

Triggers when users need help with RAG systems, retrieval-augmented generation, or knowledge-grounded LLM applications. Activate for questions about chunking strategies, embedding models, vector databases (Pinecone, Weaviate, Chroma, Qdrant, pgvector), retrieval methods (dense, sparse, hybrid, reranking), context window management, citation and attribution, RAG evaluation, context relevance, faithfulness, and answer relevance metrics.

Triggers when users need help with RLHF, alignment, or preference optimization for LLMs. Activate for questions about reward model training, PPO optimization, DPO, KTO, ORPO, preference data collection, human annotation, synthetic preferences, reward hacking, overoptimization, constitutional AI, debate, scalable oversight, and red teaming for aligned models.

Triggers when users need help with synthetic data generation using LLMs. Activate for questions about using LLMs for training data, self-instruct, Evol-Instruct, data augmentation with LLMs, quality filtering for synthetic data, constitutional AI data generation, preference pair generation, synthetic evaluation data, domain-specific synthetic data, and avoiding model collapse from synthetic data.