Prompt Engineering

Overview

Prompt engineering is the discipline of crafting inputs to large language models that reliably produce high-quality, task-appropriate outputs. It spans simple instruction writing through advanced techniques like chain-of-thought reasoning, few-shot exemplars, and structured output schemas. Effective prompt engineering reduces hallucination, improves consistency, and often eliminates the need for fine-tuning.

Use this skill when designing prompts for production systems, debugging inconsistent LLM outputs, building prompt templates for repeatable workflows, or evaluating whether a task can be solved with prompting alone versus requiring fine-tuning.

Core Framework

Prompt Architecture Layers

1. System Prompt: Sets identity, constraints, and behavioral boundaries.
2. Context Block: Provides reference material the model needs (documents, schemas, examples).
3. Task Instruction: The specific action to perform, stated clearly and unambiguously.
4. Output Specification: Format, length, structure, and any constraints on the response.
5. Guard Rails: What to avoid, edge case handling, fallback behavior.

Technique Spectrum

• Zero-shot: Instruction only, no examples. Best for simple, well-defined tasks.
• Few-shot: 2-5 input/output examples. Best for pattern matching and format consistency.
• Chain-of-thought (CoT): "Think step by step" or explicit reasoning scaffolding. Best for logic, math, multi-step tasks.
• ReAct: Interleave reasoning and actions (tool use). Best for agentic workflows.
• Self-consistency: Sample multiple reasoning paths, take majority vote. Best for accuracy-critical tasks.

Process

1. Define the exact task and success criteria before writing any prompt.
2. Write the simplest possible instruction that conveys the task.
3. Test with 5-10 diverse inputs and categorize failure modes.
4. Add specificity to address each failure mode: constraints, examples, or format instructions.
5. Structure the prompt with clear delimiters (XML tags, markdown headers, or triple backticks) to separate sections.
6. Add few-shot examples if output format is complex or nuanced.
7. Add chain-of-thought scaffolding if reasoning quality is insufficient.
8. Test again with edge cases and adversarial inputs.
9. Optimize token usage by removing redundant instructions.
10. Version-control the final prompt and document its intended scope.

Key Principles

• Be explicit rather than implicit; LLMs do not read your mind.
• Put the most important instruction at the beginning and end of the prompt (primacy and recency effects).
• Use structured delimiters (XML tags like <context>, <instructions>) to prevent context bleed.
• Provide the output format as a template or schema, not just a description.
• Negative instructions ("do not") are weaker than positive instructions ("instead, do X").
• Temperature and top-p settings are part of prompt engineering; lower temperature for deterministic tasks, higher for creative ones.
• Test prompts across model versions; behavior changes between releases.
• Few-shot examples should cover edge cases, not just the happy path.

Common Pitfalls

• Writing vague instructions and expecting the model to infer intent.
• Providing too many conflicting instructions that the model cannot satisfy simultaneously.
• Using few-shot examples that are all too similar, teaching the model a narrow pattern.
• Ignoring token limits and truncating critical context.
• Not testing with adversarial or out-of-distribution inputs.
• Over-engineering prompts when a simpler instruction would suffice.

Output Format

When delivering a prompt design:

1. Task Definition: What the prompt accomplishes.
2. Full Prompt Text: The complete prompt with all sections labeled.
3. Variable Placeholders: Clearly marked dynamic fields (e.g., {{user_input}}).
4. Recommended Parameters: Temperature, max tokens, stop sequences.
5. Test Cases: 3-5 example inputs with expected outputs.
6. Known Limitations: Scenarios where the prompt may fail.