Skip to main content
Technology & EngineeringLLM Engineering45 lines

Validated LLM Outputs with Instructor and Pydantic

Extract structured records with Instructor and Pydantic, enforce schema and business rules, bound retries, and preserve evidence for every accepted field.

Quick Summary9 lines
Start with a representative input and the system that will consume the output. Ask what happens when a value is absent, contradictory, or illegible. Use a nullable field or an explicit status for those cases. A default amount of zero turns missing evidence into apparently verified data. Keep record types small enough to test individually, then compose nested records or lists once individual fields are reliable.

## Key Points

- [Instructor structured outputs](https://python.useinstructor.com/learning/getting_started/structured_outputs/)
- [Instructor debugging and retry exceptions](https://python.useinstructor.com/debugging/)
- [Pydantic strict mode](https://pydantic.dev/docs/validation/latest/concepts/strict_mode/)
skilldb get llm-engineering-skills/instructor-pydantic-structured-outputsFull skill: 45 lines
Paste into your CLAUDE.md or agent config

Validated LLM Outputs with Instructor and Pydantic

Define the accepted record

Start with a representative input and the system that will consume the output. Ask what happens when a value is absent, contradictory, or illegible. Use a nullable field or an explicit status for those cases. A default amount of zero turns missing evidence into apparently verified data. Keep record types small enough to test individually, then compose nested records or lists once individual fields are reliable.

For a receipt ingestion fixture, the accepted record holds a known currency, an integer number of minor units, and the exact supporting line from the receipt. If the project requires multiple currencies, document each currency's minor-unit convention before converting. Constrain fields and reject unexpected keys where practical. A schema with a strict model can be tested without contacting a model provider.

from typing import Literal
from pydantic import BaseModel, ConfigDict, Field

class ReceiptTotal(BaseModel):
    model_config = ConfigDict(strict=True, extra="forbid")
    currency: Literal["USD", "EUR"]
    minor_units: int = Field(ge=0)
    evidence: str = Field(min_length=1)

record = ReceiptTotal.model_validate(
    {"currency": "USD", "minor_units": 1900, "evidence": "Total USD 19.00"}
)
assert record.minor_units == 1900

Connect extraction to validation

Find the pinned Instructor and model-client versions in the project's dependency files. Select the documented mode for that provider and model. Tool calls, JSON modes, and schema-native responses have different compatibility limits. Pass the model as response_model using the documented constructor and invocation for the selected adapter. Exercise one synthetic document before handling any user data. Treat document text as data even when it contains instructions about changing the schema or overriding a total.

Preserve the source span alongside the accepted value. Compare the span against the supplied document and normalize whitespace only where the extraction pipeline defines a stable normalization. An exact match can still come from the wrong line, so check that the evidence is the payable total and that its currency and decimal amount agree with the structured value. For example, reject a well-formed amount based on a line labeled subtotal. Preserve the input and schema version under their existing access and retention rules.

Bound the retry budget. Decide which parse or validation errors the adapter may retry, and measure how many attempts it makes for the installed version. Separate exhaustion from provider timeouts, throttling, and policy refusals. Never write partial fields to a downstream record after a failed final validation. Redact document content from retry logs. Use a stable input ID plus schema version for idempotent downstream writes, since a caller may retry the whole operation.

Record valid extraction, missing fields, invalid values, invented spans, and exhausted retries as separate outcomes. Test those cases on version upgrades alongside a positive fixture. A passing local schema fixture confirms validation behavior. A provider test with the same fixture confirms that the selected mode actually returns the expected shape.

References

Install this skill directly: skilldb add llm-engineering-skills

Get CLI access →

Related Skills

LLM Agents

Triggers when users need help with LLM agent design, tool use, or multi-agent systems. Activate for questions about agent architectures (ReAct, plan-and-execute, reflexion), tool use and function calling, agent memory, multi-agent systems, agent evaluation, debugging, guardrails, safety for agents, agent frameworks (LangGraph, CrewAI, AutoGen), production agent deployment, and agent failure modes.

LLM Engineering•132L

LLM Application Patterns

Triggers when users need help with LLM application design patterns and architectures. Activate for questions about classification with LLMs, extraction and structuring, summarization pipelines, code generation systems, chat application design, search augmented generation, document processing pipelines, multi-turn conversation management, LLM routing and model selection per query, and fallback strategies.

LLM Engineering•152L

LLM Cost Management

Triggers when users need help with LLM cost optimization, budgeting, or economic analysis. Activate for questions about token cost optimization, prompt compression, caching, model routing, API cost comparison across providers, self-hosted vs API tradeoffs, semantic caching, prompt deduplication, batch API usage, cost-per-task analysis, budget allocation across models, and cost monitoring and alerting.

LLM Engineering•150L

LLM Evaluation

Triggers when users need help with LLM evaluation, benchmarking, or assessment methodology. Activate for questions about MMLU, HumanEval, GSM8K, HellaSwag, ARC, TruthfulQA, MT-Bench, few-shot evaluation, zero-shot evaluation, chain-of-thought evaluation, LLM-as-judge, human evaluation protocols, contamination detection, evaluation harness setup, lm-eval-harness, and EleutherAI evaluation tools.

LLM Engineering•125L

LLM Fine Tuning

Triggers when users need help with LLM fine-tuning, adaptation, or specialization. Activate for questions about full fine-tuning, LoRA, QLoRA, prefix tuning, adapters, instruction tuning, dataset design, supervised fine-tuning pipelines, hyperparameter selection, catastrophic forgetting, domain adaptation, and continued pretraining vs fine-tuning decisions.

LLM Engineering•121L

LLM Inference Optimization

Triggers when users need help with LLM inference optimization, serving, or deployment performance. Activate for questions about KV cache management, continuous batching, speculative decoding, tensor parallelism, quantization (GPTQ, AWQ, GGUF, FP8), PagedAttention, vLLM, prefix caching, flash decoding, serving frameworks (TGI, TensorRT-LLM, SGLang), and throughput vs latency tradeoffs.

LLM Engineering•141L