Skip to content

Guardrails: middleware and policy hooks

Maps to: Guardrails: middleware.

Scope

Deterministic policies around model and tool calls: redaction, moderation, rate limits, fallbacks, and human gates expressed in code, not only in prompts.

Design questions

  • Which hooks run on every model call and every tool call in all interaction modes (stream, background, HITL resume)?
  • How are policies versioned and rolled out without breaking in-flight runs?
  • Where do application-specific rules live versus shared platform middleware?

Tradeoffs

  • Prompt-only guardrails fail when the model ignores instructions under pressure.
  • Aggressive redaction can degrade task quality; insufficient redaction fails compliance.
  • Middleware ordering matters when multiple policies compose.

Evaluation hooks

  • Adversarial inputs with PII; assert redaction before model and in exported traces per policy.
  • Confused model repeatedly calling paid APIs; assert hard ceiling and alerting.
  • Regression suite when middleware chain changes.

Reference notes

See LangChain runtime article (middleware model flow figure).