Prompt Layering: The Five Layers That Control Your Agent

I used to write one giant system prompt for my agents. Every rule, every tool description, every project convention, all in one block. It worked until I needed the agent to behave differently during planning versus implementation. Then I was maintaining two giant prompts with 80% overlap.

The fix was treating the prompt like a layered protocol, not a monolith.

Five layers, each with one job

Every prompt my agents receive is assembled from five distinct layers, in this order:

1. System prompt — identity, safety policy, tool rules, hard constraints. This never changes mid-session. It's the constitution.
2. Runtime prompt — current phase (context / action / verify), allowed tools this turn, iteration count, approval mode. This changes every turn.
3. Session memory — task summary, plan, past observations (summarized), decisions made. This accumulates across iterations.
4. Retrieved context — file excerpts, symbol definitions, recent diffs relevant to the current subtask. This is fresh each turn.
5. User message — the original goal or follow-up instruction.

Separation matters because each layer has a different lifecycle. The system prompt is written once. The runtime prompt changes per phase. Session memory grows. Retrieved context is recalculated. Mixing them creates a mess you can't maintain.

The response schema is the real control

The most powerful part isn't the prompt layers — it's forcing the model to respond in a structured schema:

response_schema = {
  type: "plan" | "tool_calls" | "final",
  user_summary: string,     // shown to user — brief
  internal_notes: string,   // logged, not shown — reasoning
  calls: [{ tool, args, reason }],
  output: string
}

This gives you stepwise behavior without visible chain-of-thought. The model reasons in internal_notes (which you log for debugging) and communicates with the user via user_summary (which is clean and concise). You get the benefit of reasoning without the noise.

Why the system prompt is immutable

Changing the system prompt based on model output creates feedback loops. The model's behavior affects its own instructions, which affects its next behavior. This is non-deterministic and impossible to debug.

The system prompt is the one thing that stays constant. If you need the agent to behave differently in different phases, that goes in the runtime prompt (layer 2), not the system prompt (layer 1).

The anti-pattern: no response schema

Without a schema, the model responds in free-form text. Your runtime has to parse natural language to figure out what the model wants to do. "I think we should edit the file" — is that a plan statement or a request to use the edit tool? With a schema, there's no ambiguity: it's either type: "plan" or type: "tool_calls".

The takeaway

Treat your prompt like infrastructure. Layer it. Version the system prompt. Make the runtime prompt reflect the current phase. And force structured responses — the schema is where discipline happens.

Next: Why Context Is Everything in Agent Design