AI Agent Planning in Practice: Task Decomposition, Plan Validation, Replanning, and Failure Recovery

Who This Is For

• System architects moving from single-agent workflows to complex multi-agent collaboration.
• Frontline developers focused on communication latency, network overhead, and debugging logs for multi-agent systems in distributed environments.
• Technical decision-makers who need to evaluate the implementation value and cost of multi-agent applications in specific vertical business scenarios.

I. Deep-Water Reasoning: Why Your Agent Keeps Going Off Course

The hard part of LLM planning is not generating a plausible-looking to-do list. It is controlling an agent’s execution state and self-healing path when the agent encounters an unknown external environment.

In agent development, we often use this formula: Agent = Large Language Model + Planning + Memory + Tool Use. Planning is the brain’s navigation system. Without a reliable planning mechanism, an agent is merely an obedient but blind instruction executor. Once the task chain grows longer or a tool encounters an unexpected network disruption, the agent drifts logically or spins on the same error.

A representative long-task failure looks like this: the plan needs to read several documents, but one PDF is damaged or the parser repeatedly fails. Without error classification, stop conditions, and replanning, the agent may repeat the same step; without explicit state, later steps can also continue while key evidence is missing.

This problem does not need an invented personal production incident to make the point. Planning engineering should make the plan, state, errors, stopping conditions, and replanning decisions inspectable so the system can decide whether to repair arguments, choose another path, preserve partial work, fail safely, or hand off to a person.

II. Recommended Planning Topology: Production-Grade Agent Planning Architecture

A production-grade planning architecture must isolate goal parsing, step validation, task dispatch, and dynamic replanning into separate modules.

To keep the agent on course while it executes long task chains, I designed the planning engine around the following core topology:

user (User Goal)
  │
  ▼
 (Goal Parser)
  │
  ▼
task (Task Decomposer)
  │
  ▼
validate (Plan Validator - )
  │
  ▼
state (State Store) ◄─────────┐ (state/)
  │                                 │
  ▼                                 │
tool/ (Tool Selector) │
  │                                 │
  ▼                                 │
stepExecute (Step Executor) ─────────┼─► (Replanner - trigger)
  │                                 │
  ▼                                 │
 (Stop Controller) ───────┘ ()
  │
  ▼
Final output (Final Answer)

In this topology, the user’s raw input is first parsed and decomposed into strongly typed substeps, or Task Steps. Before execution, the Plan Validator filters out invalid or high-risk actions. After each step runs, its result, or Observation, is written to the State Store. If the execution engine detects a serious tool error, it does not stop immediately. Instead, it passes the error context to the Replanner, which dynamically revises the remaining plan while preserving completed steps as Checkpoints.

III. Task Decomposition: Turning Unstructured Goals into Strongly Typed Steps

Task decomposition must produce structured JSON with dependencies, tool mappings, and risk controls, not a loose set of natural-language notes.

Many demos ask the model for a natural-language to-do list and then treat it as an executable plan. That makes dependency, permission, parameter, and recovery checks difficult, and there is no need to require the model to expose private chain-of-thought.

A better approach is an execution-oriented structured plan: step ID, goal, candidate tool, dependencies, argument schema, expected output, and risk level. The model’s private reasoning does not need to become part of the execution protocol or logs.

The following is an example of the structured step model I use in a Python planning engine:

from typing import List, Dict, Any, Optional
from pydantic import BaseModel, Field

class TaskStep(BaseModel):
    step_id: str = Field(..., description="Step id")
    step_goal: str = Field(..., description="Step goal")
    required_tool: str = Field(..., description="Required tool")
    input_arguments: Dict[str, Any] = Field(default_factory=dict, description="Dictionary of arguments passed to the tool call")
    expected_output: str = Field(..., description="Expected output")
    dependency_id: Optional[str] = Field(None, description="ID of the prerequisite step on which this step depends")
    risk_level: str = Field("low", description="Operational risk rating for this step (low, medium, high)")
    status: str = Field("pending", description="Step status (pending, running, completed, failed)")

With a strongly typed step dictionary like this, the execution engine can perform extensive validation in advance. It can check whether a dependency_id refers to a step that has already completed successfully, verify that required_tool is registered in the current Tool Registry, and confirm that the argument types in input_arguments conform to the tool’s JSON Schema. These checks make the plan reliable at the code level.

IV. Plan Validation: Run Defensive Checks Before Tool Execution

Plan validation is the first line of defense against agents abusing high-risk privileges or executing invalid actions, and it must happen before tasks are dispatched.

After decomposing a long task chain, we must never send the plan directly to the Executor. During planning, an LLM can generate incorrect parameters because it does not fully understand a local tool’s description. A malicious prompt injection can also lead it to create high-risk steps, such as deleting the root directory or downloading sensitive files without authorization.

An independent Plan Validator must therefore apply the following mandatory checks to every TaskStep after plan generation and before execution:

1. Parameter Completeness and Format Review

The validator parses each step’s input_arguments and checks for missing required parameters, such as whether an order-details step actually extracted an order_id. If a parameter is missing, the validator rejects the plan locally and asks the LLM to extract it again. It does not call the database API, thereby avoiding pointless network latency.

2. Permission and High-Risk Action Blocking

If an LLM-generated step contains a high-risk operation, such as a risk_level of high or an attempt to write to a database, issue a refund, or send a group notification, the validator forcibly suspends execution and asks the control layer for human-in-the-loop (HITL) confirmation. This prevents the agent from completing dangerous actions autonomously.

3. Tool Availability Check

The validator checks whether the required_tool supplied by the LLM exists in the system’s allowlist. If the model hallucinates a nonexistent tool such as auto_reboot_server, the validator catches the error locally, returns an error message, and forces the LLM to self-correct and replan.

V. Execution and Reasoning Loop: Take a Step, Observe, and Update

Planning is not a one-time static workflow. It is a reasoning loop that uses real-time Observations to make dynamic corrections.

As LLM planning algorithms have evolved, static One-shot Planning has often performed poorly on long task chains because the physical environment is highly dynamic and uncertain. For example, the first step may be to retrieve a web page, but the page returns a 403 access error. A static plan will press ahead with data analysis in step two and ultimately produce a meaningless empty report.

Production systems therefore need a dynamic ReAct reasoning loop: Thought -> Action -> Observation -> State Update.

1. Thought: The model considers what to do next based on the global plan and current session state.
2. Action: It calls a local tool or executes a specific subtask.
3. Observation: The system captures the actual result returned by the tool, whether successful data, error text, or a timeout state.
4. State Update: The system appends the Observation to the State Store and updates the cache of completed steps.
5. Decision: The controller determines whether the task is complete because all steps finished, needs replanning because the current step failed with low confidence, or must trip a circuit breaker because it reached the maximum step count.

By introducing dynamic Observation feedback at every step, we place the LLM’s black-box reasoning on a track that code can intercept and observe in real time, greatly improving execution safety.

VI. Architecture Selection: ReAct, Plan-and-Execute, or Hybrid Orchestration

Depending on latency constraints and branching complexity, you must weigh ReAct, static planning, adaptive replanning, and Workflow approaches.

In real projects, different planning algorithms have dramatically different communication overhead and use cases. I have organized the common approaches into the following decision table for use during architecture design:

Planning Mode	Core Logic	Latency	Token Cost	Best Use Cases	Limitations
ReAct	Chooses the next action from the latest Observation	Depends on model turns and tool latency	Depends on context, model, and turns	Exploratory work, debugging, tasks where tool results determine the next step	Needs explicit termination/no-progress detection; context may grow over long chains
Plan-and-Execute	Generates a structured plan, then executes and validates steps	Depends on whether execution still requires model decisions	Depends on plan generation, validation, and repair	Clear goals where dependencies can be described ahead of time	External changes can invalidate the plan; revalidation/replanning may still be required
Adaptive Replanning	Revises the remaining plan when new evidence changes the path	Adds replanning overhead when triggered	Depends on trigger frequency and context size	Long chains, unstable dependencies, partial-result preservation	State, idempotency, and validation of the new plan are more complex
Workflow + Agent	Code defines the deterministic skeleton; agentic nodes handle local uncertainty	Can reduce unnecessary model decisions, but must be measured	Depends on the number and complexity of agentic nodes	Approval, transactions, compliance, deterministic SOP plus local semantic decisions	Requires maintaining both workflow rules and agent boundaries
Multi-Agent Planning	Multiple specialist agents divide work, hand off, or verify one another	Often adds coordination/model calls, but not by a fixed factor	Depends on team pattern, parallelism, and termination	Complex cross-domain tasks where a single agent has proven insufficient	Harder state, authorization, debugging, and cost attribution

VII. Dynamic Replanning: Reconstructing the Path After Failure

When a tool times out or returns unexpected data, the agent must be able to revise its original plan from the latest context and recover on its own.

Tool-call failures are among the most common production problems. For example, an agent may plan to use search_google for current data, only for the search API to return a Rate Limit Exceeded error. In a basic agent demo, it may simply retry until it consumes every token. A production system with replanning should instead do the following:

1. Capture the actual error state returned by the tool.
2. Preserve the status of completed steps, such as step_1 and step_2.
3. Invoke the Replanner and send the LLM the failed step ID, the specific failure reason, such as API rate limiting, and the remaining task objective as context.
4. Have the LLM generate a revised plan under the new constraint—for example, abandoning search_google in favor of a local read-only database cache or calling search_bing as a fallback.
5. Load the new plan into the execution engine, move the execution pointer to the new step, and continue.

Replanning is valuable when new evidence actually changes the remaining path: the system can preserve confirmed work and recompute what comes next. Whether this improves task success, and by how much, must be measured on the same evaluation set with and without replanning; do not assume a 70%→95% improvement.

VIII. Stop Conditions and Hard Circuit Breakers: Prevent Infinite Loops

To prevent an agent from entering an infinite planning loop after encountering a logic flaw, the system needs strong circuit breakers for global step counts and resource budgets.

If an LLM receives unclear guidance during replanning, or if the new plan still depends on a broken tool, it can enter a Planner Loop: Generate plan -> Execution error -> Trigger replanning -> Generate the same plan -> Repeat the same error

The orchestration layer should enforce stop conditions, but the thresholds must come from workload baselines and failure cost:

• Maximum steps/model calls: size the hard budget from normal task distributions and product limits rather than fixing every task at 15 steps.
• Repeated same-tool/same-argument failures: treat repeated no-progress calls as a signal to classify the error, stop the path, or escalate. Three attempts can be an example starting point, not a standard.
• Maximum replanning count: calibrate from task length and replanning cost; terminate earlier when consecutive plans are materially equivalent.
• Token/cost/wall-clock budget: derive limits from model pricing, user value, SLOs, and task type rather than using 100,000 tokens as a generic line.

A more robust Stop Controller combines max_steps, max_model_calls, wall-clock timeout, repeated Tool+Args, repeated state with no new evidence, token/cost budget, and external cancellation.

After stopping, the system should not merely throw a generic 500 error. It should return a structured diagnosis that identifies which steps completed, where and why execution became stuck, and what information or parameters the user must provide manually, giving a human operator clear audit material for taking over.

IX. Failure Recovery and State Rollback: Build a Self-Healing System with Checkpoints

A high-confidence planning system needs state rollback so an agent can return to the previous checkpoint after a temporary network disruption.

Data consistency is one of the hardest problems in executing long, complex plans. Consider this agent plan: Step 1: Create a new user in the local database (SQL Write) -> Step 2: Register an API Key with a third-party provider (Network Call) -> Step 3: Send the customer an activation email.

If the network call in step two times out, starting from the beginning during replanning or retrying will repeat the SQL Write in step one and create dirty duplicate data in the database.

To solve this problem, the planning engine must support state Checkpoints and rollback:

1. Checkpoint mechanism: Each time a substep executes successfully and its state is updated, the system saves a state snapshot for the current tenant session in the database, including the current variable dictionary and hashes of tool return values.
2. Local idempotency check: Every tool that performs a write, such as create_user, must implement idempotency in code. It checks a unique external business identifier and returns success without writing again if the record already exists.
3. Recovery and compensation: After a failure, restore workflow state from the latest confirmed durable checkpoint—but do not treat that as a database transaction rollback. SQL/API/email side effects that may already have committed must be reconciled through an idempotency ledger, compensating action, or human review before replay.

X. Planning Evaluation Metrics

Planning evaluation must break performance down into multiple technical and business dimensions, including plan validity, tool selection accuracy, and replanning success rate.

Evaluate planning changes on a fixed task set and fixed success criteria instead of starting from production-looking percentages:

1. Technical Planning Metrics

• Plan validity rate (plan_validity_rate): the share of generated plans that pass formatting, tool, argument, and dependency checks. Release gates should come from current baselines and business risk rather than a universal 97% target.
• Logic drift / no-progress rate (loop_rate): the share of tasks with repeated actions, repeated state, or clear divergence from the requested goal.
• Replanning success rate (replan_success_rate): the share of replanned tasks that still achieve the original goal. Segment by failure class and model/version instead of quoting an unsupported “88% with Sonnet 3.5.”
• Steps per task (steps_per_task): use average and tail distributions to find abnormally long paths. Fewer steps are not automatically better; compare success, tool use, cost, and quality together.

2. Business Planning Metrics

• Final task completion rate (task_completion_rate): The percentage of tasks in which autonomous planning and execution ultimately achieve the user’s expected goal.
• Partial success rate (partial_success_rate): The percentage of tasks that successfully return completed intermediate results when a physical constraint prevents further execution. We favor partial success over a complete error.
• Average cost per task (cost_per_completed_task): The average amount in US dollars consumed to complete one complex planning task successfully.

XI. Common Design Mistakes and Error-Log Troubleshooting

A poorly designed planning engine is highly susceptible to logical drift, recursive explosion, and expensive infinite loops, so it needs precise safeguards.

The following are constructed failure scenarios for planning regression tests and troubleshooting, not claims that XBSTACK experienced these production incidents. The step counts, similarity threshold, and context length in the sample logs are illustrative; replace them with your own traces and task baselines:

1. Logic Drift

• Symptom: After executing more than 8 steps, the agent completely diverges from the user’s original objective and starts spinning through irrelevant side-path tools.
• Error message:

  Warning: [LOGIC_DRIFT_DETECTED] Task 'task-1012' current thought similarity to raw user_goal 'Generate Q2 Budget Report' has fallen below threshold 0.40. Current focus: 'Translating read_file debug comments to French'.
  

• Root cause: As execution steps and Observation logs accumulate, detailed tool results fill the model’s context window. The LLM’s native Attention mechanism decays, causing it to forget the original Goal.
• Troubleshooting: In the prompt template for each round of the Reasoning Loop, force the user’s raw_user_goal to appear at the very bottom of the Prompt, close to the model output area. Also add an assertion during the Thought stage that checks the relationship between the current action and the user’s ultimate goal. If the cosine similarity between the current action and the original goal falls below the threshold, forcibly truncate the context, keep only the Task Steps skeleton and current Observation, and inject the main goal again.

2. Vague Observation Loop

• Symptom: A tool fails with a vague error such as Error 500. Because the Agent cannot determine the cause during the Thought stage, it mechanically repeats the same tool call.
• Error message:

  Error: [PLANNER_RETRY_LOCK] Agent repeated step_3 'query_postgres' 3 times with identical arguments. Reason: Tool returned 'Internal Server Error (500)'.
  

• Root cause: The local tools passed raw HTTP status codes or null-pointer exceptions directly to the model instead of wrapping errors with semantic context. Without enough clues, the LLM cannot self-reflect effectively or adjust its parameters.
• Troubleshooting: When writing Tool functions, wrap underlying exceptions and return clear, meaningful error descriptions. For example, instead of returning 500, return: Tool execution failed: The Postgres SQL query timed out because missing indexes caused a full table scan. Try adding a LIMIT or optimizing the WHERE clause. Only an Observation with this kind of technical diagnosis allows the LLM to produce a correct self-healing plan during Replanning.

3. Recursive Decomposition Explosion

• Symptom: The LLM over-decomposes a simple task, such as “send the user an email containing an invoice,” into dozens of tiny atomic steps. The Context overflows within seconds and produces a large bill.
• Error message:

  Fatal: [CONTEXT_OVERFLOW] Planning step count reached 24. Task decomposition generated nested sub-plans. Context window exceeded 128,000 tokens at step 'step_18_verify_cc_email_format'.
  

• Root cause: The decomposition Prompt does not constrain decomposition depth or step granularity, so the LLM becomes excessively meticulous and isolates every supporting format check as a separate step.
• Troubleshooting: Constrain decomposition depth and granularity, and keep deterministic checks such as email format, filenames, and schemas in code rather than turning each mechanical validation into an agent step. A step-count budget is useful, but 3–8 is not a universal optimum; calibrate it from normal tasks and success rates.

XII. Continue Reading

To deploy high-confidence agents in production, you need to learn how to introduce robust governance rules at every level of the process.

• 👉 AI Agent Architecture in Practice: From Prompts to Production-Grade Agent Systems
• 👉 AI Agent Tool Use: A Practical Guide to Tool Calls and API Integration
• 👉 AI Agent Memory System: A Deep Dive into Long-Term and Short-Term Memory Design
• 👉 AI Agent Observability: A Guide to Opening the Agent Execution Black Box
• 👉 AI Agent Evaluation: Four Core Dimensions and Practical Metrics for Quantifying Agent Performance
• 👉 AI Agent Deployment: Production-Grade Architecture and High-Concurrency Scaling
• 👉 Multi-Agent Systems in Practice: Architecture Boundaries, State Handoffs, and Failure Control
• 👉 Multi-Agent Planning: An In-Depth Guide to Planning Strategies for Multi-Agent Collaboration
• 👉 AI Agent Frameworks Compared: LangChain vs. AutoGen vs. CrewAI
• 👉 Microsoft AutoGen Agents in Practice: A Progressive Development Guide
• 👉 MCP vs. A2A vs. Function Calling: Protocol Selection and System Integration for AI Agents
• 👉 Complete AI Agent Engineering Guide

External references:

• LangGraph overview: durable execution, state, and HITL
• LangGraph persistence: checkpoints and recovery boundaries
• AutoGen AgentChat Teams: current team orchestration and termination
• Microsoft Semantic Kernel planning
• OpenAI Structured Outputs

Topic path / AI Agents

Continue from one agent pattern to the complete production system

The AI Agent hub organizes architecture, memory, tool use, evaluation, security, deployment and multi-agent coordination into a single learning path.

Open AI Agent hub → Review the architecture guide Subscribe