Multi-Agent Systems in Practice: Architectural Boundaries, State Handoffs, and Failure Control

Who Should Read This

• System architects shifting from single-agent workflows to complex multi-agent collaborative systems.
• Frontline developers focusing on communication latency, network overhead, and debugging logs for multi-agent environments.
• Technical decision-makers who need to assess the value and cost of deploying multi-agent applications in specific verticals.

1. Architectural Choice: Don’t Use Multiple Agents for Their Own Sake

Introducing a multi-agent architecture should be a defensive response to task complexity and increasing model-attention entropy, not the default choice simply because the technology appears advanced.

With Multi-Agent Systems (MAS) surging in popularity, many developers have fallen into the misconception that more agents make a system smarter. They try to use a complex cluster containing a Planner, Coder, Tester, and Reviewer for a simple task that a single agent and a few Python tools could easily handle.

Multi-agent systems are not a free lunch. Every additional Agent that requires a separate model call or state handoff usually adds some latency, token/call cost, context maintenance, and debugging complexity, but the amplification depends on team topology, parallelism, model price, message sharing, and tool-result size. If one Agent plus tools already meets the acceptance bar, there is no reason to split roles for appearance alone. If the business path is deterministic, compare a normal code workflow against agent orchestration on real cost and maintenance effort.

The useful multi-agent benefit is usually responsibility isolation: independent retrieval, generation, review, execution, or cross-system delegation can have separate context and permissions. Whether that actually outperforms one agent must be proven on the same task set rather than assumed from agent count.

2. Scenario Fit: When Should You Introduce Multi-Agent Systems?

Multi-agent systems are best suited to complex scenarios that require independent roles, bidirectional review, and collaboration across separate systems, not linear pipelines.

Before deciding whether to refactor a system into a multi-agent architecture, I usually classify the business scenario into the following two categories in my local development environment in Guiyang:

1. Scenarios Where MAS Is Strongly Recommended

• Complex open-ended research: for example, searching a large number of web pages in real time, synthesizing technology trends, and producing competitor reports. This requires a Searcher Agent to filter for signal, a Writer Agent to draft the report, and a Critic Agent to identify errors and audit against an independent evidence trail.
• High-risk decision review: for example, legal contract audits or financial-statement reconciliation. To prevent a model from compromising with itself, a generation Agent must be paired with an independent review Agent for adversarial review and cross-examination.
• Cross-organization and cross-toolchain collaboration: when agents are implemented by different teams, languages, or frameworks and must delegate across separate security boundaries, evaluate A2A, MCP, HTTP APIs, queues, or custom event contracts. A2A is one standardized option for agent-to-agent collaboration, not the only integration pattern.

2. Scenarios Where Multi-Agent Systems Should Not Be Used

• Single-turn or low-frequency Q&A: for example, a simple enterprise intranet FAQ knowledge base can use RAG + prompts directly.
• Highly deterministic approval processes: for example, leave approvals or automatic deductions from reimbursement limits. Logic with clear rules should be implemented directly with traditional CRUD operations and conditional branches instead of consuming expensive LLM inference capacity.
• Simple single-tool calls: if the agent only needs to query the weather once or read a database in response to an instruction, a single Agent with Function Calling is the most efficient approach.

3. Five Core Collaboration Architecture Models and Their Practical Risks

Production-grade multi-agent collaboration must begin with clearly defined responsibilities for each node to prevent redundant internal work caused by overlapping logical boundaries.

In industrial-grade MAS design, five collaboration and orchestration patterns are common. Each has a different division of responsibilities and a different set of boundary risks:

1. Supervisor-Worker Pattern (Manager and Executors)

• Structure: The central Supervisor receives the user’s task, decomposes it, assigns work to specialized Workers, and formats and consolidates the Workers’ results while providing final quality control.
• Suitable scenarios: routing for multi-channel intelligent customer service and cross-domain document analysis.
• Physical risk: the Supervisor becomes a single point of failure. If it hallucinates while classifying a task—for example, routing a legal refund request to an FAQ Worker—the entire chain goes off course, and lower-level Workers cannot recover on their own.

2. Planner-Executor Pattern (Planner and Executor)

• Structure: The Planner is responsible only for decomposing a long-running task and defining its phases, then sending specific instructions to the Executor. After execution, the Executor returns the actual result to the Planner, which decides whether to proceed or replan.
• Suitable scenarios: complex automated software deployment, multi-step data cleaning.
• Physical risk: if the Planner decomposes the task too finely, communication costs consume large numbers of Tokens. If the decomposition is too coarse, the Executor frequently produces errors because local tool calls lack required inputs.

3. Researcher-Writer-Reviewer Pattern (Research, Writing, and Auditing)

• Structure: this is the classic content-production triad. The Researcher calls retrieval tools to collect verifiable facts; the Writer assembles text from those facts; and the Reviewer acts as a strict auditor, checking whether every sentence from the Writer is supported by the original documents collected by the Researcher.
• Suitable scenarios: industry research reports and automated proofreading of technical documentation.
• Physical risk: the Reviewer can compromise on quality. Without constraints, it may read only the Writer’s summary instead of checking the original Research evidence trail, allowing hallucinated facts to pass.

4. Debate / Critic Pattern (Two-Role Adversarial Debate)

• Structure: two Agents using similarly capable models debate the same proposal, such as a software architecture or investment target, from opposing positions over multiple rounds. They look for weaknesses until they reach consensus or hit the hop limit.
• Suitable scenarios: evaluating high-risk investment proposals and automated security-vulnerability discovery.
• Physical risk: meaningless pleasantries and bloated output. Without strong external factual constraints, the debate can easily degrade into a pointless loop that burns large numbers of Tokens.

5. Agent Handoff Pattern (Agent Handoff Routing)

• Structure: each Agent has its own role. When the FAQ Agent detects a refund request, it passes the current session’s history and state variables like a baton to a dedicated Refund Agent.
• Suitable scenarios: complex, distributed, multifunctional business systems.
• Physical risk: context loss. If the shared process memory or state-persistence layer fails to pass key session variables—such as an already recognized order_id—the receiving Agent will ask the user for the information again, creating a severe break in the experience.

4. State Handoff: Transitioning from Natural Language to Strongly Typed Structured Data

State transfer between Agents must use a versioned, strongly typed JSON dictionary. Vague natural-language summaries must never serve as the handoff context.

When developing a multi-agent system, the worst engineering mistake is to have Agents hand off tasks using only conversational natural language. When reading a summary written by the previous Agent, a large model often misses technical details such as exact IDs, execution times, and parameter hashes.

To make multi-agent collaboration highly deterministic, we must design a unified, versioned, strongly typed state-handoff dictionary.

Below is an example of a structured handoff-state data class that I defined in the Python collaboration gateway to preserve data integrity when switching between nodes:

from typing import Dict, List, Any, Optional
from pydantic import BaseModel, Field

class AgentState(BaseModel):
    task_id: str = Field(..., description="Globally unique task ID")
    parent_trace_id: str = Field(..., description="Trace ID for end-to-end monitoring")
    current_agent: str = Field(..., description="Name of the Agent node currently holding control")
    target_agent: str = Field(..., description="Name of the next target Agent node for the handoff")
    
    # Business context and state data
    user_goal: str = Field(..., description="User's ultimate input intent")
    task_context: Dict[str, Any] = Field(default_factory=dict, description="Business-state context key-value pairs")
    
    # Execution-pipeline tracking
    completed_steps: List[str] = Field(default_factory=list, description="List of successfully completed substeps")
    tool_results: Dict[str, str] = Field(default_factory=dict, description="Raw historical results from tool calls")
    
    # State assessment and exception metrics
    confidence: float = Field(1.0, ge=0.0, le=1.0, description="Confidence score for the current step")
    failure_reason: Optional[str] = Field(None, description="Specific error text recorded when an exception occurs")
    handoff_reason: str = Field(..., description="Semantic rationale that triggered the task transition")

At runtime, when the Coder Agent completes the code and needs to route state to the Reviewer Agent, key business fields can be serialized into a version-checked AgentState. The Reviewer can then read task_context, completed_steps, and the required tool evidence. Structured state substantially reduces the risk that a natural-language handoff drops an ID or parameter, but it cannot guarantee zero state loss: schema evolution, concurrent writes, serialization failures, stale-worker replay, and external-system state still need versioning and recovery tests.

5. Communication Overhead and Latency: The Concrete Cost of Multi-Role Interaction

Although separating roles can improve output accuracy, it also brings substantial costs in Token consumption, network-call latency, and state-synchronization entropy.

How much more expensive a multi-agent design becomes must be measured on your own task set. One unarchived seconds/token comparison is not a universal benchmark. A more defensible test keeps input, tool permissions, and pricing assumptions constant: use a single agent as the baseline, add Planner, Reviewer, or specialized Workers one at a time, and record model-call count, input/output tokens, P50/P95 latency, tool calls, task success, and human correction rate. An extra role is justified only when the quality gain covers the additional cost and complexity.

When deciding whether to use MAS, measure at least:

• Token and call amplification: attribute input, output, cache behavior, and tool-result volume to each role rather than assuming communication tokens always dominate.
• Interaction latency (P95/P99): serial model rounds accumulate delay, while parallel calls add resource and aggregation cost. Define an explicit latency SLO before enabling a multi-agent path for real-time traffic.
• State consistency overhead: when Workers update the same business object concurrently, choose an explicit concurrency strategy such as transactions, optimistic versions, queue serialization, idempotent writes, or key partitioning. A distributed lock is not universally required.

6. Failure Propagation: Local Collapses in Multi-Agent Clusters and Chain Reactions

The primary risk in a multi-agent system is the indiscriminate amplification of upstream errors. Critical nodes require hard interception points and self-healing rollback mechanisms.

In MAS, mistakes are contagious. Without intervention, a tiny local error can quickly turn into a global logical disaster.

The classic failure transmission chain is as follows:

1. Stage One (Planner drift). The user wants to check the R&D budget for Q1 2026, but the Planner hallucinates and decomposes the task for Q1 2025.
2. Stage Two (blind Worker execution). The Worker Agent responsible for SQL received instructions for 2025 and faithfully queried and returned the database’s 2025 data.
3. Stage Three (false Reviewer approval). The Reviewer compared only the Worker’s SQL with the Planner’s plan, found that both specified 2025, and passed the consistency check.
4. Stage Four (summary and return). The user ultimately receives an exceptionally well-formatted, logically structured budget report whose core factual data is entirely wrong.

To break this chain of error amplification, a multi-agent architecture needs two hard lines of defense: First, Reviewers must not inspect only the final summary. Review nodes need permission to call the underlying MCP tools and retrieve raw inputs, such as the user’s initial question, compare the Worker’s output with the user’s original intent, and perform end-to-end semantic validation. Second, critical nodes need state rollback. When the Reviewer determines that the Worker’s output is inadequate, it cannot simply retry in place. It must send a reset command and error log to the Planner, forcing the Planner to move the execution pointer back one step and decompose the plan again.

7. Evaluation Metrics: How Do You Measure the Health of Multi-Agent Collaboration?

Multi-agent evaluation must not rely solely on the final output similarity score; instead, it must break down and monitor each subnode’s call frequency, handover success rate, and marginal resource contribution.

If we only use a simple LLM-as-a-judge to score the final article or report when evaluating multi-agent systems, we cannot judge the effectiveness of the division of labor among each Agent role. We might end up spending dozens of times more tokens, introducing five agents, but the final effect is no different from a single agent.

To achieve production-level quality monitoring, we must establish a detailed evaluation indicator system within the development environment:

1. Collaboration and Performance Metrics (Technical Metrics)

• Agent Handoff Success Rate (agent_handoff_success_rate): Measures the proportion of timeouts and deadlocks caused by parsing failures or context loss during state handoffs between Agents.
• Average Steps Performed (avg_steps_per_task): A key measure of system execution efficiency. If this indicator continues to climb, it indicates that the Agent cluster is trapped in a semantic cycle of inefficiency.
• Communication Token Ratio: The proportion of tokens consumed by mutual confirmation between agents to the total token consumed for a single task, used to assess redundant communication costs.
• Local Retry Success Rate (retry_rate): Measures the proportion of outputs that Workers successfully repair after a Reviewer initiates a retry, indicating the Worker’s self-healing ability.

2. Real Business Metrics

• Marginal accuracy improvement: Compares the accuracy of multi-agent and single-agent systems on the same evaluation set. If the difference is below 2%, the system should immediately be downgraded to a single-Agent architecture to save inference costs.
• Manual intervention takeover rate (human_escalation_rate): The proportion of tasks entering the manual review queue within the total task, used to monitor the system’s autonomous closed-loop health.
• Marginal cost per task success (cost_per_successful_task): Calculate the average amount of dollars spent for each successful completion of a real business task.

8. Minimum Viable Release Recommendations

When launching a multi-agent application for the first time, the number of Agent roles should be kept within three, and fully automated closed-loop execution should be unconditionally rejected.

When building the first MAS system (MVP), avoid designing an overly grand autonomous ecosystem. I strongly recommend that you comply with the following development constraints:

• Role limit: The number of independent Agents in the system must never exceed 3. The most classic combination is: a Supervisor orchestrating routes, a Worker executing specific tools, and a Reviewer independently verifying the chain of evidence.
• Clear boundaries: Each Agent’s mounted toolset must be physically mutually exclusive. For example, the FAQ query tool and the order refund tool must belong to separate agent containers. It is strictly forbidden to mount all tools globally, otherwise it can cause serious tool hallucinations.
• Mandatory human-in-the-loop control: for actions that modify core business state, such as sending external notifications or initiating payments, do not let the Agent complete the loop autonomously. A manual review step must be added to the task queue, and execution can proceed only after a person clicks to approve it in the control panel.

9. Common Design Pitfalls and Error Troubleshooting (Error Logs)

Poorly designed multi-agent clusters are prone to typical errors such as communication deadloops, state locks, and hallucination amplification, requiring targeted interception.

During the process of bringing the MAS architecture into production, the three most common physical errors encountered are as follows. I have summarized the error texts and troubleshooting strategies as follows:

1. Infinite A2A Loop (Agent Communication Deadlock)

• Symptom: Two Agents endlessly exchange pleasantries and argue over a minor spelling mistake or state confirmation, causing token consumption to explode immediately.
• Error Text:

  Fatal: [MAS_COMMUNICATION_DEADLOCK] Infinite interaction loop detected between 'agent_code_developer' and 'agent_code_reviewer'. Session ID: sess_998877. Interchanged turns exceeded 8.
  

• Root Cause: Lack of global hop count monitoring, and the Reviewer did not provide specific code modification instructions when returning changes, causing the Worker to guess the modifications, and the Reviewer continued to report errors and return them.
• Troubleshooting Measures: Globally configure Max_Turns = 5 in the collaboration gateway. When the number of message exchanges between two nodes reaches the limit, forcibly terminate the session and throw it into the manual review channel. At the same time, enforce that the Reviewer’s error reporting format be in JSON and must include explicit audit feedback such as expected_format and failing_segments.

2. State Synchronization Lag (Concurrent State Conflict)

• Phenomenon: When multiple agents concurrently read the same file and update the state, the Agent that finishes later directly overwrites the data of the Agent that finished earlier, causing data corruption.

• Error Text:

  Error: [STATE_VERSION_CONFLICT] Optimistic lock check failed for entity 'task_state_992'. Expected version: 4, found: 5. Active agent: 'agent_document_writer'.
  

• Root Cause: Multiple concurrently executing sub-Workers share the same memory state and no optimistic locking mechanism has been introduced at the database or persistence layer.

• Troubleshooting Measures: Introduce version (version) control in the underlying State DB. Each time the state dictionary is updated, perform an optimistic lock check on WHERE version = current_version; if a write conflict occurs, catch the exception and trigger exponential backoff retry to decouple concurrent conflicts.

3. Latency Explosion (Response Delay Avalanche)

• Symptom: Front-end users perceive the system as completely hung, and the console shows HTTP connections automatically disconnected after running for 60 seconds due to gateway timeout.

• Error Text:

  Error: [HTTP_GATEWAY_TIMEOUT] Upstream multi-agent task execution did not complete within the gateway window of 60000ms. Active trace: trace_223344.
  

• Root Cause: The system uses a synchronous waiting model to handle multi-step planning across multiple agents. Any slowdown in an upstream model triggers a domino-like delay avalanche.

• Troubleshooting Strategy: Unconditionally switch task execution to a publish-subscribe (Pub/Sub) mechanism based on asynchronous task queues. After the front end submits a request, immediately disconnect and return a task credential. Use asynchronous task frameworks like Celery / BullMQ to execute tasks in the background, and track progress in real time via push notifications or long polling.

10. Further Reading

To deploy high-reliability agents in production, you need to further learn how to introduce robust governance standards at each process level.

• 👉 AI Agent Architecture Practice: From Prompt to Production-Grade Agent System Architecture
• 👉 AI Agent Tool Use: Practical Guide to Agent Tool Calls and Interface Integration
• 👉 AI Agent Observability: Guide to Opening Up the Observability of Agent Execution Black Boxes
• 👉 AI Agent Deployment: Production-Grade Agent Deployment Architecture and High Concurrency Scaling Guide
• 👉 AI Agent Evaluation: Four Core Dimensions for Quantifying Agent Performance and Practical Evaluation
• 👉 AI Agent Planning: Implementing Large Model Planning Capabilities in Agent Systems
• 👉 Multi-Agent Planning: In-Depth Analysis of Planning Strategies in Multi-Agent Collaboration
• 👉 AI Agent Framework Deep Comparison: LangChain vs AutoGen vs CrewAI
• 👉 Microsoft AutoGen Agent Practice: Step-by-Step Development Guide for Multi-Agent Systems
• 👉 MCP vs A2A vs Function Calling: AI Agent Protocol Selection and System Integration Guide
• 👉 Complete AI Agent Engineering Guide

External References:

• LangGraph Multi-Agent Architecture and State Management
• Microsoft AutoGen Multi-Agent Conversational Patterns
• CrewAI Role-Playing and Process SOP Configurations
• OpenAI Swarm Multi-Agent Orchestration Standard

Topic path / AI Agents

Continue from one agent pattern to the complete production system

The AI Agent hub organizes architecture, memory, tool use, evaluation, security, deployment and multi-agent coordination into a single learning path.

Open AI Agent hub → Review the architecture guide Subscribe