How Multi-Agent Orchestration Works in Agentic AI

Spread the love

What Is Agent Orchestration in AI?

Agent orchestration is the process of coordinating multiple AI agents, tools, and tasks to achieve a specific goal. An orchestrator decides which agent should perform a task, in what order tasks should run, how agents share results, and how the final response should be produced.

In simple words, Agent orchestration works like a project manager who assigns work to different team members, checks their results, and combines their work to complete a project.

Real-world example of Agent Orchestration

Imagine you want to build an AI system that researches a company and generates an investment report.

The system might have four specialized agents:

1. Research Agent

Collects company information, financial results, and relevant news.

2. Financial Analysis Agent

Analyzes revenue, profit, debt, valuation, and other financial metrics.

3. Risk Analysis Agent

Identifies business risks, market risks, and potential warning signs.

4. Report Generation Agent

Combines the research, financial analysis, and risk assessment into a structured report.

The orchestrator coordinates these agents. It determines which tasks are needed, starts the appropriate agents, tracks their results, and sends the collected information to the report-generation agent.

How does Agent Orchestration work?

The workflow follows these steps:

Receive the request: Understand what the user wants.
Plan the work: Break the request into smaller tasks.
Assign tasks: Select suitable agents for each task.
Execute tasks: Agents perform their assigned work, sometimes in parallel.
Collect results: The orchestrator gathers the outputs.
Validate results: Check whether information is complete and consistent; retry or request additional work if necessary.
Generate the final response: Combine the results and return the report.

How Many way Orchestrate Agent Ai

There are two way Agent Orchestration.by code and by LLMs.

By code: more predictable and deterministic.
By LLMs: more powerful.

1.How to Orchestrate Agent by code

Your code demonstrates how to build an AI sales email automation system using the OpenAI Agents SDK. It creates three AI agents with different writing styles, generates emails in parallel, selects the best email, and finally uses a tool to send the selected email.

intro = """
You are an AI-powered sales representative at ComplAI, a SaaS company that helps businesses achieve SOC 2 compliance and prepare for audits. Your responsibility is to create professional, engaging, and persuasive sales emails that highlight the value of ComplAI’s AI-driven compliance solutions.
"""

instructions1 = intro + "Your email style is professional, serious, with gravitas and credibility."
instructions2 = intro + "Your email style is witty, engaging, and humorous."
instructions3 = intro + "Your email style is concise, to the point, in the style of a busy senior executive."

sales_agent1 = Agent(name="Professional Sales Agent", instructions=instructions1, model=MODEL_NAME)
sales_agent2 = Agent(name="Sales Agent", instructions=instructions2, model=MODEL_NAME)
sales_agent3 = Agent(name="Executive Sales Agent", instructions=instructions3, model=MODEL_NAME)


result = Runner.run_streamed(sales_agent1, input="Write a cold sales email")
async for event in result.stream_events():
    if event.type == "raw_response_event" and isinstance(event.data, ResponseTextDeltaEvent):
        print(event.data.delta, end="", flush=True)


message = "Write a cold sales email"

with trace("Parallel cold emails"):
    results = await asyncio.gather(
        Runner.run(sales_agent1, message),
        Runner.run(sales_agent2, message),
        Runner.run(sales_agent3, message),
    )

outputs = [result.final_output for result in results]

for output in outputs:
    print(output + "\n\n")


    decision = """
    You pick the best cold sales email from the given options.
    Imagine you are a customer and pick the one you are most likely to respond to.
    Do not give an explanation; reply with the selected email only.
    """
    sales_picker = Agent(name="Sales_picker", instructions=decision, model=MODEL_NAME)


message = "Write a cold sales email"

with trace("Sales selection workflow"):
    results = await asyncio.gather(
        Runner.run(sales_agent1, message),
        Runner.run(sales_agent2, message),
        Runner.run(sales_agent3, message),
    )
    outputs = [result.final_output for result in results]
    emails = "Cold sales emails:\n\n" + "\n\nEmail:\n\n".join(outputs)
    best = await Runner.run(sales_picker, emails)
    print(f"Best sales email:\n{best.final_output}")


@function_tool
def send_email_tool(subject: str, text_body: str, html_body: str) -> str:
    """
    Send out an email with the given subject and body to all sales prospects
    
    Args:
        subject: The subject of the email
        text_body: The body of the email as plain text
        html_body: The HTML body of the email
    """
    send_message(subject, text_body, html_body)
    return "Email sent successfully"

send_email_tool.params_json_schema
decision = """
You pick the best cold sales email from the given options.
Imagine you are a customer and pick the one you are most likely to respond to.
Then use your tool to send the email.
"""

require_tool = ModelSettings(tool_choice="required")

sales_sender = Agent(name="Sales Sender", instructions=decision, model=MODEL_NAME, tools=[send_email_tool], model_settings=require_tool)

message = "Write a cold sales email"

with trace("Sales selection workflow with sending"):
    results = await asyncio.gather(
        Runner.run(sales_agent1, message),
        Runner.run(sales_agent2, message),
        Runner.run(sales_agent3, message),
    )
    outputs = [result.final_output for result in results]
    emails = "Cold sales emails:\n\n" + "\n\nEmail:\n\n".join(outputs)
    response = await Runner.run(sales_sender, emails)
    print(f"Final response:\n{response.final_output}")

Explanation of above code:

Let’s understand it step by step in simple language, with examples.

There are two important concepts in this code:

Agent orchestration: coordinating multiple AI agents to complete a workflow.
Tool calling: allowing an AI agent to invoke a Python function, such as an email-sending function.

2. Define the common instructions

intro = """
You are an AI-powered sales representative at ComplAI,a SaaS company that helps businesses achieve SOC 2 compliance
and prepare for audits.Your responsibility is to create professional, engaging,and persuasive sales emails that highlight the valueof ComplAI’s AI-driven compliance solutions.
"""

Electric Juicer Bottle Blender Grinder Mixer

Amazon

Food-Grade Material: Small USB blender uses food-grade non-toxic and environmengtally friendly PP& ABS materials. With the most advanced CS+C(Cold Spin+Centrifugal) Technology

Buy Now

We earn a commission if you make a purchase, at no additional cost to you.

Line-by-line explanation

intro = creates a Python variable named intro.
""" ... """ is a multiline string. It allows you to write text across multiple lines.
The text inside describes the AI agent’s identity, company, responsibility, and objective.

For example, the agent learns from these instructions that it should write sales emails for a company selling AI-powered compliance software.

Why use a variable?

You have three agents with the same basic role. Instead of repeating the entire company description three times, you store it once and reuse it.

3. Create three different writing styles

instructions1 = intro + "Your email style is professional, serious, with gravitas and credibility."

instructions2 = intro + "Your email style is witty, engaging, and humorous."

instructions3 = intro + "Your email style is concise, to the point, in the style of a busy senior executive."

4. Create the three AI agents

sales_agent1 = Agent(
    name="Professional Sales Agent",
    instructions=instructions1,
    model=MODEL_NAME
)

sales_agent2 = Agent(
    name="Sales Agent",
    instructions=instructions2,
    model=MODEL_NAME
)

sales_agent3 = Agent(
    name="Executive Sales Agent",
    instructions=instructions3,
    model=MODEL_NAME
)

The Agent() constructor creates an AI agent.

Let’s understand its parameters.

name: a human-readable name for the agent.
instructions: tells the agent how it should behave and write.
model=MODEL_NAME: specifies the AI model the agent should use. MODEL_NAME must be defined elsewhere in your program.

5. Run the first agent and stream its response

result = Runner.run_streamed(
    sales_agent1,
    input="Write a cold sales email"
)

async for event in result.stream_events():
    if event.type == "raw_response_event" and isinstance(
        event.data, ResponseTextDeltaEvent
    ):
        print(event.data.delta, end="", flush=True)

This section runs the professional sales agent and displays the generated email as it arrives, rather than waiting for the complete response.

Runner executes an agent.
run_streamed() starts an execution that exposes streaming events.
sales_agent1 is the agent to execute.
input contains the user’s request.
result holds the streaming run object, which you use to read the events.

A cold sales email is an unsolicited introductory email sent to a potential customer who has not necessarily interacted with your company before.

stream_events() provides events as the agent runs.
async for iterates over an asynchronous stream.
Each event represents something that happened during execution.

Why async for? Because the next event may arrive later, and the program can await it without using an ordinary blocking loop.

event.type == "raw_response_event" checks that the event is a raw model-response event.
isinstance(...) checks whether its data is a ResponseTextDeltaEvent.
ResponseTextDeltaEvent represents a piece of generated text, often called a delta.

In simple terms, the program says: “Print only the events that contain a new piece of response text.”

6. Generate emails from all three agents in parallel

message = "Write a cold sales email"

with trace("Parallel cold emails"):
    results = await asyncio.gather(
        Runner.run(sales_agent1, message),
        Runner.run(sales_agent2, message),
        Runner.run(sales_agent3, message),
    )

outputs = [result.final_output for result in results]

for output in outputs:
    print(output + "\n\n")

This is the main example of parallel agent orchestration.

Step 1: Define a shared request

message = "Write a cold sales email"

You store the task in a variable so that all three agents receive the same input.

Step 2: Start a trace

with trace("Parallel cold emails"):

A trace records the execution of a workflow for debugging and inspection. The label helps you identify this run.

Step 3: Execute agents concurrently

results = await asyncio.gather(
    Runner.run(sales_agent1, message),
    Runner.run(sales_agent2, message),
    Runner.run(sales_agent3, message),
)

Let’s break this down:

asyncio.gather() schedules multiple asynchronous operations and waits for them to complete.
Runner.run() runs an agent and returns its result.
Each agent receives the same message.
await waits until all three operations finish and collects their results.

The agents can work concurrently rather than being called one after another.

Parallel execution using asyncio.gather()

Step 4: Extract the final emails

outputs = [result.final_output for result in results]

This is a Python list comprehension. It takes each item in results and extracts its final_output.

outputs = [
    "Professional email...",
    "Humorous email...",
    "Executive-style email..."
]

These are illustrative placeholders, not actual model responses.

Step 5: Print all emails

for output in outputs:
    print(output + "\n\n")

for output in outputs loops through the generated emails.
output holds one email during each iteration.
"\n\n" adds two newline characters to separate the emails visually.

At this point, the program has three candidate emails. It has not selected a winner yet.

7. Create an agent that selects the best email

decision = """
You pick the best cold sales email from the given options.
Imagine you are a customer and pick the one you are most likely to respond to.
Do not give an explanation; reply with the selected email only.
"""

sales_picker = Agent(
    name="Sales_picker",
    instructions=decision,
    model=MODEL_NAME
)

What does decision do?

It defines the selection criteria for a new AI agent.

The picker is instructed to:

Read several candidate emails.
Evaluate them from a potential customer’s perspective.
Choose the email it believes is most likely to receive a response.
Return only the selected email without explaining its decision.

What does sales_picker do?

It creates a fourth agent. Unlike the first three agents, which generate emails, this agent evaluates and selects an email.

This creates a useful pattern:

Generation agents → Evaluation agent → Best candidate

One issue in your original code is that the decision and sales_picker definitions appear indented inside the earlier for output in outputs loop. If that indentation is exactly as shown, Python creates the picker repeatedly, once for each output. It would be cleaner to define it outside the loop, once, as shown above.

8. Run the selection workflow

message = "Write a cold sales email"

with trace("Sales selection workflow"):
    results = await asyncio.gather(
        Runner.run(sales_agent1, message),
        Runner.run(sales_agent2, message),
        Runner.run(sales_agent3, message),
    )

    outputs = [result.final_output for result in results]

    emails = "Cold sales emails:\n\n" + "\n\nEmail:\n\n".join(outputs)

    best = await Runner.run(sales_picker, emails)

    print(f"Best sales email:\n{best.final_output}")

Step 1: Generate the three candidates again

The asyncio.gather() call repeats the three agent runs. It does not reuse the emails generated in the earlier section.

If you have already generated and stored suitable emails in outputs, you can reuse them instead of paying for another set of model calls.

Step 2: Format the emails into one string

emails = "Cold sales emails:\n\n" + "\n\nEmail:\n\n".join(outputs)

This creates a single text input containing all three emails.

"\n\nEmail:\n\n".join(outputs) joins the email strings with a separator.
The initial "Cold sales emails:\n\n" adds a heading.
+ joins the heading and the combined email text.

Step 3: Ask the picker to select the best email

best = await Runner.run(sales_picker, emails)

This runs the fourth agent and passes all three candidates as its input. The picker evaluates the candidates according to its instructions. Its result is stored in best.

Step 4: Print the winner

print(f"Best sales email:\n{best.final_output}")

f"..." is a Python f-string, which lets you insert values into a string.
{best.final_output} inserts the selected email into the output.
\n starts the email on the next line.

The selection is based on the model’s judgment. It is not proof that the email will achieve the highest real-world response rate.

9. Create a tool to send an email

Now your code moves from generating content to performing an action.

@function_tool
def send_email_tool(
    subject: str,
    text_body: str,
    html_body: str
) -> str:
    """
    Send out an email with the given subject and body
    to all sales prospects

    Args:
        subject: The subject of the email
        text_body: The body of the email as plain text
        html_body: The HTML body of the email
    """
    send_message(subject, text_body, html_body)
    return "Email sent successfully"

10. Force the agent to use the email tool

decision = """
You pick the best cold sales email from the given options.Imagine you are a customer and pick the one you are most likely to respond to.Then use your tool to send the email.
"""

require_tool = ModelSettings(tool_choice="required")

sales_sender = Agent(
    name="Sales Sender",
    instructions=decision,
    model=MODEL_NAME,
    tools=[send_email_tool],
    model_settings=require_tool
)

This creates a new agent that can both select an email and call the sending tool.

Explanation

decision

The instructions tell the agent to select the strongest candidate and then send it using the tool.

ModelSettings(tool_choice="required")

This tells the model to make a tool call rather than respond only with ordinary text.

There is an important distinction: requiring a tool call does not necessarily mean the model must choose send_email_tool if multiple tools are available. In this example, it is the only registered tool.

tools=[send_email_tool]

This gives the agent access to the email-sending function. The agent can request a tool call, and the Agents SDK handles executing that call.

model_settings=require_tool

This applies the tool-use setting to the agent.

The tool’s arguments still need to be valid. Since your tool expects subject, text_body, and html_body, the agent needs to supply all three.

11. Complete workflow: Generate, select, and send

message = "Write a cold sales email"

with trace("Sales selection workflow with sending"):
    results = await asyncio.gather(
        Runner.run(sales_agent1, message),
        Runner.run(sales_agent2, message),
        Runner.run(sales_agent3, message),
    )

    outputs = [result.final_output for result in results]

    emails = "Cold sales emails:\n\n" + "\n\nEmail:\n\n".join(outputs)

    response = await Runner.run(sales_sender, emails)

    print(f"Final response:\n{response.final_output}")

Here is what happens in order:

Generate: Three agents produce candidate emails in parallel.
Collect: Their final_output values are stored in a list.
Combine: The candidates are formatted into one string.
Select: sales_sender receives the candidates and chooses one.
Call tool: The agent requests send_email_tool with the email details.
Execute: The S

12. Important improvements before using this in production

There are a few things to fix or consider in the current implementation.

Avoid duplicate model calls: Your code generates three emails in one section, then generates them again in the selection sections. You can reuse the first set of outputs.
Correct the indentation: Define sales_picker outside the for output in outputs loop.
Make sure the tool sends the selected email: Your current tool accepts subject, plain text, and HTML, but the selected email may contain only one text string. You need to extract or generate all three fields reliably.
Add a delivery safeguard: For real sales prospects, validate the recipient list and consider requiring human approval before sending bulk or external emails.
Handle failures: Add appropriate error handling for API errors, invalid arguments, rejected messages, and delivery failures.

2. How to Orchestrate Agent By LLMs.

Agent orchestration by LLMs (Large Language Models) is the process of coordinating one or more AI agents so they can work together to complete a task. The orchestration process decides which agent should perform a task, what tools it should use, how tasks should be executed, and how the results should be combined into a final answer.

instructions = """
You are a Sales Manager at ComplAI. You get your sales team to draft emails, then send them all to a sales picker.
"""

task = """
Follow these steps:

1. Generate Drafts: Use each of the three sales_email_writer tools to generate different email drafts.
Just instruct each to write a sales email; no further details are needed.
Do not proceed until all three drafts are ready, one from each tool.
 
2. Handoff to the sales sender to choose and send the best email.
"""

tools = [tool1, tool2, tool3]
handoffs = [sales_sender]

sales_manager = Agent(name="Sales Manager", instructions=instructions, tools=tools, handoffs=handoffs, model=MODEL_NAME)

draw_graph(sales_manager)
with trace("Sales manager"):
    result = await Runner.run(sales_manager, task)

If this article helped you learn something new, consider supporting us. Your small contribution helps us continue creating free, useful content for everyone.