AI Agents

AI agents are AI systems that use a model to decide what actions to take while working toward a goal. Instead of generating one response from one prompt, an agent can repeatedly inspect its current state, choose an action, use tools, observe the result, and decide what to do next.

For example, a traditional LLM application might answer a question about an order using information already included in the prompt. An AI agent could instead identify the order number, call an order API, inspect shipment data, check a carrier service, and then generate an answer from the collected information. The important difference is not simply tool access. An agent has a control loop in which model outputs influence what happens next.

Table of Contents

• What Is an AI Agent?
• How AI Agents Work
  • Goal and State
  • Actions and Tools
  • The Agent Loop
• AI Agent vs Chatbot
• A Simple AI Agent Example
• Agent Memory and Context
• Planning and Workflow Control
• Single-Agent vs Multi-Agent Systems
• Common AI Agent Failures
• Designing Production AI Agents
  • Limit Agent Authority
  • Make Actions Idempotent
  • Bound the Agent Loop
• Monitoring and Evaluating Agents
• Conclusion
  • More Articles to Read

What Is an AI Agent?

An AI agent is a software system in which an AI model participates in deciding the next action required to accomplish a task.

Consider a request:

Find the current status of order 18492 and explain any delivery delay.

A simple LLM cannot know the current status unless that information is already present in its context. An agent can decide that additional information is required and interact with external systems.

Goal
 ↓
LLM
 ↓
Choose Action
 ↓
Call Tool
 ↓
Observe Result
 ↓
LLM
 ↓
Choose Next Action
 ↓
Final Answer

The model might first call an order service:

{
  "tool": "get_order",
  "arguments": {
    "order_id": "18492"
  }
}

The result may show:

{
  "status": "in_transit",
  "tracking_number": "TRK-91827",
  "carrier": "fast_ship"
}

The model can then determine that carrier tracking is required and request another tool call.

This ability to choose subsequent actions based on previous observations is what makes the system agentic.

The mechanics of connecting models to application functions and APIs are covered in AI Tool Calling.

How AI Agents Work

Most AI agents can be understood through a few basic components: a goal, current state, available actions, observations, and a control loop.

The implementation can be simple. An agent does not require a large framework or dozens of specialized components.

Goal and State

The goal describes what the agent should accomplish.

Determine why shipment SH-18492 is delayed
and provide the customer with the current status.

The state contains information accumulated while performing the task. It might include the original request, tool results, completed actions, remaining steps, retry counters, or other workflow data.

A simplified state object might be:

state = {
    "goal": "Investigate shipment SH-18492",
    "steps": 0,
    "observations": [],
    "completed": False,
}

State does not need to be stored entirely inside the LLM context. Production systems can keep structured state in ordinary application storage and send only relevant information to the model.

This becomes increasingly important as agent workflows become longer because model context is finite. AI Tokens and Context Windows explains these limits.

Actions and Tools

Tools give the agent capabilities beyond text generation.

A logistics agent might have tools such as:

get_order(order_id)
get_tracking(tracking_number)
get_warehouse_status(warehouse_id)
search_support_documents(query)
create_support_ticket(...)
request_refund(...)

The model receives descriptions of available tools and determines when one is appropriate.

Tools can represent API requests, database queries, search operations, calculations, internal services, or controlled business operations.

Not every tool needs to modify data. Read-only tools are often the safest starting point because an incorrect model decision cannot directly create an external side effect.

The Agent Loop

The central component is the agent loop.

A minimal implementation can look like:

MAX_STEPS = 8


def run_agent(goal: str):
    state = create_initial_state(goal)

    for step in range(MAX_STEPS):
        decision = model.decide(
            goal=goal,
            state=state,
            tools=available_tools,
        )

        if decision.type == "final_answer":
            return decision.answer

        if decision.type != "tool_call":
            raise ValueError("Unsupported agent decision")

        result = execute_tool(
            decision.tool,
            decision.arguments,
        )

        state.add_observation(
            tool=decision.tool,
            arguments=decision.arguments,
            result=result,
        )

    raise RuntimeError("Agent exceeded maximum steps")

The loop repeatedly asks the model what to do next. If the model produces a final answer, execution stops. If it requests a tool, the application executes the tool and returns the observation to the model.

The application remains responsible for enforcing tool permissions, argument validation, timeouts, budgets, and maximum step counts.

AI Agent vs Chatbot

The terms AI agent and chatbot are sometimes used interchangeably, but they describe different architectural capabilities.

A chatbot can still use tools without becoming a highly autonomous agent. For example, an application might always call a weather API before asking an LLM to formulate the response. The application, not the model, determines the workflow.

An agentic system gives the model more control over the sequence:

Fixed Workflow:
Application → Weather API → LLM → Response

Agent:
LLM → Decide Tool → Tool Result → Decide Next Step → Response

Agentic behavior therefore exists on a spectrum. A system can give the model control over one routing decision or allow it to execute a longer sequence of actions.

More autonomy is not automatically better. Deterministic workflows are easier to test and operate when the required sequence is already known.

A Simple AI Agent Example

Consider an agent that investigates delayed shipments.

A customer asks:

Why hasn't shipment SH-18492 arrived yet?

The model does not have enough information to answer. It chooses:

{
  "tool": "get_shipment",
  "arguments": {
    "shipment_id": "SH-18492"
  }
}

The application executes the request and returns:

{
  "shipment_id": "SH-18492",
  "status": "in_transit",
  "tracking_number": "TRK-9921",
  "carrier": "FastShip",
  "estimated_delivery": "2026-09-01"
}

The estimated delivery date has passed, but the reason is still unknown. The agent chooses another action:

{
  "tool": "get_tracking_events",
  "arguments": {
    "tracking_number": "TRK-9921"
  }
}

The carrier service returns:

{
  "latest_event": "weather_delay",
  "location": "Dallas Distribution Center",
  "updated_at": "2026-09-02T08:42:00"
}

The agent now has enough information and generates the final answer:

Shipment SH-18492 is still in transit. The latest carrier update reports a weather delay at the Dallas Distribution Center.

The model did not need to know the shipment status beforehand. It needed to know how to determine which information was required and which available tools could provide it.

Agent Memory and Context

Agents often need information from previous steps, but the word memory can describe several different mechanisms.

Working memory contains information relevant to the current task: previous tool calls, observations, partial results, and the current plan.

Conversation memory preserves useful information from earlier turns in a user interaction.

Long-term memory stores information across tasks, often in a database or retrieval system.

These mechanisms should not automatically mean copying every previous message into every model request.

For a long-running agent, a structured state might contain:

{
  "task_id": "task-981",
  "shipment_id": "SH-18492",
  "shipment_status": "in_transit",
  "delay_reason": "weather_delay",
  "completed_steps": [
    "shipment_lookup",
    "carrier_lookup"
  ]
}

The application can send only the fields required for the next decision.

Long-term information can also be retrieved when needed using semantic search or other lookup methods. RAG (Retrieval-Augmented Generation) explains how external information can be retrieved and added to model context.

This leads to a useful principle: store durable state in durable systems and use the LLM context as temporary working context.

Planning and Workflow Control

Some tasks require several dependent steps. An agent can decide those steps dynamically, but not every task benefits from fully model-generated planning.

Consider processing a customer refund. The required workflow may already be known:

Load Order
   ↓
Check Refund Eligibility
   ↓
Calculate Maximum Refund
   ↓
Request Approval if Required
   ↓
Execute Refund
   ↓
Notify Customer

There is little benefit in asking an LLM to rediscover this sequence for every request. A deterministic workflow engine can enforce it reliably.

The model can still participate where judgment is useful. For example, it might classify the customer's reason, extract relevant details, or summarize evidence for an approval step.

Now consider an investigation task:

Determine why checkout conversion dropped yesterday.

The exact steps are less predictable. Depending on observations, the agent might inspect application errors, deployment history, payment failures, latency metrics, or feature-flag changes.

Dynamic planning is more valuable when the path depends on information discovered during execution.

A useful architecture therefore combines both approaches:

Known Business Process → Deterministic Workflow

Uncertain Investigation → Agent Decisions

Sensitive Side Effect → Deterministic Validation / Approval

Use model reasoning where uncertainty exists, and deterministic code where the rules are already known.

Single-Agent vs Multi-Agent Systems

A multi-agent system uses several agents with different roles or capabilities. One agent might research information, another analyze it, and another review the result.

For example:

Coordinator
   ├─→ Documentation Agent
   ├─→ Metrics Agent
   └─→ Deployment Agent
            ↓
      Combined Result

This can be useful when components genuinely require different tools, permissions, models, contexts, or ownership boundaries.

For example, a deployment agent may have access to deployment metadata while a customer-support agent should never receive those permissions.

However, splitting a simple workflow into many agents creates additional model calls, context transfers, latency, failure modes, and debugging complexity.

Suppose one agent extracts an order ID, another retrieves the order, another checks shipping, and another writes the answer. If each operation could have been handled by one agent with two tools, the multi-agent design adds coordination without adding meaningful capability.

Multi-agent architecture should therefore solve a concrete separation problem rather than being treated as the default architecture for sophisticated AI applications.

Common AI Agent Failures

Agents introduce failure modes beyond ordinary LLM responses because model decisions can affect subsequent application behavior.

Tool selection errors occur when the model chooses the wrong operation. A shipment question might trigger an order-search tool when a tracking tool is required.

Invalid arguments occur when the model generates parameters that do not satisfy the tool contract. Tool inputs should always be validated using deterministic schemas.

Loops occur when the agent repeatedly performs the same action without making progress:

Search → No Result
   ↓
Search Again → No Result
   ↓
Search Again → No Result
   ↓
...

Excessive exploration occurs when the agent continues calling tools even though enough information is already available.

Unsafe side effects are more serious. A model might incorrectly decide to cancel an order, issue a refund, delete a resource, or send a message.

Prompt injection through tool results can occur when retrieved web pages, documents, emails, or other untrusted content contains text attempting to influence subsequent model decisions.

Partial failures become important when several side effects have already occurred. If an agent successfully creates a ticket but times out before recording that result, retrying the complete workflow could create another ticket.

These failures are why production agents need stronger application controls than ordinary conversational interfaces.

Designing Production AI Agents

A production agent should operate inside explicitly defined boundaries. The model can propose actions, but application code should decide whether those actions are valid and authorized.

A useful execution boundary looks like:

def execute_agent_action(user, action):
    validate_schema(action)
    authorize(user, action)
    enforce_business_rules(action)
    enforce_rate_limits(user, action)

    return execute(action)

The model should not be able to bypass these checks by changing the wording of its request.

Limit Agent Authority

Tools should follow the principle of least privilege.

If an agent only needs shipment status, provide:

get_shipment_status(shipment_id)

rather than a generic database tool capable of reading arbitrary tables.

If refunds are allowed only up to a specific amount, enforce the limit in application code rather than writing it only in the prompt.

High-impact operations can require explicit approval:

Agent Proposes Refund
        ↓
Validate Eligibility
        ↓
Amount > Automatic Limit?
        │
        ├─ Yes → Human Approval
        │
        └─ No → Execute Refund

This allows AI to automate decision support without giving the model unrestricted control over external systems.

Make Actions Idempotent

Agents operate in distributed systems where timeouts and retries are normal. Side-effecting tools should therefore support idempotency whenever possible.

For example:

refund = payments.create_refund(
    payment_id=payment_id,
    amount=amount,
    idempotency_key=f"{task_id}:refund",
)

If the agent retries after a network timeout, the payment service can recognize the same operation instead of issuing another refund.

This principle also applies to ticket creation, emails, deployments, database writes, and other side effects.

Bound the Agent Loop

Every agent should have explicit execution limits.

Useful limits include:

• maximum number of steps;
• maximum model tokens;
• maximum tool calls;
• maximum execution time;
• maximum financial cost;
• per-tool retry limits;
• allowed tool categories.

A simple budget can be enforced directly:

MAX_STEPS = 10
MAX_TOOL_CALLS = 6

for step in range(MAX_STEPS):
    decision = get_next_decision(state)

    if state.tool_calls >= MAX_TOOL_CALLS:
        return fallback("Tool-call budget exceeded")

    process_decision(decision)

Limits prevent a malformed task or poor model decision from consuming resources indefinitely.

Fallback behavior should also be defined. When the agent cannot complete a task safely, it can return an incomplete result, request additional information, or escalate the task rather than continuing indefinitely.

Monitoring and Evaluating Agents

Evaluating only the final response is insufficient for agentic systems. The sequence of actions matters as much as the text eventually produced.

For each execution, useful telemetry can include:

{
  "task_id": "task-981",
  "agent_version": "shipment-agent-v7",
  "model_version": "model-v4",
  "steps": 3,
  "tool_calls": 2,
  "duration_ms": 1840,
  "input_tokens": 4210,
  "output_tokens": 382,
  "status": "completed"
}

Tool-level traces should show which operation was requested, whether validation succeeded, how long execution took, and whether retries occurred. Sensitive arguments and results may need redaction before logging.

Useful production metrics include:

Evaluation datasets should include complete tasks rather than only isolated prompts. A test might specify an initial request, mocked tool results, expected actions, forbidden actions, and the expected final outcome.

For example:

{
  "task": "Find why shipment SH-18492 is delayed",
  "expected_tools": [
    "get_shipment",
    "get_tracking_events"
  ],
  "forbidden_tools": [
    "cancel_shipment",
    "issue_refund"
  ],
  "expected_reason": "weather_delay"
}

This makes it possible to detect regressions when prompts, models, tool descriptions, or workflow logic change.

Production evaluation should focus on the business outcome. An agent that uses sophisticated reasoning but takes 14 tool calls to perform a task that normally requires two is not necessarily a good agent.

AI Monitoring and Evaluation covers the broader evaluation and observability practices used in production AI systems.

Conclusion

AI agents extend language models from generating responses to participating in multi-step software workflows. An agent can inspect its current state, choose tools, observe results, and adapt subsequent actions until it reaches a goal or execution limit.

This flexibility is useful for tasks where the correct path depends on information discovered during execution, such as research, troubleshooting, operational investigation, and workflows involving several external systems.

Agentic architectures also introduce additional risk and complexity. Tool calls must be validated and authorized, side effects should be idempotent, loops need explicit limits, untrusted observations must be treated carefully, and every important action should be observable.

Many workflows do not need an agent. When the sequence of operations is already known, deterministic application code is usually simpler and more reliable. The model can still be used for the parts requiring language understanding or probabilistic judgment.

A production AI agent should not be an unrestricted model controlling software. It should be a model making bounded decisions inside a deterministic system that controls permissions, state, side effects, budgets, and failure handling.

More Articles to Read

• AI Tool Calling
• AI Prompt Engineering
• RAG (Retrieval-Augmented Generation)
• AI Tokens and Context Windows
• AI Monitoring and Evaluation