What is an Agent?

An agent is anything that can perceive its environment through sensors and act upon that environment through actuators .

Human Agent

• Sensors: Eyes, ears, and other organs.
• Actuators: Hands, legs, mouth, and other body parts.

Robotic Agent

• Sensors: Cameras, infrared range finders.
• Actuators: Various motors to move arms, wheels, etc.

Software Agent

• Sensors: Keyboard input, file contents, network packets.
• Actuators: Displaying on screen, writing to a file, sending network packets.

The Agent-Environment Interaction

An agent operates in a continuous loop: it receives percepts from the environment and performs actions that change the environment.

• Agent Function : This is an abstract mathematical mapping from a history of percepts to an action. It's represented as:

  $$f: \mathcal{P}^* \rightarrow \mathcal{A}$$

  Where $\mathcal{P}^*$ is the sequence of all percepts, and $\mathcal{A}$ is the set of all possible actions.

• Agent Program : This is the concrete implementation of the agent function. It's the actual code that runs on the agent's physical architecture.

  Agent = Architecture + Program

Rationality

A rational agent is one that does the "right thing." But what does "right" mean?

Defining Rationality

For every possible sequence of percepts, a rational agent should choose an action that is expected to maximize its performance measure, given the evidence from the percepts and any built-in knowledge the agent has.

Key Aspects of a Rational Agent

When an agent decides how to act, its choice is guided by four factors:

1. Performance Measure: A well-defined metric to evaluate success.
2. Prior Knowledge: The agent's built-in understanding of the environment.
3. Available Actions: The set of all possible actions the agent can perform.
4. Percept History: All the information the agent has gathered from the environment so far.

Rationality vs. Omniscience

Rationality is not the same as omniscience . An omniscient agent knows the actual outcome of its actions and can see the future. A rational agent simply makes the best decision based on what it knows now.

• A perfectly rational poker agent can't know what cards the opponent holds, so it can still lose a hand. Its rationality is based on making the play that maximizes its expected winnings over the long run.

Autonomy and Learning

• Information Gathering: A rational agent may perform actions to explore its environment and gather more information. This is crucial for making better decisions later.
• Autonomy : An agent is autonomous if it learns from its own experiences to improve its behavior, rather than relying solely on the initial knowledge provided by its designer.

Task Environments (PEAS)

To design a rational agent, we must first specify its task environment. The PEAS framework is used for this purpose.

PRequiredPerformance Measure

How is success measured? What do we want the agent to achieve?

ERequiredEnvironment

Where does the agent operate? What are the "laws of physics" of this world?

ARequiredActuators

What can the agent do? How does it affect the environment?

SRequiredSensors

How does the agent perceive the environment? What information can it gather?

PEAS Example: Automated Taxi

Performance Measure

• Safety (e.g., +500 for arriving unharmed, -100 for a crash).
• Speed (e.g., -0.1 points per second).
• Legality (e.g., -500 per violation).
• Comfort (e.g., -10 for speed bumps, -50 for sudden turns).
• Profit (e.g., maximize fare, minimize fuel).

Environment

• Roads, traffic, pedestrians, customers, weather.

Actuators

• Steering, accelerator, brake, turn signals, horn, display screens.

Sensors

• Cameras, sonar, GPS, speedometer, odometer, engine sensors, microphone for speech recognition.

Environment Types

The design of an agent is heavily influenced by the type of environment it operates in.

Observability & Knowledge

• Fully Observable vs. Partially Observable: Can the agent's sensors detect all aspects of the environment's state that are relevant to the choice of action? If not, it's partially observable.
• Known vs. Unknown: Does the agent understand the "rules" of the environment? For example, does it know what its actions will do and how the world evolves?

Agency & Determinism

• Single-agent vs. Multi-agent: Is the agent operating by itself, or are there other agents in the environment whose actions affect the outcome?
• Deterministic vs. Stochastic : Is the next state of the environment completely determined by the current state and the agent's action? If there's an element of randomness, it's stochastic.

Time & State

• Episodic vs. Sequential : Is the agent's experience divided into independent, atomic episodes (like an image classifier identifying single images)? Or do past actions affect future decisions (like in a game of chess)?
• Static vs. Dynamic: Can the environment change while the agent is deliberating? A dynamic environment is more challenging as it "punishes" slow agents.
• Discrete vs. Continuous: Does the environment have a finite or infinite number of distinct states? This also applies to time, percepts, and actions.

Agent Types

Agents can be categorized based on the complexity of their decision-making process.

Simple Reflex Agent

Action basis: Current percept only. It ignores the rest of the percept history and operates on simple condition-action rules.
Example: A car that brakes only when the brake lights of the car in front light up.

Level 1

Model-based Reflex Agent

Action basis: Internal state, which models how the world works. This agent maintains an internal model of the world to handle partial observability. It tracks the state of things it can't currently see.
Example: A taxi agent remembering which roads it has already traveled.

Level 2

Goal-based Agent

Action basis: Model-based + explicit goals. This agent considers future outcomes. It asks, "Which of my possible actions will lead me to a goal state?"
Example: A taxi agent formulating a plan (a sequence of turns) to reach a specific destination.

Level 3

Utility-based Agent

Action basis: Model-based + a utility function. When there are multiple paths to a goal, this agent chooses the one that maximizes its "happiness" or utility. Utility is a function that maps a state to a real number representing its desirability.
Example: A taxi agent choosing the route that is not just correct, but is also the quickest and most fuel-efficient.

Level 4

Learning Agent

Action basis: All of the above, plus the ability to improve. A learning agent can operate in unknown environments and become more competent than its initial knowledge allows. It has four main components:

Performance Element: The agent itself (e.g., a utility-based agent).
Critic: Provides feedback on how well the agent is doing.
Learning Element: Uses feedback to modify the performance element.
Problem Generator: Suggests exploratory actions to gain new, informative experiences.

Level 5

Changelog

8/8/25, 1:50 PM

View All Changelog

• 22b76-feat: drop section for agentson 8/8/25

Copyright

Copyright Ownership:WARREN Y.F. LONG

License under:Attribution-NonCommercial-NoDerivatives 4.0 International (CC-BY-NC-ND-4.0)