Agents and Environments

Designing Rational Agents #

How do we actually build AI systems?

• An agent is any entity that perceives (through sensors) ****and acts (through actuators or effectors).
• One possible goal of making an AI is to create a rational agent - one that selects actions that maximize its expected utility.
• The agent function maps perception to action. It is generated by an agent program running on a machine.

Evaluating the Environment #

The task environment can be described using PEAS:

• Performance measure
• Environment
• Actuators
• Sensors

Example: Pacman #

Performance Measure: In Pacman, one potential way to measure performance is to positively weight collecting food, attacking ghosts, and winning, and negatively weight dying, taking steps, etc.

Environment: The rules and dynamics of the Pacman game (movement, ghost behavior, etc.)

Actuators: The controls to move Pacman up, down, left, and right

Sensors: The entire state of the game is visible except for the duration of the power pellet, which is random.

Environment Types #

The type of the environment largely determines the agent design.

• Fully vs. Partially observable: Whether or not the agent can see the entire state of the system. If partially observable, then the agent requires memory (internal state).
  • Pacman is nearly fully observable (only the power pellet edge case).
• Single agent or multi agent: How many independent actors there are in the environment. Pacman is multi-agent (ghosts, player…)
• Deterministic or stochastic: If the outcome is the same given the same inputs.
  • Pacman is deterministic, whereas something like a self-driving car environment would be stochastic.
• Static or dynamic: Whether or not the environment will wait for you to act.
  • Pacman is dynamic (time keeps ticking), whereas something like Backgammon would be static.
• Discrete or continuous: Whether or not the inputs are processed in time steps, or constantly.
  • Pacman is discrete (frame by frame), but self driving cars would have continuous time.
• Known physics: We know exactly how the environment behaves and the rules behind it.
  • Pacman definitely has known physics.
• Known performance measure: Whether or not the end goal can be concretely defined.
  • Pacman has a known performance measure (win or loss).

Reflex Agents #

Reflex agents directly map percepts to actions. By principle, reflex agents can act optimally in a fully observable environment.

• Pacman is not fully observable, so a reflex agent would not be fully optimal (can’t predict whether to go towards a ghost or not, since we don’t know the duration)

Reflex agents can be extended with state to account for the current status of the environment.

Goal-Based Agents #

Spectrum of Representations #

There are three main ways to represent state:

• Atomic: A collection of individual states (Markov models, etc)
• Factored: A list of factors (booleans, continuous variables, etc.) that can be used to describe a variety of states (neural nets, Bayes nets…)
• Structured: sets of rules to describe every state (programming languages…)