2.4 Decisions Under Uncertainty - Robots That Perceive

▶ Open this notebook in JupyterLab

import numpy as np
import matplotlib.pyplot as plt
plt.rcParams.update({
    "figure.figsize": (7, 3.6), "figure.dpi": 110,
    "axes.spines.top": False, "axes.spines.right": False,
    "axes.grid": True, "grid.alpha": 0.25, "font.size": 10,
})
rng = np.random.default_rng(42)  # fixed seed so the book text matches the plots

Perception gives the robot a belief. Reasoning turns that belief into an action. In decision theory we assign a cost $C(a, x)$ to taking action $a$ when the true state is $x$ , and then pick the action that is best on average under our belief.

Minimum expected cost¶

Given a belief $P(x \mid z)$ , the expected cost of action $a$ is

\bar{C}(a) = \sum_x C(a, x)\,P(x \mid z)

(1)

and the rational choice is $a^\star = \arg\min_a \bar{C}(a)$ .

Example: should the robot stop?¶

A delivery robot’s camera reports whether a person is in its path. Let $p = P(\text{person} \mid z)$ . The robot can continue or stop. Stopping costs time; continuing into a person is far worse:

	no person	person
continue	0	100
stop	2	2

C = np.array([[0, 100],     # continue
              [2,   2]])    # stop
actions = ["continue", "stop"]
p = np.linspace(0, 0.2, 400)
belief = np.stack([1 - p, p])           # shape (2 states, N)
expected = C @ belief                   # shape (2 actions, N)
for a in range(2):
    plt.plot(p, expected[a], lw=2, label=actions[a])
threshold = 2 / 100
plt.axvline(threshold, c="k", ls=":", label=f"switch at p = {threshold:.2f}")
plt.xlabel("P(person | z)"); plt.ylabel("expected cost"); plt.legend(); plt.title("Minimum expected cost decision");

Because a collision is 50 times more costly than a stop, the robot should stop whenever the probability of a person exceeds just 2%. This asymmetry is what makes safe robots cautious, and it follows directly from (1) rather than from any hand-tuned rule.

Value of information¶

Should the robot pay to take another measurement? The value of information (VOI) is the expected reduction in cost from acting after observing $z$ instead of acting on the prior alone:

\text{VOI} = \min_a \sum_x C(a,x)P(x) \;-\; \sum_z P(z)\,\min_a \sum_x C(a,x)P(x \mid z)

(2)

A measurement is worth taking when its VOI exceeds its cost.

def voi(prior_person, p_tp=0.95, p_fp=0.05):
    prior = np.array([1 - prior_person, prior_person])
    cost_now = (C @ prior).min()
    lik = np.array([[1 - p_fp, 1 - p_tp],   # P(z=0 | state)
                    [p_fp,     p_tp]])      # P(z=1 | state)
    cost_after = 0.0
    for zi in range(2):
        joint = lik[zi] * prior
        pz = joint.sum()
        cost_after += pz * (C @ (joint / pz)).min()
    return cost_now - cost_after

priors = np.linspace(0.001, 0.2, 200)
plt.plot(priors, [voi(q) for q in priors], lw=2)
plt.xlabel("prior P(person)"); plt.ylabel("value of information"); plt.title("When is it worth looking again?");

The VOI is largest near the decision threshold, where the robot is genuinely unsure what to do, and falls toward zero when it is already confident either way. Russell and Norvig give a fuller treatment of decision theory Russell & Norvig, 2020.