Source
import numpy as np
import matplotlib.pyplot as plt
plt.rcParams.update({
"figure.figsize": (7, 3.6), "figure.dpi": 110,
"axes.spines.top": False, "axes.spines.right": False,
"axes.grid": True, "grid.alpha": 0.25, "font.size": 10,
})
rng = np.random.default_rng(7) # fixed seed so the text matches the plotsThe robot now has a belief, from Sections 2.1–2.3. But a belief does nothing on its own. The last question in the chain is: given what I believe, what should I do?
Something on the road ahead¶
An electric delivery cart is carrying groceries along a narrow residential road in the evening. Two-wheelers are parked along both edges, there is a speed breaker ahead, and it drizzled earlier, so the road is wet. About 20 metres ahead, its camera picks up a shape near the middle of the road. The perception system is not sure what it is:
| Object | Pedestrian | Dog | Cow |
|---|---|---|---|
| Probability | 0.55 | 0.25 | 0.20 |
The cart has three choices: continue at its current speed, slow down and be ready to stop, or brake to a halt now.
A simple rule would be: “it is most likely a pedestrian, so do what you would do for a pedestrian”. But think about what the three possibilities mean. A mistake with a pedestrian could injure someone. A mistake with a cow could damage the cart and hurt the animal. A mistake with a dog is serious too, but a dog usually moves out of the way. The consequences of being wrong are very different, and the decision must take them into account.
Costs of being wrong¶
We describe the consequences with a cost matrix : the cost of taking action when the object is actually . Larger numbers are worse.
| Action | Pedestrian | Dog | Cow |
|---|---|---|---|
| Continue | 100 | 30 | 70 |
| Slow down | 20 | 10 | 20 |
| Brake | 5 | 5 | 5 |
Expected cost¶
The cart does not know which column it is in, but it knows the probability of each. So for each action it computes the expected cost: the average cost it would pay, weighting each possible object by its probability:
stands for “expected value”, the probability-weighted average. The rational choice is the action with the lowest expected cost:
Working it out by hand for the current belief:
| Action | Calculation | Expected cost |
|---|---|---|
| Continue | 76.5 | |
| Slow down | 17.5 | |
| Brake | 5.0 |
The cart should brake.
objects = ["pedestrian", "dog", "cow"]
actions = ["continue", "slow down", "brake"]
belief = np.array([0.55, 0.25, 0.20])
C_simple = np.array([[100, 30, 70],
[ 20, 10, 20],
[ 5, 5, 5]], dtype=float)
expected = C_simple @ belief
for a, e in zip(actions, expected):
print(f"{a:>10}: expected cost {e:5.1f}")
print("best action:", actions[int(np.argmin(expected))]) continue: expected cost 76.5
slow down: expected cost 17.5
brake: expected cost 5.0
best action: brake
A cost matrix must be honest¶
Look again at the matrix. Braking costs 5 whatever the object is, and 5 is smaller than every other entry. So braking wins for every possible belief, even if the cart were 99% sure the road was clear. A cart that brakes for every shadow and plastic bag is not safe. On a wet road, a sudden stop invites the two-wheeler behind to skid into it, and a cart that never arrives is not doing its job either.
Braking has real costs, and the table should say so. Let us revise it to charge 15 for a hard stop (rear-end risk, spilled groceries, delay), whatever the object turns out to be:
| Action | Pedestrian | Dog | Cow |
|---|---|---|---|
| Continue | 100 | 30 | 70 |
| Slow down | 20 | 10 | 20 |
| Brake | 15 | 15 | 15 |
With the original belief (0.55, 0.25, 0.20), braking still wins (15 against 17.5 for slowing). Now suppose a moment later the perception system updates its belief:
| Object | Pedestrian | Dog | Cow |
|---|---|---|---|
| Probability | 0.35 | 0.45 | 0.20 |
The most probable object is now a dog. If the cart simply acted on the most probable object, it would pick the best action for a dog, which is slow down (cost 10). But expected cost says otherwise:
C_honest = np.array([[100, 30, 70],
[ 20, 10, 20],
[ 15, 15, 15]], dtype=float)
belief2 = np.array([0.35, 0.45, 0.20])
most_probable = objects[int(np.argmax(belief2))]
naive = actions[int(np.argmin(C_honest[:, np.argmax(belief2)]))]
expected2 = C_honest @ belief2
print(f"most probable object: {most_probable} → naive choice: {naive}")
for a, e in zip(actions, expected2):
print(f"{a:>10}: expected cost {e:5.1f}")
print("expected-cost choice:", actions[int(np.argmin(expected2))])most probable object: dog → naive choice: slow down
continue: expected cost 62.5
slow down: expected cost 15.5
brake: expected cost 15.0
expected-cost choice: brake
Slowing down has expected cost 15.5, braking 15.0. The cart should brake, even though the object is most likely a dog. The reason is that a pedestrian or a cow, the two costly possibilities, together have probability 0.55. The small chance of an expensive mistake outweighs the larger chance of a cheap one.
Probability alone does not decide the action. The consequences of being wrong matter just as much.
Seeing every belief at once¶
Every belief is a point where the three probabilities add up to 1, so we can draw all of them as a triangle and colour each point by its best action:
N = 300
pp, pc = np.meshgrid(np.linspace(0, 1, N), np.linspace(0, 1, N))
pd = 1 - pp - pc
valid = pd >= 0
P = np.stack([pp, pd, pc], axis=-1)
best = np.argmin(P @ C_honest.T, axis=-1).astype(float)
best[~valid] = np.nan
from matplotlib.colors import ListedColormap
cmap = ListedColormap(["#2e8b57", "#d4a017", "#c0392b"])
fig, ax = plt.subplots(figsize=(5.2, 4.6))
ax.imshow(best, origin="lower", extent=[0, 1, 0, 1], cmap=cmap, vmin=0, vmax=2, alpha=0.85)
for b, lbl in [(belief, "first belief"), (belief2, "second belief")]:
ax.plot(b[0], b[2], "ko", ms=6); ax.annotate(lbl, (b[0], b[2]), xytext=(6, 6), textcoords="offset points", fontsize=8)
from matplotlib.patches import Patch
ax.legend(handles=[Patch(color=c, label=a) for c, a in zip(cmap.colors, actions)], fontsize=8, loc="upper right")
ax.set_xlabel("P(pedestrian)"); ax.set_ylabel("P(cow)"); ax.grid(False); ax.set_title("Best action for every belief");
Figure 1:The best action for every possible belief, using the honest cost matrix. Each point is a belief; its horizontal position is P(pedestrian) and its vertical position is P(cow), with P(dog) = 1 − P(pedestrian) − P(cow). The two marked beliefs are the examples from the text.
The picture shows the whole policy at a glance. Slow down covers beliefs dominated by “dog”, and brake takes over as soon as pedestrian and cow together become likely enough. With this matrix, braking beats slowing exactly when .
Continue never appears. Row by row, slowing down costs less than continuing whatever the object is (20 < 100, 10 < 30, 20 < 70), so continuing is dominated: some other action is better in every state, and no belief can make it the best choice. Spotting dominated actions is a quick sanity check on any cost matrix. For continue to be sensible, the table would need a state such as “the object is moving out of the way”, where slowing down wastes time for nothing.
Road context changes the numbers. Near a school zone at dismissal time, the prior probability of a pedestrian is higher, which moves beliefs to the right. On a monsoon evening, braking distances are longer, which raises the cost of continue. The method stays the same; only the inputs change.
Value of information: look again, or act now?¶
The cart is uncertain between pedestrian, dog, and cow. It has two options:
Option A: act now on its current belief, which with the belief (0.35, 0.45, 0.20) means brake, at expected cost 15.
Option B: capture one more camera frame (about 0.3 s), update its belief with Bayes’ rule, and then choose.
Is obtaining more information worth the time? A second frame could show the object’s shape more clearly. The camera’s classifier behaves like this:
| True object ↓ / Camera says → | pedestrian | dog | cow |
|---|---|---|---|
| Pedestrian | 0.85 | 0.10 | 0.05 |
| Dog | 0.10 | 0.85 | 0.05 |
| Cow | 0.05 | 0.05 | 0.90 |
Before looking, the cart does not know what the camera will say, but it can work out each possible answer, the belief that answer would produce, and the best action after it. Averaging over the possible answers gives the expected cost after looking. The value of information (VOI) is the saving:
Here is the camera’s answer, how likely that answer is, and the expected cost using the updated belief.
camera = np.array([[0.85, 0.10, 0.05], # rows: true object, columns: camera's answer
[0.10, 0.85, 0.05],
[0.05, 0.05, 0.90]])
def value_of_information(belief, C=C_honest, sensor=camera, verbose=False):
act_now = (C @ belief).min()
after = 0.0
for k, answer in enumerate(objects):
joint = sensor[:, k] * belief # P(answer | object) × P(object)
p_answer = joint.sum()
post = joint / p_answer # Bayes' rule
best_after = (C @ post).min()
after += p_answer * best_after
if verbose:
print(f"camera says {answer:>10} (prob {p_answer:.3f}): belief {np.round(post, 2)}, "
f"best action {actions[int(np.argmin(C @ post))]:>9}, cost {best_after:5.2f}")
return act_now, after, act_now - after
now, after, voi = value_of_information(belief2, verbose=True)
print(f"\nact now: {now:.2f} look first: {after:.2f} value of information: {voi:.2f}")camera says pedestrian (prob 0.352): belief [0.84 0.13 0.03], best action brake, cost 15.00
camera says dog (prob 0.427): belief [0.08 0.89 0.02], best action slow down, cost 11.05
camera says cow (prob 0.220): belief [0.08 0.1 0.82], best action brake, cost 15.00
act now: 15.00 look first: 13.31 value of information: 1.69
Looking again is worth about 1.7 cost units on average. If the camera says “dog”, the cart can slow down instead of braking hard; if it says “pedestrian” or “cow”, it brakes anyway. The information is valuable because it can change the decision.
But looking is not free. For 0.3 s the cart keeps rolling towards the object, and the distance it covers is distance it no longer has for braking. That cost depends on speed. We model it as 2 cost units per metre travelled while waiting (a teaching value, like the others):
wait_s, cost_per_m = 0.3, 2.0
speeds_kmh = np.linspace(0, 30, 200)
wait_cost = cost_per_m * (speeds_kmh / 3.6) * wait_s
plt.plot(speeds_kmh, np.full_like(speeds_kmh, voi), c="#1f3a5f", lw=2, label=f"value of information ({voi:.2f})")
plt.plot(speeds_kmh, wait_cost, c="#c0392b", lw=2, label="cost of waiting 0.3 s")
plt.fill_between(speeds_kmh, wait_cost, voi, where=wait_cost < voi, color="#2e8b57", alpha=0.15, label="worth looking again")
for v, lbl in [(8, "Case 1: 8 km/h"), (25, "Case 2: 25 km/h")]:
plt.axvline(v, c="gray", ls=":", lw=1); plt.text(v + 0.4, 3.6, lbl, fontsize=8)
plt.xlabel("cart speed [km/h]"); plt.ylabel("cost units"); plt.legend(fontsize=8, loc="upper left")
plt.title("Is another look worth it?")
for v in [8, 25]:
wc = cost_per_m * (v / 3.6) * wait_s
print(f"{v:>2} km/h: waiting costs {wc:.2f}, information is worth {voi:.2f} → "
+ ("look again" if wc < voi else "act now (brake)")) 8 km/h: waiting costs 1.33, information is worth 1.69 → look again
25 km/h: waiting costs 4.17, information is worth 1.69 → act now (brake)

Figure 2:Value of one more camera frame (flat line) against the cost of the 0.3 s it takes (rising line), as the cart’s speed changes. Below about 10 km/h, looking again pays off; above it, the cart should act on what it already knows.
The two cases from the chapter’s story come out clearly:
Case 1: moving slowly (8 km/h through a lane of parked two-wheelers). Waiting 0.3 s costs about 1.3 units, less than the 1.7 the information is worth. The cart should take another look.
Case 2: approaching quickly (25 km/h). Waiting costs about 4.2 units, far more than the information is worth. The cart should brake now on its current belief.
Three ideas sum up the value of information:
Information reduces uncertainty, but it is only valuable if it can change what the robot does. If every possible answer led to “brake”, the VOI would be zero.
Obtaining information has a cost: time, energy, computation, and sometimes risk.
Waiting can itself be dangerous. At higher speed, the robot must be willing to act on imperfect knowledge.
The chapter summary collects the ideas, equations, and exercises from all four sections.