Skip to article frontmatterSkip to article content
Site not loading correctly?

This may be due to an incorrect BASE_URL configuration. See the MyST Documentation for reference.

2.1 Probability: What Might Happen?

Source

This chapter follows one story in four steps. Every robot in this book has to answer these four questions, in this order:

We begin with the first question, in a kitchen.

The next vessel at the washing station

A vessel-washing robot stands at a sink in a busy home kitchen. After every meal, vessels arrive one at a time: steel plates, tumblers, bowls, the cooking vessels (the kadai, the tawa, the cooker), and the occasional odd item such as a sieve or a ladle.

Before the next vessel arrives, the robot has to get ready. It might preheat water for a greasy kadai or open the rack for a stack of plates. So it asks: what will the next vessel be?

A tempting answer is “a plate, because plates are the most common”. That answer is often wrong. Plates are common, but not certain; if the robot commits to “plate” and a greasy kadai arrives, it has prepared the wrong cycle. What the robot needs is not a guess but a statement of how likely each possibility is.

From a few weeks of watching this kitchen, the robot has learned how often each type appears:

Vessel type xxPlateTumblerBowlCooking vesselOther
P(X=x)P(X = x)0.400.250.200.100.05

Random variables and PMFs

We call the type of the next vessel a random variable and write it as a capital letter, XX. A random variable is a quantity whose value we do not yet know, but whose possible values and their likelihoods we can describe. A particular value is written in lower case, xx, so X=plateX = \text{plate} reads “the next vessel is a plate”.

Because XX takes one of a list of values, it is a discrete random variable, and the table above is its probability mass function (PMF). Any PMF must satisfy two rules:

P(X=x)≥0for every x,∑xP(X=x)=1P(X = x) \ge 0 \quad \text{for every } x, \qquad \sum_{x} P(X = x) = 1

The first rule says a probability can never be negative. The second says the next vessel must be something on the list, so the probabilities add up to exactly 1. The symbol ∑x\sum_x means “add up over every possible value xx”.

A probability here expresses uncertainty about an unknown future event. “P(X=plate)=0.40P(X = \text{plate}) = 0.40” does not mean the next vessel is 40% of a plate. It means that in the long run, about 40 out of every 100 vessels are plates, and that the robot should weigh “plate” accordingly when it prepares.

probabilities add up to 1.0
first ten arrivals: tumbler, cooking vessel, bowl, plate, plate, cooking vessel, plate, bowl, bowl, tumbler

The sequence looks irregular, just as a real sink does. The PMF cannot tell us what this evening will look like, but it tells us what many evenings look like on average. We can check this by simulating many arrivals and comparing the observed frequencies with the PMF:

<Figure size 770x396 with 1 Axes>

Figure 1:The vessel PMF (dark bars) compared with the observed fraction of each vessel type in simulated streams of 30 and 1000 vessels. The short stream differs noticeably from the PMF; the long one matches it closely.

Measurements that never repeat exactly

Now move to the stove. The robot heats oil for a tadka and wants it near 170 °C. It holds the thermometer probe in the oil and reads it once per second. Nothing changes the heater setting, yet the readings are:

Reading123456
Temperature (°C)168171169173170172

Why aren’t they identical? Several small effects add up. The oil is not perfectly mixed, so the probe tip sits in slightly hotter or cooler pockets. The electronics add a little electrical noise, and the display rounds to whole degrees. None of these is a fault. They are the normal behavior of a real sensor in a real kadai.

So the next reading is also a random variable, TT. Unlike the vessel type, it is continuous: it can take any value in a range, not just one of a list. We cannot predict the next reading exactly, but we can summarize where readings tend to fall.

Mean and variance

The mean is the centre of the readings, their ordinary average. For nn readings t1,t2,…,tnt_1, t_2, \dots, t_n:

tˉ=1n∑i=1nti\bar{t} = \frac{1}{n}\sum_{i=1}^{n} t_i

The bar in tˉ\bar{t} (“t-bar”) marks a value computed from data. For our six readings, tˉ=(168+171+169+173+170+172)/6=170.5\bar{t} = (168 + 171 + 169 + 173 + 170 + 172)/6 = 170.5 °C.

The variance measures how spread out the readings are: it is the average squared distance from the mean. Computed from a sample, it is

s2=1n−1∑i=1n(ti−tˉ)2s^2 = \frac{1}{n-1}\sum_{i=1}^{n} (t_i - \bar{t})^2

Squaring makes every distance positive, so readings above and below the mean both count. We divide by n−1n - 1 rather than nn because the mean itself was estimated from the same readings, which makes the readings look slightly less spread than they really are; dividing by n−1n-1 corrects for this. The standard deviation s=s2s = \sqrt{s^2} is in the same units as the readings (°C), which makes it easier to interpret.

readings      : [168. 171. 169. 173. 170. 172.]
deviations    : [-2.5  0.5 -1.5  2.5 -0.5  1.5]
mean          : 170.5 °C
variance s²   : 3.50 °C²
std. dev. s   : 1.87 °C

A model for repeated readings: the Gaussian

If we collect many more readings with the heater steady, a pattern appears: most readings cluster near the centre, and readings far from the centre become steadily rarer, in a symmetric bell shape. This shape appears whenever many small, independent effects add together, as they do in the kadai. It is called the Gaussian (or normal) distribution, and we write

T∼N(μ,σ2)T \sim \mathcal{N}(\mu, \sigma^2)

Read this as “TT is Gaussian with mean μ\mu and variance σ2\sigma^2”. Greek letters μ\mu (“mu”) and σ\sigma (“sigma”) denote the true mean and standard deviation of the process; tˉ\bar{t} and ss are our estimates of them from data. The Gaussian’s probability density is

p(t)=12πσ2exp⁡ ⁣(−(t−μ)22σ2)p(t) = \frac{1}{\sqrt{2\pi\sigma^2}} \exp\!\left(-\frac{(t-\mu)^2}{2\sigma^2}\right)

For a continuous quantity we use a density p(t)p(t) (lower-case pp) rather than a PMF. The chance that a reading falls in a range is the area under the curve over that range. Two rules of thumb are worth remembering: about 68% of readings fall within μ±σ\mu \pm \sigma, and about 95% fall within μ±2σ\mu \pm 2\sigma.

mean 169.89 °C, std. dev. 1.60 °C, fraction within ±2s: 0.97
<Figure size 770x396 with 1 Axes>

Figure 2:One hundred thermometer readings of oil held near 170 °C (histogram) and the Gaussian model fitted to them (curve). The shaded band marks the mean ± 2 standard deviations, where about 95% of readings fall.

Do we need all 100 readings?

The robot now has 100 readings. Each takes a second, and a tadka does not wait. Does it need all of them to know the average temperature?

Two words help here. The population is everything we could in principle measure: all possible readings of this oil at this setting. A sample is the readings we actually take. Here we treat the 100 readings as the population and ask how well smaller samples estimate their mean.

We draw many random samples of 5, 20, and 50 readings, compute each sample’s mean, and see how much those means vary:

mean of all 100 readings: 169.89 °C
<Figure size 1210x330 with 3 Axes>

Figure 3:Means of 2000 random samples of each size, drawn from the 100 readings. Small samples give means that scatter widely; larger samples cluster tightly around the mean of all 100 readings (dashed line).

The pattern is clear. With 5 readings, the sample mean can easily be off by more than a degree. With 50, it is almost always within a few tenths of a degree of the full-population mean. This variation from sample to sample is called sampling uncertainty.

There is a simple rule behind it. If single readings have standard deviation σ\sigma, the mean of nn independent readings has standard deviation

σtˉ=σn\sigma_{\bar{t}} = \frac{\sigma}{\sqrt{n}}

This is called the standard error of the mean. Quadrupling the number of readings only halves the uncertainty, so each extra reading helps less than the one before. For the robot, 10–20 readings (10–20 seconds) usually give a good enough average; waiting for 100 buys little extra precision while the mustard seeds burn.