lotus-and-variance

Sun Jul 05 2026

LOTUS: the shortcut that is secretly a theorem

To compute $E(X^2)$ for a die roll, the definition of expectation says I owe the PMF of $X^2$ first: values of $X^2$, weighted by probabilities of $X^2$. But everyone just writes $(1+4+9+16+25+36)/6 = 91/6$ and never finds that PMF. Why is that legal?

For $X^2$ the question barely bites. Squaring is one-to-one on $\{1,\dots,6\}$, so the six weights carry over unchanged and the formula is the definition with values relabeled. The real test is a function with collisions:

$W = (X - 3.5)^2$

Now $x=1$ and $x=6$ both map to $6.25$, $x=2$ and $5$ to $2.25$, $x=3$ and $4$ to $0.25$. The relabeling defense is dead. Compute both ways:

• Honest route, PMF of $W$ first: $E(W) = \tfrac13(0.25) + \tfrac13(2.25) + \tfrac13(6.25) = 35/12 \approx 2.92$.
• Shortcut, never forming $W$'s PMF: $\sum_x (x-3.5)^2 \cdot \tfrac16$ over the six faces.

Pair the shortcut's six terms by collision and each brace holds a constant value of $W$ with weight $\tfrac16 + \tfrac16 = \tfrac13$, which is exactly $P(W=w)$. The shortcut is the honest sum, rearranged. That regrouping is the entire proof, and it is the same move that proves linearity of expectation. The move itself lives in pebble-grouping.

The general statement is Theorem 4.5.1, the law of the unconscious statistician: for any function $g$,

$E(g(X)) = \sum_x g(x)\,P(X=x).$

The name is a joke. You swap $x$ for $g(x)$ in the expectation formula without thinking, and the theorem's content is that the thoughtless substitution is correct. For one-to-one $g$ it is pure relabeling. For many-to-one $g$, several $P(X=x)$ values pool into one $P(g(X)=w)$, like the collision pairs above. Rearranging is the word that covers both cases, relabeling only covers the first.

What LOTUS does not license

The tempting sibling shortcut, $E(g(X)) = g(E(X))$, is false for nonlinear $g$. The die already shows it: $E(X^2) = 15.17$ but $(EX)^2 = 12.25$. For linear $g$ the two agree, and that is just linearity of expectation again.

St. Petersburg shows how badly it can fail. Flip a fair coin until the first heads. Let $X$ be the number of flips, so $P(X=n) = (1/2)^n$ and $E(X) = 2$ (waiting times average $1/p$). The casino pays $2^X$ dollars. The naive shortcut prices the game at $2^{E(X)} = 4$ dollars. LOTUS prices it honestly:

$E(2^X) = \sum_{n \ge 1} 2^n \cdot (1/2)^n = 1 + 1 + 1 + \cdots = \infty.$

Every term is exactly 1 and the support never stops, since no flip count is guaranteed to end the game. So by the expectation criterion, no finite ticket price is too high. Pay a million, the game is still a bargain. My gut said a couple of dollars, and the gut is onto something real: the game almost always ends within a few flips and pays pocket change. The mean is infinite only because it is propped up by astronomically rare jackpots.

Both answers are right about different things. Expectation is a center of mass, not a typical value. It tells you where the distribution balances and nothing about how far a single play lands from that point. That gap is what variance exists to fill.

Variance: the second number

The naive spread measure fails first. Averaging the signed deviations gives $E(X - EX) = EX - EX = 0$, for every distribution, by linearity. This is not a symmetry accident. Take $X = 0$ with probability $0.9$ and $X = 10$ with probability $0.1$. The mean is 1, and the deviations are $-1$ carrying weight $0.9$ against $+9$ carrying weight $0.1$. They cancel. The mean is the point where probability-weighted pulls from left and right agree, so signed deviations always net to zero.

The fix is to kill the signs before averaging:

$\mathrm{Var}(X) = E\big[(X - EX)^2\big].$

For the die this is exactly the collision exercise: $W = (X-3.5)^2$, so $\mathrm{Var}(X) = 35/12 \approx 2.92$. I had computed the die's variance before knowing its name.

Why square instead of absolute value? $E|X - EX|$ is a legitimate spread measure (mean absolute deviation), so the square has to win on merit. Three reasons, in increasing order of importance:

1. Algebra. $(X-\mu)^2$ expands into $X^2 - 2\mu X + \mu^2$ and the expansion feeds straight into linearity. Every clean variance formula flows from this. Absolute values do not expand and give almost no formulas.
2. Additivity. For independent $X$ and $Y$, $\mathrm{Var}(X+Y) = \mathrm{Var}(X) + \mathrm{Var}(Y)$. Spreads add. Mean absolute deviation obeys no such law. This is what gives binomial variance in one line below, and it is the engine behind $\sqrt{n}$ behavior, Chebyshev, and the tail bounds crypto arguments lean on.
3. Geometry. Squared deviations give independent contributions a Pythagorean structure. They combine like perpendicular vectors, and SD behaves like a Euclidean length. The additivity in point 2 is the Pythagorean theorem in disguise.

The standing complaint, wrong units (the die's 2.92 is in squared pips), is patched by $\mathrm{SD}(X) = \sqrt{\mathrm{Var}(X)} \approx 1.71$ pips.

Expanding the definition with linearity gives the workhorse form:

$\mathrm{Var}(X) = E(X^2) - (EX)^2.$

The $E(X^2)$ piece is a LOTUS computation, which closes the loop: variance is an $E(g(X))$, and LOTUS is the license to compute it from $X$'s PMF alone. Die check: $91/6 - 3.5^2 = 15.1667 - 12.25 = 2.9167$, the same $35/12$ as the honest route. One notation trap the book itself flags: write $(EX)^2$, not $EX^2$. The two differ by exactly the variance.

What variance is asking

Variance is the expected squared distance from the mean, and it answers one question: if I draw once, how far from the center should I expect to land? It measures unpredictability, not size. A random variable that always outputs 42 is huge but perfectly predictable, so its variance is 0.

That is the whole intuition behind $\mathrm{Var}(c) = 0$. Spread is disagreement between draws. A constant never surprises: every draw lands exactly on the mean, the distance is 0 on every single draw, and the average of all zeros is 0. In the pebble picture, a constant piles every pebble on one spot, which then is the balance point, so no pebble sits any distance away. Zero variance means the distribution has collapsed to a point.

The same reading applied to the distributions I know:

• Bernoulli(p): $p(1-p)$, maximal at $p = 1/2$, the most unpredictable coin. As $p \to 0$ or $p \to 1$ the variance slides to 0 because the coin degenerates into a constant.
• Binomial(n, p): $np(1-p)$, the same per-flip uncertainty added over $n$ independent copies.
• Uniform(a, b): $(b-a)^2/12$. The spread depends only on how wide the room is, not where the room is. Shifting costs nothing, and squeezing the width to zero collapses it back to a constant.
• Fair die: $35/12 \approx 2.92$, six distinct outcomes each holding mass away from 3.5.

Indicators and binomial variance

An indicator $I$ is 1 if an event happens (probability $p$) and 0 otherwise. One coin flip, one trial, one bit of a hash output. Indicators are the only random variables whose square is themselves: $0^2 = 0$ and $1^2 = 1$, so $I^2 = I$ identically and $E(I^2) = E(I) = p$ with no computation. Then

$\mathrm{Var}(I) = p - p^2 = p(1-p).$

The shape is a sanity check by itself. Variance is 0 at $p=0$ and $p=1$ (a certain event does not vary) and maximal at $p = 1/2$, value $1/4$. An unbiased bit is the most unpredictable bit.

A binomial count $X \sim \mathrm{Bin}(n,p)$ is a sum of $n$ independent indicators, so the additivity law gives $\mathrm{Var}(X) = np(1-p)$ in one line. Compare where the assumptions land: $E(X) = np$ never consulted independence, because linearity holds unconditionally (see pebble-grouping for why). Variance addition is the step that consumes it.

What stayed shaky

The recall check at close: I blanked on which number fills the gap expectation leaves. It is the standard deviation. $\mathrm{SD}(X)$ is how far a single realization typically lands from $E(X)$, in $X$'s own units. Die: single rolls sit roughly 1.7 pips from 3.5. St. Petersburg: infinite variance, so $E$ is a uselessly bad guide to a single play. Re-test this reading first next session.

Natural continuations: Chebyshev's inequality, where SD literally becomes a tail bound (the crypto motivation), and Poisson, section 4.7, next in the book.