normal-and-exponential-distributions

Tue Jul 14 2026

The standard Normal

The standard Normal $Z \sim \mathcal{N}(0,1)$ has PDF

$\varphi(z) = \frac{1}{\sqrt{2\pi}}\, e^{-z^2/2}, \quad -\infty < z < \infty.$

The exponent is just $-z^2/2$. The $\pi$ is the normalizing constant, which exists only to make the total area 1. This PDF is special enough to get its own letter $\varphi$, and its CDF gets $\Phi$.

The mean is where the symmetry pays out. The expectation integrand $z\,\varphi(z)$ is an odd function: same magnitude at $\pm z$, opposite sign, so the positive lobe cancels the negative lobe and $E(Z) = 0$ with no antiderivative in sight. One footnote makes the cancellation legal: each lobe must be finite ($\infty - \infty$ is undefined), and it is, because $e^{-z^2/2}$ dies faster than any polynomial grows. The same argument kills every odd moment ($E(Z^k)$ for odd $k$, so $E(Z^3) = E(Z^5) = 0$ too). Variance takes an integration by parts I took on faith: $\text{Var}(Z) = 1$, hence the notation $\mathcal{N}(0, 1)$.

$\Phi$ has no closed form, so all practical work runs through identities. The first one: anchor at $\Phi(0) = 1/2$, name the slab between $-z$ and $0$, and the bookkeeping lands on

$\Phi(-z) = 1 - \Phi(z).$

The mental handle: the left tail below $-z$ equals the right tail above $+z$, so $\Phi(-z) = P(Z \ge z)$.

Transforms move mass, not points

A transform of a random variable moves probability mass, and the only honest way to follow it is to track a small segment, never a point. This is the $dx$ view from pdf-and-cdf-basics doing real work.

The warm-up was $W = -Z$. My flipping argument: negation literally maps $z = -k$ to $+k$, so the PDF flips over 0, and $\varphi$ is even, so $W \sim \mathcal{N}(0,1)$ too. The argument is valid, but it rests on a hidden lemma worth saying out loud: negation carries the mass $\varphi(z)\,dz$ sitting on $[-k-dk, -k]$ onto $[k, k+dk]$, an interval of the same width. Rigid transport, no stretching, so heights carry over unchanged. (The CDF route needs no picture at all: $P(-Z \le w) = P(Z \ge -w) = 1 - \Phi(-w) = \Phi(w)$. Two moves.)

The width caveat is the whole story. Run the same flip on $W = Z^3$ and it breaks: the segment $[k, k+dk]$ lands on $[k^3, (k+dk)^3]$, whose width is $3k^2\,dk$ to first order. The width multiplier is the derivative of the map. To keep the mass block's area fixed, the height divides by that same factor: $f_W(k^3) = \varphi(k)/3k^2$. Near 0, where $3k^2 < 1$, widths compress and the density piles up instead. That is the $U^2$ pile-up from sitting 1, now with a formula behind it.

The general Normal is defined by this transport. Take $X = \mu + \sigma Z$ in two stages. Stretch: $w = \sigma z$ scales every width $dz$ by $\sigma$, so heights divide by $\sigma$, giving $f_W(w) = \varphi(w/\sigma)/\sigma$. Shift: sliding by $\mu$ moves the curve without touching widths, giving

$f_X(x) = \varphi\!\left(\frac{x-\mu}{\sigma}\right)\frac{1}{\sigma}.$

Why $\varphi(w/\sigma)$ and not $\varphi(w)$? Read it from the new variable's side. The height that lived at $z$ now lives at $w = \sigma z$, so the density at $w$ is the original looked up at its preimage $w/\sigma$: from $w$'s perspective, $w = 1$ holds whatever lived at $z = 1/\sigma$. The shift is the same logic, the mass at $x$ came from $w = x - \mu$. The recipe for any one-to-one $Y = g(X)$: evaluate the old density at the preimage, then divide by the stretch factor $|g'|$.

Standardization and the 68-95-99.7 rule

The transport also runs backwards: if $X \sim \mathcal{N}(\mu, \sigma^2)$ then $(X-\mu)/\sigma \sim \mathcal{N}(0,1)$, and consequently $F(x) = \Phi\!\left(\frac{x-\mu}{\sigma}\right)$. There is only one Normal, and every Normal question becomes a $\Phi$ question. For quick mental math: $P(|Z| < 1) \approx 0.68$, $P(|Z| < 2) \approx 0.95$, $P(|Z| < 3) \approx 0.997$.

The worked pipeline: $X \sim \mathcal{N}(-1, 4)$, find $P(|X| < 3)$. First trap dodged: the second parameter is the variance, so $\sigma = 2$, not 4. Standardizing turns $-3 < X < 3$ into $-1 < Z < 2$, so the exact answer is $\Phi(2) - \Phi(-1) = \Phi(2) + \Phi(1) - 1$ by the tail identity. For the approximation, my decomposition: $[-3, -1]$ is the left half of the 1SD band (0.34) and $[-1, 3]$ is the right half of the 2SD band (0.475), total 0.815 against a true 0.8186. The bands measure from $\mu = -1$, not from 0. That is exactly what standardizing guards against.

The crypto anchor: Gaussians carry lattice crypto. Falcon samples discrete Gaussians, LWE hides secrets under Gaussian noise, and the security arguments are tail bounds, the 68-95-99.7 rule taken to cryptographic extremes.

The Exponential from its survival function

The Exponential $\text{Expo}(\lambda)$ models the waiting time until the first arrival when arrivals hit at rate $\lambda$. My recall worry ("doesn't $e^{-\lambda x}$ blow up on the left?") is answered by the story: waits are never negative, the support is $(0, \infty)$, and the blow-up region is simply not in it. The kernel $e^{-\lambda x}$ needs a constant, and unlike the Normal's $\sqrt{2\pi}$ this one is a freshman integral: $c = \lambda$, so

$f(x) = \lambda e^{-\lambda x}, \quad x > 0.$

Units sanity check: $\lambda$ carries per-time units, exactly the density units a PDF over time must have, and $\lambda x$ in the exponent is dimensionless.

The CDF is $F(x) = 1 - e^{-\lambda x}$, so the survival function is $P(X > x) = e^{-\lambda x}$. That survival function is the real primitive: exponential-survival-first rebuilds it from nothing but a constant rate, and everything below runs through it.

The mean and variance come from the chapter's signature move, compute for the simplest family member and transform out. One integration by parts gives $E(X) = 1$ for $\text{Expo}(1)$, and taking $E(X^2) = 2$ on faith gives $\text{Var}(X) = 1$. Scaling is the family's only knob (a shift would drag the support off $(0,\infty)$, so unlike the Normal there is no location part): $X/\lambda \sim \text{Expo}(\lambda)$, with mean $1/\lambda$ and variance $1/\lambda^2$. Faster arrivals, shorter waits. One flag from my own work: I wrote $\text{Var} = E(X^2) - E(X)$ and got the right number only because $E(X) = 1$ is its own square. The formula is $E(X^2) - (E(X))^2$. Cheap luck, flagged.

Memorylessness

The proof is two lines once you see the collapse: the event $X > s+t$ implies $X > s$, so their intersection is just the smaller event, and the conditional becomes a ratio of survivals:

$P(X > s+t \mid X > s) = \frac{e^{-\lambda(s+t)}}{e^{-\lambda s}} = e^{-\lambda t}.$

What the whole proof rests on, in one sentence: exponential functions have the property $f(x+y) = f(x) \cdot f(y)$. The survival function factorizes, so conditioning divides the past out exactly. No polynomial or Gaussian tail survives that ratio unchanged. And the converse holds: any continuous distribution on $(0,\infty)$ whose survival function satisfies $G(s+t) = G(s)G(t)$ is forced to be $e^{-\lambda x}$ for some $\lambda$. So the Exponential is the only memoryless continuous distribution. Memorylessness characterizes it.

The Poisson bridge

The definition speaks only the counting language (how many arrivals, a discrete number). The waiting language is one sentence away: "the first arrival has not happened by time $t$" is the same event as "zero arrivals landed in $(0, t]$". Identical events have identical probabilities, so

$P(T_1 > t) = P(N_t = 0) = e^{-\lambda t}\frac{(\lambda t)^0}{0!} = e^{-\lambda t}.$

The Exponential survival function falls out of a definition that never mentioned waiting: $T_1 \sim \text{Expo}(\lambda)$. After the first arrival, independence of disjoint intervals restarts the process fresh, so the gaps between arrivals are i.i.d. $\text{Expo}(\lambda)$. (Disjoint is doing the work there: overlapping intervals share arrivals, so their counts are obviously dependent.) One process, two views: count the arrivals and you get Poisson, time them and you get Exponential. That is why they share the letter $\lambda$.

The units wire I crossed twice before it settled: $\lambda$ is a rate (arrivals per unit time), $E(T_1) = 1/\lambda$ is a time. Reciprocals, not equals. Buses at rate 6 per hour have mean gap $1/6$ hour, and $6 \ne 1/6$. They coincide only at $\lambda = 1$, the deceptive special case. The other $\lambda$, the prefactor in $\lambda e^{-\lambda t}$, is the same letter doing a second consistent job: the density's per-time units showing up for work.

Bitcoin runs on this bridge. Block arrivals are engineered into a Poisson process (difficulty retargets to hold $\lambda = 1/10$ per minute), inter-block gaps are $\text{Expo}(1/10)$, and memorylessness is what "progress-free mining" means. After 25 minutes without a block, the expected wait is still 10 minutes from now. A block is never "due".

What stayed shaky: Fred's bus paradox

Buses in Blotchville have Expo gaps with mean 10 minutes, Fred arrives at a random moment, and memorylessness resets the clock: his expected wait is a fresh 10 minutes. His friend argues 5, and each premise sounds fine (you land uniformly inside a gap, the average gap is 10, so you wait half of it). The part I did get: with alternating 50/5 gaps, the dart lands in a 50-minute gap ten times more often, so the gap containing Fred is systematically longer than a typical gap. Length-biased sampling. Which sentence of the friend's argument actually dies, and how the two answers reconcile, is parked as fred-bus-waiting-paradox and opens sitting 3.