joint-distribution-mental-model

Thu Jul 16 2026

The opening question

$X$ and $Y$ are each $\text{Bern}(1/2)$, and that is everything you know about them. What is $P(X=1, Y=1)$?

Where I got stuck

I answered 1/4 without pausing:

The marked step swaps $P(Y=1 \mid X=1)$ for $P(Y=1)$, which needs independence, and independence was never given. My justification when pressed: two separate Bernoulli events are independent, right? The belief underneath was that once you know each variable's own distribution, common sense fills in the rest.

A counterexample: flip a single fair coin, let $X=1$ for heads and $Y=1$ for tails. Each marginal is $\text{Bern}(1/2)$, exactly like two separate coins, but

Same marginals, completely different joint.

The key move: one combined sample space, one table

What made the whole picture click was seeing which object the notation commits you to. In my words from the session:

One refinement: there is no moving between spaces. The moment you write $P(X=i, Y=j)$, both variables are mappings on one common sample space, and "two different experiments" just means $\Omega$ is the product of the two experiments' outcomes. $P(X=i)$ lives on that same space too, as the event "$X=i$ and $Y$ anything".

This is also what makes comparing the two variables legal at all. I raised it twice as a side worry: how can you write $X = Y$ or $Y = 1-X$ when they come from different experiments? Once both are mappings on the one combined space, $X = Y$ is just the ordinary event $\{\omega : X(\omega) = Y(\omega)\}$, so the comparison is fine the moment a joint model exists.

Draw that space as a 2D table: rows for $X$'s values, columns for $Y$'s, one cell per pair. Each outcome $\omega$ lands in exactly one cell, so the cells are mutually exclusive and exhaustive. They partition the space and sum to 1. Getting a marginal is then just projecting the table onto one axis: sum a row and you have collapsed $Y$ away, which is exactly what $P(X=i) = \sum_j P(X=i, Y=j)$ means.

Three worlds, same marginals

That projection runs one way only. Keep both marginals pinned at $\text{Bern}(1/2)$ and change only the relationship:

The chain rule shows this is the full range. $P(X=1) = 1/2$ is pinned by the marginal, and $P(Y=1 \mid X=1)$ can be pushed anywhere from 0 to 1, so the joint value sweeps $[0, 1/2]$. Finding the top end also corrected a second belief: conditioning does not always shrink a probability, since $Y = X$ gives $P(Y=1 \mid X=1) = 1$, larger than the marginal $1/2$.

The three tables have identical row and column margins but different interiors. The marginals cannot tell them apart. The joint table is the only object that can. So the honest answer to the opening question is "cannot be determined." Asking two marginals to produce a joint value is asking two shadows to rebuild the object that cast them.

The relationship information lives nowhere inside the two marginal probability mass functions (PMFs), which are just two lists of numbers, one per variable. Even "they come from different experiments" is relationship information you add to the model, never something you read off the marginals. Independence is the one case where the shadows do rebuild the object, and only because "independent" is itself the relationship being handed to you: the joint is then declared to be the product of the marginals (the PMF face in discrete-joint-distributions).

Mantra: marginals are shadows, the joint is the object. Shadows fall out of the object by projection. No pair of shadows gives the object back.