Named Distributions

One trial becomes a count.

A Bernoulli variable is a single yes/no event. A binomial is the sum of $n$ independent Bernoulli trials with the same success probability $p$.

Drag $n$ and $p$: the bars are $P(X=k)$ for $X\sim\mathrm{Binomial}(n,p)$. The center tracks $np$ and the spread tracks $np(1-p)$.

Waiting for the first success.

The geometric distribution counts trials until the first success. After failures, the remaining wait still has the same distribution: discrete memoryless waiting time.

The expectation follows from the tail sum:

\[\mathbb{E}X=\sum_{k\ge0}P(X>k)=\sum_{k\ge0}(1-p)^k=1/p.\]

Many tiny chances become a rate.

If $X_n\sim\mathrm{Binomial}(n,\lambda/n)$, then as $n\to\infty$ the mass approaches $\mathrm{Poisson}(\lambda)$. This is the count model for many independent rare opportunities.

Poisson processes are the process-level version: counts over time windows plus exponential waiting times.

The plot overlays the binomial bars with the limiting Poisson curve.

Waiting for several successes, or overdispersed counts.

A negative binomial can be seen as a sum of geometric waits. As a count model, it is what you reach for when Poisson variance is too small for the data.

The gray curve is a Poisson with the same mean. The wider bars show the extra dispersion.

The random-number source behind the others.

If $U\sim\mathrm{Uniform}(0,1)$, then $F^{-1}(U)$ has CDF $F$. This is the inverse-CDF method behind exact one-dimensional sampling.

The left axis is uniform probability. The curve is $F^{-1}(u)$ for the chosen family, turning evenly spaced $u$ values into nonuniform samples.

Sampling methods builds this into rejection and importance sampling.

Memoryless waiting in continuous time.

The exponential distribution is the continuous waiting time with no aging:

\[P(T>s+t\mid T>s)=P(T>t).\]

It is also the interarrival-time distribution in a Poisson process.

The minimum of independent exponentials is exponential again: $\min(T_1,T_2)\sim\mathrm{Exp}(\lambda_1+\lambda_2)$. Competing clocks add rates.

The shape sums with finite variance converge to.

Gaussian distributions are closed under sums: independent normals add to a normal. More broadly, normalized sums of many finite-variance variables drift toward a bell curve.

The bars show the sum of $m$ centered uniform variables rescaled to variance $\sigma^2$. The curve is the matching normal approximation.

A ratio distribution with no mean.

If $Z_1,Z_2$ are independent standard normals, then $Z_1/Z_2$ is Cauchy. Its tails are so heavy that the mean does not exist.

The plot shows the theoretical Cauchy density against a normal density. The Cauchy peak is lower and its tails decay much more slowly.

Adding independent variables convolves their densities.

The density of $X+Y$ is $f_X*f_Y$. Some families are closed under addition: Gaussian plus Gaussian is Gaussian; Poisson plus Poisson is Poisson; Cauchy plus Cauchy is Cauchy.

The plot shows two draggable family choices and their sum. A future MGF/CF page can explain the algebra behind this closure.

Extremes act on the CDF, not the density.

If $M_n=\max(X_1,\dots,X_n)$, then:

\[P(M_n\le x)=F(x)^n.\]

If $m_n=\min(X_1,\dots,X_n)$, then:

\[P(m_n\le x)=1-(1-F(x))^n.\]

The plot uses a $\mathrm{Uniform}(0,1)$ base distribution and shows how increasing $n$ pushes mass toward the right edge for maxima and the left edge for minima.

Changing variables reshapes density by the derivative.

For a monotone transform:

\[f_Y(y)=f_X(g^{-1}(y))\left|\frac{d}{dy}g^{-1}(y)\right|.\]

Non-monotone transforms sum over preimages.

Two canonical cases: $Y=X^2$ turns $N(0,1)$ into $\chi^2_1$; $Y=e^X$ turns a normal into a log-normal.

Division creates heavy tails.

The Cauchy distribution is what you get when you divide one standard normal by another. Near-zero denominators create enormous ratios.

This is the ratio story behind the $t$ and $F$ sampling distributions too: normalize by an estimated scale, and tail weight appears.

Sum squared standard normals.

If $Z_i\sim N(0,1)$ independently, then $\sum_{i=1}^k Z_i^2\sim\chi^2_k$. It is the distribution of squared Gaussian length in $k$ dimensions.

A normal divided by estimated scale.

If $Z\sim N(0,1)$ and $V\sim\chi^2_\nu$, then $T=Z/\sqrt{V/\nu}$ has Student's $t_\nu$ distribution. As $\nu$ grows, the denominator stabilizes and $t$ becomes Gaussian.

Ratio of scaled chi-squares.

If $U\sim\chi^2_{d_1}$ and $V\sim\chi^2_{d_2}$ independently, then $(U/d_1)/(V/d_2)$ has an $F_{d_1,d_2}$ distribution. It appears in variance comparisons and ANOVA-style ratios.

Slow tail decay changes which moments exist.

The Pareto tail has $P(X>x)\propto x^{-\alpha}$. On log-log axes it becomes a straight line, while exponential and Gaussian tails curve downward much faster. Smaller $\alpha$ means heavier tails and fewer finite moments.

Cauchy has no mean or variance. Pareto has finite moments only for orders below $\alpha$: mean needs $\alpha>1$, variance needs $\alpha>2$.