Student's t, Chi-Square and F Distributions, statistics lesson

A statistic comes out equal to 2. Is that large? The number alone cannot say, because a value that is rare under one model can be ordinary under another. The answer comes from a reference distribution, the distribution the statistic would have if the model were right. This lesson builds three reference families, chi-square, Student’s t and F, from independent standard normal variables, explains what their degrees of freedom count, and shows how tail areas, quantiles and critical values are read from them.

Key terms

Reference distribution

Definition A reference distribution for a statistic is the probability distribution the statistic has, exactly or approximately, when a stated model holds, for example a null hypothesis together with assumptions about how the data were produced. Placing an observed value on it shows whether the value is typical or lies far out in a tail. The most familiar reference is the standard normal distribution $N(0, 1)$ , with mean $0$ and SD $1$ , and a variable with this distribution is written $Z$ . The chi-square, t and F families below are built from independent standard normal variables, and each is indexed by parameters called degrees of freedom. A reference distribution is only as trustworthy as the model and the assumptions that lead to it.

In plain words A reference distribution is the yardstick for a statistic. It says how often values like yours, or more extreme, would turn up if the model were correct.

Example The same value 2 has very different right-tail probabilities under three references. The last two are defined in the cards below.

Reference	Probability of a value of at least 2
Standard normal $N(0, 1)$	$0.0228$
Student’s t with 3 degrees of freedom	$0.0697$
Chi-square with 4 degrees of freedom	$0.7358$

Under the chi-square reference, 2 is even below the center of the distribution. A statistic is unusual or ordinary only relative to a stated reference.

Chi-square distribution

Definition If $Z_1, \dots, Z_k$ are independent standard normal variables, then

U = Z_1^2 + Z_2^2 + \dots + Z_k^2

has the chi-square distribution with $k$ degrees of freedom, written $U \sim \chi^2_k$ . In this construction the degrees of freedom count the independent squared standard normals. A chi-square variable is never negative, has mean $k$ and variance $2k$ , and is right-skewed, strongly for small $k$ and less so as $k$ grows. A related result appears in normal-sample inference. If $X_1, \dots, X_n$ are independent $N(\mu, \sigma^2)$ with $n \ge 2$ and $S^2$ is the sample variance with denominator $n - 1$ , written with a capital letter because it varies from sample to sample, then

\frac{(n - 1) S^2}{\sigma^2} \sim \chi^2_{n-1} .

The deviations $X_i - \bar X$ add to zero, so they cannot all vary freely, and one degree of freedom is lost compared with the $n$ terms of the sum. Degrees of freedom describe the stated construction and its constraints. They are not always the sample size minus one.

In plain words Square independent standard normal values and add them. The total measures a squared distance from zero, so it is never negative, and on average it equals the number of terms.

Example The standard normal values $1$ , $-1$ , $2$ and $0$ give the realized value

\begin{aligned} u &= 1^2 + (-1)^2 + 2^2 + 0^2 \\[0.4em] &= 6, \end{aligned}

one possible value of a $\chi^2_4$ variable, whose mean is $4$ . The reference distribution describes all the values that repeated draws would produce, not this one sum. For $\chi^2_4$ the right tail has the closed form $P(U > x) = e^{-x/2}(1 + x/2)$ for $x \ge 0$ , so

P(U > 6) = 4e^{-3} \approx 0.199 .

About one value in five lies above 6, so 6 is not unusual for this reference.

Student’s t distribution

Definition If $Z \sim N(0, 1)$ and $V \sim \chi^2_\nu$ are independent, then

T = \frac{Z}{\sqrt{V / \nu}}

has Student’s t distribution with $\nu$ degrees of freedom, written $T \sim t_\nu$ . The distribution is symmetric about $0$ and takes negative as well as positive values. It has heavier tails than $N(0, 1)$ , and it approaches $N(0, 1)$ as $\nu \to \infty$ . Its variance is $\nu/(\nu - 2)$ when $\nu > 2$ , which is larger than $1$ , and with $\nu = 1$ it is the standard Cauchy distribution, which has no finite mean. The independence of $Z$ and $V$ is part of the definition. For a normal sample the mean $\bar X$ and the variance $S^2$ are independent, and later lessons use that fact to explain why the familiar t statistic has a t distribution.

In plain words A t variable is a standard normal divided by a random estimate of scale. When the denominator happens to be small, the ratio is stretched, so extreme values are more likely than under the normal curve, especially with few degrees of freedom. The sign always comes from the numerator.

Example With an independent standard normal value $2$ and the chi-square value $10$ on $8$ degrees of freedom, the construction gives

\begin{aligned} \frac{2}{\sqrt{10/8}} &= \frac{2}{\sqrt{1.25}} \\[0.4em] &\approx 1.789, \end{aligned}

a realized value from $t_8$ . It illustrates the construction and says nothing about whether any particular data set satisfies its assumptions. The heavier tails show in the quantiles. The $0.975$ quantile of $t_8$ is about $2.306$ , compared with about $1.960$ for $N(0, 1)$ .

F distribution

Definition If $U \sim \chi^2_{d_1}$ and $V \sim \chi^2_{d_2}$ are independent, then

F = \frac{U / d_1}{V / d_2}

has the F distribution with $d_1$ numerator and $d_2$ denominator degrees of freedom, written $F \sim F_{d_1, d_2}$ . An F variable is never negative and is right-skewed. The order of the degrees of freedom matters, since $1/F$ has the distribution $F_{d_2, d_1}$ . If $T \sim t_\nu$ , then $T^2 \sim F_{1, \nu}$ . F distributions appear when two suitably scaled, independent sources of variation are compared, such as two sample variances from independent normal samples or explained and residual variation in a model. The F reference applies to a statistic that can be written in this form, with independent chi-square parts. Dividing two numbers does not by itself produce an F statistic.

In plain words An F variable compares two independent sums of squares after each is divided by its own degrees of freedom, which turns each into an average squared size. A value near $1$ says the two scaled quantities are about the same size.

Example With the chi-square values $6$ on $4$ degrees of freedom and $10$ on $8$ degrees of freedom, taken as independent,

\frac{6/4}{10/8} = \frac{1.5}{1.25} = 1.2,

a realized value from $F_{4, 8}$ . The probability that an $F_{4, 8}$ variable exceeds $1.2$ is about $0.381$ , so this value is ordinary. Swapping numerator and denominator gives $1/1.2 \approx 0.833$ , a value from $F_{8, 4}$ . Squaring the earlier t value gives $1.789^2 \approx 3.2$ , and exactly $(2^2/1)/(10/8) = 3.2$ , a value from $F_{1, 8}$ .

Critical value

Definition For a random variable $X$ with a continuous distribution, a tail area is the probability beyond a chosen point, for example the right-tail area $P(X > c)$ . A quantile reverses the question. For $0 < p < 1$ , the $p$ quantile is the point $q_p$ with $P(X \le q_p) = p$ , which is unique for the normal, chi-square, t and F distributions because their cumulative distribution functions are continuous and strictly increasing where the density is positive. A critical value is a quantile used as a boundary in a specified test or interval. A right-tail boundary with tail area $\alpha$ is the $1 - \alpha$ quantile. A central interval with total tail area $\alpha$ for a distribution symmetric about $0$ runs between the $\alpha/2$ quantile and the $1 - \alpha/2$ quantile, which are negatives of each other, leaving $\alpha/2$ in each tail. A critical value belongs to one family with one set of degrees of freedom.

In plain words A tail area turns a point into a probability. A quantile turns a probability back into a point. A critical value is the quantile a procedure has chosen as its cutoff.

Example Each cutoff below was computed for its own reference.

Reference	Cutoff	Tail area
$\chi^2_4$	$9.488$	$0.05$ in the right tail
$F_{4, 8}$	$3.838$	$0.05$ in the right tail
$t_8$	$-2.306$ and $2.306$	$0.025$ in each tail
$N(0, 1)$	$-1.960$ and $1.960$	$0.025$ in each tail

So $P(-2.306 \le T \le 2.306) \approx 0.95$ for $T \sim t_8$ . The chi-square value $u = 6$ from the earlier card lies below its cutoff $9.488$ , and the F value $1.2$ lies below its cutoff $3.838$ , which matches their large tail areas $0.199$ and $0.381$ .

Check yourself

A statistic equals $-1.3$ , and you are told it came from a chi-square, a t or an F distribution. Which of the three can produce it?

Show the answer

Only Student’s t. A chi-square variable is a sum of squares, so it is never negative. An F variable is a ratio of two nonnegative quantities whose denominator is positive with probability $1$ , so it is never negative either. A t variable has a standard normal numerator and a denominator that is positive with probability $1$ , so it is negative with probability $\tfrac12$ .

For a central 95 percent range of $t_8$ , a student uses the cutoffs $-1.860$ and $1.860$ , where $1.860$ is the $0.95$ quantile. What probability does that range actually cover, and which cutoff is correct?

Show the answer

Since $1.860$ is the $0.95$ quantile, the right tail beyond it has probability $0.05$ , and by symmetry the left tail below $-1.860$ has $0.05$ as well. The range covers

1 - 0.05 - 0.05 = 0.90,

so it is a central 90 percent range. A central 95 percent range leaves $0.025$ in each tail, so it needs the $0.975$ quantile, about $2.306$ . The normal cutoff $1.960$ would also be wrong here, because it covers only about $0.914$ of the $t_8$ distribution.

Let $Z \sim N(0, 1)$ and set $V = Z^2$ , which on its own has the $\chi^2_1$ distribution. Form $T = Z / \sqrt{V/1}$ . What values can $T$ take, and does it have the $t_1$ distribution?

Show the answer

Since $\sqrt{Z^2} = \lvert Z \rvert$ , the ratio is $T = Z / \lvert Z \rvert$ . With probability $1$ the value of $Z$ is not $0$ , so $T$ equals $1$ when $Z > 0$ and $-1$ when $Z < 0$ , each with probability $\tfrac12$ .

A $t_1$ variable is the standard Cauchy, a continuous distribution with, for example, $P(\lvert T \rvert > 10) \approx 0.0635$ . So this $T$ does not have the $t_1$ distribution, even though its numerator and its denominator each have the right distribution on their own. The construction requires $Z$ and $V$ to be independent, and here $V$ is computed from $Z$ . The same failure happens for F. Using one chi-square variable in both places gives $(V/1)/(V/1) = 1$ whenever $V > 0$ , which happens with probability $1$ , so the ratio is a constant and not an $F_{1, 1}$ variable.

Student's t, Chi-Square and F Distributions

Key terms

Reference distribution

Chi-square distribution

Student’s t distribution

F distribution

Critical value

Check yourself

How this lesson connects

Builds on

Used later in