One-Sample t-Tests

Definition Let $x_1, \dots, x_n$, with $n \ge 2$, be observations from a population with mean $\mu$, and let $\mu_0$ be a benchmark fixed without using these observations. The one-sample t test tests $H_0: \mu = \mu_0$ against an alternative chosen before the data are seen, either the two-sided $H_1: \mu \ne \mu_0$ or one of the one-sided $H_1: \mu > \mu_0$ and $H_1: \mu < \mu_0$. It computes the t statistic $t$ of the next card and, with $T_{n-1}$ a random variable having the t distribution with $n - 1$ degrees of freedom, reports the p-value for the chosen alternative.

These p-values are exact when the observations are independent draws from a normal population. For independent draws from other populations they are approximations, whose quality the card on conditions discusses.

Definition For a sample of size $n \ge 2$ with mean $\bar{x}$ and sample standard deviation $s > 0$, computed with the $n - 1$ denominator, the t statistic for the benchmark $\mu_0$ is

where $s/\sqrt{n}$ is the estimated standard error of the sample mean. If the observations are independent draws from $N(\mu_0, \sigma^2)$ with $\sigma > 0$, the statistic, viewed as a random variable, has the t distribution with $n - 1$ degrees of freedom, written $t_{n-1}$. This distribution is symmetric about $0$, has heavier tails than $N(0, 1)$ for every finite number of degrees of freedom, and approaches $N(0, 1)$ as the degrees of freedom grow. The count is $n - 1$ because the deviations $x_i - \bar{x}$ always sum to zero, so the standard deviation rests on only $n - 1$ freely varying deviations.

Definition The p-values of the one-sample t test are exact when three conditions hold. The observations are independent, they come from a single normal population with unknown standard deviation $\sigma > 0$, and $\mu_0$ is a fixed number chosen without using the sample. Without normality the test is approximate. For independent observations from one population with finite positive variance, the distribution of the t statistic under $H_0$ approaches $N(0, 1)$ as $n$ grows, and so does $t_{n-1}$, so the p-values become accurate for large enough samples. How large depends on the population, and strong skewness or heavy tails call for larger samples, especially for one-sided tests. Independence, and the link between the sample and the population it is meant to describe, come from how the data were collected.

Definition For a one-sample comparison with benchmark $\mu_0$, the standardized mean difference is

the difference between the sample mean and the benchmark in units of the sample standard deviation of individual observations. It estimates the population quantity $(\mu - \mu_0)/\sigma$. It has no units, its sign gives the direction of the difference, and it is related to the t statistic by

Because $d$ estimates a fixed population quantity, more data make it more precise rather than larger, while $\lvert t \rvert$ tends to grow like $\sqrt{n}$ whenever $\mu \ne \mu_0$.

Definition For a level $\alpha$ with $0 < \alpha < 1$, the two-sided $100(1 - \alpha)$ percent t interval for $\mu$ is

where $t_{1-\alpha/2,\,n-1}$ is the $1 - \alpha/2$ quantile of the t distribution with $n - 1$ degrees of freedom. For every benchmark $\mu_0$, the two-sided one-sample t test of $H_0: \mu = \mu_0$ on the same data has $p > \alpha$ exactly when $\mu_0$ lies strictly inside this interval, $p = \alpha$ exactly when $\mu_0$ is an endpoint, and $p < \alpha$ exactly when $\mu_0$ lies outside. The interval is therefore the set of benchmarks that the test at level $\alpha$ does not reject, apart from the two endpoints, where the decision depends on the rule for $p = \alpha$. The match needs the same data, model, standard error, degrees of freedom, level and sidedness. A one-sided test against $\mu > \mu_0$ at level $\alpha$ matches the one-sided lower confidence bound $\bar{x} - t_{1-\alpha,\,n-1}\, s/\sqrt{n}$ instead.