Chi-Square Goodness-of-Fit Tests

Definition A multinomial model describes $n$ observations, each falling into exactly one of $k \ge 2$ categories. The observations are independent, and each one falls into category $i$ with the same probability $p_i$, where every $p_i \ge 0$ and $p_1 + \dots + p_k = 1$. The category counts $O_1, \dots, O_k$ then add to $n$ and have the multinomial distribution

for whole numbers $o_i \ge 0$ with $o_1 + \dots + o_k = n$. Each count on its own is binomial with $n$ trials and success probability $p_i$, so $E(O_i) = np_i$.

Definition For counts from a multinomial model with $k$ categories, a goodness-of-fit test of a specified model tests

against the alternative $H_1$ that $p_i \ne p_{i0}$ for at least one $i$. The specified probabilities $p_{10}, \dots, p_{k0}$ are positive, add to 1 and are fixed before the data are seen. The test measures how far the observed counts are from the counts the model predicts. A small p-value is evidence against the specified model. A large p-value means the data are compatible with the model, not that the model is true.

Definition Under a specified model the expected frequency of category $i$ is

the mean of the count $O_i$ when $H_0$ holds. The expected frequencies add to $n$, and they need not be whole numbers.

Definition Pearson’s chi-square statistic for observed counts $O_i$ and expected frequencies $E_i$ is

It equals $0$ exactly when every count equals its expected frequency, and larger values mean a larger discrepancy. If $H_0$ holds, the specified probabilities are all positive and the expected frequencies are large, then $X^2$ has approximately the chi-square distribution with $k - 1$ degrees of freedom. The p-value is $P(\chi^2_{k-1} \ge x)$ at the observed value $x$, the approximate probability, computed assuming $H_0$, of a statistic at least as large as the one observed. The number $k - 1$ reflects the constraint that the counts add to $n$, so once $k - 1$ of them are known the last one is determined. The approximation can be poor when expected frequencies are small. A common guideline asks for all of them to be at least 5, but that is a rule of thumb, not a theorem. For sparse counts the exact multinomial distribution of $X^2$ can be used instead, or a simulation that draws many samples from the specified model and recomputes the same statistic. Merging categories changes the hypothesis being tested, so a merge should follow from the question, be decided before looking at the counts and be reported.

Definition Sometimes $H_0$ says only that the probabilities belong to a family $p_i(\theta)$ with $r$ unknown parameters, such as a binomial or Poisson family. The parameters are then estimated from the same data, and the expected frequencies become $E_i = n\, p_i(\hat{\theta})$. If $\hat{\theta}$ is the maximum likelihood estimate computed from the category counts and regularity conditions hold, then under $H_0$ the statistic $X^2$ has approximately the chi-square distribution with

degrees of freedom, which must be at least 1. The conditions include identifiable parameters, meaning that different parameter values give different category probabilities, a true parameter value inside the parameter range rather than on its boundary, positive category probabilities, and categories fixed before the data are seen. If the parameters are estimated from the raw, ungrouped observations instead, the large-sample distribution of $X^2$ lies between the chi-square distributions with $k - 1 - r$ and $k - 1$ degrees of freedom, and in general neither one is exact.