Standard Deviation and Standard Error, statistics lesson

Individual measurements from a constructed process vary with a standard deviation of 12 units. The mean of 100 independent measurements from the same process varies from sample to sample with a standard deviation of only 1.2 units. Both numbers describe spread, but the spread of different things, and reporting one in place of the other makes an estimate look far more or far less certain than it is. This lesson defines the standard error, shows where the square root of the sample size comes from, and explains how estimating $\sigma$ and how dependence between observations change the calculation.

Key terms

Standard error

Definition The standard error of a statistic is the standard deviation of its sampling distribution, for a stated population, sample size and sampling design. It is written $\operatorname{SE}$ and has the same units as the statistic. A standard deviation without further qualification describes a distribution of individual values, such as the population distribution or the values in one data set.

In plain words The standard deviation measures how widely individual observations spread around their mean. The standard error measures how much a summary would change if the whole study were repeated in the same way. Both are standard deviations, but of different distributions.

Example Take a population of three cards labeled $0$ , $2$ and $4$ . One randomly drawn card has standard deviation $\sigma = \sqrt{8/3} \approx 1.63$ . The mean of two cards drawn at random with replacement has a sampling distribution with variance $4/3$ , so the standard error of that mean is $\sqrt{4/3} \approx 1.15$ . The proportion of cards labeled $4$ in the same samples has a sampling distribution with variance $1/9$ , so its standard error is $1/3$ . Each statistic has its own standard error.

Standard error of the mean

Definition Let $X_1, \dots, X_n$ be independent and identically distributed with mean $\mu$ and finite standard deviation $\sigma$ . Independence makes the variances add, so the sum has variance $n\sigma^2$ , and dividing the sum by $n$ divides its variance by $n^2$ . Hence

\begin{gathered} \operatorname{Var}(\bar{X}) = \frac{n\sigma^2}{n^2} = \frac{\sigma^2}{n}, \\[0.4em] \operatorname{SE}(\bar{X}) = \frac{\sigma}{\sqrt{n}} . \end{gathered}

For a simple random sample of $n$ units drawn without replacement from a population of $N \ge 2$ units, the draws are in general not independent. With the population variance $\sigma^2$ defined with denominator $N$ , the standard error is then

\operatorname{SE}(\bar{X}) = \frac{\sigma}{\sqrt{n}}\sqrt{\frac{N - n}{N - 1}} ,

and with $S^2$ defined with denominator $N - 1$ the same value is $\sqrt{(S^2/n)(1 - n/N)}$ .

In plain words For independent observations with a common standard deviation, the standard error of the mean shrinks like one over the square root of the sample size. Quadrupling $n$ halves it, and doubling $n$ divides it only by $\sqrt{2} \approx 1.41$ . The standard deviation $\sigma$ of individual observations does not depend on $n$ at all. A larger sample makes the average more precise, not the process more consistent, and the formula assumes that the data are collected in the same way at every sample size.

Example A constructed process has $\sigma = 12$ units, and its measurements are independent. With $n = 25$ the standard error of the mean is $12/\sqrt{25} = 2.4$ units, and with $n = 100$ it is $12/10 = 1.2$ units, while single measurements have standard deviation $12$ units in both cases. To make the standard error at most $1$ unit, solve $12/\sqrt{n} \le 1$ , which gives $n \ge 144$ . If instead the $100$ units are drawn without replacement from a finite population of $N = 200$ units with $\sigma = 12$ , computed with denominator $N$ , the standard error in units is

\frac{12}{\sqrt{100}}\sqrt{\frac{100}{199}} \approx 0.851 .

It is smaller than $1.2$ because half of the population has been measured.

Estimated standard error

Definition When $\sigma$ is unknown, it is replaced by the sample standard deviation

s = \sqrt{\frac{1}{n - 1}\sum_{i=1}^{n}(x_i - \bar{x})^2} ,

which needs $n \ge 2$ . For independent observations, the estimated standard error of the mean is $\widehat{\operatorname{SE}}(\bar{X}) = s/\sqrt{n}$ . It is a statistic computed from the same sample, so it has its own sampling distribution and in general changes from sample to sample. It estimates $\operatorname{SE}(\bar{X})$ and is generally not equal to it. For a sample proportion the usual estimate is $\sqrt{\hat{p}(1 - \hat{p})/n}$ . Under simple random sampling without replacement, where $s^2$ estimates $S^2$ , the estimate for the mean becomes $\sqrt{(s^2/n)(1 - n/N)}$ .

In plain words The estimated standard error is a sample-based estimate of how much the mean moves between samples. A sample that happens to show little variation gives a small estimate even when the population is more variable. Later procedures, such as the t interval, account for this extra uncertainty under stated conditions.

Example Twenty-five constructed measurements have sample standard deviation $s = 10$ units. Their estimated standard error of the mean is $10/\sqrt{25} = 2$ units. That number estimates how much the sample mean would vary over repeated samples. It does not say that the true standard error is exactly $2$ , and it does not say that most single measurements lie within $2$ units of the mean, since their sample standard deviation is $10$ . An error bar on a graph should state whether it shows a standard deviation, a standard error or an interval.

Cluster dependence

Definition Data have cluster dependence when the observations come in groups, such as several readings from one device, several pupils in one class or repeated measurements of one person, and observations in the same group are correlated while different groups are independent. The variance of a sum then includes covariance terms,

\begin{aligned} &\operatorname{Var}\Bigl(\sum_{i} X_i\Bigr) = \sum_{i} \operatorname{Var}(X_i) \\[0.4em] &\quad + \sum_{i \ne j} \operatorname{Cov}(X_i, X_j) . \end{aligned}

A simple model has $k$ clusters of $m$ observations each, so $n = km$ , with every observation having variance $\sigma^2$ and every two observations in the same cluster having correlation $\rho$ . Then

\begin{gathered} \operatorname{Var}(\bar{X}) = D\,\frac{\sigma^2}{n}, \\[0.4em] D = 1 + (m - 1)\rho . \end{gathered}

The factor $D$ is the design effect of this model. When $m \ge 2$ and $\rho > 0$ it exceeds $1$ , and the standard error of the mean is larger than $\sigma/\sqrt{n}$ .

In plain words Ten readings from one device partly repeat the same information, because they share whatever is particular to that device. When readings within a cluster are positively correlated, a formula that treats every row as independent understates the uncertainty of the mean.

Example One hundred readings come from ten devices, ten readings each, with $\sigma = 12$ units and within-device correlation $\rho = 0.2$ in the model above. The design effect is $D = 1 + 9 \cdot 0.2 = 2.8$ , so

\operatorname{SE}(\bar{X}) = 12\sqrt{\frac{2.8}{100}} \approx 2.008

units, compared with $1.2$ units if all $100$ readings were independent.

Effective sample size

Definition For a mean of $n$ dependent observations, each with variance $\sigma^2$ , the effective sample size is the number of independent observations with the same variance that would give the mean the same variance,

n_{\text{eff}} = \frac{\sigma^2}{\operatorname{Var}(\bar{X})} .

In the equal-cluster model it equals $n/D = n/\bigl(1 + (m - 1)\rho\bigr)$ . It describes precision under a specific model. It is not a count of actual units, and it need not be a whole number.

In plain words Effective sample size converts dependent data into the number of independent observations they are worth for estimating the mean. In the equal-cluster model with a correlation between $0$ and $1$ , it lies between the number of clusters and the number of rows.

Example For the hundred readings from ten devices, $n_{\text{eff}} = 100/2.8 \approx 35.7$ , and $12\big/\sqrt{100/2.8} \approx 2.008$ units reproduces the standard error above. If the readings from each device were identical, so that $\rho = 1$ , then $D = 10$ and $n_{\text{eff}} = 10$ , the number of devices. If they were uncorrelated, $n_{\text{eff}}$ would be $100$ . Other dependence structures, such as serial correlation in a time series, need their own models.

Check yourself

A process has standard deviation $\sigma = 18$ units, and its measurements are independent. Find the standard error of the mean for $n = 9$ and for $n = 36$ , and the smallest sample size that makes it at most $2$ units.

Show the answer

The standard errors are $18/\sqrt{9} = 6$ units and $18/\sqrt{36} = 3$ units, so quadrupling the sample size halves the standard error. The condition $18/\sqrt{n} \le 2$ means $\sqrt{n} \ge 9$ , so $n \ge 81$ . Individual measurements keep standard deviation $18$ units at every sample size.

A report on $16$ independent measurements gives a mean of $50$ units and a sample standard deviation of $s = 8$ units, and then says that a typical measurement lies within about $2$ units of the mean. Where does the $2$ come from, and what is wrong with the sentence?

Show the answer

The number $2$ is the estimated standard error of the mean, $8/\sqrt{16} = 2$ units. It estimates how much the sample mean would vary over repeated samples of $16$ measurements.

The spread of individual measurements is described by the sample standard deviation, $8$ units, four times as large. A correct report says that individual measurements have a sample standard deviation of $8$ units and that the mean of $50$ units has an estimated standard error of $2$ units.

A researcher measures the same five people ten times each and treats the $50$ readings as $50$ independent people. In a model where every reading has variance $\sigma^2$ , two readings of the same person have correlation $0.5$ , and different people are independent, compare the standard error of the mean with that from $50$ independent people measured once each. What happens if each of the five people is measured $100$ times instead?

Show the answer

For five people with ten readings each, $m = 10$ and $\rho = 0.5$ , so $D = 1 + 9 \cdot 0.5 = 5.5$ . The standard error is $\sigma\sqrt{5.5/50} \approx 0.332\,\sigma$ , and the effective sample size is $50/5.5 \approx 9.09$ . Fifty independent people give $\sigma/\sqrt{50} \approx 0.141\,\sigma$ , smaller by a factor of $\sqrt{5.5} \approx 2.35$ . Treating the $50$ rows as $50$ people would report $0.141\,\sigma$ and understate the uncertainty.

With $m$ readings per person the effective sample size is

\frac{5m}{1 + 0.5(m - 1)} = \frac{10m}{m + 1},

which is about $9.90$ at $m = 100$ and stays below $10$ for every $m$ . Repeated readings can reduce measurement noise for each person, but in this model they never make five people worth more than ten independent people.

Standard Deviation and Standard Error

Key terms

Standard error

Standard error of the mean

Estimated standard error

Cluster dependence

Effective sample size

Check yourself

How this lesson connects

Builds on

Used later in