Sign Tests and Wilcoxon Signed-Rank Tests, statistics lesson

Five people are measured before and after a change, and their differences, after minus before, are $-3$ , $-1$ , $2$ , $4$ and $5$ units. With five values there is little to check a normal model against, so it is natural to ask a question that needs less, using only the direction of each change or the direction together with the order of the sizes. The first idea gives the sign test and the second gives the Wilcoxon signed-rank test. This lesson builds both, shows that they test different null hypotheses under different conditions, and settles how zero differences and tied magnitudes are handled.

Key terms

Sign test

Definition Let $d_1, \dots, d_n$ be independent observations from one population of differences $D$ . In paired data each $d_i$ is a within-pair difference such as after minus before. For a one-sample question about a value $c_0$ , use $d_i = x_i - c_0$ . The sign test of

H_0 : P(D > 0) = P(D < 0)

drops the zero differences and counts the positive ones. If $m$ of the differences are nonzero and $K$ of them are positive, then under $H_0$ , conditional on $m$ ,

K \sim \operatorname{Binomial}\bigl(m, \tfrac12\bigr).

The p-value is computed from this binomial distribution, with the rule for what counts as extreme fixed in advance. For a two-sided test it is the probability of a count at least as far from $m/2$ as the observed one. When $D$ has a continuous distribution, $P(D = 0) = 0$ and $H_0$ says exactly that $P(D > 0) = \tfrac12$ , so $0$ is a median of $D$ . This is why the sign test is used as a test about a population median.

In plain words Each nonzero difference is like a coin toss between up and down. If ups and downs are equally likely in the population, the number of ups behaves like the number of heads in $m$ fair tosses. The test never looks at how large the differences are.

Example For the differences $-3$ , $-1$ , $2$ , $4$ and $5$ there are $m = 5$ nonzero differences and $K = 3$ positive ones. Under $H_0$ the expected count is $2.5$ . The observed count is $0.5$ away from it, and every possible count from $0$ to $5$ is at least that far away, so

\begin{aligned} p &= P\bigl(\lvert K - 2.5 \rvert \ge 0.5\bigr) \\[0.4em] &= 1. \end{aligned}

The count is as close to the center as a count out of five can be. The p-value of $1$ does not mean that $H_0$ has probability $1$ . The sample median is $2$ and the sample mean is $1.4$ , and neither size enters the test.

Signed rank

Definition Take the $m$ nonzero differences and rank their absolute values $\lvert d_i \rvert$ from $1$ for the smallest to $m$ for the largest. The signed rank of $d_i$ is that rank with the sign of $d_i$ attached. The positive rank sum $W_+$ adds the ranks of the positive differences, and $W_-$ adds the ranks of the negative ones. Every rank goes into exactly one of the two sums, and the ranks $1, 2, \dots, m$ add to $m(m + 1)/2$ , so

W_+ + W_- = \frac{m(m + 1)}{2}.

In plain words A rank keeps the order of the sizes and forgets the gaps between them. The largest difference gets rank $m$ whether it is slightly or enormously larger than the next one, so a single extreme value cannot dominate $W_+$ .

Example For $-3$ , $-1$ , $2$ , $4$ and $5$ the absolute values $3$ , $1$ , $2$ , $4$ and $5$ have ranks $3$ , $1$ , $2$ , $4$ and $5$ . Ranks and sizes agree here only because the sizes happen to be the whole numbers $1$ to $5$ . The signed ranks are $-3$ , $-1$ , $2$ , $4$ and $5$ , so

\begin{gathered} W_+ = 2 + 4 + 5 = 11, \\[0.4em] W_- = 3 + 1 = 4, \end{gathered}

and the two sums add to $15 = 5 \cdot 6 / 2$ . If the largest difference were $50$ instead of $5$ , the mean would jump from $1.4$ to $10.4$ , but every rank and the value $W_+ = 11$ would stay the same.

Wilcoxon signed-rank test

Definition Let $d_1, \dots, d_m$ be independent observations from a continuous distribution, so that zero differences and tied absolute values occur with probability $0$ . The Wilcoxon signed-rank test uses the statistic $W_+$ to test the null hypothesis $H_0$ that $D$ and $-D$ have the same distribution, which says that the distribution of $D$ is symmetric about $0$ . Under $H_0$ , given the absolute values, the signs are independent and each is positive or negative with probability $\tfrac12$ . So all $2^m$ ways of attaching signs to the ranks $1, \dots, m$ are equally likely, and computing $W_+$ for each of them gives its exact null distribution, with

\begin{aligned} E_0(W_+) &= \frac{m(m + 1)}{4}, \\[0.4em] \operatorname{Var}_0(W_+) &= \frac{m(m + 1)}{4} \\[0.4em] &\quad \times \frac{2m + 1}{6}. \end{aligned}

If the differences are assumed to be symmetric about some center $c$ , under the null and the alternative alike, the test is a test of $c = 0$ . The center $c$ is then a median of $D$ , and it is also the mean when the mean exists.

In plain words Symmetry about $0$ means that every interval of values is exactly as likely as its mirror image on the other side of $0$ . Then each rank is equally likely to carry either sign, and $W_+$ should land near half the total rank sum. Symmetry is a stronger condition than a median of $0$ . If the median is $0$ but the positive differences are typically larger in size than the negative ones, the positive differences collect the high ranks and $W_+$ tends to exceed its null center. So the usual signed-rank test is not a test of the median alone.

Example For $m = 5$ there are $2^5 = 32$ equally likely sign patterns, and the null center is $15/2 = 7.5$ . The observed $W_+ = 11$ lies $3.5$ above it, so the two-sided rule counts the patterns with $W_+ \ge 11$ or $W_+ \le 4$ . The low values arise as follows.

$W_+$	$0$	$1$	$2$	$3$	$4$
Sign patterns	$1$	$1$	$1$	$2$	$2$

That is $7$ patterns, and by symmetry another $7$ give $W_+ \ge 11$ , so

p = \frac{14}{32} = 0.4375.

Some software reports the smaller sum $W_- = 4$ instead, which gives the same p-value. The sign test gave $p = 1$ for the same data. The two tests use different information and rest on different conditions, so the test should be chosen for its question before either p-value is seen. With five nonzero differences the smallest possible two-sided p-value is $2/32 = 0.0625$ for both tests, so neither two-sided test can reject at the $5$ percent level, whatever the data.

Zero differences

Definition A zero difference has no sign, so each procedure needs a stated convention for it. The sign test and the usual Wilcoxon procedure drop the zeros and use the number $m$ of nonzero differences in place of $n$ . Pratt’s method instead ranks the zeros together with the nonzero absolute values, so that the zeros take the smallest ranks, and then leaves those ranks out of both $W_+$ and $W_-$ . The two conventions can give different statistics and different reference distributions. For the sign test with zeros dropped, the binomial reference rests on the null $P(D > 0) = P(D < 0)$ . When $D$ equals $0$ with positive probability, a median of $0$ does not imply this null.

In plain words Dropping a zero from the test does not erase it from the data. Report how many zeros there were and which convention was used. With discrete or rounded measurements zeros can be common, and the null of the sign test then has to be stated as a comparison of the chances of a positive and a negative difference, not as a median of $0$ .

Example For the differences $0$ , $-2$ and $3$ , dropping the zero leaves the ranks $1$ and $2$ for the sizes $2$ and $3$ , so $W_+ = 2$ and $W_- = 1$ . Pratt’s method ranks the sizes $0$ , $2$ and $3$ as $1$ , $2$ and $3$ and discards the rank of the zero, so $W_+ = 3$ and $W_- = 2$ . Now take a discrete difference with

\begin{gathered} P(D = -1) = 0.2, \\[0.4em] P(D = 0) = 0.4, \\[0.4em] P(D = 1) = 0.4. \end{gathered}

Its median is $0$ , because $P(D \le 0) = 0.6$ and $P(D \ge 0) = 0.8$ are both at least $\tfrac12$ . Yet a nonzero difference is positive with probability $0.4/0.6 = \tfrac23$ , so the sign test rejects with probability approaching $1$ as the sample grows. It correctly rejects $P(D > 0) = P(D < 0)$ , which is false here, but the rejection is not evidence against a median of $0$ .

Tied ranks

Definition When several nonzero absolute differences are equal, each of them gets the average of the rank positions the group occupies, called a midrank. Midranks keep the total rank sum at $m(m + 1)/2$ . If the signs are independent and each is positive with probability $\tfrac12$ given the absolute values, as under the signed-rank null, then with midranks $r_1, \dots, r_m$

\operatorname{Var}_0(W_+) = \frac{1}{4} \sum_{i=1}^{m} r_i^2 ,

which is smaller than the untied value whenever a tie occurs. Ties also change which values of $W_+$ are possible and how likely each one is. An exact reference therefore attaches all sign patterns to the actual midranks, and a normal approximation uses the variance above. A table built for the ranks $1, \dots, m$ no longer applies.

In plain words Tied sizes share their positions evenly. The sum of the ranks does not change, but the possible values of $W_+$ do, so the p-value has to come from a calculation that knows about the ties. A p-value that software labels exact is worth checking for how it treated ties and zeros.

Example For the differences $1$ , $-1$ , $2$ and $-3$ the absolute values are $1$ , $1$ , $2$ and $3$ . The two values $1$ occupy positions $1$ and $2$ , so each gets rank $1.5$ , and the others get ranks $3$ and $4$ . Then $W_+ = 1.5 + 3 = 4.5$ and $W_- = 1.5 + 4 = 5.5$ , which add to $10 = 4 \cdot 5 / 2$ . Attaching all $16$ sign patterns to the ranks $1.5$ , $1.5$ , $3$ and $4$ never produces $W_+ = 5$ , a value that occurs in $2$ of the $16$ patterns for the untied ranks $1$ , $2$ , $3$ and $4$ . The squared midranks $2.25$ , $2.25$ , $9$ and $16$ add to $29.5$ , so the null variance is

\begin{aligned} \operatorname{Var}_0(W_+) &= \frac{29.5}{4} \\[0.4em] &= 7.375, \end{aligned}

below the untied $4 \cdot 5 \cdot 9 / 24 = 7.5$ . Every attainable value is at least $0.5$ from the center $5$ , so the observed $4.5$ gives a two-sided p-value of $1$ .

Distribution-free method

Definition A test is distribution-free over a stated class of null distributions if its test statistic has the same null distribution for every distribution in that class. The reference distribution can then be computed without knowing the shape of the population. Given the number of nonzero differences, the sign count is distribution-free over all distributions with $P(D > 0) = P(D < 0)$ , for independent observations. The signed-rank statistic $W_+$ is distribution-free over all continuous distributions symmetric about $0$ , for independent observations. Methods of this kind are often called nonparametric. Distribution-free does not mean assumption-free, since independence, the class of distributions that defines the null, and a convention for zeros and ties remain conditions of the method.

In plain words The null calculation needs no normal curve, but that does not mean anything goes. If the observations are dependent, or if the differences are skewed and the signed-rank test is read as a test of a median of $0$ , the reference distribution need not match how the statistic behaves when the null of interest is true, even though no normal model was used.

Example For five untied differences, $W_+$ has the same $32$ -pattern null distribution whether the differences come from a normal, a uniform or any other continuous distribution symmetric about $0$ . The paired t test instead takes its reference, the t distribution with $4$ degrees of freedom, from a normal model for the differences. Both rank methods still need independence. If two of the five pairs were the same person measured twice, the five signs need not behave like independent coin tosses, and both the binomial reference and the $32$ equally likely patterns would lose their justification.

Check yourself

Twelve pairs give $9$ positive differences, $1$ negative difference and $2$ zero differences. Find the two-sided sign test p-value with zeros dropped.

Show the answer

Dropping the zeros leaves $m = 10$ nonzero differences with $K = 9$ positive. Under $H_0$ the count $K$ follows $\operatorname{Binomial}(10, \tfrac12)$ , so

\begin{aligned} P(K \ge 9) &= \frac{\binom{10}{9} + \binom{10}{10}}{2^{10}} \\[0.4em] &= \frac{11}{1024}. \end{aligned}

The counts at least as far from $5$ as the observed $9$ are $0$ , $1$ , $9$ and $10$ , and the distribution is symmetric, so the two-sided p-value is $22/1024 \approx 0.0215$ . The report should also say that $2$ of the $12$ differences were zero and were left out of the test.

Paired differences are $1.5$ , $-0.5$ , $3$ , $2$ and $4$ . Find $W_+$ and the exact two-sided signed-rank p-value, compare it with the two-sided sign test p-value, and name the condition the signed-rank analysis needs.

Show the answer

The absolute values $1.5$ , $0.5$ , $3$ , $2$ and $4$ have ranks $2$ , $1$ , $4$ , $3$ and $5$ . The only negative difference has rank $1$ , so $W_- = 1$ and $W_+ = 14$ . The null center is $7.5$ and the observed value is $6.5$ above it. Among the $32$ sign patterns, $W_+ \ge 14$ happens for $2$ of them (all positive, or all positive except rank $1$ ) and $W_+ \le 1$ for another $2$ , so

p = \frac{4}{32} = 0.125.

The sign test sees $K = 4$ positive signs out of $5$ . Counts at least $1.5$ from $2.5$ are $0$ , $1$ , $4$ and $5$ , with probability $(1 + 5 + 5 + 1)/32 = 0.375$ . The signed-rank p-value is smaller because it also uses the fact that the one negative difference is the smallest. Using that information is justified by the condition that, under $H_0$ , the differences are independent and symmetric about $0$ . Neither test rejects at the $5$ percent level, and the choice between them belongs before the data are seen.

Paired changes come from a skewed population. With probability $\tfrac12$ a change is uniform on $(-1, 0)$ , and with probability $\tfrac12$ it is uniform on $(0, 3)$ . Is $0$ the population median? Is the usual signed-rank test a valid test of a median of $0$ here, and which test fits that question better?

Show the answer

Yes. Exactly half of the probability lies on each side of $0$ , and the density is positive just below and just above $0$ , so $0$ is the unique median. The distribution is not symmetric, though. A positive change is uniform on $(0, 3)$ and the size of a negative change is uniform on $(0, 1)$ , so a positive change is larger in size than a negative one with probability $1 - \tfrac16 = \tfrac56$ . Positive changes therefore tend to take the high ranks, and $W_+$ is pushed above its null center although the median is $0$ .

The symmetry condition of the signed-rank test fails, so it is not a valid test of a median of $0$ here. In a simulation of $400{,}000$ samples of $20$ changes from this population, the exact two-sided signed-rank test at the $5$ percent level rejected in about $22$ percent of the samples. The sign test needs only $P(D > 0) = P(D < 0)$ , which holds, and it rejected in about $4$ percent of the samples, matching its exact size of about $0.041$ for $20$ differences. The sign test fits the median question, at the cost of ignoring the sizes of the changes.

Sign Tests and Wilcoxon Signed-Rank Tests

Key terms

Sign test

Signed rank

Wilcoxon signed-rank test

Zero differences

Tied ranks

Distribution-free method

Check yourself

How this lesson connects

Builds on

Used later in