Time Series, Trend, Seasonality and Autocorrelation, statistics lesson

A shop records its sales every day for a year and ends up with 365 numbers. Busy days may follow busy days, weekends may differ from weekdays, and the level may drift upward over the months. Shuffling the 365 numbers would keep their mean and standard deviation and destroy all of that structure, so treating them as an independent sample of 365 days can be badly misleading. This lesson names the structure in ordered data, trend, seasonality and autocorrelation, defines stationarity, and shows why a standard error that treats ordered observations as independent can be far too small.

Key terms

Time series

Definition A time series is a set of observations of one variable recorded at successive times, written $y_1, y_2, \dots, y_n$ in time order. The times, the spacing between them and the order are part of the data. A series is equally spaced when consecutive observations are a fixed interval apart, such as one hour or one day. Statistically, the series is treated as one realization of a stochastic process $Y_1, Y_2, \dots$ , a sequence of random variables, and the process has serial dependence when observations at different times are not independent.

In plain words A time series is one long record of a single process, not a pile of separate draws. Summaries that ignore order, such as a histogram, the mean and the standard deviation, cannot see a pattern over time. A long series is therefore not automatically a large independent sample, and a reading every minute need not bring a new independent piece of information every minute.

Example The sequences $(1, 2, 3, 4, 5)$ and $(1, 5, 2, 4, 3)$ contain the same five values. Both have mean $3$ and sample standard deviation $\sqrt{2.5} \approx 1.581$ , and their histograms are identical. As time series they differ, since the first rises steadily and the second zigzags around $3$ .

Trend

Definition A trend is a sustained change in the level of a series over time, the slowly varying part of its mean. It may be roughly linear over an interval, curved, or broken by a structural change such as a new instrument. In the additive decomposition

Y_t = m_t + s_t + \varepsilon_t ,

the trend component is $m_t$ , the seasonal component is $s_t$ and $\varepsilon_t$ is the irregular remainder.

In plain words The trend says where the level is heading over many periods. Individual steps can go against it, because seasonal effects and noise are added on top. A trend describes the data without explaining them, and a change in level that begins at some date is not by itself evidence that an event on that date caused it.

Example Twelve constructed readings were built as $Y_t = 20 + t + s_t$ for $t = 1, \dots, 12$ , with trend $m_t = 20 + t$ and seasonal offsets that repeat every four steps. Written one cycle per row, they are

Cycle	Position 1	Position 2	Position 3	Position 4
Steps 1 to 4	19	23	22	26
Steps 5 to 8	23	27	26	30
Steps 9 to 12	27	31	30	34

The cycle averages are $22.5$ , $26.5$ and $30.5$ , rising by $4$ per cycle, one unit per step. Yet the reading falls from $23$ to $22$ between steps 2 and 3, and from $26$ to $23$ between steps 4 and 5. Here the trend is known because the data were built from it. With real data the trend must be estimated under a model, and its separation from the other components can be uncertain.

Seasonality

Definition Seasonality is a pattern that repeats with a fixed, known period of $d$ observation steps. In an additive model the seasonal component satisfies $s_{t+d} = s_t$ for every $t$ , usually with $s_1 + \dots + s_d = 0$ so that the seasonal effects do not shift the level. The period comes from the measurement frequency and the process, for example $d = 24$ for a pattern within the day in hourly data, $d = 7$ for a weekly pattern in daily data and $d = 12$ for a yearly pattern in monthly data. Rises and falls without a fixed period, such as business cycles, are not seasonality in this sense.

In plain words Seasonality calls for comparing like with like. The same position in different cycles, the same weekday or the same month, is the fair comparison, and the seasonal difference $Y_t - Y_{t-d}$ removes a fixed additive seasonal pattern. When the seasonal swings grow with the level, a multiplicative model, or an additive model for the logarithm of a positive series, may describe them better.

Example In the constructed series the offsets are $(s_1, s_2, s_3, s_4) = (-2, 1, -1, 2)$ , which sum to $0$ , and the period is $d = 4$ steps. Each column of the table above holds one seasonal position, and down each column the values rise by exactly $4$ . Since $s_t = s_{t-4}$ , for $t = 5, \dots, 12$

\begin{aligned} Y_t - Y_{t-4} &= 4 + s_t - s_{t-4} \\[0.4em] &= 4 . \end{aligned}

The comparison of neighboring readings mixes the trend with the change in seasonal position, while the comparison four steps apart isolates the trend.

Lag

Definition In an equally spaced series, lag $k$ pairs each observation $y_t$ with the observation $y_{t-k}$ taken $k$ steps earlier, so the time separation is $k$ times the spacing. A lag- $k$ plot shows the $n - k$ points $(y_t, y_{t+k})$ for $t = 1, \dots, n - k$ . For irregularly spaced observations a fixed number of rows need not correspond to a fixed time separation.

In plain words A lag counts observation steps. With hourly data lag $4$ means four hours, and with daily data it means four days. The pairs in a lag plot overlap, because an observation can appear in two pairs, once as the earlier value and once as the later one. The points of a lag plot are therefore not independently sampled pairs.

Example The twelve constructed readings give $11$ pairs at lag $1$ and $8$ pairs at lag $4$ . The lag-4 pairs are $(19, 23)$ , $(23, 27)$ , $(22, 26)$ , $(26, 30)$ , $(23, 27)$ , $(27, 31)$ , $(26, 30)$ and $(30, 34)$ , and in each of them the later value is the earlier value plus $4$ . The reading $23$ at step 5 appears twice, as the later value of the first pair and the earlier value of the fifth.

Autocorrelation

Definition For a weakly stationary process, as defined under stationarity below, with autocovariance $\gamma_k = \operatorname{Cov}(Y_t, Y_{t+k})$ and variance $\gamma_0 > 0$ , the lag- $k$ autocorrelation is

\rho_k = \frac{\gamma_k}{\gamma_0} = \frac{\operatorname{Cov}(Y_t, Y_{t+k})}{\operatorname{Var}(Y_t)} ,

which does not depend on $t$ . The autocorrelation function, or ACF, is the sequence $\rho_1, \rho_2, \dots$ . A common sample version, used on this page, centers at the overall mean and divides by the total sum of squares,

\begin{gathered} d_t = y_t - \bar{y}, \\[0.4em] r_k = \frac{\sum_{t=1}^{n-k} d_t\, d_{t+k}}{\sum_{t=1}^{n} d_t^2} . \end{gathered}

Software packages use several conventions, so the one used should be stated. Autocorrelation measures linear association only, so autocorrelations of $0$ at every lag do not in general imply independence.

In plain words A positive lag-1 autocorrelation means that neighbors tend to lie on the same side of the mean, and a negative one means that the series tends to alternate. A seasonal pattern tends to produce peaks in the sample ACF at multiples of the period, and a trend tends to make sample autocorrelations positive and slow to die out. The sample ACF has sampling noise, especially in short series, and its shape is a clue for building a model, not a test that singles out one correct model.

Example For $(1, 2, 3, 4, 5)$ the mean is $3$ and the deviations are $-2, -1, 0, 1, 2$ , whose squares add to $4 + 1 + 0 + 1 + 4 = 10$ . The products of neighbors are $2, 0, 0, 2$ , which add to $4$ , so

r_1 = \frac{4}{10} = 0.4 .

For $(1, 5, 2, 4, 3)$ the deviations are $-2, 2, -1, 1, 0$ , the products of neighbors are $-4, -2, -1, 0$ and $r_1 = -7/10 = -0.7$ . The values and the denominator are the same, and only the order changed. Five points are far too few to estimate a process autocorrelation, and a steadily rising sequence gives no reason to assume the stationarity on which $\rho_1$ is defined.

Stationarity

Definition A stochastic process $Y_1, Y_2, \dots$ with finite variances is weakly stationary if for all times $t$ and lags $k$

\begin{gathered} E(Y_t) = \mu, \\[0.4em] \operatorname{Var}(Y_t) = \sigma^2, \\[0.4em] \operatorname{Cov}(Y_t, Y_{t+k}) = \gamma_k , \end{gathered}

with $\mu$ , $\sigma^2$ and $\gamma_k$ not depending on $t$ . Strict stationarity asks that the joint distribution of any stretch of the process is unchanged by a shift in time, which implies weak stationarity when the variances are finite. The difference $\Delta Y_t = Y_t - Y_{t-1}$ and the seasonal difference $Y_t - Y_{t-d}$ are common ways to build a series that is closer to stationary, and they change the variable under study from levels to changes. For a weakly stationary process with $\sigma^2 > 0$ and autocorrelations $\rho_k$ , the mean $\bar{Y}$ of $n$ consecutive observations has

\begin{aligned} \operatorname{Var}(\bar{Y}) &= \frac{\sigma^2}{n}\, M_n, \\[0.4em] M_n &= 1 + 2\sum_{k=1}^{n-1} w_k\, \rho_k, \\[0.4em] w_k &= 1 - \frac{k}{n} , \end{aligned}

where the weight $w_k$ reflects that only $n - k$ pairs of the observations are $k$ steps apart.

In plain words Stationarity means that the probability rules of the process stay the same over time, and in the weak sense only its mean, variance and autocovariances have to. It does not mean that the observed values are constant, and it does not mean that they are independent. A stationary process can fluctuate widely and be strongly correlated from one step to the next. A trend in the mean, a seasonal mean or a variance that changes over time each rule it out. Stationarity is an assumption about the process, so a plot or a test can make it more or less plausible but cannot prove it, and subtracting a fitted line can leave seasonality, changing variance or breaks behind. When $n \ge 2$ and every autocorrelation is positive, the factor $M_n$ exceeds $1$ and the independent formula $\sigma^2/n$ understates the variance of the mean.

Example Take a declared stationary model with $\sigma^2 = 4$ and $\rho_k = 0.5^k$ , not fitted to any data. For $n = 10$ consecutive observations the factor is $M_{10} = 6657/2560 \approx 2.600$ , so

\begin{aligned} \operatorname{SE}(\bar{Y}) &= \sqrt{\frac{4}{10} \cdot 2.600} \\[0.4em] &\approx 1.020 , \end{aligned}

while the independent formula gives $\sqrt{4/10} \approx 0.632$ . Dividing $10$ by the factor gives an effective sample size of about $3.85$ , a variance comparison under this model rather than a count of observations. For $\rho_k = \phi^k$ with $-1 < \phi < 1$ the factor approaches $(1 + \phi)/(1 - \phi)$ as $n$ grows, which is $3$ here. The constructed seasonal series above is not stationary, since its mean $20 + t + s_t$ changes with $t$ .

Check yourself

For the series $(4, 2, 4, 2, 3)$ , compute $r_1$ and $r_2$ with the convention of this page.

Show the answer

The mean is $3$ , the deviations are $1, -1, 1, -1, 0$ and the denominator is $1 + 1 + 1 + 1 + 0 = 4$ . The lag-1 products are $-1, -1, -1, 0$ , which add to $-3$ , so $r_1 = -3/4 = -0.75$ . The lag-2 products are $1 \cdot 1$ , $(-1)(-1)$ and $1 \cdot 0$ , which add to $2$ , so $r_2 = 2/4 = 0.5$ . This series alternates around its mean, which gives a negative lag-1 and a positive lag-2 autocorrelation. With five values these are descriptive numbers, not estimates of a process autocorrelation.

Suppose the twelve constructed readings come from a process, and a new operating instruction is introduced right after step 8. The mean of steps 9 to 12 is $30.5$ and the mean of steps 5 to 8 is $26.5$ . Does the increase of $4$ show that the instruction raised the readings? What does the fall from $30$ at step 8 to $27$ at step 9 show?

Show the answer

The increase does not show an effect of the instruction. The series was built with a trend of one unit per step, so every block of four steps averages $4$ more than the block before it, as the steps 1 to 4 and 5 to 8 already show without any instruction. The fall from $30$ to $27$ is not evidence of a harmful effect either. It is the move from seasonal position 4, with offset $2$ , to position 1, with offset $-2$ , partly made up by one step of trend.

A before-after comparison in one series cannot separate an intervention from an existing trend, the seasonal timing, other changes at the same date and random variation. Separating them needs more, for example a model of the trend and seasonality checked on the data before the change, a comparison series that did not receive the instruction, or a randomized design.

A sensor is read once a minute for two hours. The readings drift upward, neighboring readings are very similar, and an analyst reports a narrow confidence interval for the mean using the standard error $s/\sqrt{120}$ . Give two reasons to doubt the interval. What does lag 5 mean here, and would subtracting a fitted line prove that what remains is stationary? Finally, suppose instead that the readings had no drift and followed a stationary model with $\rho_k = 0.8^k$ . Roughly how many independent readings are the 120 worth for estimating the mean?

Show the answer

First, the upward drift means the mean changes over the two hours, so it is unclear which mean the interval is supposed to estimate. Second, very similar neighbors suggest positive autocorrelation, and then $s/\sqrt{120}$ tends to understate the standard error of the average because it ignores the covariance terms.

The readings are equally spaced one minute apart, so lag 5 pairs readings taken five minutes apart. Subtracting a fitted line does not prove stationarity. It can leave autocorrelation, changing variance, a recurring pattern or a break, so the remainder still has to be examined and stationarity remains an assumption.

With $\phi = 0.8$ the factor approaches $(1 + 0.8)/(1 - 0.8) = 9$ , so the effective sample size is roughly $120/9 \approx 13.3$ . The exact factor for $n = 120$ is about $8.67$ , which gives about $13.8$ . The standard error of the mean is about $\sqrt{8.67} \approx 2.94$ times the value $\sigma/\sqrt{120}$ for independent readings.

Time Series, Trend, Seasonality and Autocorrelation

Key terms

Time series

Trend

Seasonality

Lag

Autocorrelation

Stationarity

Check yourself

How this lesson connects

Builds on

Used later in