Unit 11, lesson 76

Smoothing, Forecasting and Time-Aware Validation

How naive, seasonal naive, moving average and exponential smoothing forecasts work, how rolling-origin evaluation at a stated forecast horizon avoids temporal leakage, and what a forecast interval needs.

  • Video 76
  • Time Series and Forecasting

A shop wants to predict tomorrow’s demand for bread, and a smooth curve drawn through last month’s sales follows the data closely. That closeness says little about forecasting, because the curve at each day may have been built from sales recorded after that day. Every forecasting claim depends on what was known when each prediction would have been made. This lesson compares simple forecasting baselines and smoothers on a short constructed series, evaluates them in time order, and shows what a forecast interval rests on.

Key terms

Forecast horizon

Definition The forecast origin tt is the last time whose observation is available when a forecast is made, and the forecast horizon h≥1h \ge 1 is the number of steps ahead being predicted. The forecast of yt+hy_{t+h} made at origin tt is written y^t+h∣t\hat{y}_{t+h \mid t}, and it may use only information available at time tt. On this page the forecast error is the actual value minus the forecast,

et+h∣t=yt+h−y^t+h∣t,e_{t+h \mid t} = y_{t+h} - \hat{y}_{t+h \mid t},

so a negative error means that the forecast was too high.

In plain words The origin fixes what the forecaster knows, and the horizon fixes how far ahead the forecast reaches. A method can do well one step ahead and badly ten steps ahead, so an accuracy figure should name its horizon, and methods should be compared on the same origins and horizons. A step means whatever spacing the series has, such as an hour, a day or a month.

Example Eight constructed, equally spaced temperature readings, in degrees, are listed below. They are invented for transparent arithmetic and come from no real study.

TimeReadingTimeReading
110512
212614
311713
413815

From origin 44, the forecast of y5y_5 has horizon 11 and the forecast of y7y_7 has horizon 33. Both may use y1,…,y4y_1, \dots, y_4 and nothing later.

Naive and seasonal naive forecasts

Definition The naive forecast repeats the last observation, y^t+h∣t=yt\hat{y}_{t+h \mid t} = y_t for every horizon hh. For a series with a seasonal period of dd steps fixed in advance, the seasonal naive forecast at an origin t≥dt \ge d repeats the last observed value from the same seasonal position,

y^t+h∣t=yt+h−d⌈h/d⌉,\hat{y}_{t+h \mid t} = y_{t + h - d\lceil h/d \rceil},

where ⌈h/d⌉\lceil h/d \rceil is the smallest integer that is at least h/dh/d. The index always falls within the last dd observations, so forecasts more than dd steps ahead repeat the last observed cycle again.

In plain words These baselines have no fitted parameters and can be computed as soon as enough history exists, so a more elaborate method should beat them on the same evaluation before its extra complexity is worth having. The naive forecast suits a level that wanders unpredictably, and the seasonal naive forecast suits a strong repeating pattern. Neither continues a trend, so both tend to fall behind a series that keeps rising.

Example At origin 44 the naive forecast for time 55 is y4=13y_4 = 13, and the observed y5=12y_5 = 12 gives an error of −1-1 degree. Suppose a period of d=2d = 2 was specified in advance because operating conditions alternate. The seasonal naive forecast for time 55 is then y3=11y_3 = 11, with error +1+1 degree. At origin 88 the seasonal naive forecasts for times 99, 1010 and 1111 are 1313, 1515 and 1313, copied from times 77, 88 and 77. The period is an assumption here, since eight invented readings cannot establish a seasonal mechanism.

Moving average

Definition A trailing moving average with window ww estimates the level at time tt by the mean of the current observation and the w−1w - 1 before it,

ℓt=1w∑j=0w−1yt−j.\ell_t = \frac{1}{w}\sum_{j=0}^{w-1} y_{t-j} .

It uses only data up to time tt, so it is a one-sided smoother, and as a forecast it gives y^t+h∣t=ℓt\hat{y}_{t+h \mid t} = \ell_t for every horizon. A centered moving average with odd window w=2q+1w = 2q + 1 averages yt−q,…,yt+qy_{t-q}, \dots, y_{t+q}, so it needs qq observations after time tt.

In plain words A trailing average is known at time tt. A centered average can describe the past well once later data have arrived, but it cannot be computed at time tt, so presenting it as a forecast hides a look into the future. A longer window damps isolated fluctuations more but reacts more slowly to a change in level, and in a series that rises at every step, a trailing average with w≥2w \ge 2 stays below the current value. The moving-average part of an ARIMA model is a different idea, a weighted sum of current and past random shocks, and ARIMA models form a larger family that this lesson only names.

Example With w=3w = 3 the trailing average at time 44 is (12+11+13)/3=12(12 + 11 + 13)/3 = 12, and it becomes the forecast for time 55, where y5=12y_5 = 12. At time 33 the trailing average is (10+12+11)/3=11(10 + 12 + 11)/3 = 11, while the centered average is (12+11+13)/3=12(12 + 11 + 13)/3 = 12. The two numbers differ because the centered one uses y4y_4, which was not available at time 33.

Exponential smoothing

Definition Simple exponential smoothing with smoothing parameter 0≤α≤10 \le \alpha \le 1 updates a level by

ℓt=αyt+(1−α)ℓt−1,\ell_t = \alpha y_t + (1 - \alpha)\ell_{t-1},

starting from a stated initial level such as ℓ1=y1\ell_1 = y_1, and forecasts every horizon by the current level, y^t+h∣t=ℓt\hat{y}_{t+h \mid t} = \ell_t. Unrolling the recursion gives

ℓt=∑j=0t−2α(1−α)jyt−j+(1−α)t−1ℓ1,\begin{aligned} \ell_t &= \sum_{j=0}^{t-2} \alpha(1 - \alpha)^j y_{t-j} \\[0.4em] &\quad + (1 - \alpha)^{t-1} \ell_1 , \end{aligned}

so for 0<α<10 < \alpha < 1 the weights on past observations decline geometrically.

In plain words Each new level moves part of the way from the old level toward the newest observation. A larger α\alpha follows recent readings more closely, and a smaller one keeps more of the past. For 0<α<10 < \alpha < 1, unlike in a moving average, no observation ever drops out completely, and its weight only keeps shrinking. With α=1\alpha = 1 the method is the naive forecast, and with α=0\alpha = 0 the level never leaves its starting value. Simple exponential smoothing has no trend or seasonal component, so its forecasts are flat. The starting value matters in short series, and choosing α\alpha by looking at the whole series before evaluating earlier forecasts leaks later information into the evaluation.

Example With α=0.5\alpha = 0.5 and ℓ1=10\ell_1 = 10, the first update is ℓ2=0.5⋅12+0.5⋅10=11\ell_2 = 0.5 \cdot 12 + 0.5 \cdot 10 = 11. Continuing through the series gives the levels 11,12,12,13,13,1411, 12, 12, 13, 13, 14 at times 33 to 88. At time 88 the weights on y8,y7,…,y2y_8, y_7, \dots, y_2 are 12,14,…,1128\tfrac12, \tfrac14, \dots, \tfrac{1}{128}, and the starting level gets the remaining 1128\tfrac{1}{128}. Every forecast from origin 88 is 1414 degrees, whatever the horizon.

Rolling-origin evaluation and temporal leakage

Definition Rolling-origin evaluation measures forecast accuracy by moving the forecast origin forward through time. At each origin tt the whole procedure, including any preprocessing and choice of settings, is run using only y1,…,yty_1, \dots, y_t, a forecast is made for a declared horizon hh, and it is compared with yt+hy_{t+h}. The errors e1,…,eme_1, \dots, e_m collected over the origins are summarized, for example, by

MAE⁡=1m∑i=1m∣ei∣,RMSE⁡=1m∑i=1mei2.\begin{gathered} \operatorname{MAE} = \frac{1}{m}\sum_{i=1}^{m} \lvert e_i \rvert, \\[0.4em] \operatorname{RMSE} = \sqrt{\frac{1}{m}\sum_{i=1}^{m} e_i^2} . \end{gathered}

Temporal leakage is any use, in producing a forecast or in choosing how it is produced, of information that would not have been available at its origin. The later value yt+hy_{t+h} may enter only the comparison that scores the forecast.

In plain words Each forecast is judged as if it had really been issued at its origin. Leakage hides in many places, such as a centered smoother, a mean or scale computed from the whole series, a missing value filled in from later data, a smoothing parameter tuned on the evaluation period, or a data value revised after the forecast date. Mean absolute error counts every miss in proportion to its size, while root mean squared error counts large misses more heavily, so the measure should match which mistakes are costly. If many settings are tried on one evaluation period, that period has become part of the selection, and an honest estimate for the chosen workflow needs a later period that played no part in it.

Example Forecast one step ahead from origins 44, 55, 66 and 77, so the actual values are 1212, 1414, 1313 and 1515 degrees.

MethodForecastsErrorsMAERMSE
Naive13, 12, 14, 13−1,2,−1,2-1, 2, -1, 21.51.581
Seasonal naive11, 13, 12, 141, 1, 1, 111
Trailing average12, 12, 13, 130, 2, 0, 211.414
Exponential smoothing12, 12, 13, 130, 2, 0, 211.414

The seasonal naive rule uses d=2d = 2, the trailing average uses w=3w = 3 and exponential smoothing uses α=0.5\alpha = 0.5, all fixed in advance. The root mean squared errors are 2.5\sqrt{2.5}, 11, 2\sqrt{2} and 2\sqrt{2}. Three methods tie on mean absolute error, and the seasonal naive rule has the smallest root mean squared error, although its errors are all +1+1, so it underpredicts every time while the level rises. The trailing average and exponential smoothing give identical forecasts only by coincidence of these data, this α\alpha and this starting value. Four errors are a tiny demonstration and say little about which method is better in the long run, and forecast errors in a time series can be correlated, so their uncertainty needs more than independent-error formulas.

Forecast interval

Definition A forecast interval, also called a prediction interval, for Yt+hY_{t+h} is an interval, computed from the information at origin tt, that has a stated probability of containing the future value under a stated model. In the Gaussian random walk without drift, Ys=Ys−1+εsY_s = Y_{s-1} + \varepsilon_s with independent increments εs∼N(0,σ2)\varepsilon_s \sim N(0, \sigma^2) and σ\sigma known, the value hh steps ahead is the current value plus hh independent increments. Given Yt=ytY_t = y_t, the future value Yt+hY_{t+h} is therefore normal with mean yty_t and variance hσ2h\sigma^2, and

yt±1.96 σhy_t \pm 1.96\,\sigma\sqrt{h}

is a central 95 percent forecast interval, where 1.961.96 rounds the standard normal 0.9750.975 quantile 1.959961.95996. This random walk is the ARIMA⁡(0,1,0)\operatorname{ARIMA}(0, 1, 0) model, since one difference leaves independent noise.

In plain words A point forecast says nothing about how far off it may be, and an interval is only as trustworthy as the model behind it. In the random walk the margin grows like h\sqrt{h}, because the variances of independent increments add, and other models give other patterns. A forecast interval is a probability statement about a future observation, not a confidence interval for a fixed mean. When σ\sigma or other parameters are estimated, their uncertainty and the fit of the model also affect whether the stated probability is achieved.

Example Take the declared model with current value 1515 degrees and known increment standard deviation 22 degrees, not estimated from the eight readings. One step ahead the interval is 15±1.96⋅215 \pm 1.96 \cdot 2, about [11.08,18.92][11.08, 18.92] degrees. Four steps ahead the margin is 1.96⋅2⋅4≈7.841.96 \cdot 2 \cdot \sqrt{4} \approx 7.84, which gives about [7.16,22.84][7.16, 22.84] degrees, twice as wide as one step ahead and not four times. Forecasts from the seasonal naive rule or from exponential smoothing need an error model of their own before an interval like this can be attached to them.

Check yourself

  1. Using the eight readings, give the naive, seasonal naive with d=2d = 2 and trailing three-point average forecasts made at origin 66 for times 77 and 88, and their errors.
Show the answer

At origin 66 only y1,…,y6y_1, \dots, y_6 are available. The naive forecast is y6=14y_6 = 14 for both times, with errors 13−14=−113 - 14 = -1 and 15−14=115 - 14 = 1. The seasonal naive forecast for time 77 copies time 55, giving 1212, and for time 88 copies time 66, giving 1414, with errors 11 and 11. The trailing average at time 66 is (13+12+14)/3=13(13 + 12 + 14)/3 = 13, used for both horizons, with errors 00 and 22. The forecast for time 88 has horizon 22 and still uses nothing after time 66.

  1. An analyst computes a centered three-point average of the eight readings, chooses the window and the smoothing settings by trying several on all eight readings, and reports the centered values at times 2 to 7 as one-step forecasts. Identify the information problems and describe a repaired evaluation. Which error measure fits better if large misses are especially costly?
Show the answer

The centered value at time tt averages yt−1y_{t-1}, yty_t and yt+1y_{t+1}, so it uses the very value it claims to predict and the one after it. A one-step forecast of yty_t from origin t−1t - 1 may use only y1,…,yt−1y_1, \dots, y_{t-1}. At time 33, for example, the centered value 1212 uses y3=11y_3 = 11 and y4=13y_4 = 13. Choosing the settings on all eight readings adds a second leak, because later readings influence every earlier evaluation.

A repaired evaluation fixes the horizon, moves the origin forward, and at each origin computes a trailing smoother and chooses any settings using only the data up to that origin. Each forecast is then compared with the value that arrives afterward. To judge the whole workflow, including the choice of settings, a final later stretch of the series is kept aside and used once. If large misses are especially costly, root mean squared error matches that concern better than mean absolute error, because squaring weights large errors more heavily.

  1. In a Gaussian random walk without drift, the current value is 2020 and the increments have known standard deviation 33. Find the central 95 percent forecast intervals for h=1h = 1 and h=9h = 9, and compare their widths. Is either one a confidence interval for a mean?
Show the answer

For h=1h = 1 the margin is 1.96⋅3=5.881.96 \cdot 3 = 5.88, so the interval is about [14.12,25.88][14.12, 25.88]. For h=9h = 9 the margin is 1.96⋅3⋅9=17.641.96 \cdot 3 \cdot \sqrt{9} = 17.64, so the interval is about [2.36,37.64][2.36, 37.64]. The second is 9=3\sqrt{9} = 3 times as wide as the first, not 99 times, because the variance grows in proportion to hh and the standard deviation in proportion to h\sqrt{h}.

Neither is a confidence interval for a mean. Each is a probability statement about the single future value Yt+hY_{t+h}, and under this model with known σ\sigma, given the current value 2020, the future value falls inside with probability 0.950.95, up to the rounding of 1.961.96. That probability comes from the model, so it is only as reliable as the random walk assumption.

How this lesson connects