Exponential smoothing: from simple smoothing to Holt-Winters and ETS

Before this lesson: lesson 01 (level, trend, seasonal period $m$, additive and multiplicative seasons) and lesson 03 (benchmarks, rolling origin, MASE). One-dimensional minimisation helps (optimization lesson 01). Time: about 75 minutes, plus the notebook.

---

Why this matters #

• It is used everywhere in operations. Inventory systems, demand planning and capacity planning have used exponential smoothing since the 1950s (Brown, Holt, Winters). Spreadsheet software has it built in: Excel's FORECAST.ETS uses an additive ETS model.
• It is a strong benchmark. In the M4 competition, the benchmark "Comb", the average of SES, Holt and damped trend forecasts, was more accurate than all six pure machine-learning methods (Makridakis et al. 2020). The Theta method, the winner of the M3 competition, is equivalent to SES with drift (Hyndman and Billah 2003).
• It is the simplest state-space model. Its update "state = prediction + gain × error" is the same structure as the Kalman filter (lesson 07), and SES is a special case of ARIMA (lesson 06). Recurrent neural networks (lesson 09) also carry a state forward and update it with each new value; the ES-RNN hybrid that won M4 combined Holt-Winters equations with an LSTM (Smyl 2020).
• Libraries: statsmodels (ExponentialSmoothing, ETSModel), statsforecast (AutoETS), R fable (ETS()) and forecast (ets()).

1. Predict first #

2. The picture #

An average with fading weights #

Take the last observed values $y_t, y_{t-1}, y_{t-2}, \dots$. The SES forecast of the next value is

with $0 < \alpha \le 1$. The weights $\alpha, \alpha(1-\alpha), \alpha(1-\alpha)^2, \dots$ shrink by the factor $1 - \alpha$ at each step back: they decay exponentially, hence the name. With $\alpha = 0.2$ the weights are $0.2, 0.16, 0.128, \dots$: many past values matter. With $\alpha = 0.8$ they are $0.8, 0.16, 0.032, \dots$: almost only the newest value matters.

Interactive figure: loading…

State plus gain times error #

The same forecast can be computed with one number in memory, the level $\ell_t$. When the new value $y_t$ arrives:

1. the forecast was $\hat y_{t\mid t-1} = \ell_{t-1}$;
2. the error is $e_t = y_t - \ell_{t-1}$;
3. move the level a fraction $\alpha$ of the way: $\ell_t = \ell_{t-1} + \alpha e_t$.

If the forecast was too low ($e_t > 0$), the level goes up; if it was right, nothing changes. Holt's and Winters' methods keep more states (a trend $b_t$ and seasonal states $s_t$) and update each of them with its own gain times the same error. Section 3.5 shows these updates step by step.

3. Definitions and main results #

3.1 Simple exponential smoothing: three equal forms #

Definition (SES). Choose $0 < \alpha \le 1$ and a starting level $\ell_0$. For $t = 1, 2, \dots, n$:

This is the component form. The forecast is flat: the same value $\ell_n$ for every horizon, because SES has no trend and no season.

Theorem (three equal forms). For every $t \ge 1$:

1. error-correction form: $\ell_t = \ell_{t-1} + \alpha e_t$ with $e_t = y_t - \ell_{t-1}$;
2. weighted-average form: $\ell_t = \sum_{j=0}^{t-1} \alpha(1-\alpha)^j y_{t-j} + (1-\alpha)^t \ell_0$, and the weights sum to 1.

Proof.

1. $\alpha y_t + (1-\alpha)\ell_{t-1} = \ell_{t-1} + \alpha(y_t - \ell_{t-1}) = \ell_{t-1} + \alpha e_t$.
2. By induction on $t$. For $t = 1$: $\ell_1 = \alpha y_1 + (1-\alpha)\ell_0$, which is the formula. If the formula holds for $t - 1$, then $\ell_t = \alpha y_t + (1-\alpha)\Big[\sum_{j=0}^{t-2}\alpha(1-\alpha)^j y_{t-1-j} + (1-\alpha)^{t-1}\ell_0\Big]$. Put $i = j + 1$ in the sum: it becomes $\sum_{i=1}^{t-1}\alpha(1-\alpha)^{i} y_{t-i}$. Adding the term $\alpha y_t$ (the case $i = 0$) gives the formula for $t$.
3. The weights sum to $\alpha\sum_{j=0}^{t-1}(1-\alpha)^j + (1-\alpha)^t = \alpha\,\frac{1 - (1-\alpha)^t}{1 - (1-\alpha)} + (1-\alpha)^t = 1$ (a geometric sum). $\blacksquare$

So SES is a weighted average of the data and $\ell_0$, and the weight of $\ell_0$ dies out like $(1-\alpha)^t$.

Memory. The weights $\alpha(1-\alpha)^j$ form a geometric distribution over the age $j$. Its mean is $\sum_j j\,\alpha(1-\alpha)^j = (1-\alpha)/\alpha$. So the average age of the data in the forecast is $(1-\alpha)/\alpha$ steps, and roughly the last $1/\alpha$ observations carry most of the weight. $\alpha = 0.1$: about 10 observations; $\alpha = 0.5$: about 2.

3.2 Choosing $\alpha$ and $\ell_0$ #

The parameters are chosen to make the one-step errors small on the training data:

There is no closed-form minimiser, because $\hat y_{t\mid t-1}$ depends on $\alpha$ through all earlier steps. But SSE is cheap to evaluate (one pass through the data), so a one-dimensional search works: a grid over $\alpha \in (0, 1]$, then a bounded scalar minimiser such as golden section or Brent's method (scipy.optimize.minimize_scalar). Libraries estimate $\ell_0$ together with $\alpha$, or set it to $y_1$ or to the mean of the first few values.

What the minimiser finds depends on the data:

• white noise around a constant mean: the best forecast is the overall mean, so $\hat\alpha$ is close to 0 (average everything);
• a random walk: the best forecast is the last value, so $\hat\alpha$ is close to 1;
• a local level (a slowly wandering mean plus noise): $\hat\alpha$ is in between. For the model $y_t = \mu_t + \varepsilon_t$, $\mu_t = \mu_{t-1} + \eta_t$ with the ratio $q = \operatorname{Var}(\eta_t)/\operatorname{Var}(\varepsilon_t)$, the best $\alpha$ is $(-q + \sqrt{q^2 + 4q})/2$; for $q = 0.1$ this is about 0.27. Lesson 07 derives this formula as the steady-state Kalman gain.

Interactive figure: loading…

3.3 Holt's linear trend and the damped trend #

Holt's linear trend method (Holt 1957) adds a trend state $b_t$, the current slope:

with $0 < \alpha \le 1$ and $0 \le \beta^* \le 1$. Read the level equation as "a weighted average of the new value and the previous level moved one step along the trend". The trend equation is "a weighted average of the newest change in level and the previous slope".

In error-correction form, with $e_t = y_t - (\ell_{t-1} + b_{t-1})$:

(For the trend: $\ell_t - \ell_{t-1} - b_{t-1} = \alpha e_t$, so $b_t = b_{t-1} + \beta^*(\ell_t - \ell_{t-1} - b_{t-1}) = b_{t-1} + \alpha\beta^* e_t$.)

The forecast is a straight line with slope $b_n$. A straight line goes on forever, so at long horizons Holt's forecasts are often too high (or too low). The damped trend method (Gardner and McKenzie 1985) bends the line towards a flat one:

with $0 < \phi < 1$. The sum is geometric: $\phi + \dots + \phi^h = \phi(1 - \phi^h)/(1 - \phi)$. As $h \to \infty$ it tends to $\phi/(1-\phi)$, so the forecast tends to the constant $\ell_t + \frac{\phi}{1-\phi}\,b_t$. With $\phi = 1$ the method is Holt's method; with $\phi = 0$ it is SES. In practice $\phi$ is restricted to $0.8 \le \phi \le 0.98$ (FPP3): below 0.8 the damping is very strong, and above 0.98 the method is almost Holt's. On many business series, damped trend is one of the most accurate single methods (Gardner 2006).

3.4 Holt-Winters: adding a season #

Winters (1960) added seasonal states $s_t$ with period $m$. In the additive method the season is added; it has a fixed size:

Each line has a plain meaning:

• level: a weighted average of the seasonally adjusted value $y_t - s_{t-m}$ and the previous level moved along the trend;
• trend: as in Holt's method;
• season: a weighted average of the newest seasonal deviation $y_t - \ell_{t-1} - b_{t-1}$ and the seasonal state of the same season one period earlier, $s_{t-m}$.

The seasonal index. There are $m$ seasonal states, one per season, and each one is updated once per period. The forecast for horizon $h$ uses the last updated state of the season of $t + h$, which is $s_{t+h-m(k+1)}$. For $h \le m$ this is $s_{t+h-m}$; for $m < h \le 2m$ it is $s_{t+h-2m}$, the same state again. The integer $k$ counts the full periods inside the horizon, exactly as for the seasonal naive forecast of lesson 03.

The usual ranges are $0 < \alpha < 1$, $0 < \beta^* < 1$ and $0 < \gamma < 1 - \alpha$ (FPP3, §8.6). With $\gamma = 0$, the seasonal states never change: the seasonal pattern is frozen at its start values.

The multiplicative method is for swings that grow with the level. The season multiplies the level, and the seasonal states are ratios around 1 instead of deviations around 0:

	additive	multiplicative
forecast	$\ell_t + h b_t + s_{t+h-m(k+1)}$	$(\ell_t + h b_t)\, s_{t+h-m(k+1)}$
level	$\alpha(y_t - s_{t-m}) + (1-\alpha)(\ell_{t-1} + b_{t-1})$	$\alpha\, y_t / s_{t-m} + (1-\alpha)(\ell_{t-1} + b_{t-1})$
trend	$\beta^*(\ell_t - \ell_{t-1}) + (1-\beta^*)\, b_{t-1}$	the same
season	$\gamma(y_t - \ell_{t-1} - b_{t-1}) + (1-\gamma)\, s_{t-m}$	$\gamma\, y_t / (\ell_{t-1} + b_{t-1}) + (1-\gamma)\, s_{t-m}$
seasonal states	sum to about 0 over one period	average about 1 over one period

The damped versions replace $b_{t-1}$ by $\phi b_{t-1}$ and $h b_t$ by $(\phi + \dots + \phi^h) b_t$. A third option for growing swings: take logarithms (lesson 01) and use the additive method on $\log y_t$.

Start values. The recursions need $\ell$, $b$ and $m$ seasonal states before the first update. A simple rule, used in the figures and the notebook of this lesson, uses the first two periods. Let $\bar y^{(1)}$ and $\bar y^{(2)}$ be the means of the first and the second period. Then $b = (\bar y^{(2)} - \bar y^{(1)})/m$; the level at the end of the first period is $\ell = \bar y^{(1)} + b\,(m-1)/2$; and the seasonal states are the first period's values minus the trend line, $s_i = y_i - (\bar y^{(1)} + b\,(i - (m+1)/2))$ for $i = 1, \dots, m$ (divide instead of subtract for the multiplicative method). The updates start at $t = m + 1$. Libraries usually estimate the start values together with the parameters.

Interactive figure: loading…

3.5 ETS: exponential smoothing as a state-space model #

So far exponential smoothing is a recipe: equations that produce point forecasts. To get a likelihood and prediction intervals we need a statistical model: a rule that says how the data are generated, including the randomness.

ETS(A,N,N): the model behind SES. Let $\varepsilon_t$ be independent $\mathcal N(0, \sigma^2)$ errors. The innovations state-space model is

The state $\ell_{t-1}$ is the one-step forecast, and $\varepsilon_t = y_t - \ell_{t-1}$ is the one-step error $e_t$ of SES. So the model generates exactly the SES recursions: the point forecasts of ETS(A,N,N) are the SES forecasts. The same error $\varepsilon_t$ drives both equations (one source of randomness); such errors are called innovations.

The name ETS. ETS stands for (Error, Trend, Seasonal), and also for "ExponenTial Smoothing". Each letter is a component:

• Error: A (additive) or M (multiplicative, $y_t = \hat y_{t\mid t-1}(1 + \varepsilon_t)$);
• Trend: N (none), A (additive) or $\mathrm{A_d}$ (additive damped);
• Seasonal: N, A or M.

So SES is ETS(A,N,N) or ETS(M,N,N); Holt is ETS(A,A,N); damped trend is ETS(A,$\mathrm{A_d}$,N); additive Holt-Winters is ETS(A,A,A); multiplicative Holt-Winters is ETS(M,A,M). There are $2 \times 3 \times 3 = 18$ combinations (FPP3, §8.5). Three of them, ETS(A,N,M), ETS(A,A,M) and ETS(A,$\mathrm{A_d}$,M), can be numerically unstable (they divide by states that can be close to 0), so automatic selection usually skips them (FPP3, §8.6). Additive and multiplicative errors give the same point forecasts but different intervals.

What the model adds.

1. A likelihood. For additive errors, $\log L = -\frac{n}{2}\log(2\pi\sigma^2) - \frac{1}{2\sigma^2}\sum_t \varepsilon_t^2$. Maximising over $\sigma^2$ gives $\hat\sigma^2 = \text{SSE}/n$, and then maximising the likelihood is the same as minimising the SSE. For multiplicative errors the likelihood has an extra term, so the two criteria differ.
2. Model selection. With a likelihood you can compare different models with the AIC, $-2\log L + 2k$ (or the corrected AICc), where $k$ counts the parameters, the start values and $\sigma^2$. Automatic ETS (AutoETS, ets()) fits all allowed models and keeps the one with the smallest AICc. The SSE cannot do this: a model with more states always has a smaller SSE.
3. Prediction intervals. The model says how far the future can move. This needs the distribution of the future errors, derived next.

Theorem (ETS(A,N,N) forecast variance). For ETS(A,N,N), the $h$-step forecast error has mean 0 and variance

Proof.

1. Apply the state equation $j$ times: $\ell_{n+j} = \ell_n + \alpha(\varepsilon_{n+1} + \dots + \varepsilon_{n+j})$.
2. The observation equation at time $n + h$ gives $y_{n+h} = \ell_{n+h-1} + \varepsilon_{n+h} = \ell_n + \alpha\sum_{i=1}^{h-1}\varepsilon_{n+i} + \varepsilon_{n+h}$.
3. The future errors are independent of the past and of each other, with mean 0. So the conditional mean is $\ell_n$ (the SES forecast), and the variance is $\alpha^2(h-1)\sigma^2 + \sigma^2$. $\blacksquare$

A 95 % prediction interval is $\ell_n \pm 1.96\,\hat\sigma\sqrt{1 + \alpha^2(h-1)}$. With $\alpha = 1$ this is the naive forecast's $\hat\sigma\sqrt h$ from lesson 03; with small $\alpha$ the interval grows slowly, because a single shock moves the level only a little.

The same argument works for every model with additive errors and no multiplicative part: the $h$-step error is $\varepsilon_{n+h} + \sum_{j=1}^{h-1} c_j\,\varepsilon_{n+h-j}$, where $c_j$ is the effect of one unit of error on the forecast $j$ steps later, so the variance is $\sigma^2\big(1 + \sum_{j=1}^{h-1} c_j^2\big)$. For Holt's method $c_j = \alpha(1 + \beta^* j)$; for additive Holt-Winters an extra $\gamma$ is added when $j$ is a multiple of $m$. FPP3 Table 8.7 lists the closed forms. For multiplicative models there is no simple formula; libraries simulate many future paths, as the figure above does for the multiplicative method.

Error correction, step by step. Additive Holt-Winters in error-correction form (ETS(A,A,A)): with $e_t = y_t - (\ell_{t-1} + b_{t-1} + s_{t-m})$,

Each state is its prediction plus a gain times the one-step error. The gains are $\alpha$, $\alpha\beta^*$ and $\gamma$.

Interactive figure: loading…

SES is ARIMA(0,1,1), and the Kalman filter of a local level model

From $y_t = \ell_{t-1} + e_t$ and $\ell_t = \ell_{t-1} + \alpha e_t$: $y_t - y_{t-1} = (\ell_{t-1} + e_t) - (\ell_{t-2} + e_{t-1}) = \alpha e_{t-1} + e_t - e_{t-1}$. So

$$y_t - y_{t-1} = e_t + (\alpha - 1)\,e_{t-1},$$

which is an ARIMA(0,1,1) model with MA coefficient $\theta = \alpha - 1$ (lesson 06, plus-sign convention). Every additive ETS model without a multiplicative part is a special case of an ARIMA model in this way.

The local level model $y_t = \mu_t + \varepsilon_t$, $\mu_t = \mu_{t-1} + \eta_t$ has two independent noise sources. Its Kalman filter (lesson 07) updates the estimate of $\mu_t$ as "prediction + gain × innovation", and the gain converges to a constant. In the steady state the filter is SES with $\alpha$ equal to that gain, $(-q + \sqrt{q^2 + 4q})/2$.

3.6 How well does it work? A backtest on CO₂ #

We fit the parameters of each method once on the first 120 months of the CO₂ data (1980–1989), by minimising the SSE. Then we run a rolling origin with an expanding window: at the origins 120, 132, …, 252 (12 folds, one per year), the states are updated with all data up to the origin and we forecast the next 12 months. The table gives the MASE, averaged over the 12 folds and the 12 horizons (lesson 03).

Method	ETS form	MASE
seasonal naive	—	1.00
SES	ETS(A,N,N)	1.41
Holt	ETS(A,A,N)	4.93
damped trend	ETS(A,$\mathrm{A_d}$,N)	1.71
Holt-Winters additive	ETS(A,A,A)	0.27
Holt-Winters multiplicative	ETS(M,A,M)	0.27

The non-seasonal methods lose to the seasonal naive benchmark, because CO₂ has a strong yearly cycle. Holt's method is the worst: fitted to seasonal data, its slope follows the last month-to-month swing (its fitted $\alpha$ and $\beta^*$ are near 1) and the straight line extrapolates it. Both Holt-Winters methods cut the error of the seasonal naive forecast by about three quarters. Additive and multiplicative are equally good here: the seasonal swings of CO₂ (about ±3 ppm on a level of 340–370 ppm) barely change in size. Damping, often a good default, does not help a series with a steady trend.

(Keeping the parameters fixed and only updating the states is a common, cheap backtest. Refitting the parameters at every origin is more expensive and closer to production; both are honest because the parameters never see data after the origin.)

Answers to section 1. (1) $0.5 \cdot 0.5^3 = 0.0625$ (the newest value has weight 0.5, then 0.25, 0.125, 0.0625). (2) A flat line at the last level. SES has no trend state; on trending data the level also lags behind the data, so the flat forecasts are too low (for an upward trend) and get worse with the horizon. Use Holt's or the damped trend method. (3) Multiplicative (or additive on the log scale).

4. Worked example #

(a) SES by hand. $\alpha = 0.5$, $\ell_0 = 10$, data $y_1, \dots, y_4 = 12, 9, 13, 11$.

$t$	$y_t$	forecast $\ell_{t-1}$	error $e_t$	$\ell_t = \ell_{t-1} + 0.5\,e_t$
1	12	10	2	11
2	9	11	−2	10
3	13	10	3	11.5
4	11	11.5	−0.5	11.25

The forecast for every horizon is $\ell_4 = 11.25$. Check with the weighted-average form:

The weights are $0.5, 0.25, 0.125, 0.0625$ for $y_4, y_3, y_2, y_1$, and $(1 - 0.5)^4 = 0.0625$ for $\ell_0$; they sum to 1.

Interval. SSE $= 4 + 4 + 9 + 0.25 = 17.25$, so $\hat\sigma^2 = 17.25/4 = 4.3125$ and $\hat\sigma \approx 2.077$. For $h = 1$: $11.25 \pm 1.96 \cdot 2.077 = 11.25 \pm 4.07$. For $h = 3$: $\sqrt{1 + 0.25 \cdot 2} = \sqrt{1.5} \approx 1.225$, so $11.25 \pm 4.99$. (With only four observations this is only an illustration: $\hat\sigma$ is very uncertain.)

(b) Holt by hand. $\alpha = 0.5$, $\beta^* = 0.4$ (so the trend gain is $\alpha\beta^* = 0.2$), $\ell_0 = 10$, $b_0 = 1$, data $11.5, 12, 14$.

$t$	$y_t$	forecast $\ell_{t-1} + b_{t-1}$	$e_t$	$\ell_t = \text{forecast} + 0.5\,e_t$	$b_t = b_{t-1} + 0.2\,e_t$
1	11.5	11	0.5	11.25	1.1
2	12	12.35	−0.35	12.175	1.03
3	14	13.205	0.795	13.6025	1.189

Check step 1 with the component form: $\ell_1 = 0.5 \cdot 11.5 + 0.5\,(10 + 1) = 11.25$ and $b_1 = 0.4\,(11.25 - 10) + 0.6 \cdot 1 = 0.5 + 0.6 = 1.1$. The forecasts are the line $13.6025 + 1.189\,h$: $14.79, 15.98, 17.17$ for $h = 1, 2, 3$.

(c) Damped forecasts. Suppose a damped trend fit ends with $\ell_n = 13.6$, $b_n = 1.2$ and $\phi = 0.9$.

$h$	$\phi + \dots + \phi^h$	damped forecast	Holt line $\ell_n + h b_n$
1	0.9	14.68	14.8
2	1.71	15.65	16.0
10	$9(1 - 0.9^{10}) \approx 5.86$	20.63	25.6
$\infty$	$0.9/0.1 = 9$	24.4	$\infty$

(d) Which seasonal state? Quarterly data, $m = 4$, origin $n$. The forecast for horizon $h$ uses $s_{n+h-4(k+1)}$ with $k = \lfloor (h-1)/4 \rfloor$:

$h$	1	2	3	4	5	6
$k$	0	0	0	0	1	1
state	$s_{n-3}$	$s_{n-2}$	$s_{n-1}$	$s_n$	$s_{n-3}$	$s_{n-2}$

Horizons $h$ and $h + 4$ use the same state: the last estimate of that quarter.

5. In code #

NumPy reproduces the worked examples and checks the ETS(A,N,N) interval by simulation. The notebook has these cells, a general ETS function and the exercises.

import numpy as np

def ses(y, alpha, l0):
    l, levels = l0, []
    for v in y:
        l = l + alpha * (v - l)          # error-correction form
        levels.append(l)
    return np.array(levels)

y = np.array([12, 9, 13, 11.])
print(ses(y, 0.5, 10.0))                                   # [11.   10.   11.5  11.25]
w = 0.5 * 0.5 ** np.arange(4)                              # weights of y4, y3, y2, y1
print(w @ y[::-1] + 0.5**4 * 10.0)                         # 11.25

# Holt, error-correction form
l, b, alpha, beta_star = 10.0, 1.0, 0.5, 0.4
for v in [11.5, 12, 14]:
    e = v - (l + b)
    l, b = l + b + alpha * e, b + alpha * beta_star * e
print(round(l, 4), round(b, 4), np.round(l + b * np.arange(1, 4), 4))
# 13.6025 1.189 [14.7915 15.9805 17.1695]

# ETS(A,N,N): simulate 200000 futures and compare the h-step variance with 1 + alpha^2 (h - 1)
rng = np.random.default_rng(0)
alpha, H, N = 0.3, 6, 200_000
eps = rng.normal(size=(N, H))
level = np.zeros(N)
ys = np.empty((N, H))
for k in range(H):
    ys[:, k] = level + eps[:, k]          # y = previous level + error
    level = level + alpha * eps[:, k]     # level update
print(np.round(ys.var(axis=0), 3))                        # [1.001 1.088 1.183 1.268 1.357 1.447]
print(1 + alpha**2 * np.arange(H))                        # [1.   1.09 1.18 1.27 1.36 1.45]

With statsmodels installed, ETSModel(y, error="add", trend="add", seasonal="add", seasonal_periods=12) fits ETS(A,A,A) by maximum likelihood; with the same parameters and start values it gives the same fitted values and forecasts as the recursions above (the notebook checks this).

6. Where it shows up #

• Demand planning and inventory. Retail and supply-chain systems forecast thousands of items with ETS models chosen automatically by AICc; the prediction intervals set safety stocks.
• Benchmarks for ML. The M4 "Comb" benchmark (SES, Holt and damped trend, averaged) and automatic ETS are the standard statistical baselines in papers on machine learning for forecasting (lessons 08 and 09).
• Hybrid models. ES-RNN, the winner of M4, used Holt-Winters-style level and seasonal states to normalise each series and an LSTM to forecast the normalised values (Smyl 2020).
• Smoothing in machine learning. The exponential moving average is SES under another name: the running statistics of batch normalisation, the moment estimates of Adam (optimization lesson 06), and EMA weights of trained models all use $\ell_t = \beta\,\ell_{t-1} + (1 - \beta)\,x_t$, that is SES with $\alpha = 1 - \beta$.
• Monitoring and science. EWMA control charts in quality control and anomaly detection (lesson 11) smooth a signal with SES; sensor and environmental data are often gap-filled and smoothed with Holt-Winters before analysis.

7. Common mistakes #

• Using SES on trending or seasonal data. Its forecast is flat and lags behind a trend. Use Holt, damped trend or Holt-Winters, and compare with the seasonal naive forecast.
• Forgetting that Holt's line never bends. Long-horizon forecasts of Holt's method often overshoot; try the damped trend.
• Confusing $\beta^*$ and $\beta$. In component form the trend parameter is $\beta^*$; in ETS state-space form the trend gain is $\beta = \alpha\beta^*$. Check which one a library reports (ETSModel reports $\beta$).
• Choosing additive seasonality for growing swings. Look at the plot (lesson 01): if the swings grow with the level, use the multiplicative method or a log transform.
• Picking the model with the smallest SSE. More states always fit better in-sample. Compare models with AICc or, better, with a rolling-origin backtest.
• Reading the seasonal index wrongly. The forecast at horizon $h$ uses $s_{n+h-m(k+1)}$, the last estimate of that season, not $s_{n+h-m}$ for $h > m$ (which lies in the future).
• Treating the recursions as the model. Without the ETS error model there is no likelihood and no valid interval. A "±1.96 σ" band with constant width ignores that the error grows with $h$.

8. Check yourself #

1. Show that $\alpha = 1$ turns SES into the naive method, and say what $\alpha \to 0$ does.
   Answer

   With $\alpha = 1$, $\ell_t = y_t$, so the forecast is the last value. As $\alpha \to 0$ the level barely moves: the forecast stays near $\ell_0$. If $\ell_0$ is the mean of the data, SES with very small $\alpha$ behaves like the mean method.
2. A series has the SSE-optimal $\alpha = 0.95$. What does this tell you about the series, and which benchmark will SES resemble?
   Answer

   The best forecast is almost the last value: the series behaves like a random walk (or has a trend or season that SES cannot follow, so it chases the last value). SES then almost equals the naive forecast. If there is a trend or season, try Holt or Holt-Winters.
3. Damped trend with $\ell_n = 50$, $b_n = 2$, $\phi = 0.8$. Give the forecasts for $h = 1$ and $h = 2$, and the limit as $h \to \infty$.
   Answer

   $h = 1$: $50 + 0.8 \cdot 2 = 51.6$. $h = 2$: $50 + (0.8 + 0.64) \cdot 2 = 52.88$. Limit: $50 + \frac{0.8}{0.2} \cdot 2 = 58$.
4. Monthly data, $m = 12$, origin $n$. Which seasonal state does additive Holt-Winters use for $h = 13$ and for $h = 24$?
   Answer

   $h = 13$: $k = \lfloor 12/12 \rfloor = 1$, state $s_{n+13-24} = s_{n-11}$, the same as for $h = 1$. $h = 24$: $k = \lfloor 23/12 \rfloor = 1$, state $s_{n+24-24} = s_n$, the same as for $h = 12$.
5. Why does ETS give prediction intervals while the Holt-Winters recursions alone do not?
   Answer

   The recursions only produce point forecasts. ETS adds a probability model for the errors ($\varepsilon_t$ independent $\mathcal N(0, \sigma^2)$, entering the observation and the state equations). From this model the distribution of $y_{n+h}$ follows: for ETS(A,N,N) its variance is $\sigma^2[1 + \alpha^2(h-1)]$.
6. For ETS(A,N,N) with $\alpha = 0.2$ and $\hat\sigma = 3$, how wide is the 95 % interval at $h = 1$ and at $h = 26$?
   Answer

   $h = 1$: $\pm 1.96 \cdot 3 = \pm 5.88$. $h = 26$: $1 + 0.04 \cdot 25 = 2$, so $\pm 1.96 \cdot 3\sqrt 2 \approx \pm 8.32$.

Recall cardsAnswer in your head first, then reveal. Add the cards to your review deck to see them again just before you would forget.

1. Write SES in component form and in error-correction form.

   Show answer

2. What weight does SES give to $y_{t-j}$?

   Show answer

3. About how many recent observations carry most of the weight in SES?

   Show answer

4. How are the SES parameters usually chosen?

   Show answer

5. Write Holt's forecast and the damped trend forecast.

   Show answer

6. Which seasonal state does Holt-Winters use at horizon $h$?

   Show answer

7. What does ETS stand for, and what is ETS(A,N,N)?

   Show answer

8. Why does ETS give prediction intervals and the Holt-Winters recursions alone do not?

   Show answer

9. What is the $h$-step forecast variance of ETS(A,N,N)?

   Show answer

10. Additive or multiplicative seasonality: how do you choose?

    Show answer

Add these cards to my reviewOpen the review →

9. Interview angle #

• "Explain exponential smoothing in one minute." — A weighted average of the past with weights $\alpha(1-\alpha)^j$; equivalently a level updated by $\alpha$ times the forecast error. Holt adds a trend state, Winters a seasonal state; ETS adds an error model, which gives a likelihood and intervals.
• "How would you choose $\alpha$?" — Minimise the one-step SSE (maximum likelihood for additive errors) on the training data, then check with a rolling-origin backtest.
• "SES vs ARIMA?" — SES is ARIMA(0,1,1) with $\theta = \alpha - 1$; additive ETS models are special ARIMA models, while multiplicative ETS models have no ARIMA equivalent.
• "What is an exponential moving average in deep learning?" — SES: Adam's moment estimates, batch-norm running statistics and EMA of weights all use $\ell_t = \beta\ell_{t-1} + (1-\beta)x_t$.
• "Your Holt forecast explodes after a few months. What do you do?" — Use a damped trend ($\phi \approx 0.9$), check whether the series is seasonal (Holt chases seasonal swings), and compare with seasonal naive in a backtest.

10. Practice #

The notebook for this lesson runs in Colab. It has five exercises, each with a check cell and a folded solution:

1. Write ses(y, alpha, l0) that returns the levels and the one-step forecasts, and check it against the weighted-average formula.
2. Write fit_ses(y): a grid over $\alpha$ followed by scipy.optimize.minimize_scalar on the SSE. Recover $\alpha \approx 0.27$ on a simulated local-level series.
3. Write holt_winters_additive(y, m, alpha, beta, gamma, H) and check that it reproduces a series built exactly from level, trend and season.
4. Write the damped trend forecast and show that it converges to $\ell_t + \frac{\phi}{1-\phi} b_t$.
5. Backtest SES, Holt, damped trend and Holt-Winters against seasonal naive on the CO₂ data with the rolling origin of lesson 03 (and, if statsmodels is installed, compare with ETSModel).

11. Watch and read #

Forecasting Principles & Practice: 8.1 Simple exponential smoothingOTexts · Forecasting: Principles and Practice · 13 min

The weighted average, the component form and how alpha and the start level are estimated.

Forecasting Principles & Practice: 8.2 Methods with trendOTexts · Forecasting: Principles and Practice · 14 min

Holt's linear trend and the damped trend on real data.

Forecasting Principles & Practice: 8.3 Methods with seasonalityOTexts · Forecasting: Principles and Practice · 14 min

Additive and multiplicative Holt-Winters, and how to choose between them.

Forecasting Principles & Practice: 8.5 Innovations state space modelsOTexts · Forecasting: Principles and Practice · 15 min

How an error model turns the recursions into ETS, with a likelihood.

What are Exponential Smoothing ModelsAric LaBarr · 5 min

A short overview of the whole family, from SES to Holt-Winters.

• Hyndman and Athanasopoulos, Forecasting: Principles and Practice, 3rd ed., chapter 8, free online: §8.1 SES, §8.2 trend, §8.3 seasonality, §8.4 taxonomy, §8.5 ETS models, §8.7 forecasting with ETS.
• Hyndman, Koehler, Ord and Snyder (2008), Forecasting with Exponential Smoothing: The State Space Approach, Springer: the reference for ETS (not free).
• Gardner (2006), Exponential smoothing: The state of the art — Part II, International Journal of Forecasting 22(4).
• Makridakis, Spiliotis and Assimakopoulos (2020), The M4 Competition: 100,000 time series and 61 forecasting methods, International Journal of Forecasting 36(1).
• statsmodels, ETSModel.