Time series data: trend, seasonality, noise and decomposition
Before this lesson: Python and NumPy arrays (foundations track); mean and variance (probability-and-statistics track). Nothing else. Time: about 60 minutes, plus the notebook.
Why this matters #
Forecasting is everywhere: product demand, electricity load, weather, traffic, server metrics, hospital admissions. Each of these is a time series, and the first step is always to look at the data and find its trend and its seasons.
Ordinary machine learning assumes that examples are i.i.d. (independent and identically distributed). Time series break both parts. Neighbouring values are dependent, and the distribution can change over time (a trend, a new regime). So a random train/test split puts future values into the training set. Lesson 03 shows how to evaluate honestly with a split in time order.
Decomposition is used directly in practice and in models:
- Official statistics publish seasonally adjusted unemployment and retail figures, made with X-13ARIMA-SEATS or similar decomposition methods.
- Forecasting models are built from the same parts. Prophet (Taylor and Letham, 2018) is an additive model of trend, seasonality and holiday effects. The deep models Autoformer (Wu et al., 2021) and DLinear (Zeng et al., 2023) split their input into a moving-average trend and a remainder before the main layers (lesson 09).
- Features and monitoring. Trend and seasonal strength are features for ML models (lesson 08), and large remainders flag anomalies (lesson 11).
1. Predict first #
2. The picture #
What a time series is #
A time series is a sequence of values observed in time order. The index is time and is the length. Most series in this track are regularly sampled: one value every hour, day, week, month or quarter. This spacing is the frequency of the series.
Three plots to make first #
The time plot shows against . Look for a long-term rise or fall (trend), a pattern that repeats (seasonality), sudden jumps, and single unusual values.
The seasonal plot draws one line per year (or per week) against the position inside the period: month 1 to 12, or Monday to Sunday. When the lines have the same shape, the series has a stable seasonal pattern. When the lines are stacked upward, there is also a trend.
The lag plot is a scatter plot of against , the value steps earlier. The number is the lag. If the points lie close to a rising line, values steps apart are similar. A seasonal series with period 12 lines up well at , because January is compared with January.
Move the lag and watch the right panel. Each blue dot is one pair ; the yellow pair is marked in both panels. For "season + noise" the dots line up at (and ), and they form a falling cloud at , because a seasonal high is followed by a low half a year later. Switch to "trend + season": the trend makes every lag look strongly correlated. The readout is the correlation of the pairs; lesson 02 turns it into the autocorrelation function.
Trend, season and remainder #
Most series are a combination of three components:
- the trend-cycle : the slow, long-term movement of the level (it includes slow cycles, see below);
- the seasonal component : a pattern that repeats with a fixed, known period , for example for monthly data with a yearly pattern, for daily data with a weekly pattern, for quarterly data;
- the remainder : everything else, that is noise and unusual events.
The simplest combination is a sum: . The figure builds a series from these three parts. Then it takes the series apart again with the method of section 3 and draws the recovered parts as dashed lines.
Build a series from its parts, then take it apart. The top panel is the observed series . Below are the true trend, seasonal and remainder parts (solid) and the parts recovered by classical decomposition (dashed). Raise the noise or shorten the series to : the recovered seasonal pattern becomes wrong, because each seasonal index is then an average of only one or two noisy values. Note that the recovered trend has no values at both ends. Switch to "multiplicative": the seasonal swing grows with the level.
Monthly ice-cream sales (no trend) are high every summer and low every winter. In the lag plot of against , where do the points lie?
Six months after a summer month comes a winter month. So high values of go with low values of : the correlation is strongly negative. The points lie close to the line at the full period , not at .
3. Definitions and main results #
3.1 Time series data in practice #
- A series is univariate when one quantity is measured, and multivariate when several quantities are measured at the same times (temperature, pressure and wind at one station).
- Often there are many related series, for example the sales of 10 000 products. This is called panel data. One model can be trained on all of them together: a global model (lesson 08).
- Exogenous variables (also called covariates) are other inputs that help to forecast . Some are known in advance: the calendar, public holidays, planned promotions. Others are only observed: the weather. To use an observed-only variable for forecasting, you need a forecast of it too.
In pandas, a time series is a Series with a DatetimeIndex. Four tools cover most data cleaning:
pd.date_range("2024-01-01", periods=36, freq="MS")makes a regular index (here: month starts).s.resample("MS").mean()changes the frequency, for example from weekly to monthly means. Note thatmean()skips missing values: a month with one valid week still gets a value.s.asfreq("D")puts the series on a regular daily grid. A missing time stamp (a row that does not exist) becomes a visible missing value (NaN). These are different problems: many methods silently assume that consecutive rows are one step apart.s.tz_localize("Europe/Zurich").tz_convert("UTC")handles time zones. With daylight saving time, the day when the clocks change has 23 hours (spring) or 25 hours (autumn). Store time stamps in UTC and convert only for display.
3.2 Components and the two decomposition models #
Seasonal or cyclic? A seasonal pattern has a fixed, known period that comes from the calendar or the clock (year, week, day). A cycle is a rise and fall without a fixed period, for example business cycles of a few years. The sunspot cycle is about 11 years long, but its length varies (roughly 9 to 14 years), so it is cyclic, not seasonal. Cycles cannot be separated reliably from the trend over a few years of data, so decomposition puts them into the trend-cycle .
Definition (decomposition models).
In the additive model the seasonal swing has the same size at every level. In the multiplicative model and are factors near 1: a seasonal factor of means "20 % above the trend", so the swing in units grows with the level.
The log transform turns multiplicative into additive. Take logs of both sides:
This is an additive model for . So the standard recipe is: if the swing grows with the level, take logs, decompose additively, and transform back with .
The Box–Cox transform is a family that contains the log as a special case. For and a parameter ,
only shifts the series by ; gives , a square root up to a shift and a scale. The case is the limit of the first line as , because for small . A value between 0 and 1 helps when the swing grows with the level, but more slowly than in proportion.
Move from 1 to 0. The top panel is the transformed series (blue). The bars show the seasonal swing (maximum minus minimum) inside each year (violet). On the original scale () the last swing is about 9 times the first one. At (log) the bars are almost equal: the multiplicative pattern has become additive.
A shop's monthly sales swing by about 20 % above and below the trend, at every level. Which decomposition should you use?
A constant percentage swing means the swing in units grows with the level, which is the multiplicative model . The log turns it into , an additive model. The additive model on raw data assumes a constant swing in units.
3.3 Moving averages #
A moving average replaces each value by the mean of the values around it. Noise averages out, and a seasonal pattern averages out when the window covers exactly one period.
Definition. For odd , the centred moving average of order (-MA) is
The trailing moving average of order uses only the past: .
Result 1 (a trailing MA lags behind a trend). For a straight line the trailing MA of order equals the line at time . So it is late by steps.
Proof.
- .
- , so the result is .
A centred MA has symmetric weights that sum to 1. By the same computation with from to , the sum is 0, so a centred MA returns a straight line unchanged: no lag.
Result 2 (one full period removes the seasonal pattern). Let repeat with period () and sum to 0 over one period. Then the average of any consecutive values of is 0.
Proof. Any consecutive values contain each of the positions of the period exactly once, so their sum is the sum over one period, which is 0.
The problem with even periods. For monthly data, is even. A window of 12 values, say months , has its centre at , between two months. The fix is to average two neighbouring 12-MAs: one centred at and one at . The result is centred at . It is called the MA.
Result 3 (weights of the MA). For even ,
For : the weights are on 13 values.
Proof.
- The first 12-MA covers , the second covers . Each has weight on each of its values.
- Their average gives weight to and to (each is in one window only), and to the 11 values in between (they are in both windows). The same argument works for any even .
Months and are the same calendar month, so each calendar month gets total weight . Therefore the MA also removes a period-12 pattern exactly (Result 2), and its weights are symmetric, so it keeps a straight-line trend.
The end effect. A centred MA of order , or a MA with , has no value at the first and the last time steps: the window does not fit. This is a serious limit for forecasting, because the newest values are the most important ones.
Change the window and the type. The aqua line is the moving average of the data; the dashed line is the true trend. The yellow band shows the window used at one time, and the lower panel shows its weights. With "centred" and an even , the widget uses the MA (note the half weights at both ends). At the leftover seasonal amplitude is 0. With "trailing" and the season is also removed, but the line is 5.5 steps late.
You have daily data with a weekly pattern. Which moving average removes the weekly pattern and is centred?
The window must cover one full period (7 days) to remove the pattern. Because 7 is odd, the window is already centred. The trick is needed only for even . A trailing 7-MA removes the pattern too, but it lags a trend by 3 days.
Moving averages as filters
A moving average is a linear filter . Feed it a pure wave and the output is the same wave times the number , the frequency response. For the -MA, is a geometric sum that is exactly 0 at . These are the frequencies of a period- pattern and of all its harmonics. This is the frequency view of Result 2. The MA is the -MA applied after a 2-MA, so its response is a product of the two responses and also has these zeros. Lesson 02 meets frequencies again in the periodogram.
3.4 Classical decomposition #
Algorithm (classical additive decomposition) with seasonal period :
- Trend. = the MA if is even, the centred -MA if is odd.
- Detrend. wherever exists.
- Seasonal indices. For each season , average over all times in season . Then subtract the mean of the averages, so the indices sum to 0. The seasonal component repeats these indices.
- Remainder. .
For the multiplicative version, divide instead of subtract: , scale the indices so they average to 1, and . The seasonally adjusted series is (or ): the data with the season removed.
Weaknesses (FPP3 §3.4):
- no trend (and no remainder) at the first and last time steps;
- the moving average smooths over fast rises and falls of the trend;
- the seasonal pattern is the same in every year, so a slowly changing season is not captured;
- unusual values (outliers) enter the averages and spoil the trend and the seasonal indices.
3.5 STL and other modern methods #
STL (Seasonal and Trend decomposition using Loess; Cleveland et al., 1990) fixes most of these weaknesses. Loess is local regression: at each time it fits a low-degree polynomial to the nearby points, with more weight on the closest ones. STL repeats two smoothing steps:
- remove the current trend; for each season (all Januaries, all Februaries, …), smooth its values over the years with loess. This gives a seasonal pattern that may change slowly over time;
- remove that seasonal pattern and smooth the rest with loess to get the trend.
An outer loop computes robustness weights: points with large remainders get small weights, so outliers do not spoil the trend and the season. Its main parameters are the seasonal window (how many years each seasonal smoothing uses: larger means a more stable pattern) and the trend window (larger means a smoother trend). STL works for any period, gives values at the ends, and is the usual default. It is additive, so use it on for multiplicative series. National statistics offices use X-13ARIMA-SEATS (US Census Bureau) and SEATS, which are built for monthly and quarterly data with calendar effects.
Strength of trend and seasonality (FPP3 §4.3). With the remainder of a decomposition:
Both lie in . If the seasonal part is large compared with the remainder, varies much more than and is close to 1. With no seasonality, and is near 0.
3.6 What decomposition is for #
Decomposition describes the past. It does not forecast by itself: the trend from a centred MA stops steps before the end. To forecast with a decomposition (FPP3 §5.7), forecast the seasonally adjusted series with a non-seasonal method, forecast the seasonal part by repeating the last year, and add the two (lessons 03 and 04). Other uses: seasonal adjustment for reports, features for ML models, and residual-based anomaly detection.
Note that the "trend" is not a fixed property of the data. It depends on the method and on the window: a longer window gives a smoother trend and pushes more movement into the remainder.
Answers to section 1. (1) The yearly pattern is removed exactly (Result 2). But the plain 12-MA is centred between two months; the MA fixes this. (2) Multiplied: the swing doubles when the level doubles, so the model is multiplicative; the log makes it additive. (3) It is late by steps, so it is units below the line (Result 1).
A trailing 5-MA is applied to a series with a straight-line trend. By how many time steps is the smoothed trend late?
By Result 1, a trailing MA of order equals the line at time , which is for . The average of is , the middle of the window. Only a centred MA returns the line unchanged.
4. Worked example #
Quarterly data () for three years, :
| 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| quarter | Q1 | Q2 | Q3 | Q4 | Q1 | Q2 | Q3 | Q4 | Q1 | Q2 | Q3 | Q4 |
| 14 | 9 | 6 | 15 | 16 | 11 | 8 | 17 | 19 | 12 | 10 | 21 |
The swing has about the same size every year, so we use the additive model.
Step 1: trend with the MA. The weights are on . For :
For : . In the same way:
| 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | |
|---|---|---|---|---|---|---|---|---|
| 11.25 | 11.75 | 12.25 | 12.75 | 13.375 | 13.875 | 14.25 | 15 | |
| 3.25 | 3.75 | 3.125 | 4.75 |
There is no trend at (the end effect, at each end).
Step 2: seasonal indices. Average the detrended values of each quarter:
- Q1 ():
- Q2 ():
- Q3 ():
- Q4 ():
Their sum is , so their mean is . Subtract it from each:
and now the four indices sum to 0. Q4 is 3.25 above the trend on average, Q3 is 5.25 below it.
Step 3: remainder. :
| 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | |
|---|---|---|---|---|---|---|---|---|
| 0 | 0 | 0.5625 | 0.4375 |
For example . The remainders are small compared with the seasonal indices.
Step 4: strength. On : , and . So (very strong seasonality) and (a clear trend).
Step 5: seasonally adjusted series. , for example . Note that the seasonally adjusted series exists at the ends too, because the seasonal indices are known for every quarter.
5. In code #
NumPy reproduces every number of the worked example; the last lines show resample on a weekly
series with gaps.
import numpy as np
import pandas as pd
y = np.array([14, 9, 6, 15, 16, 11, 8, 17, 19, 12, 10, 21.]) # 3 years of quarters
m = 4
# 2x4 moving average: weights 1/8, 1/4, 1/4, 1/4, 1/8
w = np.r_[0.5, np.ones(m - 1), 0.5] / m
trend = np.full(len(y), np.nan)
trend[m // 2 : -(m // 2)] = np.convolve(y, w, mode="valid")
print(trend[2:10]) # [11.25 11.75 12.25 12.75 13.375 13.875 14.25 15. ]
detr = y - trend
raw = np.array([np.nanmean(detr[j::m]) for j in range(m)])
idx = raw - raw.mean() # centred seasonal indices
print(raw, idx) # [ 4.25 -2.375 -5.3125 3.1875] [ 4.3125 -2.3125 -5.25 3.25 ]
seasonal = np.tile(idx, len(y) // m)
remainder = y - trend - seasonal
print(remainder[2:10]) # [ 0. 0. -0.5625 0.5625 -0.125 -0.125 0.4375 -0.6875]
ok = ~np.isnan(trend)
F_T = max(0, 1 - remainder[ok].var() / (trend[ok] + remainder[ok]).var())
F_S = max(0, 1 - remainder[ok].var() / (seasonal[ok] + remainder[ok]).var())
print(round(F_T, 2), round(F_S, 2)) # 0.89 0.99
# pandas: a weekly series with gaps -> monthly means (NaN weeks are skipped)
idx_w = pd.date_range("2024-01-06", periods=10, freq="W-SAT")
s = pd.Series([5., 6, np.nan, 7, 8, 8, 9, np.nan, 10, 11], index=idx_w)
print(s.resample("MS").mean().round(2).tolist()) # [6.0, 8.33, 10.5]
np.convolve flips the weights, which does not matter here because they are symmetric. The
library version is statsmodels.tsa.seasonal.seasonal_decompose (classical) and
statsmodels.tsa.seasonal.STL; the notebook compares them with your own code.
6. Where it shows up #
- Seasonal adjustment. Unemployment, retail sales and GDP are published seasonally adjusted, so that a change from one month to the next is not just the season.
- Forecasting by components. Prophet fits trend + seasonality + holidays. "STL + ETS" (FPP3 §5.7) forecasts the seasonally adjusted series and adds the seasonal part back. MSTL extends STL to several seasonal periods, such as daily and weekly patterns in hourly electricity load.
- Deep learning. Autoformer and FEDformer contain a moving-average "series decomposition block" inside the network. DLinear splits the input window into a moving-average trend and a remainder, and applies one linear layer to each (lesson 09).
- Features for ML. , , the peak month and the remainder variance are standard features for classifying or clustering many series, and lags of seasonal indices are inputs for gradient-boosting forecasters (lesson 08).
- ML for science. Climate scientists remove the mean annual cycle ("climatology") before they study anomalies such as El Niño. The Mauna Loa CO₂ record is the standard example of a trend with a stable yearly cycle; you decompose it in the notebook.
- Anomaly detection. A large remainder after STL is a simple and strong anomaly score (lesson 11).
7. Common mistakes #
- Shuffling time series data. A random split puts future values into training, and the test error is too optimistic. Split in time order (lesson 03).
- Using a plain 12-MA as a centred trend. Its centre is half a month off. Use the MA (or any MA for even ).
- Using a trailing MA as "the trend" in a plot. It is late by steps, so peaks of the trend appear too late.
- Additive decomposition of a multiplicative series. The seasonal swing then leaks into the remainder: large remainders at the end, where the level is high. Take logs first.
- Calling every wave "seasonal". A season has a fixed calendar period. Business cycles and the sunspot cycle are cyclic and belong to the trend-cycle.
- Treating decomposition as a forecast. The centred trend does not reach the last points; you still need a forecasting model.
- Ignoring missing time stamps. Code that uses "the previous row" as "the previous time step"
is wrong when rows are missing. Use
asfreqto make gaps visible.
8. Check yourself #
- Daily data has a weekly pattern. Which moving average do you use for the trend, and at how many
time steps at each end is it missing?
Answer
The centred 7-MA (7 is odd, so no step is needed). Its half-width is , so the first 3 and the last 3 days have no trend value. - Show that the MA has the weights .
Answer
The 4-MA over and the 4-MA over each give weight . Their average gives to and (in one window each), and to (in both windows). The weights sum to 1. - The raw quarterly averages of a detrended series are . What are the
seasonal indices?
Answer
The sum is , so the mean is . Subtract it: . Check: . - Hourly electricity load has a daily and a weekly pattern. What are the periods, and can
classical decomposition handle both?
Answer
The periods are (daily) and (weekly). Classical decomposition handles one period only. Use a method for multiple seasonality, such as MSTL, which applies STL once for each period. - A decomposition gives and .
What is , and what does it say?
Answer
. The seasonal part adds almost nothing to the variation: the series has hardly any seasonality. - Why is the remainder of an additive decomposition of an exponentially growing, multiplicative
series a bad anomaly score?
Answer
The additive model assumes a constant seasonal swing. When the true swing grows with the level, the part of the swing that the fixed indices do not explain goes into the remainder. So the remainders grow over time and flag normal, high-level seasons as anomalies. Decompose instead.
What are the three components of a classical decomposition?
The trend-cycle , the seasonal component with fixed period , and the remainder .
When should you use a multiplicative decomposition?
When the seasonal swing (and the noise) grow in proportion to the level of the series: .
Why does the log transform turn a multiplicative model into an additive one?
.
Give the weights of the moving average.
on and , and on the 11 values in between (13 values in total).
Why do we need a MA when is even?
A window of an even number of values is centred between two time steps. Averaging two neighbouring -MAs moves the centre back onto a time step.
By how many steps does a trailing moving average of order lag a linear trend?
steps.
Compute the MA of the values 14, 9, 6, 15, 16 (centred on 6).
.
What is the difference between a seasonal pattern and a cycle?
A seasonal pattern has a fixed, known period from the calendar (year, week, day). A cycle rises and falls without a fixed period, for example business cycles.
What does STL stand for, and name one advantage over classical decomposition.
Seasonal and Trend decomposition using Loess. It lets the seasonal pattern change over time, is robust to outliers, and gives values at the ends.
Write the strength of seasonality .
.
9. Interview angle #
- "How do you check whether a series is seasonal?" — Time plot and seasonal plot; peaks of the ACF at multiples of (lesson 02); the strength from an STL decomposition.
- "Additive or multiplicative?" — Compare the seasonal swing at low and high levels. If it grows with the level, use multiplicative, or log + additive. Box–Cox with is in between.
- "Why can you not use ordinary k-fold cross-validation on time series?" — Neighbouring values are dependent and the future must not be in the training data; use rolling-origin evaluation (lesson 03).
- "What does STL do better than classical decomposition?" — Changing seasonality, robustness to outliers, estimates at the ends, any period.
- "What is a centred moving average of even order?" — The MA, with half weights at both ends; explain why the plain -MA is half a step off.
10. Practice #
The notebook for this lesson runs in Colab. Each exercise has a check cell and a folded solution:
- Arrange a monthly series into a years × months table for the seasonal plot (
seasonal_matrix), and draw the time, seasonal and lag plots of the "monthly demand" series. - Write
centred_ma(y, k)for odd andma_2xm(y, m)for even , and check them against a convolution with the explicit weights. - Write
classical_decompose(y, m)(additive) and check that the indices sum to 0 and that the three parts add up to . - Decompose a multiplicative series through and recover its seasonal factors.
- Resample the weekly Mauna Loa CO₂ record to monthly means, fill the gaps, decompose it with , and compare with STL (optional cell).
11. Watch and read #
- Hyndman and Athanasopoulos, Forecasting: Principles and Practice, 3rd ed. (free online): chapter 2, time series graphics (time, seasonal and lag plots), chapter 3, decomposition (§3.1 transformations, §3.3 moving averages, §3.4 classical decomposition, §3.6 STL) and §4.3, STL features (strength of trend and seasonality).
- Cleveland, Cleveland, McRae and Terpenning (1990), STL: A Seasonal-Trend Decomposition Procedure Based on Loess, Journal of Official Statistics 6(1) — the original STL paper.
- pandas documentation, Time series / date functionality:
DatetimeIndex,resample,asfreqand time zones. - statsmodels, Seasonal-Trend decomposition using LOESS (STL): a worked example on the CO₂ data.
- The CO₂ record: C. D. Keeling and the Scripps CO₂ program, distributed with statsmodels as
sm.datasets.co2; current data at NOAA Global Monitoring Laboratory.
Runs in your browser with nothing to install. The notebook repeats the key demos and has exercises with checks; solutions are inside, folded.
No Google account needed: Python runs inside your browser. The first start downloads about 40 MB; your changes are saved in this browser only.