Skip to main content

Chapter 26: Time Series Forecasting

MODULE 5 | WEEK 26​

Chapter 26: Time Series Forecasting​

Time series is data that is recorded over time : daily sales, monthly electricity consumption, hourly weather, the price of a share. In business the most frequently asked question is: "What will happen in the future?" In this chapter we will learn the real principles of forecasting : never breaking the time order, building a baseline , turning a time series into supervised learning through windowing , and then forecasting with ML and LSTM/GRU .

26.1 What is a Time Series?​

In a time series, every observation has a time attached, and nearby observations are linked to each other (today's sales resemble yesterday's sales). This is why the usual ML rules (random shuffling) do not work here.

We will build a synthetic series that still looks like real life: 10 years (120 months) of sales of a shop, with a rising trend , a yearly seasonal effect (seasonality) and a little noise . The benefit of synthetic data is that we know what is hidden inside.

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
rng = np.random.RandomState(42)
months = pd.date_range("2015-01-01", periods=120, freq="MS") # the first day of every month tareekh
t = np.arange(120)
trend = 100 + 0.8 * t # rising a little every month
season = 15 * np.sin(2 * np.pi * t / 12) # one cycle in 12 months
noise = rng.normal(0, 3, 120)
sales = trend + season + noise
ts = pd.Series(sales, index=months, name="sales")
print(ts.head(4).round(1))
print("Total months:", len(ts), "| Min:", ts.min().round(1), "| Max:", ts.max().round(1))
fig, ax = plt.subplots(figsize=(10, 3.6))
ax.plot(ts, color="#6B5B95", lw=1.8)
ax.set_title("Monthly sales (synthetic data): trend + seasonality + noise")
ax.set_ylabel("sales")
plt.show()

OUTPUT 2015-01-01 101.5 2015-02-01 107.9 2015-03-01 116.5

2015-04-01 122.0 Freq: MS, Name: sales, dtype: float64 Total months: 120 | Min: 92.0 | Max: 205.3 OUTPUT (chart)

26.1 What is a Time Series?: Monthly sales (synthetic data):trend +seasonality + noise 200 180 160 sales 140 120 100 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025
26.1 What is a Time Series?: Monthly sales (synthetic data):trend +seasonality + noise 200 180 160 sales 140 120 100 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025
PartMeaningIn this series
TrendA long-term upward or downward movementAn increase of +0.8 every month
SeasonalityA repeating seasonal patternThe same shape after every 12 months
NoiseIrregular small ups and downsRandom noise (std = 3)
Separating these parts from the real series ( decomposition ) is the very first step:​
trend_est = ts.rolling(window=12, center=True).mean() # 12-month
average =
trend
detrended = ts - trend_est
season_est = detrended.groupby(detrended.index.month).mean() # the usual effect of each month
print("Seasonal effect of each month (first 6 months):")
print(season_est.round(1).head(6))
print("Month with the largest effect:", int(season_est.idxmax()), " | smallest:",
int(season_est.idxmin()))
fig, axes = plt.subplots(1, 3, figsize=(10, 2.8)) axes[0].plot(ts.index, trend_est, color="#6B5B95")
axes[0].set_title("Trend") axes[1].bar(season_est.index, season_est.values, color="#A4457A")
axes[1].set_title("Seasonality (by month)") axes[2].plot(ts.index, (detrended - detrended.index.month.map(season_est)), color="#2F7D4F", lw=1) axes[2].set_title("Residual (noise)")
plt.tight_layout()
plt.show()

OUTPUT Seasonal effect of each month (first 6 months): 1 0.6 2 6.8 3 10.9 4 15.6 5 13.0 6 9.8 Name: sales, dtype: float64 Month with the largest effect: 4 | smallest: 10

OUTPUT (chart)​
26.1 What is a Time Series?: Trend Seasonality (by month) Residual (noise) 180 5.0 10 2.5 160 0.0 140 -2.5 120 -10 -5.0 201B01201201902D022022022024 10 201B01201201902D022022022024
26.1 What is a Time Series?: Trend Seasonality (by month) Residual (noise) 180 5.0 10 2.5 160 0.0 140 -2.5 120 -10 -5.0 201B01201201902D022022022024 10 201B01201201902D022022022024

26.2 The golden rules of forecasting​

26.3 Baseline models and Metrics​

BaselineIdea
NaiveNext month's sales = last month's sales
Seasonal NaiveNext month's sales = the sales ofthe same month last year
Moving AverageThe average of the last 12 months
MetricFormula (short)
Meaning
MAEmean(|actual - forecast|)
Average error, in the same unit as the data
RMSEsqrt(mean((actual - forecast)^2))
Punishes large errors more
MAPEmean(|actual - forecast| / |actual|) x 100
Error in percent (wrong if the actual is near 0)
def metrics(y_true, y_pred):
y_true, y_pred = np.asarray(y_true), np.asarray(y_pred)
mae = np.mean(np.abs(y_true - y_pred))
rmse = np.sqrt(np.mean((y_true - y_pred) ** 2))
mape = np.mean(np.abs((y_true - y_pred) / y_true)) * 100
return round(mae, 2), round(rmse, 2), round(mape, 2)

full = ts.values
n_tr = len(train)
y_test = test.values
# One-step baselines: the previous ACTUAL values are used for every test month
naive = full[n_tr - 1:-1] # the previous month
seasonal_naive = full[n_tr - 12:-12] # 12 months earlier
moving_avg = np.array([full[i - 12:i].mean() for i in range(n_tr, len(full))])
print("Baseline".ljust(16), "MAE RMSE MAPE%")
for name, pred in [("Naive", naive), ("Seasonal naive", seasonal_naive),
("Moving avg (12)", moving_avg)]:
print(name.ljust(16), *metrics(y_test, pred))
OUTPUT​
Baseline MAE RMSE MAPE%
Naive 5.65 6.78 3.05
Seasonal naive 10.03 10.67 5.39
Moving avg (12) 9.8 11.77 5.16

These numbers are our line to beat . Any model that cannot beat them is not needed.

26.4 Windowing: turning a time series into supervised learning​

An ML model needs (X, y) . In a time series we take the previous window steps as X and the next step as y . This method is called a sliding window .

Example window

Series: 10, 12, 11, 14, 15, 17, 16
Window: 3
[10, 12, 11] -> 14
[12, 11, 14] -> 15
[11, 14, 15] -> 17
[14, 15, 17] -> 16
def make_windows(values, window):
X, y = [], []
for i in range(window, len(values)):
X.append(values[i - window:i])
y.append(values[i])
return np.array(X), np.array(y)

demo = np.array([10, 12, 11, 14, 15, 17, 16], dtype=float)
Xd, yd = make_windows(demo, 3)
print(Xd)
print("y:", yd)
print("X shape:", Xd.shape, " y shape:", yd.shape)
OUTPUT​
[[10. 12. 11.]
[12. 11. 14.]
[11. 14. 15.]
[14. 15. 17.]]
y: [14. 15. 17. 16.]
X shape: (4, 3) y shape: (4,)

RNN/LSTM/GRU need 3D input: (samples, timesteps, features) . Our series has only 1 feature (sales):

Xd3 = Xd.reshape(Xd.shape[0], Xd.shape[1], 1)
print("Shape for an RNN:", Xd3.shape, "-> (samples, timesteps, features)")

OUTPUT Shape for an RNN: (4, 3, 1) -> (samples, timesteps, features)

26.5 Scaling and the full preparation​

Neural networks need scaled data. The scaler is fitted on the train data only (otherwise future information would leak):

from sklearn.preprocessing import MinMaxScaler
window = 12
scaler = MinMaxScaler().fit(train.values.reshape(-1, 1)) # on train only
scaled = scaler.transform(ts.values.reshape(-1, 1)).ravel()
X_all, y_all = make_windows(scaled, window) # the target of a window is the month at index (window + i)
target_idx = np.arange(window, len(scaled))
is_test = target_idx >= n_tr # those whose target is in the test period
X_tr, y_tr = X_all[~is_test], y_all[~is_test]
X_te, y_te = X_all[is_test], y_all[is_test]
print("Train windows:", X_tr.shape, " Test windows:", X_te.shape)
print("Scaled train range:", scaled[:n_tr].min().round(2), "to", scaled[:n_tr].max().round(2))
print("Scaled test range :", scaled[n_tr:].min().round(2), "to", scaled[n_tr:].max().round(2))

OUTPUT Train windows: (84, 12) Test windows: (24, 12) Scaled train range: 0.0 to 1.0 Scaled test range : 0.83 to 1.21 REMEMBER

The scaled range of the test went above 1 , because the series is rising and the train data simply did not have such large values. This is perfectly fine (there is no leakage), but it means that the model will have to extrapolate outside the train range . This is the real problem with series that have a trend.

26.6 Forecasting with ML and a Neural Network​

Three models: Ridge Regression (linear), Random Forest , and a small neural network ( MLPRegressor , a real neural net). All three are given the same previous 12 months:

from sklearn.linear_model import Ridge
from sklearn.ensemble import RandomForestRegressor
from sklearn.neural_network import MLPRegressor
import warnings
warnings.filterwarnings("ignore")
models = {
"Ridge": Ridge(alpha=0.1),
"Random Forest": RandomForestRegressor(n_estimators=200, random_state=0),
"Neural Net (MLP)": MLPRegressor(hidden_layer_sizes=(32, 16), max_iter=3000,
random_state=0),
}
preds = {}
print("Model".ljust(18), "MAE RMSE MAPE%")
for name, m in models.items():
m.fit(X_tr, y_tr)
p_scaled = m.predict(X_te)
p = scaler.inverse_transform(p_scaled.reshape(-1, 1)).ravel() # back to the original unit
preds[name] = p
print(name.ljust(18), *metrics(y_test, p))
OUTPUT​
Model MAE RMSE MAPE%
Ridge 2.62 3.42 1.4
Random Forest 7.76 9.95 4.04
Neural Net (MLP) 8.59 10.08 4.67
fig, ax = plt.subplots(figsize=(10, 3.8))
ax.plot(train.index[-36:], train.values[-36:], color="gray", lw=1.5, label="Train (last 3 years)")
ax.plot(test.index, y_test, color="black", lw=2, label="Actual (test)")
cols = {"Ridge": "#6B5B95", "Random Forest": "#B7791F", "Neural Net (MLP)": "#A4457A"}
for name, p in preds.items():
ax.plot(test.index, p, color=cols[name], lw=1.6, ls="--", label=name)
ax.set_title("One-step forecast: test period")
ax.legend(ncol=2, fontsize=8)
plt.show()

OUTPUT (chart)

26.6 Forecasting with ML and a Neural Network: One-stepforecast:testperiod Train (last 3 years) Random Forest 200 Actual (test) Neural Net (MLP) Ridge 190 180 170 160 150 140 2021 2020 2022 2023 2024 2025
26.6 Forecasting with ML and a Neural Network: One-stepforecast:testperiod Train (last 3 years) Random Forest 200 Actual (test) Neural Net (MLP) Ridge 190 180 170 160 150 140 2021 2020 2022 2023 2024 2025

Notice that Ridge (linear) came out the best, and Random Forest and Neural Net both stayed behind even the naive baseline. The reason: a Random Forest gives the average of values it saw in training inside its trees, so it cannot extrapolate beyond the largest value of the train data ; a neural net is also weak outside the train range. A linear model catches the trend directly.

Differencing: remove the trend and try again​

diff = np.diff(ts.values) # the change of every month
d_tr_end = n_tr - 1 # this many elements of diff are train
sc2 = MinMaxScaler().fit(diff[:d_tr_end].reshape(-1, 1))
d_scaled = sc2.transform(diff.reshape(-1, 1)).ravel()
Xd_all, yd_all = make_windows(d_scaled, window)
d_target_idx = np.arange(window, len(d_scaled)) # indices of diff
is_test_d = d_target_idx >= d_tr_end
Xd_tr, yd_tr = Xd_all[~is_test_d], yd_all[~is_test_d]
Xd_te, yd_te = Xd_all[is_test_d], yd_all[is_test_d]
print("Differenced train windows:", Xd_tr.shape, " test windows:", Xd_te.shape)
prev_values = full[n_tr - 1:-1] # the previous ACTUAL value (to add the
change back to)
for name, m in [("Random Forest (diff)", RandomForestRegressor(n_estimators=200, random_state=0)), ("Neural Net (diff)", MLPRegressor(hidden_layer_sizes=(32, 16), max_iter=3000, random_state=0)), ("Ridge (diff)", Ridge(alpha=0.1))]:
m.fit(Xd_tr, yd_tr)
d_pred = sc2.inverse_transform(m.predict(Xd_te).reshape(-1, 1)).ravel()
p = prev_values + d_pred # previous value + predicted change preds[name] = p
print(name.ljust(22), *metrics(y_test, p))

OUTPUT Differenced train windows: (83, 12) test windows: (24, 12) Random Forest (diff) 3.42 4.14 1.83 Neural Net (diff) 3.36 4.71 1.8 Ridge (diff) 2.89 3.54 1.54 After differencing, the RMSE of both Random Forest and Neural Net came down from around 10 to around 4 , because now the model does not have to learn the trend, only the change and the seasonality , which have already been seen in the train range. (In this series Ridge still remains simple and strong.)

26.7 Multi-step forecast: several months ahead​

So far we predicted one step ahead (one-step) and gave the actual previous values every time. In real life we often need several months ahead , when the actual values are not known. The recursive method: make the first forecast, add it to the window, and make the next one.

rm = Ridge(alpha=0.1).fit(X_tr, y_tr) # a model learned on the train windows
history = list(scaled[:n_tr]) # only the data up to the train end is known
forecast = []
for step in range(test_size):
x = np.array(history[-window:]).reshape(1, -1)
p = rm.predict(x)[0]
forecast.append(p)
history.append(p) # its own prediction becomes the next step
input
forecast = scaler.inverse_transform(np.array(forecast).reshape(-1, 1)).ravel()
print("24-month recursive forecast (Ridge):", *metrics(y_test, forecast))
print("Naive (repeat last value, baseline):", *metrics(y_test, [full[n_tr - 1]] * test_size))
fig, ax = plt.subplots(figsize=(10, 3.6))
ax.plot(train.index[-36:], train.values[-36:], color="gray", lw=1.5, label="Train")
ax.plot(test.index, y_test, color="black", lw=2, label="Actual")
ax.plot(test.index, forecast, color="#A4457A", lw=2, ls="--", label="24-month forecast")
ax.set_title("Multi-step (recursive) forecast")
ax.legend()
plt.show()

OUTPUT 24-month recursive forecast (Ridge): 2.48 3.17 1.33 Naive (repeat last value, baseline): 22.04 24.15 11.59

OUTPUT (chart)​
26.7 Multi-step forecast: several months ahead: Multi-step(recursive)forecast Train 200 Actual 190 24-monthforecast 180 170 160 150 140 2020 2021 2022 2023 2024 2025
26.7 Multi-step forecast: several months ahead: Multi-step(recursive)forecast Train 200 Actual 190 24-monthforecast 180 170 160 150 140 2020 2021 2022 2023 2024 2025

26.8 Walk-forward Validation (TimeSeriesSplit)​

In time series K-Fold (with shuffling) is wrong. The right method: old data for training, the data after it for validation , and each time make the train part a little longer.

from sklearn.model_selection import TimeSeriesSplit
tscv = TimeSeriesSplit(n_splits=4, test_size=12)
for k, (tr_idx, va_idx) in enumerate(tscv.split(X_tr), 1):
m = Ridge(alpha=0.1).fit(X_tr[tr_idx], y_tr[tr_idx])
p = m.predict(X_tr[va_idx])
rmse = np.sqrt(np.mean((y_tr[va_idx] - p) ** 2))
print("Fold", k, "| train:", tr_idx[0], "to", tr_idx[-1], "| validate:", va_idx[0], "to", va_idx[-1], "| RMSE (scaled):", round(rmse, 4))

OUTPUT Fold 1 | train: 0 to 35 | validate: 36 to 47 | RMSE (scaled): 0.0302 Fold 2 | train: 0 to 47 | validate: 48 to 59 | RMSE (scaled): 0.0298 Fold 3 | train: 0 to 59 | validate: 60 to 71 | RMSE (scaled): 0.0437 Fold 4 | train: 0 to 71 | validate: 72 to 83 | RMSE (scaled): 0.0305

26.9 LSTM, GRU and Conv1D forecasting in Keras​

Now the same problem in Keras. The input shape is (window, 1) . The data is just the X_tr of the previous section; we only need to add an axis for one feature.

import numpy as np
from tensorflow import keras
from tensorflow.keras import layers
X_tr3 = X_tr.reshape(-1, 12, 1) # (samples, 12 timesteps, 1 feature)
X_te3 = X_te.reshape(-1, 12, 1)
lstm_model = keras.Sequential([ keras.Input(shape=(12, 1)), layers.LSTM(32), layers.Dense(1), # regression: no activation ])
lstm_model.compile(optimizer="adam", loss="mse")
lstm_model.summary()

OUTPUT (sample) Model: "sequential" ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓ ┃ ┃ ┃ ┃ Layer (type) Output Shape Param # ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩ │ lstm (LSTM) │ (None, 32) │ 4,352 │ │ dense (Dense) │ (None, 1) │ 33 │ └───────────────────────────────────┴──────────────────────────┴─────────────────┘

Total params: 4,385 (17.13 KB)
Trainable params: 4,385 (17.13 KB)
Non-trainable params: 0 (0.00 B)
OUTPUT (sample)​
3393 161

26.10 Common Mistakes and Hands-on​

Hands-on​

  • Change the series: raise the noise std from 3 to 10 and see which model stays the best.

  • Change the window: try 6, 12, 24 and write down the comparison of RMSE.

  • Remove the seasonality: take the seasonal part out of the series and see how the baselines change.

  • Your own data: write down 24 months of your home's electricity bill (or any data of your own) and build a seasonal naive baseline.

  • Keras: in Colab, make a table of the parameters, time and RMSE of LSTM, GRU and Conv1D.

KEY TAKEAWAYS​
  • Time series = trend + seasonality + noise. Never break the order; no shuffling.

  • Always a baseline first: naive, seasonal naive, moving average.

  • Metrics: MAE, RMSE, MAPE. Measure after bringing scaled values back to the original unit.

  • Use windowing to turn the series into (X, y); the shape for an RNN is (samples, timesteps, features).

  • Fit the scaler on train only. The test range can fall outside the train range.

  • On a series with a trend, differencing makes tree-based models work fine too.

  • Multi-step: recursive (errors add up) or direct.

  • Validation: TimeSeriesSplit / walk-forward, never shuffled K-Fold.