Overview01
If the people who run a data cache know traffic will spike, they can provision for it. If they miss the spike, everything downstream slows down.
My work has two halves. One is building a forecaster that beats a hard-to-beat naive baseline. The other is finding out, with evidence, how predictable the peaks are at all, so the team designs features and models around a real limit. I present to the group every week, and each week's question comes out of the last one's result.
- 01
Implemented PatchTST (PyTorch / NeuralForecast) for multivariate forecasting of cache traffic at 1-, 7- and 30-day horizons, choosing the lookback window with rolling cross-validation and beating a persistence baseline on every variable and horizon, by up to 24% RMSE.
- 02
Showed that the missed traffic peaks are a limit on what past traffic can predict: changing the loss or forecasting high on purpose missed the same spikes, which redirected the work toward new input signals.
- 03
Built reproducible Pandas EDA pipelines over several years of XCache logs from three regional caches, and showed that the best amount of history depends on whether traffic is calm or erratic.
The research at a glance
| Question | How I tested it | What I concluded |
|---|---|---|
| Can a deep forecaster beat the naive baseline at all? | PatchTST vs. persistence and linear regression on a chronological hold-out | Yes on the headline metric, but the average error hid that it smoothed over every spike |
| How much history should the model see? | Lookback sweep per horizon, scored with rolling-origin cross-validation | Long context helps a day ahead; at a month ahead short windows win |
| Is that result real or luck? | Hyperparameter grid on the two finalists, then retraining on several random seeds | Stable across seeds; the long window specifically controls large errors |
| Are the missed peaks a training problem? | Five losses from soft to strict, then deliberately high percentile forecasts | No: every variant misses the same spikes, so the signal isn't in past traffic |
| Does the right setup depend on the kind of traffic? | Separate models for a calm era and an erratic era of the same cache | Yes: calm traffic rewards longer history, erratic traffic rewards the last few days |
What PatchTST is & how it works02
PatchTST (Nie et al., ICLR 2023, "A Time Series is Worth 64 Words") is a Transformer built for long-horizon forecasting. Given the last L days of a series (the lookback), it predicts the next H days (the horizon) in one shot, instead of predicting one day and feeding that guess back in. One set of weights serves every variable.
Its main idea is borrowed from vision transformers, which cut an image into small squares and treat each square as a "word". PatchTST cuts a time series into short patches of consecutive days and treats each patch as a token.
How a forecast is made, step by step
- Normalize per window (RevIN). Traffic levels drift over months. Removing each window's own mean and scale lets the model learn the shape of what comes next, then the level is added back to the output.
- Patch. A 128-day window with 16-day patches and an 8-day stride becomes about 15 tokens instead of 128. Each token now carries a local shape (a ramp, a weekly dip, a burst) rather than a single number with no context.
- Attend. Self-attention learns which stretches of the past matter for the forecast: last week, the same point last month, the start of a burst. Its cost grows with the square of the number of tokens, so patching is what makes a long lookback affordable.
- Channel independence. Each of the six traffic variables goes through the same network separately. That multiplies the training examples and reduces overfitting on a short history, at a cost: the model doesn't know that hits plus misses must equal accesses.
- Direct multi-step output. The linear head predicts every future day at once, so errors don't compound the way they do when a model feeds its own predictions back in.
| A Transformer on daily values | PatchTST | |
|---|---|---|
| One token is | one day (a single number) | a patch of consecutive days |
| Tokens for a long lookback | one per day | about one per stride |
| What attention compares | individual days, noisy | local shapes, more meaningful |
| Variables | often mixed into one token | encoded separately, shared weights |
How I set it up
- Framework: NeuralForecast's PatchTST on PyTorch Lightning, with RevIN on and early stopping on validation loss.
- Patches follow the lookback: 16-day patches with an 8-day stride once the window is long enough; for very short windows the patch shrinks to half the window, since a patch can't be longer than its input.
- Scaling: each variable standardised, because counts and data volumes live on very different scales.
- Chosen deliberately: lookback, horizon, model size, dropout and loss were each tested as experiments (below).
The problem & the data03
Experiments like the Large Hadron Collider store their data at a few large sites, while the physicists analysing it are spread across the country, and ESnet carries that traffic. Much of a popular dataset is read again and again, so regional caches (XCache) keep copies close to the users: fewer repeated transfers, lower latency, less wide-area traffic. Forecasting how hard each cache will be used is what lets operators size them.
The data is a daily record from three CMS caches (Southern California, Chicago and Boston) covering several years. Each day has six variables: how many file accesses there were and how much data they moved, split into cache hits (served locally) and misses (fetched over the network). Operators care most about the volume served from the cache.
Exploring before modelling
I built reproducible Pandas pipelines that load, clean, resample and plot every site the same way and rerun end to end on new data. What the exploration found:
The first model and its baselines04
Step 1 · Baselines and the hold-out split
For traffic, "tomorrow looks like today" (persistence) is a strong baseline because most of the signal is momentum and weekly rhythm. I compared PatchTST with persistence and a linear model on a chronological hold-out (never a random split, which would leak the future), with early stopping on validation loss to avoid overfitting.
Step 2 · Plot the forecasts against the actual traffic
The first model beat both baselines on most variables, including the one operators care about. Plotting the forecasts against reality showed a problem: the model tracked the typical level but smoothed straight through the spikes, and on the low-volume miss variables it collapsed toward zero. Those variables only looked fine because their true values are mostly near zero too.
Conclusion: RMSE rewards being right on average, so a model that plays it safe near the typical value can score well while missing every spike. From then on I judged every experiment on both the average error and the peaks.

Choosing how much history to use05
I set each hyperparameter by testing it, and ran the one that mattered most, the amount of history, as an experiment of its own.
Step 3 · Choose the lookback with rolling cross-validation
A single test period can flatter one setting by chance, so every lookback was scored across many rolling windows: train on the past, forecast the next block, slide forward, repeat. The persistence baseline was scored on the identical windows, so every comparison is like for like.
Walk-forward only: no fold ever peeks at the future.
What I saw: one day ahead, error was lowest with roughly three months of history, long enough to span a monthly cycle. A week to a month of history did worst (too short to see a cycle, long enough to add noise), and among the short windows, 3 days stood out. A week ahead, 3 days was best.

At each variable's best setting, PatchTST beat persistence on every variable at both horizons, with the largest gains on data volumes one day ahead and the smallest on miss counts a week ahead.

Step 4 · Check the result isn't luck
- Head-to-head grid. I tuned model size and dropout for the two finalists (3 vs. 100 days of history). On average error their configurations overlapped; on RMSE, which weights big misses, every 100-day configuration beat every 3-day one. So long context specifically controls large errors.
- Seeds. Retraining the chosen model on several random seeds changed its error only slightly, so the choice is a stable finding rather than a lucky initialisation.


Step 5 · Push the horizon out to a month
If long history helps a day ahead because it captures seasonality, it should help even more a month ahead. It didn't: at 30 days the short windows won and the longest ones never did (the dashed line in the lookback chart above).
So a longer lookback is always better?
Not a month ahead. Short windows win there, and the longest ones never do.
But long history wins a day ahead?
Right, and the seed check says that's real. The seasonal advantage just doesn't carry out to 30 days.
Testing why peaks are missed06
Every experiment so far tracked ordinary days well and under-predicted the rare peaks. Rather than keep tuning, I turned the two obvious explanations into hypotheses and tested each directly.
Hypothesis 1 · The loss function is to blame
If training rewards average correctness, a stricter loss should chase peaks. I trained the same model under five losses from softest to strictest (MAE, three Huber settings, MSE). All five produced nearly identical forecasts and missed the same spikes by the same amount. Measured against the MAE-trained model, every stricter loss was equal or slightly worse, on peak days too. Rejected.


All five losses under-predict the biggest spikes.
Hypothesis 2 · Just forecast high on purpose
For provisioning, over-shooting is cheaper than falling short, so I trained percentile forecasts that aim above the median. Raising the target added spare capacity on ordinary days but barely changed how often a busy day was missed: the whole forecast shifted up without learning to see spikes coming. Rejected.
Conclusion · A limit on what history can predict
If neither a stricter objective nor a deliberately high forecast catches the spikes, the information isn't in the past traffic. The peaks are infrequent and rarely foreshadowed, so a model trained on history alone can't anticipate them. That changed the team's question from "which model predicts peaks?" to "what signal would make peaks predictable?", and it now drives feature design: calendar features, rolling variance and recent miss activity.
Calm vs. erratic traffic07
The most recent test period was far more erratic than earlier years, which raised a new question: is there one best setup at all, or does it depend on the kind of traffic?
Step 6 · Split by regime and compare
I divided the longest-running cache's history into a calmer earlier era and an erratic recent era, trained a separate model on each with the same chronological split, and repeated the lookback-by-horizon comparison inside both.

Conclusion: rather than one model for all traffic, detect the regime and use a model tuned for it. The next step is a measurable seasonality or regime signal that says in advance whether traffic will be calm or erratic, and fixing patch length and stride for each lookback.
Where it went next08
The first model answered one question: can a Transformer beat persistence? The work since then asked a harder one. What is the best honest next-day forecast, and does anything beyond a single PatchTST actually add to it?
A small blend, chosen the hard way
The reported one-day model is a fixed three-part blend: an hourly-input PatchTST that reads two weeks of hourly traffic and predicts the next day's total, the original daily PatchTST, and a shallow decision tree on the daily features. The hourly signal is where most of the extra accuracy came from. Across the rolling-origin folds it averages about 14.5 TB/day of error against persistence's 17.6, roughly 17% lower, and it wins on most of the held-out folds.
What didn't help
Plenty, and each is written up as a null rather than dropped quietly. Zero-shot time-series foundation models (Amazon's Chronos-2 and Chronos-Bolt) made the same mistakes the blend already made and were worse on busy days, so they earned zero weight. Heavier networks (NHITS, TiDE, an LSTM, TimeXer) and gradient-boosted trees added nothing over the blend either.
Rules fixed before the data was read
Every candidate is pre-registered before a single model is fit. Models are chosen on rolling-origin folds with leave-one-fold-out; a separate block of earlier days can only veto a choice, never make one; and the test window is read only after the choice is locked to a hash. The model's one real weakness is stated plainly, that it under-forecasts the rare record-high days, and the whole pipeline is guarded by 566 automated checks.