Skip to content
All work

Case studyIndependent prototype

Network performance forecasting and early warning

An independent prototype that forecasts network throughput, latency and packet loss with statistical and deep-learning models, raises early warnings and drafts incident reports. Built as a tested Python package and developed on synthetic perfSONAR-style data.

ARIMA · PROPHET · LSTM · TRANSFORMERobservedforecastthreshold
Schematic diagram of the forecasting and alerting flow.
My role
Sole developer: data generation, models, API, tests and deployment manifests
Period
2026
Type
Independent prototype
Reading depth

View repository (opens in a new tab)

The 30-second version

Problem
Degradation on research networks often goes unnoticed for hours; perfSONAR history could be used to predict it and warn operators early.
My contribution
Independent prototype; I wrote the whole repository: data generator, features, four forecasters, ensemble, early warning, reports, API, tests and deployment manifests.
Approach
ARIMA, Prophet, LSTM and Transformer forecasters on the same horizon, a weighted ensemble, threshold- and forecast-based alerts, and a FastAPI service with Docker and Kubernetes manifests.
Results
A working, tested end-to-end prototype on synthetic data. No accuracy figures are claimed. A code review for this page found an ensemble weight-alignment bug (below).
Stack
  • Python
  • PyTorch
  • statsmodels
  • Prophet
  • FastAPI
  • pytest
  • Docker
  • Kubernetes

Problem

Research and education networks move very large scientific datasets, and a slow link or rising packet loss can stall work for hours before anyone notices. Monitoring tools such as perfSONAR record throughput, latency and loss continuously. The opportunity is to use that history to predict degradation and warn operators before users are affected.

Objective

Build an end-to-end forecasting system rather than a single model in a notebook: realistic data, several forecasting approaches compared fairly, an early-warning layer, readable incident reports, and a service that could be deployed and tested like production software.

My contribution

I wrote the whole repository: the data generator, feature engineering, the four forecasting models, the ensemble, the early-warning and reporting modules, the API, the tests, the Docker image, the Kubernetes manifests and the documentation.

Forecasting models
4
+ weighted ensemble
Metrics forecast
3
throughput · latency · loss
Test modules
7
pytest
Kubernetes manifests
4
incl. GPU training job

Data and feature engineering

Real perfSONAR archives were not available for the prototype, so a generator produces hourly telemetry (throughput, latency, packet loss and retransmits) with daily and weekly seasonality, trend shifts and a configurable share of injected anomalies. Because the generator is seeded, every experiment is reproducible.

On top of the raw series, the preprocessing module adds:

  • cyclical encodings of hour-of-day and day-of-week, so 23:00 and 00:00 are close together;
  • rolling mean, standard deviation, minimum and maximum over several windows;
  • lag features to capture autocorrelation;
  • first differences to capture the rate of change.

Architecture

Forecasting system · select a stage

Stage 1 of 7 · Data

Telemetry and features

Hourly perfSONAR-style measurements are loaded (synthetic in this prototype) and enriched with time encodings, rolling statistics, lags and differences.

  • Seeded, reproducible generator
  • Anomaly injection for testing alerts

Stage 2 of 7 · Models

Four forecasters

Statistical and deep-learning models are trained side by side on the same data and horizon, so they can be compared fairly.

  • ARIMA / SARIMA and Prophet
  • LSTM and Transformer encoder (PyTorch)

Stage 3 of 7 · Ensemble

Weighted combination

Forecasts are combined with configurable weights and normalised to sum to one.

  • Weights live in configuration, not code
  • Known issue: weights misalign if Prophet is unavailable

Stage 4 of 7 · Adapt

Few-shot adaptation

A Reptile-style meta-learning routine trains across several network topologies, then adapts quickly to a new topology from a small number of examples.

  • Meta-training across topology tasks
  • Fast fine-tuning on a small support set

Stage 5 of 7 · Detect

Early warning

Current values and forecasts are checked against configurable thresholds, producing warnings with severity levels.

  • Threshold-based and prediction-based checks
  • Critical and warning severities

Stage 6 of 7 · Report

Incident reports

Warnings become structured incident reports with likely causes and remediation steps, from a deterministic template or an optional LLM endpoint.

  • Template fallback when no LLM key is set
  • Markdown and JSON output

Stage 7 of 7 · Serve

API and deployment

A FastAPI service exposes forecasts, warnings and reports. It ships with a Dockerfile and Kubernetes manifests.

  • Deployment, service and config map
  • Separate GPU training job

Models and engineering decisions

Model Why it is included Implementation
ARIMA / SARIMA Strong, interpretable baseline for linear and seasonal patterns statsmodels
Prophet Handles trend changes and missing data with little tuning Prophet
LSTM Learns non-linear temporal dependencies PyTorch, 2 layers, 64 hidden units
Transformer Captures longer-range dependencies with self-attention PyTorch encoder, 2 layers, 4 heads
Ensemble No single model is best on every metric Configurable weighted average
  • Compare simple and complex models together. Classical baselines are fast and explainable. Deep models only earn their cost if they beat them under the same conditions.
  • Ensemble rather than pick a winner. Different models tend to win on different metrics, so a weighted combination is more robust than betting on one.
  • Few-shot adaptation for new topologies. Networks change. Meta-learning aims to make a forecaster useful on a new path after seeing only a little of its data.

Code review: a weight-alignment bug

The ensemble collects each model’s forecast in a dictionary and takes the matching slice of the configured weights. When Prophet is not fitted, its forecast is skipped and its weight set to zero:

if self.prophet.is_fitted:
    prophet_result = self.prophet.predict(horizon)
    model_preds["prophet"] = prophet_result.predictions
else:
    weights[1] = 0.0
...
active_weights = np.array(weights[:len(model_preds)])
active_weights = active_weights / active_weights.sum()
all_preds = np.stack(list(model_preds.values()))
ensemble_pred = np.average(all_preds, axis=0, weights=active_weights)

With Prophet missing, model_preds holds three forecasts (ARIMA, LSTM, Transformer) but weights[:3] is ARIMA’s, Prophet’s zeroed weight and LSTM’s. The LSTM forecast is silently weighted zero and the Transformer receives the LSTM’s weight. With all four models available the slice lines up and the ensemble is correct. The robust fix is to key weights by model name rather than by position, plus a unit test for the missing-model path. This has not been fixed yet; I found it while preparing this page, and it is exactly the kind of bug a test for the fallback path would have caught.

Evaluation methodology

The evaluation module compares models on a held-out forecast horizon with MAE, RMSE, MAPE and directional accuracy (whether the forecast gets the direction of change right). Directional accuracy matters for alerting, because operators care about whether things are getting worse.

Results from synthetic data say little about real networks, so they are not reported here. The next step is to run the same evaluation on real perfSONAR archives with rolling-origin backtesting.

Software engineering

  • Package layout under src/ with a pyproject.toml, separate modules for data, models, evaluation, early warning, reporting and the API.
  • Seven pytest modules covering data generation, preprocessing, models, evaluation, early warning, reporting and the API.
  • Repeatable setup: declared dependencies, a Makefile for common tasks, and a Docker image.
  • Kubernetes manifests for the API deployment, service, configuration and a GPU training job.
  • Documentation of the architecture and deployment in docs/.

Challenges and solutions

  • No access to real data. A seeded generator with realistic seasonality and injected anomalies made it possible to build and test the whole pipeline, including the alerting path, while being explicit that results are not real-world evidence.
  • Very different model interfaces. Statistical and neural models fit, predict and fail differently. A common forecasting interface kept the rest of the system simple, but the fallback path for a missing model needed more care than it got (see the weight-alignment bug above).
  • Making alerts actionable. A raw threshold breach is not enough for an operator, so warnings are turned into reports with severity, likely causes and suggested remediation.

Lessons learned

  1. Building the boring parts (tests, configuration, deployment) first made every later model change cheaper and safer.
  2. Synthetic data is excellent for engineering and dangerous for conclusions. Separating the two keeps a project honest.
  3. Directional accuracy is often more useful than raw error for operational forecasting.

Repository

The code, tests and documentation are on GitHub.

100%