Synthetic Streamflow Forecasting Benchmark

Read as a ranking, the table says gradient boosting beats classical statistics by a wide margin. Read against the code, the three rows are not answering the same question. XGBoost is scored one step ahead with the previous day’s measured discharge supplied as an input feature; SARIMAX forecasts 25 monthly means without seeing any of them; the baseline repeats a seasonal average that has no trend term at all.

Two things explain the table. The series was written rather than measured — a fixed-seed generator whose recursion is y[t] = 0.85·y[t−1] + …, so its day-to-day autocorrelation is 0.9941 and the two most recent lags carry 94.4% of the fitted model while rainfall carries 0.0007. And the baseline is undefended against trend: the same generator adds 0.005 a day, which its own feedback multiplies by 6.67, so the series climbs 89.91 between the training and test periods while the baseline’s mean bias is 89.94. Its MAE (89.935) and RMSE (90.299) are nearly equal because the error is an offset, not scatter.

Where it is worst is measurable rather than impressionistic. On the highest tenth of test days — observed flow above 266.28 — the mean absolute error is 5.591 against 2.562 everywhere else, and the signed mean is -5.506: the misses are undershoots. Those days are 10% of the holdout and carry 29% of the squared error. The worst single day observed 273.12 and came in 12.99 low.

Record

5,475 days

Holdout

730 days

Features

26

Horizon

One step

• Synthetic data, not a catchmentThe river was written by a 40-line generator with a fixed seed. Nothing here is evidence about any real basin, and the repository's own scope note says so first.
• One step ahead, with the answer in the featuresThe XGBoost score is next-day prediction given discharge already observed at prior timestamps. It is not a recursive multi-day forecast and was never evaluated as one.
• The three rows are not the same taskThe naive baseline and XGBoost are scored on 730 daily values; SARIMAX is scored on 25 monthly means it forecast without seeing any of them. A single table makes them look comparable.
• The baseline is undefended against trendA day-of-year mean has no trend term, and the generator injects one. Most of the gap between the baseline and everything else is that, not model quality.
• No uncertainty of any kindEvery prediction here is a point. The benchmark produces no intervals, no coverage and no calibration, so none is shown.
• One seed, one runThe tracked figures and JSON record a single reference run. The README notes that library versions may shift the numbers slightly.