Forecast Benchmark

Model forecasts vs. operator forecasts

The GridPulse column scores the model named in each row; the published default may be a different model or baseline. Operator comparisons use EIA-930 forecasts and latest-observed demand on matched hours.

Forecast leads are not fully matched. This measures demand error, not operational value.

Authorities with scored rows

47 / 51

3 excluded for fairness, 1 still accumulating — each with a published reason; 46 in the aggregate group

Operator forecast error

3.63%

median per-authority MAPE · 46 authorities

GridPulse model error

4.94%

median per-authority MAPE · 46 authorities

30-day model-comparison benchmark (~24h lead), across 46 operators: GridPulse model median error 4.94%; operator forecasts 3.63%. GridPulse’s median error is 1.31 percentage points higher.Lower error by operator: GridPulse on 17 of 46; operator forecasts on 29 of 46.

More comparison statistics

Best-to-worst per-operator error ratio: GridPulse 8.5×; operator forecasts 41.9×. Re-scored on median APE or WAPE instead, the verdict is the same.

GridPulse model comparison vs each grid operator’s own day-ahead forecast, nominal 24-hour lead, 30-day window. MAPE on paired hours against settled actuals. This does not score the published fallback baseline.
BA	Model scored	Operator error	Model error	Closer
AECI	Ensemble	4.77%	4.90%	Operator forecast
AVA	Ensemble	1.39%	2.70%	Operator forecast
BANC	Ensemble	2.10%	6.63%	Operator forecast
BPAT	Ensemble	1.23%	3.80%	Operator forecast
CAISO	Ensemble	9.69%	5.51%	GridPulse model
CHPD	Ensemble	2.63%	3.63%	Operator forecast
CPLE	Ensemble	3.26%	5.12%	Operator forecast
CPLW	Ensemble	2.60%	4.46%	Operator forecast
DOPD	Ensemble	3.15%	2.80%	GridPulse model
DUK	Ensemble	1.77%	4.86%	Operator forecast
EPE	Ensemble	3.19%	5.21%	Operator forecast
ERCOT	Ensemble	1.44%	4.25%	Operator forecast
FMPP	Ensemble	3.74%	5.17%	Operator forecast
FPC	Ensemble	24.64%	5.48%	GridPulse model
FPL	Ensemble	3.25%	5.42%	Operator forecast
GCPD	Ensemble	3.12%	3.01%	GridPulse model
GVL	Ensemble	12.56%	6.00%	GridPulse model
HST	Ensemble	6.33%	7.66%	Operator forecast
IID	Ensemble	6.30%	11.15%	Operator forecast
IPCO	Ensemble	4.27%	3.43%	GridPulse model
ISONE	Ensemble	3.59%	4.34%	Operator forecast
JEA	Ensemble	4.89%	5.92%	Operator forecast
LGEE	Ensemble	6.54%	5.08%	GridPulse model
NEVP	Ensemble	8.21%	4.66%	GridPulse model
NWMT	Ensemble	1.93%	3.23%	Operator forecast
NYISO	Ensemble	3.04%	4.37%	Operator forecast
PACE	Ensemble	8.60%	4.64%	GridPulse model
PACW	Ensemble	2.44%	2.94%	Operator forecast
PGE	Ensemble	1.26%	2.32%	Operator forecast
PJM	Ensemble	2.70%	3.29%	Operator forecast
PNM	Ensemble	5.74%	2.64%	GridPulse model
PSCO	Ensemble	25.11%	8.65%	GridPulse model
PSEI	Ensemble	51.43%	3.45%	GridPulse model
SC	Ensemble	4.44%	4.23%	GridPulse model
SCEG	Ensemble	4.78%	5.38%	Operator forecast
SCL	Ensemble	3.67%	4.95%	Operator forecast
SEC	Ensemble	22.02%	19.01%	GridPulse model
SOCO	Ensemble	3.34%	6.08%	Operator forecast
SPP	Ensemble	6.18%	3.98%	GridPulse model
SRP	Ensemble	25.61%	19.76%	GridPulse model
TAL	Ensemble	3.71%	5.43%	Operator forecast
TEC	Ensemble	3.29%	5.36%	Operator forecast
TEPC	Ensemble	3.11%	5.87%	Operator forecast
TIDC	Ensemble	3.10%	4.93%	Operator forecast
TPWR	Ensemble	1.75%	3.73%	Operator forecast
TVA	Ensemble	2.21%	5.42%	Operator forecast
WALC	Ensemble	14.65%	10.23%	GridPulse model

The summary above is rendered server-side. The full sortable table, the per-BA drop counts and the exclusion detail come from the public JSON endpoint /api/v1/benchmark, which needs JavaScript to fetch — the endpoint itself is plain JSON and can be read directly.

Fleet

Per balancing authority

Each row compares the two forecasts' mean MAPE (the average miss, as a share of actual demand) over the paired hours in the 30-day window, at the nominal 24-hour lead. Lower error is better. Hours scored counts the hours where both forecasts and a usable actual exist. Those are the only hours either side is graded on. Expand a row to see which hours were dropped and why.

Excluded, and why

A balancing authority is left out only when it cannot be scored fairly — a feed whose readings revise too heavily to score intraday, or one that has stopped publishing a day-ahead forecast. Each row carries its reason.

Balancing authority	Reason	Why it cannot be scored fairly

Not yet scoreable — still accumulating

These balancing authorities qualify on every fairness rule but have not yet built up enough comparable hours for a verdict. Each day adds at most 24 newly scoreable hours per horizon, so a newly scored BA needs just over a week before it can appear above. A BA with no verdict yet has not accumulated evidence yet, which is not the same as losing.

Balancing authority	Paired hours so far	Needed for a verdict

What this does not show

Published so the numbers above can be checked. The full list lives in the methodology.

The 24-hour arm is not lead-matched

Operators submit day-ahead forecasts 17–41 hours out by EIA's documentation; ours run just under 24 (the measured range is under “How this is measured”). On a typical hour the operator plausibly forecast from further out than we did — on the very horizon every comparison here is taken from.

The hour set depends on our availability too

An hour is dropped when we have no matured prediction for it, even though the operator's forecast for that hour exists. Not corrected for; the drop counts are published per row instead.

This is not how operators forecast

The day-ahead value is what a BA publishes to EIA. It need not be the forecast it dispatches on, and several operate more sophisticated internal models. A large error here is evidence about the published series, nothing more.

Seven days does not mean complete coverage

The above band mark comes from a separate seven-day drift monitor. The table’s Hours scored counts paired observations in the 30-day benchmark; these are different samples. The drift count describes observed samples within a seven-day window, not seven complete days. Its 24h diagnostic uses an offset from the first forecast target, not an exact publication-to-target lead.

Repeated origins and missing inputs can leave target hours unscored. Available counters do not establish every cause of the missing hours. Read each row’s count as its sample size; the coverage investigation remains open.

Not a claim about market value

Demand error only. Nothing here is scored on prices, reserve margins, or dispatch outcomes, and two leads is not the whole horizon curve.

How this is measured

Comparison: ours vs the earliest day-ahead we observed
Truth: Latest-observed EIA demand — used for both sides; may still be revised
Applied to: both sides, same hours
Window: 30 days
Horizon: nominal 24h
Realized leadhours actually ahead: —
Verdict metric: mean MAPE — the one statistic that decides
Updated: —

The “24h” diagnostic selects a target 24 hours after the first forecast target. That offset does not establish an exact lead from forecast publication. The realized-lead field reports a measured generation-to-target interval where available.

Rules that shaped the numbers

Hours where EIA published the forecast as the actual are excluded — scoring them would credit the operator with a perfect prediction on hours it never made.
The operator's column is the earliest day-ahead forecast we observed, not necessarily the first they issued: a revision landing before we first saw the hour is invisible to us.
Both forecasts use the latest demand values captured from EIA. These can still be revised; no final-status flag is available.
Hours we first saw after they had already passed are excluded: their earliest-observed forecast is already a post-revision value, so scoring it as “as-issued” would erase the distinction the two operator columns exist to draw.
The operator's forecast is scored twice — earliest-observed and as-revised — because a value revised after the target hour carries hindsight. A row where the two scorings disagree about the winner is marked on its Difference cell; everywhere else they agree.
Verdicts are decided on mean MAPE alone. The other statistics are published for interpretation and decide nothing.
A row marked above band exceeds the configured error band for that forecast lead. This is an error assessment, not a record that a rollback occurred.
A row marked not served scores the ensemble while the dashboard displays a seasonal-naive baseline for that authority. This row does not score the displayed baseline.

Read the rules yourself

Methodology — every rule, and every known limit
Scoreability — which BAs qualify, and why not
Provenance — what we measured about our own inputs

Every number on this page is read from one public endpoint. Call it yourself:

GET /api/v1/benchmark