Transformer-Based Framework For Sequential And Probabilistic Energy Load Forecasting

Abstract

Modern power systems require accurate and uncertainty-aware forecasts of electricity demand to maintain stability and optimize resource scheduling. Traditional models such as ARIMA and recurrent neural networks (RNNs) often struggle to capture longrange temporal dependencies and to quantify prediction uncertainty. This paper presents a Transformer-based forecasting framework that integrates self-attention mechanisms with probabilistic quantile regression to model sequential dependencies in short-term and mediumterm energy-load data. Historical consumption, weather, and calendar attributes are pre-processed through regime-aware segmentation to enhance robustness across seasonal and behavioral variations. The proposed system generates both point and interval forecasts— providing operators with calibrated P10, P50, and P90 values—while simultaneously benchmarking against ARIMA, Random Forest, and Gradient Boosting models using deterministic (MAPE, RMSE, MAE) and probabilistic (CRPS, PICP) metrics. Expected results indicate a 20–25 % improvement in forecasting accuracy and a reduction of MAPE below 2.5 % for day-ahead and 5 % for week-ahead horizons. The unified framework demonstrates how Transformer architectures can advance data-driven energy analytics through reliable, explainable, and risk-sensitive forecasting.