КРАТКОСРОЧНОЕ ПРОГНОЗИРОВАНИЕ НАГРУЗКИ С ИСПОЛЬЗОВАНИЕМ ДАННЫХ ИНТЕЛЛЕКТУАЛЬНЫХ СЧЕТЧИКОВ И МАШИННОГО ОБУЧЕНИЯ

 

УДК 621.311.1

Abstract

Short-term load forecasting (STLF) for residential consumption poses a significant challenge due to uncertainty in consumption behavior and irregular appliance usage. This study proposes a machine learning (ML) framework that does not require external variables, utilizing 30-minute data from smart meters. Approximately 1 million supervised samples were created using the windowing method from 31 households in the public dataset collected from London households, enabling the generalization of consumption behaviors among users. Five ML models were compared to predict consumption 30, 60, and 120 minutes later. The results showed that ensemble-based methods delivered superior performance; XGBoost stood out with an MAE value of 0.0747 kWh/hh, and Random Forest with an RMSE value of 0.1561 kWh/hh. LightGBM showed high performance in terms of accuracy-computational cost with a training time of 12 seconds. LightGBM's MAE value was measured as 0.0749 kWh/hh for 30 min, 0.0876 for 60 min, and 0.0961 for 120 min. One significant issue affecting prediction performance is sudden load increases caused by appliances. Findings show that ML models produce accurate, stable, and computationally efficient predictions using residential consumption data. The proposed framework offers a robust and practical solution for real-time energy management, demand-side programs, and smart grid control applications.

Аннотация

Краткосрочное прогнозирование нагрузки (STLF) для жилого сектора представляет собой сложную задачу из-за неопределенности в поведении потребителей и нерегулярного использования бытовых приборов. В данном исследовании предлагается модель машинного обучения (ML), не требующая внешних переменных, основанная на данных интеллектуальных счетчиков с 30-минутным интервалом. Используя метод скользящего окна, было создано около 1 миллиона обучающих выборок на основе данных 31 домохозяйства из общедоступного набора данных (Лондон), что позволило обобщить модели потребления пользователей. Были сравнены пять моделей машинного обучения для прогнозирования потребления на 30, 60 и 120 минут вперед. Результаты показали, что ансамблевые методы продемонстрировали превосходную производительность: XGBoost показал лучший результат по показателю MAE (0,0747 кВт⋅ч/дом), а Random Forest — по показателю RMSE (0,1561 кВт⋅ч/дом). LightGBM показал высокую эффективность с точки зрения соотношения точности и вычислительных затрат, при этом время обучения составило 12 секунд. Значения MAE для модели LightGBM составили 0,0749 кВт⋅ч/дом для прогноза на 30 минут, 0,0876 — на 60 минут и 0,0961 — на 120 минут. Одной из существенных проблем, влияющих на точность прогнозирования, являются резкие скачки нагрузки, вызванные включением бытовых приборов. Результаты исследования показывают, что модели машинного обучения позволяют получать точные, стабильные и вычислительно эффективные прогнозы на основе данных о потреблении в жилом секторе. Предложенная модель представляет собой надежное и практическое решение для управления энергопотреблением в режиме реального времени, программ управления спросом и систем контроля интеллектуальных сетей.

 

Keywords: Short-term load forecasting; smart meter data; machine learning; time series forecasting.

Ключевые слова: краткосрочное прогнозирование нагрузки; данные интеллектуальных счетчиков; машинное обучение; прогнозирование временных рядов.

 

1. Introduction

Short-term load forecasting (STLF) has become a critical component in modern power systems with the rapid proliferation of distributed energy resources, rooftop-scale photovoltaic systems, and electric vehicles [1], [2]. Accurately prediction of load fluctuations occurring at the distribution level is crucial for demand-side management, dynamic pricing, flexible resource planning, and microgrid operation [3]. The half-hourly or higher resolution consumption data provided by smart meters has enabled the development of data-driven approaches to understand consumption behavior at the residential level [4], [5].

Various STLF approaches based on both machine learning and deep learning exist in the literature [6], [7]. Machine learning-based models such as Gradient Boosting Machines (GBM), XGBoost, Random Forest, and Support Vector Regression stand out with their high accuracy, low computational cost, and interpretability advantages [8], [9], [10]. More complex models such as LSTM, GRU, CNN-LSTM, and Transformer-based deep learning architectures are successful in capturing long-term dependencies [11], [12]. However, high data requirements, training costs, and excessive volatility at the individual meter level prevent them from always delivering superior performance in practical applications [13], [14], [15] .

The majority of studies conducted to date have focused on models powered by large-scale multi-meter data [16]. However, the electricity consumption time series of a single residence presents a more challenging problem due to random user behavior, sudden spikes in device usage, low repeatability, and the absence of external variables [5], [7], [17]. Although some studies have proposed supporting methods such as transfer learning [18], profile similarity analysis [19], early-day classification [20], or data set reduction techniques [21], most of these approaches still require a large amount of multi-meter data.

The difficulty of the residential STLF problem is not limited to the noisy and low repeatability of the data structure; consumption patterns are also closely tied to user habits, daily routines, and the operational dynamics of devices [22], [23]. Therefore, the ability of past 24-hour information to represent future short-term consumption is often limited, especially as the timing of sudden power demands becomes largely unpredictable [24]. Furthermore, when external variables such as meteorological data or tariff-based behavioral signals are not used, whether models based solely on raw smart meter data have sufficient generalization capacity emerges as an important research question[25], [26]. These reasons make it critical to evaluate the competitiveness of lightweight machine learning methods with deep models in terms of both data efficiency and computational cost [27], [28].

Limitations have been observed in studies evaluating the performance of lightweight models based solely on individual residential consumption and not utilizing external variables. In particular, systematic machine learning comparisons using sliding window sample sets generated from multiple meters with 30-minute resolution smart meter data have rarely been addressed in the literature. This study aims to fill this gap in the literature and comprehensively evaluates five machine learning models (LightGBM, XGBoost, Random Forest, Linear Regression, and kNN) for multiple time horizons under a windowing framework derived from half-hourly time series of 31 randomly selected residential properties from the London LCL dataset.

The main contributions of this study are as follows:

I. At the residential level only, an STLF framework that does not require external variables has been presented using 30-minute resolution smart meter data.

II. Approximately 1 million samples were generated from multi-residential data using windowing technique, and the performance of 5 different machine learning models was systematically compared for 3 different horizons (30, 60, 120 min).

III. Model accuracy, computational cost, horizon sensitivity, and residual analyses were investigated in detail.

IV. The robustness of strong and lightweight ML models under high volatility in individual meter consumption was demonstrated, and important implications for practical applications were presented.

2. Dataset and Environment

2.1.Smart Meter and Household Loads

The consumption data used in this study consists of smart meter measurements collected as part of the “Low Carbon London (LCL)” project in the United Kingdom. The LCL dataset contains half-hourly electricity consumption values obtained from 5,567 households between 2011 and 2014. Smart meters measure the instantaneous electricity consumption of the entire household and transmit the total consumption (kWh/hh) information for each 30-minute time interval to the central system.

Residential electricity loads are evaluated under three main categories: base loads, appliance loads, and seasonal loads. Base loads include outlets, lighting elements, and electronic devices; appliance loads cover periodically operating devices such as washing machines, ovens, refrigerators, and irons. Seasonal loads consist of heating and cooling systems and cause consumption patterns that vary significantly with the seasons. All of these load types are measured by the Smart Meter as a single time series.

Fig. 1 visually presents the system architecture used in this study, showing the smart meter infrastructure and residential load distribution.

 

Figure 1 Smart meter data acquisition and household electrical load distribution

 

2.2.Dataset Description

This study uses time series data from 31 randomly selected residences in the LCL dataset. Consumption values (kWh/hh) are available at 30-minute intervals for each residence. The main columns in the dataset are as follows:

LCLid          : Meter ID

stdorTOU   : Time-of-use classification for the tariff

DateTime    : Half-hourly time stamp

KWH/hh     : Energy consumption during the relevant 30-minute period

During the data preprocessing phase, the date-time field was converted to DateTime format, missing or incorrect values are marked as NaN, and the data is properly cleaned before analysis. All counters are combined based on the timestamp and aligned to a common time axis.

2.3.Windowing and Sample Generation for STLF

Fig. 2 shows the annual original consumption distributions for the 31 residences used in the study. Each box plot represents the minimum–maximum range, median, and quartile values of the annual consumption for a smart meter. The graphic shows that there is significant consumption diversity among the residences; consumption is quite low in some households and higher in others. This situation indicates that high heterogeneity due to user behavior must be considered in model training.

 

Figure 2 Annual consumption distribution of 31 households in the LCL dataset

 

Figure Y shows the hourly average consumption distribution of data obtained from all residences. A typical residential consumption profile has been observed, showing a moderate increase in the morning hours, a relatively low level in the afternoon, and a distinct peak in the evening hours. The width of the hourly box plots indicates that user behavior is much more variable, especially in the evening hours, making the forecasting problem more challenging during these time periods. This analysis confirms that the robustness of STLF models to hourly behavioral variability is critical.

 

Figure 3 Hourly consumption distribution across 31 households in the LCL dataset

 

2.4.Consumption Characteristics of the LCL Smart Meter Dataset.

Raw time series data cannot be used directly for short-term load forecasting; therefore, within the scope of this study, all smart meter consumption series were converted into supervised learning examples using a sliding window method. In this configuration, the input of each example is defined as a vector 𝑥_𝑡 consisting of the last 48 time steps (24 hours of past consumption), and the output is defined as the consumption value 𝑦_𝑡+𝐻 at the next step H. Prediction horizons of 30, 60, and 120 minutes were created for 𝐻=1,2,4, respectively; thousands of windows were generated for all meters, producing a total of approximately one million examples. This enabled the models to learn both individual meter dynamics and heterogeneous consumption behaviors between households. The general flow of this windowing process and the transformation of the data structure are summarized visually in Fig. 4.

Raw time series data cannot be provided directly to the model for short-term load forecasting. Therefore, using the sliding window approach, each sample is defined as follows:

Input (X): Last 48 time steps

48 × 30 min = 24 hours of past consumption

Output (y): Consumption at the next H step

H = 1 → 30 minutes later

H = 2 → 60 minutes later

H = 4 → 120 minutes later

Thousands of samples are created for each meter using this method, resulting in approximately 1 million window samples when all meters are combined. This large sample set enables the models to learn a more general consumption pattern rather than individual meter behavior.

The time series windowing process is formulated as follows:

Input vector:

                            

Prediction target:

Where

 : consumption value at time t,

: horizon (1, 2 or 4),

: represents the 48-dimensional input vector given to the model.

This definition forms the basic STLF structure of the study.

 

Figure 4. Overview of the dataset structure and the sliding-window mechanism used to generate supervised learning samples. The raw 30-minute smart-meter time series is transformed into 48-step input windows (representing the previous 24 hours), and the forecasting

 

3. Methodology

3.1. Window-Based Short-Term Load Forecasting Framework

This study is based on converting short-term load forecasting into supervised learning examples using the sliding-window method on raw smart meter time series and training them with different machine learning models. The general methodology consists of the following steps: (i) data windowing, (ii) model training, (iii) multi-horizon forecasting, and (iv) performance evaluation. The overall flow of this process is summarized in Figure 5.

 

Figure 5. Overview of the proposed methodology, which converts raw smart meter time series into supervised learning samples using the sliding-window approach, trains multiple machine learning models, and produces multi-horizon short-term load forecasts

 

3.2.Machine Learning Models

In this study, five different machine learning models are evaluated for short-term residential electricity consumption forecasting. The models are selected to encompass both lightweight and computationally fast methods as well as more advanced ensemble-based models. The theoretical structure, mathematical expressions, and working principle of each model are presented below.

a. Gradient Boosting Models

Gradient Boosting is a powerful ensemble learning method based on the principle of sequentially training multiple weak learners, typically shallow decision trees, with each learner correcting the errors of the previous model. This structure minimizes the total model error by making iterative improvements in the direction of the negative gradient of the loss function.

The general logic of Gradient Boosting can be summarized in four steps:

(i) a simple initial model is constructed,

(ii) the prediction errors (residuals) produced by the current model are calculated,

(iii) a new decision tree is trained to predict these errors,

(iv) the trained tree is added to the current model with a small learning rate.

This process continues over many iterations, and the resulting final model is the sum of the scaled predictions of all trees:

The general formulation is as follows:

                             (2)

Where:

: m. weak learner (decision tree),

: total number of trees.

Gradient Boosting stands out for its high accuracy, especially with complex, non-linear, and noisy datasets. It is resistant to overfitting with its regularization mechanisms and offers strong interpretability in terms of feature importance evaluation. However, its computational cost is higher than other methods, and hyperparameter tuning requires careful attention.

The negative gradient of the loss function is learned in each iteration:

                          (3)

The Gradient Boosting mechanism relies on sequentially training each weak learner to reduce the error terms of the previous model. This sequential error correction process and the model's collective structure are shown schematically in Fig. 6.

 

Figure 6. Conceptual illustration of the Gradient Boosting framework

 

(1) LightGBM

LightGBM is an optimized boosting algorithm developed by Microsoft Research that aims to efficiently implement gradient boosting decision trees (GBDT). Designed to overcome the limitations of traditional boosting methods, it offers significant advantages in terms of speed and memory usage, particularly for large-scale datasets.

LightGBM's key innovations are based on two main components: a leaf-wise tree growth strategy and a histogram-based splitting algorithm. Leaf-wise growth enables the model to learn more aggressively by expanding the leaf that minimizes loss the most, and this approach often produces fast convergence with fewer levels but deeper trees. Histogram-based splitting significantly reduces the cost of searching for the optimal splitting point by dividing continuous variables into predefined bins.

LightGBM also uses additional mechanisms such as gradient-based sampling techniques and feature subsampling to improve efficiency and reduce overfitting. Furthermore, its ability to naturally handle categorical variables, its architecture suitable for parallel and distributed computing, and its L1–L2 regularization support make the algorithm a flexible solution that delivers high accuracy.

These features make LightGBM stand out among gradient boosting algorithms in terms of speed, scalability, memory efficiency, and prediction accuracy, making it a suitable option for short-term load forecasting in large-scale smart meter time series.

Nodes are continuously expanded in the direction that maximizes loss reduction:

Leaf-wise Split Criterion

                           (4)

Advantages of LightGBM:

• Very fast on large data sets
• High accuracy with deep trees
• Low memory requirements

(2) XGBoost

Extreme Gradient Boosting (XGBoost ) is an optimized version of gradient boosting decision trees and was developed specifically to provide high accuracy, speed, and stability on large-scale datasets. The algorithm extends standard boosting methods with advanced regularization, second-order gradient information, and efficient tree-building strategies to offer a more powerful and general learning mechanism.

XGBoost's key innovation is optimizing through gradient and Hessian information using a second-order Taylor expansion of the loss function. This approach enables more accurate evaluation of tree splits and contributes to faster model convergence. Additionally, L₁ and L₂ regularization terms balance leaf weights, reducing overfitting.

The “shrinkage” , “column subsampling” , and “pruning” mechanisms used for tree structure optimization help reduce the algorithm's computational cost and increase model robustness. XGBoost also delivers high performance with parallel tree building, sparse data processing, and cache-optimized design.

These features make XGBoost a widely adopted boosting method for regression and classification problems, producing successful results in complex, noisy, and nonlinear time series problems such as short-term load forecasting.

XGBoost Objective Function

                               (5)  

                                 (6)

Here:

• : Number of nodes in the leaf
• : Leaf weights
• : Regularization parameters

XGBoost grows decision trees using a level-wise strategy, expanding all leaves of the same depth in each iteration. This approach ensures more balanced and stable model growth but incurs high computational costs. LightGBM, on the other hand, uses a leaf-wise strategy, expanding only the leaf with the highest gain. This mechanism produces deeper, more selective, and generally faster-converging trees; it provides a significant speed advantage on large datasets. Fig. 7 shows a comparative view of the growth structures of the two algorithms.

 

Figure 7 Structural comparison of XGBoost and LightGBM tree growth strategies

 

b. Bagging-Based Model

(3) Random Forest

Random Forest takes the average of numerous decision trees created using bootstrap sampling and feature subsampling:

Random Forest Prediction:

                                      (7)

Here:

• : total number of trees
• : decision tree trained with bootstrap

Principle of Variance Reduction

                             (8)

RF is extremely robust, especially in noisy time series with sudden consumption spikes.

Random Forest is an ensemble method based on the bagging (bootstrap aggregation) principle. Random and resampled subsets are created from the training data; each subset is learned with an independent decision tree. Since the trees are independent of each other, model variance is significantly reduced and high robustness against noisy time series is achieved. During the prediction phase, the outputs of all trees are combined using averaging or majority voting to produce the final result. Fig. 8 schematically summarizes this multi-tree structure and the combined prediction process.

 

Figure 8. Random Forest architecture. Multiple bootstrap-sampled decision trees are trained independently, and their outputs are combined through majority voting or averaging to obtain the final prediction

 

c. Baseline Models

(4) Linear Regression

Linear regression is a simple yet powerful baseline that models the linear relationship between the consumption vector and the target.

Linear Model

                                         (9)

Model parameters are learned using the following optimization:

                      (10)

Advantages:

• Interpretable
• Very fast
• Ideal for baseline comparison

(5) k-Nearest Neighbors (kNN)

kNN is a parameter-free (lazy learner) method. Predictions are calculated using the average of the nearest neighbors.

kNN Regression

                                        (11)

The Euclidean metric is generally used for distance measurement:

                    (12)

kNN is quite effective on low-dimensional data sets, but computational cost increases on large data sets.

Table 1. Summary Of The Machine Learning Models Evaluated in This Study

Model	Learning Principle	Advantages	Limitations
LightGBM	Leaf-wise gradient boosting, histogram-based splitting	High accuracy, very fast training, low memory usage, suitable for large data	Risk of overfitting; requires careful hyperparameter tuning
XGBoost	Regularized gradient boosting with second-order optimization	Stable performance, strong regularization, handles noisy data well	Higher computational cost; slower than LightGBM
Random Forest	Bagging with bootstrap-sampled decision trees	Robust to noise, reduced variance, strong generalization	Large model size; less interpretable than boosting models
Linear Regression	Linear mapping between inputs and output	Simple, interpretable, extremely fast	Cannot model nonlinear patterns; limited performance on volatile load series
kNN Regression	Instance-based learning using distance metrics	No training time, effective for small datasets, sensitive to local patterns	High inference cost, performance decreases on large datasets and high dimension

 

4. Experimental Setup

This section details the hardware-software environment used to conduct the study, the data processing steps, the training-test split, model configurations, and evaluation criteria. All experiments were performed on the same computational environment, using a standardized pipeline, to ensure comparability between models.

4.1. Hardware and Software Environment

Experimental studies were conducted on a computer capable of meeting the computational requirements of the machine learning models selected for short-term load forecasting. The hardware and software components used are summarized below:

CPU: 8-core Intel Core i9 processor

RAM: 32 GB

GPU: Not used (all models were trained on the CPU)

Operating System: Windows 11 / Ubuntu 22.04

Python Version: Python 3.13 (Conda environment)

Libraries Used: NumPy, Pandas, Scikit-Learn, LightGBM, XGBoost, Matplotlib

This environment is fully compatible with the lightweight, fast, and GPU-free nature of the machine learning models used in the study.

All samples created using sliding window sampling are arranged in chronological order, and no random shuffling was performed to preserve time series properties. Training–test split:

80% training data

20% test data

was performed using time-based splitting.

Separate models were trained for three different prediction horizons:

H = 1 → 30 minutes later

H = 2 → 60 minutes later

H = 4 → 120 minutes later

Each horizon was treated as an independent learning problem, and the models were retrained based on the horizon.

	LightGBM	XGBoost	Random Forest	Linear Regression	kNN
Training Duration	12 s	16-18 s	60-120 s
	n_estimators = 300 learning_rate = 0.05 max_depth = −1 num_leaves = 31 subsample = 0.9 feature_fraction = 0.9	n_estimators = 400 learning_rate = 0.05 max_depth = 6 subsample = 0.9 colsample_bytree = 0.9 reg_lambda = 1 reg_alpha = 0	n_estimators = 300 max_depth = None max_features = “sqrt”	Ordinary Least Squares (OLS)	k = 5 distance metric = Euclidean

 

4.2. Evaluation Metrics

Three common regression metrics were used to evaluate model performance.

Mean Absolute Error (MAE):

                             (13)

Root Mean Squared Error (RMSE):

                       (14)

Symmetric Mean Absolute Percentage Error (sMAPE):

                 (15)

Here  has been added to prevent division by zero errors.

The proposed LightGBM-based STLF framework exhibits strong predictive performance across all evaluated short-term horizons. For the 15-minute-ahead forecast, the model achieves an MAE of 0.0749 kWh/hh and an RMSE of 0.1583 kWh/hh, indicating high accuracy in capturing short-term consumption dynamics. As the prediction horizon extends to 30 and 60 minutes, the error metrics increase gradually and consistently (MAE rising to 0.0876 and 0.0961 kWh/hh, respectively). This monotonic performance degradation is fully aligned with the inherent increase in uncertainty at longer horizons in household electricity consumption. Overall, the results confirm that the LightGBM framework maintains robust and generalizable forecasting capability even under progressively more challenging prediction intervals.

5. Results and Discussion

The LightGBM model accurately tracks the real-time consumption curve for the 30-minute prediction horizon shown in Fig. 9. The model produced predictions very close to the actual data, particularly in low and medium consumption areas during the day, thus successfully capturing the overall patterns. However, as seen at the points marked with red markers in the graph, the model predicted sudden and high-amplitude consumption spikes behind the actual data. This situation stems from the fact that sudden device activation behaviors (e.g., kettle, oven, heater) frequently seen at the residential level cannot be fully extracted from the past 24-hour window information.

Nevertheless, despite the partial delay at peak points, LightGBM rapidly approaches the actual value during the declines following spikes and reduces the prediction error in a short time. This demonstrates the model's robustness against high-variance consumption patterns. Overall, the LightGBM model stands out for its low absolute error and strong overall trend capture success for a 30-minute prediction horizon; it produces highly stable and reliable predictions, especially outside of sudden fluctuations.

 

Figure 9 LightGBM model predictions for the 30-minute forecasting horizon

 

The residual distribution is presented in Fig. 10. The histogram reveals that the vast majority of errors are concentrated around 0 and that the distribution exhibits a symmetric structure. This indicates that the LightGBM model does not exhibit a systematic over- or under-prediction bias, and that prediction errors are low in magnitude and stable. The relatively narrow distribution confirms that the model performs consistently even on noisy residential consumption data. Furthermore, the limited number of outliers indicates that the model only amplifies errors during sudden consumption spikes, while successfully capturing general consumption patterns otherwise.

 

Figure 10. Residual distribution for the 30-minute LightGBM forecast

 

Fig. 11 shows the prediction performance of the LightGBM model for load values 60 minutes ahead. Compared to the 30-minute horizon, the differences between the prediction curve and actual consumption are seen to increase significantly. In particular, the effect of high-frequency oscillations and short-term consumption spikes negatively impacts the model's prediction accuracy as the horizon lengthens. The model successfully captures general daily patterns (low consumption at night – medium level during the day – peak loads in the evening); however, delays in the timing of high peaks and weak amplitude estimation at peak values are noticeable. This situation stems from the limited predictability of sudden device usage behaviors in individual residential consumption for a 1-hour forecast period.

Nevertheless, even if LightGBM misses the peak points, it manages to preserve the trend structure and shows convergence towards real values in the decline regions following the peak. This demonstrates that the model can accurately represent structural consumption trends even under a long horizon. Overall, LightGBM captured short-term patterns at an intermediate level for the 60-minute horizon; it struggled to predict sudden and sharp load increases but consistently modeled the continuous consumption characteristic. These results show that error naturally increases as the horizon lengthens, but the model maintains its stability.

 

Figure 11. LightGBM model predictions for the 60-minute forecasting horizon

 

Figure 12. Residual distribution for the 60-minute LightGBM forecast

 

Fig. 13 shows the short-term load forecasting performance of the LightGBM model 2 hours later (H = 4) and the residual distribution obtained for these forecasts. As the horizon increases, uncertainty rises, and the model is seen to fail to accurately capture the response time, particularly for sudden peak loads, and systematically underestimate peak magnitudes. In contrast, the model is quite stable across baseline consumption levels and successfully tracks consumption patterns in low load regions. This indicates that while short-term appliance-driven surges become harder to predict at longer horizons, the overall consumption trend is meaningfully preserved.

The error distribution (Figure Y) reveals that the vast majority of residual values are concentrated around zero and that the distribution exhibits a slightly right-skewed structure. The main reason for the skewness is the systematic underestimation of high consumption peaks. However, the narrow residual width indicates that the model consistently captures the general structure even in 2-hour forward forecasts and avoids large generalization errors. These results confirm that LightGBM exhibits strong robustness for short-term residential load forecasting even when the horizon increases.

 

Figure 13. LightGBM model predictions for the 120-minute forecasting horizon

 

Figure 14. Residual distribution for the 120-minute LightGBM forecast

 

The results show that short-term load forecasting performance varies significantly among five different machine learning models. According to Table 1, XGBoost achieved the lowest MAE value (0.07466), making it the most successful model in terms of absolute error. In contrast, Random Forest provided the lowest RMSE value thanks to its strong reduction in model variance and stood out in terms of the magnitude of prediction errors.

LightGBM was evaluated as the most efficient model in terms of speed-accuracy balance due to both its short training time and low level of errors. On the other hand, Linear Regression showed moderate performance due to the limiting effect of its linear assumptions; kNN produced the weakest results due to the high-dimensional data structure.

In general, ensemble-based models (LightGBM, XGBoost, Random Forest) were found to provide more stable and lower-error results in residential energy consumption time series containing noise and sudden jumps. This situation demonstrates that boosting and bagging methods provide a clear advantage in capturing non-linear consumption patterns at the residential level.

Table 2. Performance Comparison of The Five Machine Learning Models

Model	MAE	RMSE	sMAPE	Açıklama
LightGBM	0.07521	0.07521	0.07521	En Hızlı
XGBoost	0.07466	0.07466	0.07466	MAE lideri
Random Forest	0.07490	0.07490	0.07490	RMSE lideri, Yavaş
Linear Regression	0.08154	0.08154	0.08154	Orta
kNN	0.08347	0.08347	0.08347	Zayıf

 

Conclusion

This study presents a comprehensive short-term load forecasting (STLF) framework using only 30-minute smart meter measurements from residential households. The large-scale sample set (approximately 1 million samples from 31 residential units) created using a sliding window approach enabled the systematic comparison of five machine learning models under three different prediction horizons (30, 60, 120 minutes). Experimental analyses revealed a number of important findings and practical implications.

First, the results show that ensemble-based models (LightGBM, XGBoost, and Random Forest) consistently outperformed baseline methods (Linear Regression and kNN) across all horizons. XGBoost achieved the lowest MAE value (0.07466), making it the most successful model in terms of absolute error. Random Forest produced the lowest RMSE value thanks to its low-variance prediction structure and stood out in terms of error magnitude. LightGBM provided the best balance between accuracy and computational cost, offering advantages for large-scale applications and real-time predictions by maintaining high accuracy with the fastest training time.

Furthermore, horizon-based performance evaluation shows that error values increase steadily and consistently. For LightGBM, MAE rose from 0.0749 at a 30-minute horizon to 0.0876 at 60 minutes and 0.0961 at 120 minutes. This increase is consistent with the natural uncertainty inherent in short-term residential consumption. Despite the increasing horizon, the models successfully preserved the daily consumption structure (low consumption at night, moderate consumption during the day, evening peaks); however, all models systematically underestimated sharp load spikes associated with sudden device usage. This indicates that behavioral randomness in individual residential consumption complicates estimation, independent of model complexity.

Residual analyses confirmed model robustness. The majority of error distributions were concentrated around zero, with no systematic over- or under-estimation bias observed. Even with an extended horizon, the spread of residuals remained narrow, with error growth observed only during extreme peak moments. This demonstrates that the models successfully maintained structural integrity even on highly volatile residential consumption data.

Overall, the findings show that lightweight machine learning models; particularly gradient boosting-based approaches offer robust, stable, and computationally efficient solutions for short-term residential load forecasting, even without the use of external variables. The study confirms that large-scale window-based sampling from multiple residences helps learn the overall consumption pattern despite high randomness at the individual meter level.

Future work may focus on:

(i) incorporating external variables such as temperature, workday, occupancy,

(ii) hybrid use of deep learning + ensemble structures,

(iii) transfer learning-based rapid adaptation for new meters,

(iv) integration of probabilistic forecasting methods providing uncertainty analysis.

The results obtained demonstrate that the proposed framework offers a scalable, data-driven, and application-ready STLF solution in modern smart grid environments.

 

References:

1. W. Shi, Y. F. Wang, and T. Wang, “A short-term residential load forecasting framework based on dynamic spatio-temporal graph neural networks,” Energy Build, vol. 347, 2025, doi: 10.1016/j.enbuild.2025.116311.
2. F. L. Quilumba, W. J. Lee, H. Huang, D. Y. Wang, and R. L. Szabados, “Using smart meter data to improve the accuracy of intraday load forecasting considering customer behavior similarities,” IEEE Trans Smart Grid, vol. 6, no. 2, 2015, doi: 10.1109/TSG.2014.2364233.
3. A. K. Aktar and M. Karakiliç, “Enhancement of Grid Sustainability by Integrating Renewable Energy Sources and Power-to-Gas (P2G)Facilities,” in Proceedings - 2025 IEEE 7th Global Power, Energy and Communication Conference, GPECOM 2025, 2025. doi: 10.1109/GPECOM65896.2025.11061959.
4. H. Li, M. Heleno, W. N. Zhang, L. R. Garcia, and T. Z. Hong, “A cross-dimensional analysis of data-driven short-term load forecasting methods with large-scale smart meter data,” Energy Build, vol. 344, 2025, doi: 10.1016/j.enbuild.2025.115909.
5. A. Mohammadi, G. Nápoles, and Y. Salgueiro, “Deep Learning for Time-Series Forecasting With Exogenous Variables in Energy Consumption: A Performance and Interpretability Analysis,” IEEE ACCESS, vol. 13, pp. 86746–86767, 2025, doi: 10.1109/ACCESS.2025.3570618.
6. P. Koukaras, A. Mustapha, A. Mystakidis, and C. Tjortjis, “Optimizing Building Short-Term Load Forecasting: A Comparative Analysis of Machine Learning Models,” Energies (Basel), vol. 17, no. 6, 2024, doi: 10.3390/en17061450.
7. J. V. J. Melo, G. R. S. Lira, E. G. Costa, A. F. L. Neto, and I. B. Oliveira, “Short-Term Load Forecasting on Individual Consumers,” Energies (Basel), vol. 15, no. 16, 2022, doi: 10.3390/en15165856.
8. S. M. Sulaiman, P. A. Jeyanthy, D. Devaraj, and K. Shihabudheen, “A novel hybrid short-term electricity forecasting technique for residential loads using Empirical Mode Decomposition and Extreme Learning Machines,” COMPUTERS & ELECTRICAL ENGINEERING, vol. 98, 2022, doi: 10.1016/j.compeleceng.2021.107663.
9. D. Syed, S. S. Refaat, H. Abu-Rub, and O. Bouhali, “Short-term Power Forecasting Model Based on Dimensionality Reduction and Deep Learning Techniques for Smart Grid,” in 2020 IEEE Kansas Power and Energy Conference, KPEC 2020, 2020. doi: 10.1109/KPEC47870.2020.9167560.
10. M. Karakilic, H. Hatas, and I. Pacal, “Open-Circuit Fault Detection in T-Type MLI Using XGBoost: A Machine Learning-Based Approach,” in ICHORA 2025 - 2025 7th International Congress on Human-Computer Interaction, Optimization and Robotic Applications, Proceedings, 2025. doi: 10.1109/ICHORA65333.2025.11017205.
11.  L. Wang et al., “Optimal allocation of customer energy storage based on power big data and improved LSTM load forecasting,” Energy Reports, vol. 11, 2024, doi: 10.1016/j.egyr.2024.03.046.
12. M. Abumohsen, A. Y. Owda, and M. Owda, “Electrical Load Forecasting Using LSTM, GRU, and RNN Algorithms,” Energies (Basel), vol. 16, no. 5, 2023, doi: 10.3390/en16052283.
13. T. N. Da, M. Y. Cho, and P. N. Thanh, “Hourly load prediction based feature selection scheme and hybrid CNN-LSTM method for building’s smart solar microgrid,” Expert Syst, vol. 41, no. 7, 2024, doi: 10.1111/exsy.13539.
14. M. Jurado, M. Samper, and R. Rosés, “An improved encoder-decoder-based CNN model for probabilistic short-term load and PV forecasting,” ELECTRIC POWER SYSTEMS RESEARCH, vol. 217, 2023, doi: 10.1016/j.epsr.2023.109153.
15. C. H. Lee, P. N. Thanh, C. T. Yeh, and M. Y. Cho, “Three-Phase Load Prediction-Based Hybrid Convolution Neural Network Combined Bidirectional Long Short-Term Memory in Solar Power Plant,” INTERNATIONAL TRANSACTIONS ON ELECTRICAL ENERGY SYSTEMS, vol. 2022, 2022, doi: 10.1155/2022/2870668.
16. M. Hertel et al., “Transformer training strategies for forecasting multiple load time series,” Energy Informatics, vol. 6, 2023, doi: 10.1186/s42162-023-00278-z.
17. A. M. Alonso, F. J. Nogales, and C. Ruiz, “A Single Scalable LSTM Model for Short-Term Forecasting of Massive Electricity Time Series,” Energies (Basel), vol. 13, no. 20, 2020, doi: 10.3390/en13205328.
18. I. Ozer, S. B. Efe, and H. Ozbay, “A combined deep learning application for short term load forecasting,” ALEXANDRIA ENGINEERING JOURNAL, vol. 60, no. 4, pp. 3807–3818, 2021, doi: 10.1016/j.aej.2021.02.050.
19. D. Syed et al., “Deep Learning-Based Short-Term Load Forecasting Approach in Smart Grid With Clustering and Consumption Pattern Recognition,” IEEE ACCESS, vol. 9, pp. 54992–55008, 2021, doi: 10.1109/ACCESS.2021.3071654.
20. Z. Chen, Y. B. Chen, T. Xiao, H. L. Wang, and P. W. Hou, “A novel short-term load forecasting framework based on time-series clustering and early classification algorithm,” Energy Build, vol. 251, 2021, doi: 10.1016/j.enbuild.2021.111375.
21. I. Manojlovic, G. Svenda, and A. Erdeljan, “Load Pattern Recognition Method for Probabilistic Short-Term Load Forecasting at Low Voltage Level,” in IEEE PES Innovative Smart Grid Technologies Conference Europe, 2022. doi: 10.1109/ISGT-Europe54678.2022.9960310.
22. Z. B. Jiang et al., “A Bottom-up Method for Probabilistic Short-Term Load Forecasting Based on Medium Voltage Load Patterns,” IEEE ACCESS, vol. 9, pp. 76551–76563, 2021, doi: 10.1109/ACCESS.2021.3082926.
23. Y. H. Lin, H. S. Tang, T. Y. Shen, and C. H. Hsia, “A Smart Home Energy Management System Utilizing Neurocomputing-Based Time-Series Load Modeling and Forecasting Facilitated by Energy Decomposition for Smart Home Automation,” IEEE ACCESS, vol. 10, pp. 116747–116765, 2022, doi: 10.1109/ACCESS.2022.3219068.
24. A. Rafati, M. Joorabian, E. Mashhour, and H. R. Shaker, “Machine learning-based very short-term load forecasting in microgrid environment: evaluating the impact of high penetration of PV systems,” ELECTRICAL ENGINEERING, vol. 104, no. 4, pp. 2667–2677, 2022, doi: 10.1007/s00202-022-01509-4.
25. S. Rai and M. De, “Data-Driven Resilient Load Forecasting Model for Smart Metered Distribution System,” ELECTRIC POWER COMPONENTS AND SYSTEMS, 2023, doi: 10.1080/15325008.2023.2185832.
26. C. C. Xu and G. Chen, “Interpretable transformer-based model for probabilistic short-term forecasting of residential net load,” INTERNATIONAL JOURNAL OF ELECTRICAL POWER & ENERGY SYSTEMS, vol. 155, 2024, doi: 10.1016/j.ijepes.2023.109515.
27.  A. Nespoli, S. Leva, M. Mussetta, and E. G. C. Ogliari, “A Selective Ensemble Approach for Accuracy Improvement and Computational Load Reduction in ANN-Based PV Power Forecasting,” IEEE Access, vol. 10, 2022, doi: 10.1109/ACCESS.2022.3158364.
28. W. Lin, D. Wu, and M. Jenkin, “Electric Load Forecasting for Individual Households via Spatial-Temporal Knowledge Distillation,” IEEE Transactions on Power Systems, vol. 40, no. 1, 2025, doi: 10.1109/TPWRS.2024.3393926.