Solar power generation forecasting using ensemble approach based on deep learning and statistical methods

Abstract

Solar power forecasting will have a significant impact on the future of large-scale renewable energy plants. Predicting photovoltaic power generation depends heavily on climate conditions, which fluctuate over time. In this research, we propose a hybrid model that combines machine-learning methods with Theta statistical method for more accurate prediction of future solar power generation from renewable energy plants. The machine learning models include long short-term memory (LSTM), gate recurrent unit (GRU), AutoEncoder LSTM (Auto-LSTM) and a newly proposed Auto-GRU. To enhance the accuracy of the proposed Machine learning and Statistical Hybrid Model (MLSHM), we employ two diversity techniques, i.e. structural diversity and data diversity. To combine the prediction of the ensemble members in the proposed MLSHM, we exploit four combining methods: simple averaging approach, weighted averaging using linear approach and using non-linear approach, and combination through variance using inverse approach. The proposed MLSHM scheme was validated on two real-time series datasets, that sre Shagaya in Kuwait and Cocoa in the USA. The experiments show that the proposed MLSHM, using all the combination methods, achieved higher accuracy compared to the prediction of the traditional individual models. Results demonstrate that a hybrid model combining machine-learning methods with statistical method outperformed a hybrid model that only combines machine-learning models without statistical method.

Citation

AlKandari, M. and Ahmad, I. (2020), "Solar power generation forecasting using ensemble approach based on deep learning and statistical methods", Applied Computing and Informatics, Vol. ahead-of-print No. ahead-of-print. https://doi.org/10.1016/j.aci.2019.11.002

1. Introduction

Photovoltaic (PV) technology has been one of the most common types of renewable energy technologies being pursued to fulfil the increasing electricity demand, and decreasing the amount of CO2 emission at the same time conserving fossil fuels and natural resources [1]. A PV panel converts the solar radiation into electrical energy directly by semiconducting materials. Use of a PV panel highly depends on solar radiation, ambient temperature, weather and the geographic location of the PV panel. Furthermore, the electrical operators need to balance the electrical consumption and generation in order to conserve and avoid the waste of electrical energy produced from the renewable energy plants. In addition, they need to ensure that there will be a sufficient capacity of power to cover the requirements of the consumers [2]. Thus, the industry of renewable energy is moving toward connecting solar energy power plants with the electrical grid. However, this causes instability to the grid, making it the greatest challenge to the industry. Hence, predicting power generation from renewable energy farms in general and PV plants specifically is a necessary step for future development. Thus, it is essential to be able to predict the capacity of PV plants power that will be produced for the next few days.

Solar power prediction is not an easy process because it largely depends on climate conditions, which fluctuate over time. To overcome the above-mentioned challenges, it is important to employ new, and intelligent methods to obtain valid and accurate results. To date, machine learning (ML) methods have received significant attention from many researchers and developers in the solar power generation forecasting field [3–9] in addition to other fields such as solving partial differential eqautions [10,11]. More details of the characteristics and performance of ML techniques employed in solar PV generation forecasting can be found in recent surveys [12,13]. Besides, ensemble learning is a promising model that grabbed the attention of many researchers in the past few years [14]. Notably, the authors in [12] mentioned that diversity between the ensemble members creates a significant benefit and improvement over the traditional models. In addition to ML models, statistical methods are traditional mathematical formulas and techniques that have been used historically for forecasting purposes [15]. Yang et al. [16] combined the forecasting results of different statistical methods into more accurate prediction.

As mentioned earlier, numerous research papers have been published discussing different ML techniques, ensemble methods to combine these ML models, and ensemble methods for integrating different statistical methods. However, we could not find research that combines ML models and statistical method, especially in renewable energy forecasting applications. Therefore, in this research, we propose a hybrid model (MLSHM) that combines ML models and statistical method to predict solar power generation of a PV plant for more efficient and accurate results. As diversity is the main characteristic towards the success of the ensemble approach, we explore two types of diversity: structural diversity since we combine ML models and statistical model that have different structures, and data diversity by dividing the training set among various ML models. Moreover, four different ensemble approaches are explored to combine and aggregate the solar PV power predicted from ML models and statistical method: simple averaging approach, weighted averaging using linear approach, weighted averaging using non-linear approach, and a combination through variance using the inverse approach.

To test and evaluate the proposed hybrid model, we implement various ML predicting models: long short-term memory (LSTM) [17], gate recurrent unit (GRU) [18], and AutoEncoder long short-term memory model (Auto-LSTM) [6]. Moreover, we propose a new ML model, called AutoEncoder gate recurrent unit (Auto-GRU), which is similar to Auto-LSTM except it uses GRU cells in the ML network. In addition, we design and implement a statistical method to forecast the power of a PV farm, which is the Theta model that was proposed in [19]. This statistical model is considered the most accurate and the simplest model compared with the other statistical models in M3 Competition [15]. During the experiments, the proposed hybrid model achieved better accuracy and outperformed other individual ML and statistical models. Moreover, the experimental results demonstrated that a hybrid model combining both types of ML methods and statistical method achieved better accuracy than hybrid models that only combines machine learning models. The proposed MLSHM and Auto-GRU can be generalized to solve other time series problems such as financial markets, industrial markets, control engineering and astronomy.

The remainder of this paper is structured as follows. Section 2 presents the problem statement and the related work. Section 3 illustrates the methodology of ML models, the proposed Auto-GRU model, statistical model, and the proposed hybrid model. Section 4 shows the experimental results and performance validation of the proposed hybrid model. Finally, this research concludes in Section 5.

2. Problem statement and related work

The prediction problem can be formulated as follows: Given a time series of historical weather data in the form (x1, x2…xn), where n represents the number of weather parameters. To predict day-ahead solar PV power y^, a conventional approach is to define a mapping function f between the historical weather data and future solar PV power as follows:

(1)y^=f(x1, x2…,xn)

Figure 1 shows a simple representation of the solar PV power prediction system with n=6 weather parameters.

Numerous research studies have introduced ML algorithms as forecasting models in different application related to the field of renewable energy. Several ML methodologies such as support vector machine (SVM) [20], long short-term memory (LSTM) [21], and K-nearest neighbor (K-NN) [22], have been applied to predict solar irradiance, which can be considered as the first step towards solar PV power forecasting. A gradient boosted regression tree model (GBRT) was conducted by Persson et al. in [3] to predict multi-site solar power generation on a forecast horizon of one to six hours ahead. The GBRT model was mainly designed for classification; however, it has been extended to regression. Furthermore, GBRT is a ML model that combines the output of many small regression trees of fixed size to generate better result. Unlike the conventional time series methods, the proposed model has no updating procedure or recursive version as new observations arrive, which add a considerable limitation to their proposed model. The authors in [4] proposed a least absolute shrinkage and selection operator (LASSO) based forecasting model for solar power generation. LASSO based model assists in variable selection by minimizing the weights of less important variables and maximizing the sparsity of the overall coefficient vector. They compared the predicted solar power from their proposed algorithm with two representative schemes, SVM and a time-series based method known as TLLE method. The results showed that LASSO based algorithm achieved more accurate forecasts of solar power than the representative schemes using fewer training dataset. As a supplement to the proposed algorithm by Tang et al. [4], the authors in [5] integrated LASSO with LSTM as a forecasting model for solar intensity prediction. Their proposed model attained better performance in short-term solar intensity prediction rather than long-term prediction.

Furthermore, power forecasting of solar power plants using AutoEncoder (AE) and LSTM neural network was developed by [6]. They used the encoding side of the AE to realize the most effective features for learning and attached it to the LSTM network. Accordingly, the LSTM uses the learned encoding data as an input to predict the solar power generation of a PV plant. The model showed powerful results compared to multilayer perceptrons (MLP), LSTM, deep belief networks, and AE. Different from [6], the authors in [7] combined the AE with LSTM into augmented long short-term memory (A-LSTM) forecasting model. The algorithm was tested on different datasets, and it performed well on time series datasets. De et al. developed LSTM based model to predict PV power using limited dataset [8]. Although the LSTM model predicted accurate results, it was shown that increasing the amount of data and features will improve the performance of the LSTM model. Wang et al. proposed a GRU based short-term PV power forecasting algorithm [9]. GRU model is used to reduce the long training time compared to LSTM model as well as improve the accuracy of the output. They concluded that GRU outperformed the traditional ML models such as SVM, autoregressive integrated moving average model (ARIMA), LSTM, and back propagation neural network.

The authors in [23] surveyed state-of-the-art ML models used in different renewable energy systems. Moreover, they identified and classified many ML models applied in different applications in energy applications and explored various research in different energy systems. Rana et al. demonstrated a comprehensive comparison on various methods that is used for prediction purposes [24]. They explored six different methods: Neural Network (NN), SVM, K-NN, Multiple Linear Regression (MLR) and two persistent methods. The results showed that ensemble of NNs is the most promising accurate method compared to other predicting methods. Statistical approaches have not gathered the attention of researchers as much as ML models, especially in solar PV power forecasting. As mentioned earlier, Makridakis et al. illustrated that Theta method was the most accurate and simpler statistical method that performed partially well in M3 Competition [15].

The authors in [25] classified the ensemble methods into two categories: competitive ensemble forecasting and cooperative ensemble forecasting. Ensemble learning is a promising model which has attracted lot of attention in recent years. Ahmed et al. [14] proposed three different ensemble approaches to predict day-ahead solar power generation namely:

1. (i)

   linear,

2. (ii)

   normal distribution, and

3. (iii)

   normal distribution with additional features.

They concluded that all the ensemble methods when combined together showed better performance than the individual ML models. Gigoni et al. compared several ML forecasting methodologies, e.g., K-NN, support vector regression (SVR), and quantile random forest and evaluate their prediction accuracy in solar PV power application [26]. The experimental results showed that aggregating the output of single prediction models surpassed all the ML learning models explored in their research under any weather condition. Feng et al. grouped the weather data into hourly similarity-based approach and used those grouped data to train a two-layer ML hybrid model to be able to predict one hour ahead solar irradiance [27]. Their results showed that the hybrid model performed better than any single ML model used in the hybrid model. Another study by Koprinska et al. illustrated static and dynamic approaches to ensemble the solar power prediction of NNs [28]. Their experiment showed overwhelming results for the ensemble approaches compared to bagging, boosting, random forest, and four single prediction models (NN, SVM, K-NN, and a persistence model). Limited research can be found on ensemble statistical models for superior performance and accuracy. The authors in [16] explored eight ensemble techniques to combine the results of six best models from each family of statistical models: SARIMA (36 models), ETS (30 models), MLP (1 model), STL decomposition (2 models), TBATS (72 models) and the Theta model (1 model). Although ensemble learning showed high efficient performance and accurate results in many papers, the results in [16] showed marginal enhancement from the best model. The reason being that the models tested resulted in highly correlated errors. This led us to emphasizes that diversity is the key toward major enhancement of ensemble methods. The authors in [29] presented a cluster-based approach applied to the global solar radiation. Their approach then predict the horizontal global solar radiation using a combination of two Ml models: SVM and ANN. The results showed higher predicting accuracy compared to the conventional ANN and SVM.

In summary, we explored much research in solar power forecasting that combine ML models to enhance prediction accuracy. Substantially, all the research we explored confirmed that diversity is an essential and fundamental procedure for a powerful ensemble model. Moreover, it was found that most ensemble based studies used the conventional ensemble-ML methods such as bagging, AdaBoost, and stacked generalization to apply diversity, train the same ML models and aggregate the results into one complete model. However, this study incorporates the prediction resulted from ML models and the prediction resulted from the statistical model, which cannot be achieved by the conventional ML-based ensemble models. Furthermore, research that aggregates ML model and statistical models in solar PV power forecasting is non existent. Hence, this study could be considered the first in solar power forecasting which adds a value to this field of research.

3. Design methodology

In this section, proposed solutions to enhance and reinforce the performance of solar PV power forecasting algorithms are presented.

3.1 ML forecasting algorithms

In this section, we introduce various ML algorithms adopted in this study to build and construct the hybrid model. We have chosen the most accurate ML models that have been applied in forecasting applications such as LSTM [17], GRU [30], and Auto-LSTM [6]. Additionally, a new ML model (Auto-GRU) has been proposed to the exiting ML forecasting algorithms bundle.

3.1.1 LSTM

The long short-term memory networks (LSTM) are special recurrent neural networks (RNNs) that were first introduced in [17]. RNN is a neural network (NN) with recurrent connections between the neurons that enables it to learn from the current and the previous information to find a better solution. However, when two cells in RNN are far away from each other, it is difficult to obtain useful information due to the gradient vanishing and explosion problems. The solution to this are special neurons called memory cells. Using these special neurons, the LSTM is able to store useful information over an arbitrary period of time. Moreover, LSTM cells have the ability to learn what data needs to be read, stored, and erased from the memory by adjusting three different controlling gates, namely forget gate f(t), input gate i(t), and output gate o(t) as shown in Figure 2.

The forget gate (f(t)) decides either to keep or discard the information from the cell state, shown in Eq. (2). A logistical function generates either 0 or 1 as a value for f(t) that represents either to abandon or keep the current cell state at time step t, respectively. Where w, h, x, and b are the weight, output, input, and bias, respectively. The activation function is represented by σ. On the other hand, the input gate controls what input values can be stored in the cell state as shown in Eq. 4. Where i(t) represents the signal (0 or 1) that controls the updating procedure, g(t) is new candidate value and c(t) is the new state of the cell. Moreover, the output gate (o(t)) is responsible for releasing the stored information to the next neurons as shown in Eq. 6. Hence, LSTM is a powerful ML model to capture long-term dependency as well as the non-linear relationship in a complex dataset.

(2)f(t)=σ(wf[h(t−1), x(t)]+bf)

(3)c(t)=f(t)∗c(t−1)+i(t)∗g(t)

(4)i(t)=σ(wi[h(t−1), x(t)]+bi)

(5)g(t)=tanh(wg[h(t−1), x(t)]+bg)

(6)o(t)=σ(wo[h(t−1), x(t)]+bo)

(7)h(t)=o(t)∗tanh(c(t))

3.1.2 GRU

The gated recurrent unit (GRU) is a special case of LSTM introduced by Cho et al. [30] to reduce the long training time of LSTM. Compared to LSTM, GRU has fewer controlling gates as it lacks an output gate. As shown in Figure 3, GRU is much simpler than LSTM since it includes only two gates, the reset gate and update gate, that control the information flow inside the units. The transition functions between neurons of GRU are given as follows:

(8)r(t)=σ(wrx(t)+urh(t−1)+br)

(9)z(t)=σ(wzx(t)+uzh(t−1)+bz)

(10)h^(t)=σ(whx(t)+uh(r(t)∗h(t−1))+bh)

(11)h(t)=(1−z(t))∗h(t−1)+z(t)∗h^(t)

where r(t) is the reset gate, z(t) is the update gate, and w and u represent the parameter matrices in GRU. Furthermore, h(t), h^(t), and b are the output, candidate output, and bias, respectively. The activation function is represented as σ.

3.1.3 Auto-LSTM

The Auto-LSTM model proposed by [6] consists of two ML algorithms: AutoEncoder (AE) [31] and LSTM. An AE is an unsupervised neural network where the input and the output layers have the same size. An AE tries to learn the identity function so that the input x is approximately similar to the output x^ with some constraints applied to the network, e.g., a limited number of neurons in the hidden layer compared to the input layer. Therefore, an AE acts as a compressor and a decompressor consisting of two parts separated by a bottleneck at the center:

1. (i)

   encoding side, where the neurons are reduced from the input layer to the hidden layer, and

2. (ii)

   decoding side, where the layers in the encoding side are reflected as shown in Figure 4.

Thus, an AE is able to learn and discover the correlations in the input features and the special structure of the data. Gensler et al. [6] have utilized the encoding side of the AE to realize the feature extraction and attached it to the LSTM model. Hence, the LSTM model is trained and fitted using the historical encoded weather data produced by the encoding side of the AE as well as the corresponding solar PV power. The end result of this Auto-LSTM model is an ML algorithm that is able to predict the solar PV power generated from a PV farm given the meteorological data.

3.1.4 Auto-GRU

The proposed Auto-GRU model has similar characteristics of the Auto-LSTM model presented in [6]. As explained earlier, the AE is used to shrink the meteorological data by discovering the structure of data and attach the encoded data to the GRU network. The encoded meteorological data as well as the corresponding historical solar PV power will be fed to the GRU network to fit and train the neurons to be able to predict the desired output. Specifically, the Auto-GRU model is trained by a set of historical encoded meteorological data and the corresponding PV power (encoded meteorological parameters, PV power). Figure 5, represents the block diagram of the proposed Auto-GRU model. The forecasting process using Auto-GRU can be summarized as follows:

1. 1.

   The historical weather data is encoded using the encoding side of the AE.

2. 2.

   The encoded weather data is split into a training set and a testing set. A small percentage of training data is preserved for validation.

3. 3.

   All the training, validating and testing sets are reorganized into chunks or windows that represent the number of historical previous days (window size).

4. 4.

   The proposed model is trained and at the same time validated using windows of historical encoded weather data and the corresponding PV power.

5. 5.

   The proposed model is tested on the testing set that was rearranged into chunks of previous samples.

6. 6.

   Finally, the proposed Auto-GRU model is a powerful ML model that is ready to predict the solar PV power given a chunk of previous meteorological data.

3.2 Statistical forecasting algorithm

Based on research by Makridakis et al. [15], we decided to use Theta model to represent the statistical part of the study. This statistical model is considered to be the most accurate and the simplest model compared to the other statistical models examined in M3 Competition [15]. Assimakopoulos et al. [19] have proposed a Theta model as a decomposition approach for forecasting applications. This model is based on modifying the local curvature of the time series data using a coefficient called Theta(θ) that is applied to the second derivative of the data as shown in Eq. (12).

(12)Xnew″(θ)=θ⋅Xdata″, where Xdata″=(Xt−2Xt−1+Xt−2)at time t.

The new time series lines are called theta lines and maintain the mean and the slope of the original time series. Moreover, the deflations of the new time series curvatures depend on the value of Theta coefficient, i.e., to identify the long-term behaviors of the time series dataset programed the Theta coefficient to be between 0 and 1 (0 < θ < 1). However, when θ > 1 the new theta line is more dilated, it affects the short-term trends. The theta lines are then extrapolated separately and combined to generate the forecasted solar PV power. The authors in [19] decomposed the original time series into two theta lines by setting the Theta coefficient to θ = 0 and θ= 2. The first line (L (θ = 0)) represents the linear regression line of the original time series magnifying the long-term trends. The second line (L (θ = 2)) doubles the original curvature, magnifying the short-term trends. In this, the forecasting process is accomplished by linearly extrapolating the first theta-line while extrapolating the second line using simple exponential smoothing (SES). Afterward, the forecasted time series of the two theta-lines are simply combined via equal weights resulting in the final forecast of a specific time series dataset.

3.3 Ensemble forecasting algorithms

Ensemble of the prediction models is a key enhancement to the prediction accuracy that has grabbed the attention of many researchers in recent years. In our study, we proposed ML and statistical hybrid model (MLSHM) that combines the solar PV power predicted from various ML models and statistical model. Moreover, several ensemble methods were employed to combine the predictions of different models and generates the final solar PV power prediction. To boost and raise the benefits of aggregating ML models with statistical method, enforcing diversity between the combined models, is an essential procedure. In this research, the diversity in ensemble methods falls onto two categories:

1. (i)

   data diversity, and

2. (ii)

   structural diversity.

Data diversity is achieved by generating multiple datasets from the original dataset to train the ML models [25,32]. Structural diversity is attained by having different architectures of the prediction models [32]. In our study, we introduced data diversity within the ML models and structural diversity by combining two differently structured algorithms, i.e., ML models and statistical model. As a result, data diversity was applied to the combined ML models as follows:

1. 1.

   After the dataset was split into a training set and a testing set, the training set was further divided into n training sub-sets, where in our study n = 2.

2. 2.

   Each ML model is trained on one of the training sub-sets; hence, we have n ML models trained on different sets of data, where in our study n = 2, hence we have two ML models trained on different sets of data.

3. 3.

   All the ML models are tested on the same testing set for comparison purposes to ensure equality of results between the models.

Figure 6 represents the block diagram of the proposed MLSHM showing the data diversity applied to the training set. There are n ML models and k statistical methods predicting the solar PV power.

In the first stage, the ML models and the statistical models predicted the solar PV power separately, and then we combined these results to get the final forecast. We explored four different ensemble methods to test their effectiveness on the proposed MLSHM. The ensemble methods are described as follows:

1. 1.

   EN1: simple averaging approach, which is the simplest and the most natural method that generates the final forecasted solar PV power by taking the mean value of the forecasts resulted from the ML models and statistical models. The final solar PV power is generated as follows:

(13)y^=∑j=1my^j/m

Here m denotes the number of ensemble members which is m = n + k and y^ is the predicted solar PV power, y^j represents the predicted solar PV power of ensemble member j, and y^ is the final solar PV power forecast.

1. 2.

   EN2: weighted averaging using linear approach, where the final solar power is accomplished by assigning different weights to the combined models based on the its accuracy. Accordingly, a model with a higher error rate is given a lower weight so that it will minimally contribute in the final prediction. Conversely, a model with a lower error rate (more accurate) is given a higher weight; therefore, it will add more value to the final forecast. The weights are calculated while ensuring that the summation of all the ensemble models’ weights equal 1. Eq. 14 demonstrates the model weighting equation used in EN1 and Eq. 15 represents the weighted averaging calculation.

(14)wi=1−nMAEi/∑j=1m(1−nMAEj)

(15)y^=∑i=1my^i⋅wi

Here wi represents the weight given to model i, and nMAEi is the normalized Mean Absolute Error of model i.

1. 3.

   EN3: weighted averaging using non-linear approach, where it uses the same concept of EN1 but the models’ weights are calculated as a softmax function of the negative of its error (nMAE):

(16)wi=exp−nMAEi/∑j=1mexp−nMAEj

Here exp denotes the exponential function. The final solar power forecast is calculated by Eq. (15).

1. 4.

   EN4: combination through variance using inverse approach this is simply the weighted averaging using the following weighting equation:

(17)wi=1/nMAEi/∑j=1m1/nMAEj

The final prediction of solar PV power is calculated by the weighted averaging as in Eq. 15.

4. Experimental results and discussions

In this section, we present a comprehensive experimental study to evaluate these proposed methods in solar PV power forecasting application. We compared a new ML method (Auto-GRU) with other ML algorithms (GRU [30], LSTM [17], and Auto-LSTM [6]) and statistical method (Theta model [19]) in terms of accuracy. Four different ensemble methods were used to build the MLSHM and will be compared with the accuracy with the traditional methods. The proposed algorithms were tested on different solar PV systems starting from a single PV panel to large-scale PV farms.

4.1 Dataset description

The time series datasets used in this research were collected from two different geographical locations: Shagaya in Kuwait and Cocoa, in Florida, USA. The first dataset was collected from Shagaya Renewable Energy Park in Kuwait from May 5, 2017, to June 30, 2018 [33]. The solar PV plant in Shagaya is about 10 MW capacity and consists of two sub-plants of two different PV technologies, Polycrystalline solar panel (Poly-SI) and Thinfilm solar cells (TFSC). The Shagaya dataset includes the total solar power generated from each sub-plant (kW) as well as the meteorological data of the site which are Global Horizontal Irradiance (GHI) (kW/m²), Global Tilt Irradiance (GTI) (kW/m²), wind speed (m/s), wind direction (°), air temperature (° C), and panel’s temperature (° C). The time series dataset is available in different resolutions, i.e., five-minute, hourly, daily, and monthly. In this research we conducted the experiments on five-minute resolution data.

Marion et al. [34], collected data for different technologies of PV modules at three climatically diverse locations in the USA (Cocoa, Florida; Eugene, Oregon; and Golden, Colorado). The time series data are available in five-minute resolution from January 21, 2011, to March 4, 2012. We conducted our experiments on one sample of the USA dataset and tested our proposed methods on a single Poly-SI module from Cocoa, Florida. The dataset consists of both the total power produced from Poly-SI module (W) and the weather data gathered from the site such as solar irradiance (W/m²), ambient temperature (° C), humidity (%RH), and precipitation (mm).

As collecting data from outdoor data acquisition devices could suffer from incorrect and missing readings from the sensors, data preprocessing is an essential procedure that cleans the data before verifying the effectiveness of the proposed methods. In this research, all time-series datasets were reclaimed from missing, undesired, duplicated, and incorrect readings. The data were then grouped into daily resolution and normalized between 0 and 1 using min-max normalization in order to decrease the training time and facilitate the comparison of the different PV plants’ size. Afterward, the dataset was divided into a training set and a testing set consisting of 90% and 10% of the data, respectively. A 10% of the training data was preserved for validation. Table 1 summaries the time series datasets used in this paper.

4.2 Design of experiments

To evaluate and verify the effectiveness of the proposed ML algorithm (Auto-GRU) and MLSHM, we conducted several experimental simulations to forecast day-ahead solar power generation. All ML algorithms and ensemble methods were implemented using Keras API [35] for Python v3.7 along with TensorFlow framework v1.10 [36]. Theta model was constructed using thetaf function of the forecast R statistical package. Furthermore, we implemented GRU, LSTM, Auto-GRU, and Auto-LSTM models with 3-layer topology, that is, one input layer, one hidden layer with 20 neurons, and one output layer. The selection of 20 neurons in the hidden layer is based on testing different configurations and selecting the one which provided the best accuracy. Accordingly, the input layer represents the meteorological data while the output layer represents the forecasted solar PV power. Hyperbolic Tangent (tanh) was added as an activation function to introduce non-linearity in the network. The models were fitted using efficient RMSProp optimization algorithm and the mean squared error loss function. In addition, various hyper-parameters were used to fit ML models in order to carry out several experimental scenarios. These experimental configurations were used to fine tune the Auto-GRU model on the three datasets (Shagaya Poly-SI, Shagaya TFSC, and Cocoa single Poly-SI). The design of the experiments were tested such that the parameters are set as follows: window size is 2, update weights after 20 batches and execute 50 epochs to train the model.

In this research, we used two popular evaluation measures to compare the accuracy of the predicted normalized power from Auto-GRU model and MLSHM. Specifically, normalized Mean Absolute Error (nMAE) as shown in Eq. 18 and normalized Mean Square Error (nMSE) as shown in Eq. 19. Here, y is the normalized actual power, y^ is the normalized predicted power, and N is time series samples.

(18)nMAE=1/N∑i=1N|yi−y^i|

(19)nMSE=1/N∑i=1N(yi−y^i)2

4.3 Experimental results

As mentioned earlier, four ensemble methods (EN1, EN2, EN3, and EN4) were implemented to combine the solar PV power predicted from different ML methods and Theta model. Various combinations were applied to comprehensively evaluate the proposed MLSHM. Figure 7 depicts different integrations of ML methods with Theta statistical model. It can be seen that the hybrid model can be either an ML and statistical hybrid model (MLSHM) or an ML hybrid model without the statistical method (MLHM). Moreover, MLSHM and MLHM were further divided into a hybrid model with auto-encoded ML models (Auto-MLSHM and Auto-MLHM) or a hybrid model with traditional ML models (T-MLSHM and T-MLHM).

The accuracy measures of Theta model, LSTM, GRU, Auto-LSTM, Auto-GRU and several combinations of these algorithms using the four ensemble models mentioned in Section 3 are summarized in Tables 2,3 for each of the three datasets. Of note is that our proposed MLSHM using four different ensemble equations outperformed other traditional methods and MLHM. Hence, Theta model consolidated the performance and the accuracy of the forecasted solar PV power.

4.4 Analysis of results

By examining Tables 2,3 it can be seen that MLSHM is a powerful hybrid model that outperforms all other hybrid models and traditional single models tested. Moreover, MLSHM is more accurate than MLHM, which means that Theta model has significantly improved the accuracy of the prediction. We have highlighted the least nMAE and nMSE in bold as shown in Tables 2,3. The lowest nMSE for Shagaya Poly-SI is 0.0197 using EN4 for T-MLSHM which is 18.93% better than GRU model as shown in Table 2. Furthermore, the power prediction for Shagaya TFSC farm attained an nMSE of 0.00185 using EN4 for T-MLSHM, a 36.21% improvement over GRU model. Table 4 highlighted the least nMSE for Cocoa single Poly-SI dataset, which reached 0.0168 for EN4 of T-MLSHM that resulted in a 4.55% reduction of error rate and improved performance. Therefore, T-MLSTHM using EN4 was the most accurate ensemble method since it increased the accuracy of solar power prediction range between 4.55% and 36.21% over the single traditional models. On the other hand, Auto-MLSHM using EN4 increased the accuracy of the solar power prediction range between 8.15% and 28.90% compared to Auto-GRU model. One reason for this is that EN4 distributes the weights efficiently where the most accurate model has the greatest weight value and is not close in value to the other weights. From the single prediction models, GRU achieved the best accuracy over other traditional ML models and the Theta statistical model.

Figure 8 shows the actual solar PV power generation compared to the predicted solar PV power from different models tested in this study on the three datasets; Shagaya Poly-SI, Shagaya TFSC, and Cocoa single Poly-SI, respectively. We can see that the prediction models perform better for Shagaya dataset rather than Cocoa dataset because it contains more relative weather data that affects the performance of a PV model. The Cocoa dataset represents the power of a single Poly-SI module, which appeared to be unstable. In contrast, Shagaya dataset represents the power of a large scale solar PV farm that is more stable. Hence, the prediction of a single PV module is more difficult than the prediction on a large scale PV farm since a single PV panel can easily be affected by a passing cloud. We can conclude that the proposed hybrid model (MLSHM) is a powerful, efficient, valuable, and accurate prediction model.

We can summarize the results obtained during this research as follows:

• Hybrid models outperform all tested traditional ML models and the Theta statistical model.

• Hybrid models with Theta model (MLSHM) obtained better accuracy than hybrid models without the Theta statistical model (MLHM).

• All ensemble methods (EN1, EN2, EN3, and EN4) achieved better accuracy than any single ML algorithm and theta model.

• Almost all ensemble methods achieved similar accuracies; however, EN4 was found to be the most accurate ensemble method used in this research.

• GRU model is the most accurate ML model followed by Auto-GRU, LSTM, and Auto-LSTM.

• ML models performed better than theta statistical model in predicting solar power generation.

5. Conclusions

Integrating large-scale PV plants into the power grid poses considerable problems and challenges to the electric operators, as it causes instability to the electric grid causing the electrical operators to balance the electrical consumption and power generation in order to avoid waste of energy. Therefore, an accurate solar power forecast is a fundamental requirement toward the future of renewable energy plants. In this research, we proposed a hybrid model (MLSHM) that combines the prediction results of both ML models and statistical method. For our study we developed a new ML model, Auto-GRU, that learns from historical time series data to predict the desired solar PV power. In order to boost the hybrid model, two diversity techniques were conducted in this study, i.e., structural diversity between the ensemble members and data diversity between the training sets of the ML models. Four different combination methods illustrate to combine the prediction of ML models and statistical method. The proposed hybrid model and Auto-GRU model tested on two real-time series datasets of solar PV power and weather data collected from Shagaya located in Kuwait [33] and Cocoa, Florida, USA [34]. The experiments allow us to conclude that a hybrid model combining the prediction of ML models and statistical method obtain higher accuracy than a hybrid model combining the prediction of ML models without statistical method. Our future work is to test and validate the hybrid model on other ML models and statistical method. Moreover, we will try other diversity techniques such as dividing the training set by parameters as well as testing both data and parameters diversity. In addition, we plan to develop other ensemble techniques to boost the accuracy of the prediction.

Figures

Figure 1

Solar PV power prediction model.

Figure 2

The structure of a long short-term memory cell.

Figure 3

The structure of a gated recurrent unit cell.

Figure 4

An example of AutoEncoder topology.

Figure 5

Auto-GRU block diagram.

Figure 6

The block diagram of the proposed Machine Learning and Statistical Hybrid Model (MLSHM).

Figure 7

Hybrid model configuration.

Figure 8

Actual versus predicted solar PV power of the three datasets.

Table 1

Time series datasets summary.

Dataset	Shagaya Poly-SI [33]	Shagaya TFSC [33]	Cocoa Poly-SI [34]
System Size	5 MW	5 MW	166 W
No. of Days Before Preprocessing	420	420	316
No. of Days After Preprocessing	376	409	316
Training Set	304	330	255
Validating Set	34	37	29
Testing Set	38	42	32
Weather Parameters	GHI, GTI, wind speed, wind direction, air & panel’s temperature	GHI, GTI, wind speed, wind direction, air & panel’s temperature	Solar irradiance, ambient temperature, humidity & precipitation

Table 2

Prediction performance on nMAE and nMSE of shagaya poly-SI dataset.

						Auto-MLSHM				T-MLSHM				Auto-MLHM				T-MLHM
Approach	Theta Model	LSTM	GRU	Auto-LSTM	Auto-GRU	EN1	EN2	EN3	EN4	EN1	EN2	EN3	EN4	EN1	EN2	EN3	EN4	EN1	EN2	EN3	EN4
nMAE	0.057	0.0536	0.0346	0.0806	0.0526	0.044	0.0438	0.0438	0.0424	0.0341	0.0341	0.0341	0.0327	0.045	0.0447	0.0447	0.0423	0.0514	0.0513	0.0513	0.0499
nMSE	0.00695	0.0037	0.00243	0.00891	0.00429	0.00318	0.00316	0.00316	0.00305	0.00213	0.00214	0.00215	0.00197	0.00274	0.00271	0.00271	0.00241	0.00393	0.00393	0.00393	0.00396

Table 3

Prediction performance on nMAE and nMSE of shagaya TFSC dataset.

						Auto-MLSHM				T-MLSHM				Auto-MLHM				T-MLHM
Approach	Theta Model	LSTM	GRU	Auto-LSTM	Auto-GRU	EN1	EN2	EN3	EN4	EN1	EN2	EN3	EN4	EN1	EN2	EN3	EN4	EN1	EN2	EN3	EN4
nMAE	0.0574	0.0698	0.0358	0.0831	0.0394	0.0411	0.0409	0.0409	0.0389	0.0354	0.0352	0.0352	0.0317	0.0435	0.0428	0.0428	0.0356	0.0374	0.0373	0.0373	0.0368
nMSE	0.00656	0.00577	0.0029	0.00961	0.00251	0.00303	0.003	0.003	0.00265	0.00209	0.00207	0.00207	0.00185	0.00332	0.00324	0.00325	0.00241	0.00194	0.00194	0.00194	0.00197

Table 4

Prediction performance on nMAE and nMSE of cocoa single poly-SI dataset.

						Auto-MLSHM				T-MLSHM				Auto-MLHM				T-MLHM
Approach	Theta Model	LSTM	GRU	Auto-LSTM	Auto-GRU	EN1	EN2	EN3	EN4	EN1	EN2	EN3	EN4	EN1	EN2	EN3	EN4	EN1	EN2	EN3	EN4
nMAE	0.207	0.0739	0.0866	0.0778	0.0952	0.112	0.109	0.109	0.0948	0.166	0.103	0.103	0.0877	0.0839	0.0838	0.0838	0.0828	0.0794	0.0794	0.0794	0.0794
nMSE	0.0556	0.0176	0.0193	0.0184	0.019	0.0191	0.0185	0.0186	0.0169	0.0187	0.0181	0.0182	0.0168	0.0179	0.0179	0.0179	0.0177	0.0185	0.0185	0.0185	0.0185