Prediction of Wind Speed Using a Hybrid Regression-Optimization Approach ()
1. Introduction
Wind power is important because demand for clean, renewable energy is rapidly increasing. In addition, the invention of the smart grid allows for better integration of renewable energy sources. Wind power is directly proportional to wind speed. As the wind drives the blades of the windmill, the generator outputs energy. The more torque the wind produces, the more electricity is generated. Because wind power is directly proportional to wind speed, predicting wind speed is vital. If there is a drop in wind speed, and therefore a drop in wind power, other energy sources need to be prepared to compensate for that power loss. Likewise, if there is an abundance of wind speed and wind power, other energy sources can reduce their output, reducing cost for both the utilities and the consumers.
Wind speed can be predicted in the short-term using time series forecasting models. As early as 2010, neural networks were being used to model wind speed forecasting methods. Xiaojuan et al.’s research showed that these neural networks can continuously improve themselves [1] . Bilgili and Sahin found that artificial neural networks perform better than both linear and nonlinear regression [2] .
Researchers were also combining statistical and intelligent models. Studies were done with the ARIMA model such as the ones conducted by Liu et al. [3] . Xiao et al. studied this direction of research as well [4] . However, research was still being conducted on statistical and intelligent methods independently. Seo and Hyeon researched a statistical method [5] while Khodayar and Teshnehlab researched neural networks [6] as well as Ramasamy et al. [7] . All of whom confirmed the accuracy of wind speed prediction using intelligent systems.
Statistical methods were still being used independently, however, such as the Auto-Regressive Moving Average (ARMA) model used by Ling-ling et al. [8] , and other statistical models (e.g., MLR) were used in multiple architecture systems (MASs) like the one Bouzgou and Benoudjit designed [9] . The ARIMA (Auto-Regressive Integrated Moving Average) model was a popular statistical method, also being researched by Ye et al. [10] as well as Radziukynas and Klementavicius [11] .
Another popular statistical forecasting model applied to wind power was Empirical Mode Decomposition (EMD). Fei performed experiments on this basic model and confirmed that using EMD improved the accuracy of wind speed forecasting models [12] . The process evolved into Fast Ensemble Empirical Mode Decomposition (FEEMD) and Complete Ensemble Empirical Mode Decomposition (CEEMD). Hui Liu et al. researched both the former [13] and the latter [14] . The CEEMD model was built upon and researchers such as Zhenkun Liu et al. used an Improved Complete Ensemble Empirical Mode Decomposition with Adaptive Noise (ICEEMDAN) in their combined model [15] . The model proposed in this paper opts for a simple Multiple Linear Regression (MLR) method for forecasting and then an optimization technique for improving the accuracy of results.
There are many researchers who have used different soft computing techniques in various fields, also the most commonly techniques such as MLR and ANN are used in many fields. Bilgili and Sahin [2] compared ANN with MLR and found that wind speed prediction can be done with some accuracy by LR, NLR and ANN methods. Tsakiri et al. predicted water discharge time series by describing each component using the multiple linear regression model (MLR), and the artificial neural network (ANN) model. Barhmi et al. mentioned that models of both ANN and MLR are efficient in estimation of wind speed when the determination coefficient is high between actual and predicted wind speed.
In the past, several optimization methods have been used in wind speed prediction to improve the accuracy of the models. Grey Wolf Optimizer (GWO) was originally designed by Mirjalili et al. in 2014 [16] . Mirjalili confirmed the accuracy of GWO in training multi-layer perceptron by comparing it to five other evolutionary trainers [17] . Xie et al. applied GWO to time series forecasting [18] . Liu et al. applied GWO to wind speed forecasting and found that it improved the performance of all models tested [19] .
In this paper, the authors have proposed a Grey Wolf Optimizer based Wind Speed Prediction (GWO-WSP). This approach is tested on the 2016, 2017, 2018, and 2019 Raw Data files from the Great Lakes Environmental Research Laboratories and the National Oceanic and Atmospheric Administration’s (GLERL-NOAA) Chicago Metadata Archive. The test results show that the implementation is successful and the approach yields accurate and feasible results. The computation time for execution of the algorithm is also superior compared to the existing methods in literature.
The MLR is a simple, powerful and flexible tool for prediction compared to existing models [3] - [15] . Usually, MLR is a statistical technique used to model the relationship between a dependent variable and one or more independent variables. MLR assumes that there is a linear relationship between the independent variables and the dependent variable, and seeks to estimate the coefficients of the linear equation that best fit the data. The wind speed series are non-stationary signals, by using GWO technique which may improve the learning efficiency of the MLR. Hence in this paper, GWO and MLR for WSP are proposed.
The paper is organized as follows. Section II introduces the wind speed prediction model used in this paper. Section III discusses the principles of optimization used in wind speed prediction and how this approach can be used to minimize the error between predicted and actual wind speeds. Section IV explains the GWO method adopted in this paper and explains the advantages of using this method. Section V discusses the implementation of GWO and MLR for wind speed prediction on the Chicago wind speed data and shows the accuracy, practicality, and computation time of the proposed approach. Section VI tabulates the results. Section VII interprets and analyzes the results obtained along with graphs and tables. Section VIII concludes the paper and narrates possible scope for extension of this research idea.
2. Literature Review
2.1. Wind Speed Prediction: Formulation of Mathematical Model
Mathematically forecasting time series begins with the model. The model is used to analyze the data to determine an underlying relationship between independent variables and the dependent variable. Regression analysis is a simple yet effective method for modeling these underlying relationships. Regression analysis involves minimizing the sum of squared differences between the actual data and a unique line of best fit (also known as the hyperplane). This line is represented by the following equation
. (1)
where Y is the dependent variable, X is the independent variable, β is the unknown parameter, and ε is the error term.
Regression can be either linear or nonlinear depending on the situation and is determined by the combination of parameters. Regression with a linear combination of parameters is called linear regression. The combination of independent variables does not need to be linear. The formula for a simple linear regression with one independent variable and two parameters is as follows:
. (2)
where Y is the dependent variable, X is the independent variable, β0 is one unknown parameter, β1 is the other unknown parameter and εi is the error term.
Multiple Linear Regression (MLR) has been proven to be reliable at finding the relationship between one variable and multiple independent variables. The general formula for MLR is as follows:
. (3)
for all observations indexed as
and for all dependent variables indexed as
. Where Y is the dependent variable, Xij are the independent variables, βj represent parameters to be estimated, and εi is the independent identically distributed normal error (also known as the residuals). The benefits of MLR are that multiple independent variables can be used for a more accurate prediction. In this case, the independent variables chosen were the year, day of the year, time of day, air temperature, wind gust, wind direction, and relative humidity.
2.2. Optimization Methods Used in WSP Accuracy Enhancement
Over time, there have been significant developments in methods for optimization. Marcello Farina and Luigi Piroddi found that the prediction error minimization method (PEM), although adequate for linear situations, gave poor long term prediction accuracy and the simulation error minimization (SEM) would probably be superior [20] . It was also around this time that swarm intelligence was being utilized for optimization. One of the original swarm intelligence optimization methods was particle swarm optimization (PSO). However, PSO, according to Na Li and Song Zhu., had the common problem for optimizers of getting trapped in local minima, especially for complex problems [21] .
While conventional methods had lower accuracy, as Fuqiang Zhou, Yi Cui, Bin Peng, and Yexin Wang demonstrated [22] , many of these new swarm intelligence methods were rapidly gaining popularity. This led to the development of the ever-growing branch of metaheuristics that involve nature-inspired algorithms. New optimizers were developed and built-upon like ant colony optimization (ACO) which Bayi Cheng, QiWan, Shanlin Yang, and Xiaoxuan Hu found to have less exploration potential [23] . Similarly, Jiangshao Gu and Kunmei Wen discovered that the slow, barely adaptable glowworm swarm optimization (GSO) algorithm could be improved by combining it with some quantum-behaved properties (physics-based optimizers being another branch of metaheuristics) [24] . Another swarm intelligence optimizer that found itself being combined with another algorithm was the krill herd (KH) algorithm. Again, KH often got trapped in a local optimum and Ahmed M. E. Khalil, Seif-Eddeen K. Fateen, and Adrián Bonilla-Petriciolet had to improve it by combining it with a Monkey Algorithm (MA) [25] . The cuckoo search (CS) algorithm was rather popular, but, as discussed by Kamel Zeltni and Souham Meshoul, was only suited for unconstrained multi-objective problems [26] .
On the conventional side of optimization, Dilini Delgoda, Hector Malano, Syed K. Saleem, and Malka N. Halgamuge found another algorithm, the generic algorithm (GA), also needed more adaptability [27] . Erkan Tanyildizi found another algorithm, the golden sine algorithm (Gold-SA) that had struggles with adaptability, this time with unconstrained problems [28] . C. Chellaswamy et al. found that many of these conventional optimization methods had insufficient randomness, which impared their exploration ability [29] .
Based on the literature, the hallmarks of a good optimization algorithm are as follows:
• Being able to find the global optimum instead of getting trapped in local optima.
• A good balance between exploration and exploitation.
• A fast convergence time.
• Being adaptable enough to be used in a variety of problems.
2.3. Grey Wolf Optimization
Seyedali Mirjalili discovered Grey Wolf Optimization in 2014. It is an optimization method based on the behavior of hunting grey wolf packs. Grey wolves will search for their prey by spreading out, then converging on an area of interest. They will form a perimeter around their prey, then attack. This exploration/exploitation-based behavior makes them good candidates for having a swarm intelligence metaheuristic optimization method modeled after them [16] . GWO was shown to provide competitive or better results to other metaheuristics like PSO and Evolutionary Programming (EP). Along with [17] , researchers [30] continued to develop and prove the accuracy of this optimizer by creating a multi-objective version.
GWO has many applications both in and outside of engineering. Some authors applied an intelligent version of GWO to bidding in the energy market [31] . GWO was also used to design a fuel cell [32] . Sharma and Saikia optimized the controllers of a solar power plant with GWO [33] . In 2018, Daniel used a hybrid generic-GWO algorithm to optimize his novel medical image fusion technique [34] . A Virtual Force Algorithm-Levy-Embedded Grey Wolf Optimizer was proposed for wireless sensor networks [35] . Ma, Mei, Wu, Wu, and Zeng used Grey Wolf Optimization to find the optimal parameters of a function used to predict fossil fuel consumption [36] . Jangir and Jangir designed a Non-dominated Sorting Grey Wolf Optimizer for wind power integration [37] . An, You, Zhou, and Wang used a Grey Wolf Optimizer-based Relevance Vector Machine to predict protein-to-protein interactions [38] .
As stated before, GWO is modeled after grey wolf pack behavior. The advantages this model gives the algorithm include better exploitation by mimicking grey wolf hunting behavior (that is, searching for, surrounding, and finally attacking the prey). The algorithm has additional operators to aid in exploration. GWO is also modeled after the social hierarchy of the grey wolf pack. The social hierarchy part of the model allows for a wider, yet more accurate search capability because the “alphas” (the leaders of the pack) are mathematically modeled as the best solution, the “betas” (enforcers) of the pack are the second-best solution while the “deltas” are the third. The pack’s “omegas”, which include all other solutions, follow the rest of the pack.
The algorithm for GWO is as follows:
1) Initialize the grey wolf population.
2) Initialize coefficients.
3) Calculate the fitness (objective function) of each search agent.
4) Xα = best search agent
5) Xβ = second best search agent
6) Xδ = third best search agent
7) While (t < Max number of iterations)
a) For each search agent
b) Update the position of the current search agent
c) Update coefficients.
d) Calculate the fitness off all search agents
e) Update alpha, beta, and delta positions.
f) t = t + 1
8) Return Xα.
Figure 1 depicts the flowchart for Grey Wolf Optimization.
The above detailed GWO algorithm was applied to solve the wind power prediction problem and the implementation details including the flowchart are explained in the next section.
3. Methodology for Implementation of GWO and MLR
Grey Wolf Optimizer (GWO) is population-based optimization algorithm inspired by the social hierarchy and hunting behavior of grey wolves. It has shown competitive performance in solving various optimization problems, including wind speed prediction.
![]()
Figure 1. Flowchart for grey wolf optimization.
Here are the steps for implementing a GWO for WSP model.
1) Load in Wind Speed data from NOAA-GLERL Chicago metadata archive 2018-2019.
2) Apply K-fold cross-validation to get the training set and the testing set. This paper used 10 folds without shuffling the data.
3) Isolate the dependent variable (wind speed) in the training set.
4) Initialize thetas to zero, the learning rate, and the number of gradient descent iterations.
5) Calculate the theta values of the independent variables using the normal equation.
6) Normalize the independent variables by (X − μ)/σ where X is the dimensions of the data (number of training examples by number of independent variables), μ is the mean of the data, and σ is the standard deviation.
7) Calculate the theta values of the cost value of the dependent variable using gradient descent.
8) Predict the wind speed for a particular example via the thetas obtained from gradient descent.
9) Calculate MAE, MAPE, RMSE, and SSE of actual wind speed values versus the predicted ones.
10) Initialize the number of search agents, the maximum number of GWO iterations, and select which error measurement (MAE, MAPE, RMSE, or SSE) to minimize.
11) Initialize the upper bound, lower bound, and number of variables based on the function selected.
12) Initialize the alpha, beta, and delta positions to zeros and their scores of infinity.
13) Initialize the positions of the first population of search agents.
14) For each iteration of GWO, return back the search agents that go beyond the boundaries of the search space.
15) For each iteration of GWO, calculate the objective function for each search agent.
16) For each iteration of GWO, update alpha, beta, and delta.
17) At the end of the iteration loop, update the position of the search agents including the omegas.
18) Return the alpha position and the alpha score.
Figure 2 shows the flowchart for implementation of GWO and MLR for wind speed prediction.
The Grey Wolf Optimizer (GWO) is a metaheuristic optimization algorithm that was inspired by the hunting behavior of grey wolves. It was proposed by Mirjalili et al. in 2014 and has since been applied to a variety of optimization problems, including function optimization, feature selection, and machine learning.
Multiple Linear Regression (MLR) is a statistical technique used to model the relationship between a dependent variable and one or more independent variables. MLR assumes that there is a linear relationship between the independent variables and the dependent variable, and seeks to estimate the coefficients of the linear equation that best fit the data.
The GWO can be used to optimize the coefficients of the MLR model by minimizing the sum of squared errors between the predicted values of the dependent variable and the actual values in the training data. The GWO starts with an initial population of candidate solutions (wolf pack) and iteratively updates the solutions based on the fitness function (objective function) that measures the quality of the solutions. In the case of MLR, the fitness function is the sum of squared errors.
![]()
Figure 2. Flowchart for the implementation of GWO and MLR.
The GWO algorithm is initialized with a population of grey wolves, which are divided into alpha, beta, delta, and omega wolves. Each wolf represents a candidate solution to the optimization problem. The GWO then simulates the hunting behavior of grey wolves, where alpha wolves lead the hunting and guide other wolves towards the prey. In the GWO algorithm, alpha wolves are updated based on the best solution found so far, while beta, delta, and omega wolves are updated based on the position of the alpha wolf.
The GWO algorithm continues to update the population of wolves until a stopping criterion is met, such as a maximum number of iterations or a minimum error threshold. At the end of the optimization process, the coefficients of the MLR model are obtained from the position of the alpha wolf.
In summary, the GWO algorithm can be used to optimize the coefficients of the MLR model by minimizing the sum of squared errors. The GWO simulates the hunting behavior of grey wolves to iteratively update the candidate solutions until a stopping criterion is met. The optimized coefficients can then be used to predict the dependent variable in the test data.
The wind data used comes from the National Oceanic and Atmospheric Administration (NOAA) - Great Lakes Environmental Research Laboratories (GLERL) Chicago metadata archive. The years 2018 and 2019 were used [39] .
4. Research Results
4.1. Results of GWO-WSP
See Tables 1-3 and Figures 3-6.
![]()
Figure 3. Variation of MAE alpha score with each iteration.
![]()
Table 1. Parameter settings of the GWO and MLR models.
![]()
Figure 4. Variation of MAPE alpha score with each iteration.
![]()
Figure 5. Variation of RMSE alpha score with each iteration.
![]()
Figure 6. Variation of SSE alpha score with each iteration.
![]()
Table 2. Time taken to execute the GWO and MLR models.
![]()
Table 3. Implementation of GWO and MLR.
4.2. Discussion
The results of using Grey Wolf Optimizer (GWO) to optimize the coefficients of Multiple Linear Regression (MLR) can depend on several factors, including the size and complexity of the dataset, the chosen fitness function, and the stopping criterion used.
In general, GWO can be an effective optimization algorithm for MLR, especially when dealing with large and complex datasets. GWO has been shown to perform better than other metaheuristic algorithms such as Genetic Algorithm and Particle Swarm Optimization in terms of both convergence speed and solution quality.
One advantage of using GWO for MLR is that it can handle a large number of independent variables, as it can search for the optimal combination of coefficients among a large number of possible solutions. However, GWO can also suffer from the common problem of metaheuristic algorithms, which is the risk of getting trapped in local optima. Therefore, it is important to set appropriate parameters and run multiple trials with different initial conditions to ensure that the algorithm finds the global optimal solution.
Another consideration is the choice of fitness function. The fitness function in MLR is typically the sum of squared errors between the predicted values and the actual values in the training data. However, there are other metrics such as mean absolute error and R-squared that can be used to evaluate the performance of the MLR model.
Table 1 shows the parameter settings of the GWO and MLR models. Table 2 shows the time taken by MLR and GWO in arriving at the optimal solution. Table 3 presents the performance metrics of the GWO and MLR model, such as the mean absolute error, mean absolute percentage error, root mean square error and sum squared error, before and after applying the GWO optimization. Table 3 can show how the GWO optimization has improved the performance of the MLR model compared to the execution time.
Figures 3-6 help to visualize the convergence behavior of the GWO algorithm over time, such as the change in fitness function (i.e., mean absolute error, mean absolute percentage error, root mean square error and sum squared error) over the iterations. These figures show how the GWO algorithm improves the solution over time and can help in determining the appropriate stopping criterion for the optimization process. Additionally, these figures show the relationship between the predicted values and the actual values in the test data, which can provide an indication of the accuracy of the MLR model after GWO optimization.
Overall, the tables and figures from Section VI can be used to provide a clear and concise summary of the results obtained from using GWO to optimize the coefficients of MLR.
5. Conclusion
In conclusion, in this research paper, a hybrid optimization approach has been used to improve the accuracy of wind speed prediction models. Here, the authors proposed a Grey Wolf Optimizer based Wind Speed Prediction (GWO-WSP). The test results show that the implementation is successful and the approach yields accurate and feasible results. The computation time for execution of the algorithm is found to be fast. The GWO is a powerful optimization algorithm for MLR, but the results depend on various factors such as the dataset size and complexity, the fitness function used, and the stopping criterion. It is important to carefully choose the parameters and evaluate the performance of the MLR model using appropriate metrics to ensure that the algorithm provides accurate and reliable results. The wind data used in the case study was obtained from the National Oceanic and Atmospheric Administration (NOAA) - Great Lakes Environmental Research Laboratories (GLERL) Chicago metadata archive. Both the GWO and MLR have certain limitations when applied to wind speed prediction. It is worth noting that the limitations do not imply that GWO and MLR are ineffective for WSP. However, researchers should be aware of the limitations and consider them when selecting the appropriate modeling techniques and interpreting the results. Additionally, exploring alternative algorithms or incorporating additional features and data preprocessing techniques can help mitigate some of the limitations.
Acknowledgements
The authors wish to thank the University of West Florida, USA for their support and extensive facilities.