Multi-Model Approach for Assessing the Influence of Calibration Criteria on the Water Balance in Ouémé Basin ()
1. Introduction
It is often heard on the news that floods have caused considerable damage and killed several people. This was the case in October 2010 in Benin. These phenomena, far from being extraordinary, have more consequences at present because humans, in search of more space, build their habitats in the major riverbeds. It then becomes necessary to predict the rise in the level of the rivers, to quickly warn the populations in case of danger and thus avoid victims and excessive damage.
To this end, institutions use hydrological models that help, based on rainfall forecasts and knowledge of past rainfall, to predict flows and therefore water levels at a point in a river. Nowadays, climate change has and will continue to have significant hydrological impacts. New modelling approaches therefore need to be explored to fill the current gaps [1] . Thus, rather than creating new models for often limited performance gains [2] , a multi-model approach combining several global hydrological models has been preferred.
These models contain several parameters that cannot be directly measured and must therefore be determined by calibration. To do this, objective functions are used, which are numerical criteria to optimize and measure the gap between observations and simulations. These criteria include the Nash-Sutcliffe criterion, the R2 coefficient of determination and the King-Gupta Efficiency (KGE).
In this context, “How to evaluate the influence of calibration criteria on the water balance resulting from modelling?” is a still-lasting question. The fact is that the best calibration criteria allow for better simulation of the elements of the water balance.
This study aims to evaluate the influence of calibration criteria on the water balance based on several hydrological models. It will thus be a contribution to knowledge on hydrological modelling. The study will specifically (i) model flows with HBV (Hydrologiska Byråns Vattenbalansavdelning, in Swedish) and GR4J (Modèle du Génie Rural à 4 paramètres au pas de temps journalier, in French) models and evaluate the components of the water balance from the simulations, (ii) determine the influence of the performance criteria on the low and high flows with the HBV and GR4J models, (iii) identify the calibration criteria that best influence the water balance components.
2.1. Study Area and Data
Study Area
The Ouémé watershed is located in West Africa in the Republic of Benin (Figure 1). This watershed covers the departments of Donga, Borgou, Collines, Zou, Ouémé, Plateau and Littoral and includes 38 communes, i.e. 49.35% of Benin’s 77 communes. Together with its tributaries, including the Zou and Okpara rivers, it covers an area of 46,200 km2 spread over the Dahomean basement, the upper part of the basin, and the sedimentary formations of the coastal basin, known as the lower Ouémé [3] . The diversity of these geological bedrocks directly or indirectly influences the manifestation of extreme hydroclimatic events [4] .
The Ouémé basin comprises two geological units: an upper basin with an entirely Precambrian substrate and pronounced relief; and a lower basin with soft sedimentary substrate, a low and monotonous relief that favours the spreading and drifting of rivers, erosion and alluvial waterways.
![]()
Figure 1. Presentation of the study area.
The Ouémé basin is located in the dry tropical zone, also known as the Sudanian zone, with a dry and wet season, and is characterized by the coexistence of hortonian runoff and subsurface and subsurface flows on saturated surfaces. The aquifers are shallow and outcropping and would contribute to the flow. It covers the upstream part of the Ouémé which has its source in Atacora. The upper basin of the Ouémé rests on a fractured granite-gneissic basement known as Dahomean, consisting mainly of migmatites and gneisses [5] . Thus, this basement is altered, fractured and cracked in its upper part and then less and less deep and acts as a storage reservoir, with a low transmissivity of the order of 10−4 m²s¹ (Kamagaté, 2006) [6] . The Ouémé basin communicates with the Atlantic Ocean via the channels of Lagos and Cotonou.
2.2. Data Used
Hydrometric data (flows) were obtained from the Benin National Directory of Water (DGEau). The daily precipitation, evapotranspiration, and temperature data were obtained from the Meteo-Benin agency. These data come from rainfall and hydrometric stations and cover the period from 1986 to 2016. The study focused on the data series from 1986 to 1989 for start-up, from 1989 to 2002 for calibration and finally from 2003 to 2016 for the validation period.
3. Methodology
3.1. Hydrological Models
Two hydrological models have been used namely the HBV model and the GR4J model. In this sub-section, we provided some details about the two models and the performance criteria used during the calibration and validation process.
3.1.1. Model HBV
The HBV model and its variants have been applied in a wide range of countries and environments [7] . Bergström [8] mentions applications to more than 200 basins in 30 countries, including snow and glacial basins in the Alps, Himalayas, New Zealand, Greenland and North America, Arctic basins in Scandinavia and northern Alaska, and basins with both humid and semi-arid low-latitude environments. The HBV model has also been applied to basins in Tunisia [9] .
The HBV model consists of different routines and simulates the flow of a watershed, typically at a daily time step, based on time series of precipitation and air temperature as well as estimates of long-term monthly potential evapotranspiration rates. It takes into account topographic features such as area and elevation for the spatial discretization of the field of study into homogeneous zones. The HBV model consists of a production function for vertical flow where the recharge of the water table and the actual evapotranspiration are functions of the current water storage in the ground reservoir and where the formation of the surface flow occurs through three reservoirs, the first of which is non-linear. For soil production functions, the calculations are carried out for each zone. Thus, the model is semi-distributed, whereas the parameters of the transfer function are global by sub-basin. The snow module was deactivated in the current study.
3.1.2. Model GR4J
The development of the GR4J model was initiated by Claude Michel in the early 1980 s at the Cemagref, a public research institute in France. The first version of the model had only one parameter. Further development of the GR4J model was undertaken using a modelling approach with a large number of watersheds used to evaluate and improve the model [10] . The GR4J modelling approach is mainly empirical [11] , and consists of searching for the most efficient data, to obtain a general, efficient and robust model. The result is a parsimonious hydrological model, with successive improved versions.
The GR4J model is a catchment water balance model that links runoff to precipitation and evapotranspiration using daily data. This model works at the scale of a watershed with a daily time step. To use the model on a given watershed, the area of the watershed in km2, daily precipitation time series (mm), daily potential evapotranspiration time series (mm) and daily discharge time series must be provided. Flow data is used for calibration and evaluation of the model. The version of Perrin et al. (2000) was used in this study with four parameters namely the X1 (Production Magazine Capacity (mm)), the X2 (Water Exchange Coefficient (mm)), the X3 (Routing Store Capacity (mm)) and X4 (Base Time (Day)).
3.1.3. Model Evaluation Criteria
Nash-Sutcliffe Efficiency (NSE): it is a widely used criterion in hydrology for modelling. The Nash-Sutcliffe efficiency coefficient expresses the proportion of the residual variance between the simulated and observed values compared to the explained variance of the observed values. An NSE value greater or equal to 0.5 for a comparison of daily flows is considered an acceptable value in some hydrological studies [12] .
Coefficient of determination: it ranges from 0 to 1, and describes how often the distribution of simulated flows agrees with those observed. A value of zero indicates that there is no correlation while a value of 1 shows that the dispersion of the simulated flows is equal to those observed. Authors such as [12] suggest that any R2 value greater than 0.5 for daily flow comparisons is an acceptable threshold in hydrological simulation.
Kling Gupta Efficiency (KGE): it was developed by Kling et al. [13] to provide an interesting decomposition of the Nash criterion. It combines the three components of Nash-Sutcliffe efficiency (NSE) of model errors (i.e. correlation, bias, ratio of variances or coefficients of variation) in a more balanced way, and has been widely used for calibration and evaluation of hydrological models in recent years.
3.2. Modelling Approach
Modelling was done with HBV and GR4J models. The two models were calibrated over a period from 1989 to 2002 and then validated over a period from 2003 to 2016 for the two gauging stations. The calibration period was chosen to encompass all hydrological variability in both watersheds (wet years, dry years, and average years). The general principle of performance analysis is to compare the calculated flow rates to the observed flow rates. In this study, it was considered the three calibration criteria presented above. The simulations were carried out by simultaneously considering the result of the three calibration criteria while focusing each time on the best result of a single efficiency criterion chosen beforehand among the three criteria for the simulation. After obtaining the results of the simulations and the elements of the water balance, we now move on to the extraction of the simulated flows of the extreme periods, i.e. the peak flows and the low flows from the modelling, with each calibration criterion for the two models. For peak flows, simulated flows that correspond to observed flows above the third quartile are extracted, and for low-water flows, simulated flows that correspond to observed flows below the first quartile are extracted. The criteria for calibrating extreme periods are recalculated to determine which one best influences the water balance in the watershed.
4. Results and Discussion
4.1. Water Balance Components Simulated from HBV and GR4J Models
Table 1 presents the results of the calibration simulations and the water balance components from the outputs of the simulation with HBV and GR4J models at Savè outlet and Bonou outlet. The analysis of these tables shows that both models indicate very good calibration results. Indeed, the criteria for Nash, R2 and KGE vary from 0.58 to 0.64, from 0.5 to 0.7 and from 0.5 to 0.53 respectively. These values are greater than 0.5, which is the acceptable threshold in rainfall-runoff modelling on a daily scale [12] . This indicates a very high degree of similarity between the observed and simulated flows (the percentage of observed flows is close to that of simulated flows) and therefore indirectly a very good reproduction of the components of water balance over the catchment. It can also be seen from the analysis of Table 1 that the efficiency criterion R2, despite its results between 0.5 and 0.7 for calibration during the HBV simulation at Savé, has a low similarity between the observed and simulated flows and therefore a poor influence on the results of the water balance elements.
Table 2 presents the results of the validation and the water balance elements from the outputs of the simulation with the HBV and GR4J models at Savè and Bonou outlets. In validation, there is a general decrease in the performance of the efficiency criteria. The Nash, R2, and KGE range from 0.45 to 0.51, 0.34 to 0.67, and 0.36 to 0.62, respectively. Despite this trend, on the one hand, the calibration criteria are generally greater than 0.5 for the two models at the Savè outlet and reproduce very well the water balance components; except the KGE less than 0.5 for HBV modelling at Savè. Despite its relatively poor performance, it shows a very good similarity between the observed and the simulated flows. On the other hand, for the two models at the outlet of Bonou, the criteria of Nash and R2 are less than 0.5, which is below the acceptable threshold in rainfall-runoff modelling. However, the latter shows a very high similarity between the observed and simulated flows and subsequently a very good reproduction of water balance components, except for the R2 for modelling with GR4J at Bonou. As for the KGE, it is greater than 0.5 for the two models at Bonou and therefore a homogeneity between the observed and simulated flows and consequently a good influence on the elements of the water balance.
In Table 1 and Table 2, %∆Q, %∆ AET, and %∆ CS represent the percentage difference in simulated flows, AET, and changes in storage relative to the reference simulation, respectively. Independently of the models (Table 1), there is a high variability in the change of the simulated discharge relative to the observed discharge. The calibration using the R2 indicated the highest deviation in both models and both stations respectively, except for the calibration of HBV at Bonou with a deviation of 1.41%. For Savè station, the deviation is around 20% when the calibration is done with KGE or NSE for both models while at Bonou station, the variability is high among the deviation values. For AET, except for the calibration at Savè with HBV showing an absolute deviation of around 14%, there is generally a small deviation with the remaining models and station compared to the observation, around 4% in absolute value. The deviation in the storage change is very high for all the models and stations. For the validation period (Table 2), similar results in terms of variability were obtained.
![]()
Table 1. Performance of Calibrated Models. Values indicated in red correspond to calibration criteria used.
![]()
Table 2. Performance of Models in Validation. Values indicated in red correspond to the validation criteria used.
The Figure 2 and Figure 3 present the comparative hydrographs of the observed and simulated flows of the three calibration and validation efficiency criteria. It can be noted that there are some overestimation and underestimation of the observed flow for both models whatever the calibration or validation criteria considered. However, the GR4J model seems to perform better than the HBV model for both Savè and Bonou stations.
![]()
Figure 2. Observed and simulated discharge at Savè (mm/d) during the calibration and validation period for different criteria performances.
![]()
Figure 3. Observed and simulated discharge at Bonou (mm/d) during the calibration and validation period using different criteria performances.
4.2. Simulating Extreme Discharges in Wet and Dry Periods
4.2.1. Model Performance in Wet Periods
In descriptive statistics, the 3rd quartile is the data in the series that separates the bottom 75% of the data. The third quartiles of flows observed in the calibration period from 1989 to 2002 for Bonou and Savè are respectively 0.42 mm/d and 0.23 mm/d. For the validation period from 2003 to 2016, the third quartiles of the Bonou and Savè data are 0.47mm/d and 0.269mm/d, respectively. In the current study framework, the observed flows greater than the third quartile and the simulated flows corresponding to the observed flows greater than the third quartile are used for the calculation of the efficiency criteria for wet periods. Table 3 shows the calibration and validation performance of the two models for peak flows.
Indeed, except for the Nash and KGE criteria which are less than zero when the calibration is done using R2 with HBV at Savè, all the other simulations, i.e. calibration done using the Nash and KGE respectively at Bonou and Savè gave acceptable results for the reproduction of the flows of the wet periods. It can also be seen that the KGE and Nash reproduce peak flows very well.
4.2.2. Model Performance in Dry Periods
The 1st quartile is the data in the series that separates the bottom 25% of the data. The first quartiles of flows observed in the calibration period from 1989 to 2002 for Bonou and Savè are respectively 0.017 mm/d and 0.001 mm/d. For the validation period from 2003 to 2016, the first quartiles of the Bonou and Savè data are 0.014 mm/d and 0.006 mm/d respectively. In the current context, the dry periods used to calculate the efficiency criteria are defined by observed flows below the first quartile and simulated flows corresponding to observed flows below the first quartile. There was a drastic decrease in the performance of the three calibration criteria used as can be seen from Table 4.
![]()
Table 3. Performance during wet periods.
![]()
![]()
Table 4. Performance of efficiency criteria in dry periods.
4.3. Comparison and Identification of Calibration Criteria That Best Influence the Water Balance Components
· After the analysis of Table 2 and Table 3, it can be seen that the three calibration criteria, the Nash, the KGE and the R2, simulate very well the long-term flows and therefore indirectly the water balance components.
· After the analysis of Table 3 and Table 4, it can be seen that the KGE simulates peak flows better than the other criteria and the Nash, the R2 simulates peak flows better than those of low flows.
· The analysis of Table 4 shows that KGE, Nash and R2 have a very poor performance for low-flow simulation.
· Nash has a better performance on long-term flows and low flow rates than R2.
· The Nash simulates long-term flow rates more than the KGE.
The analysis of the influence of the calibration criteria on the water balance through two models leads to a general simulation of flows and a good representation of the water balance components, the Nash and the KGE are the best adapted compared to the R2. However, when looking at the peak flow rate, the KGE leads to a very good simulation compared to the Nash criterion and the R2. On the other hand, the three calibration criteria have a very poor performance on low water flows.
5. Conclusions
The interest of this work is to evaluate the influence of calibration criteria on the water balance through several models. The hydrological modelling exercise carried out using hydrometric and rainfall data from the Ouémé basin at Bonou and Savè helped to identify efficiency criteria that better simulate the elements of the long-term hydrological balance and peak flows.
The calibration period was from 1989 to 2002 and the validation period was from 2003 to 2016. The results obtained help to adequately fill the three specific objectives. First of all, the simulations with both models lead to fairly good criteria values. The HBV model and the GR4J model simulate acceptably the hydrological behaviour of the Ouémé watershed in Bonou and Savè gauging stations.
The Nash, KGE and R2 simulate long-term flows very well and therefore indirectly each element of the water balance. It has been retained that the criterion that best influences the water balance components and, the criterion for which the parameters best simulate peak flows, is the KGE. It was also noted that the R2 criterion is not recommended for the simulation of peak flows. The Nash values were not good for peak and low flows compared to their value when a general flow simulation is considered.
Acknowledgements
The authors are grateful to the Centre of Excellence in Water and Sanitation (C2EA) at the National Water Institute of the University of Abomey Calavi, Benin, for the financial support provided for the publication of the paper.