Comparative Evaluation of the Performance of SWAT, SWAT+, and APEX Models in Simulating Edge of Field Hydrological Processes ()
1. Introduction
Hydrological models are widely used in the understanding and management of both surface and below-surface water flow processes, water-induced soil erosion, and pollutant transport processes. Models are also integral to watershed management planning processes and are often used to estimate load reductions due to the implementation of water source protection measures. Many models exist for the consideration of these assessments. However, they differ in terms of complexity, requirements, underlying equations, and assumptions [1] and, as such, their performance in simulating hydrological processes varies. A review of past studies shows that there is limited research on the performance and application of commonly used models at the field scale level, despite land use and planning activities being undertaken on small-sized areas such as on farm plots. Most comparison studies have been undertaken on relatively larger spatial scales [2] [3] [4] [5]. A comparison of the Water Erosion Prediction Project (WEPP) model [6] and the Soil Water Assessment Tool (SWAT) [7] in [8] for modeling soil erosion in a small watershed (1.62 km2) showed that the performance of models can, indeed be different in micro watersheds.
This study compares the performance of hydrological models at even a smaller spatial scale, in a 0.066 km2 (6.6 ha) micro watershed which realistically represents the size of farmlands in the agricultural sector. Globally, 94% of farmlands are smaller than 5 ha [9]. Specifically, the study assesses the performance of the latest revamped version of the SWAT model, SWAT+ described in [10] vis a vis the widely used old version of SWAT in simulating water yield, soil, and phosphorous loss from a field plot. The performance comparison is aimed at establishing whether modifications incorporated in the new version improve model performance, particularly at the field scale level. Additionally, simulated edge-of-field outputs by SWAT and SWAT+ are compared with those simulated by the Agricultural Policy/Environmental eXtender (APEX) model [11]. APEX has been widely used to simulate satisfactorily landscape processes in small watersheds [12] [13] [14].
An Overview of SWAT, SWAT+, and APEX Models
Whereas SWAT was developed as a river basin scale model suited for large complex watersheds [15], APEX is better suited for small watersheds [16]. SWAT+ adopts most of the theoretical and empirical equations and assumptions in SWAT albeit with a few significant changes incorporated to address the limitations of the older version [10]. Descriptions of model capabilities, underlying equations, and assumptions are detailed in [15] for SWAT and in [16] for APEX. The new structural changes and improvements incorporated in SWAT+ are described in [10]. The models are therefore not described further in detail in this article, but a review of the limitations and strengths of the models is presented.
SWAT is a comprehensive physically based hydrological model that operates on a daily time step at a basin scale [17]. The model is considered one of the most suitable models for predicting the long-term impacts of land management measures on water, sediment, and soil nutrient loss in large complex ungauged watersheds [17] [18] [19]. The model uses a two-level disaggregation scheme; a preliminary sub-basin and stream network delineation based on the watershed’s topography, and further discretization based on land use, slope, and soil type heterogeneity. Areas with the same topographic characteristics, soil type, land use, and management form a Hydrologic Response Unit (HRU), a basic computational unit assumed to be homogeneous in hydrologic response to land cover change. SWAT performance is assessed in this study particularly because of the strengths it has over the other two models. Notably, the model enjoys strong technical support with detailed documentation, several interfaces, tools, and software supporting the pre-and post-processing of data. The main weakness of the SWAT model is the lack of connectivity and interaction of hydrological processes amongst HRUs [20]. The modeling framework ignores flow and pollutant routing between HRUs (Figure 1). Instead, individual processes are simulated for each HRU, and then flow/pollutants are aggregated for entire the sub-basin. Additionally, SWAT does not allow simulations of multicultural plant communities and its simulation of groundwater processes is limited [21]. Finally, numerous additions and modifications to the model over the years have increasingly made the code complicated, bulky, and hard to manage [10].
SWAT+ is a new revised version of the SWAT model whose development was aimed at addressing the weaknesses and limitations of older versions of the model. Even though the basic algorithms used to calculate the processes in the model have not changed, the structure and organization of both the code (object-based) and the input files (relational-based) have undergone considerable modification [10]. The structure of SWAT+ improves the connectivity and interaction of elements and processes within the watershed allowing for flow and pollutant outputs from one landscape unit (LSU) to be routed through another unit (Figure 1). This is accomplished by the delineation of the watershed into landscape units (LSUs).
![]()
Figure 1. Schematic representation of flow and pollutants routing amongst computational units in (a) SWAT (b) APEX and (c) SWAT+ models. Unit 1 is conceptually upstream and unit 3 is the most downstream.
Like SWAT and SWAT+, the APEX model is a continuous, daily time-step model. The individual field simulation component of APEX is generally a field-size subarea. The subarea in APEX behaves functionally the same as an HRU in SWAT. In both spatial units, the weather, soils, and management systems are assumed to be homogeneous [11] [15]. Subareas can be interconnected allowing runoff, sediment, nutrients, and pesticides to route from one subarea to another (Figure 1), just like LSUs in SWAT+.
2. Methods and Materials
2.1. Initial Model Set-Up and Parameterization
All three models were set up for the 6.6 ha Y6 watershed (31.47N, 96.8W, ~168 masl), located within the USDA-ARS Grassland, Soil and Water Research Laboratory Watershed Network near Riesel, TX. The Riesel experimental watersheds consist of small, single land-use watersheds (1.2 - 8.4 ha) with hydrologic, sediment, and nutrient monitoring stations at the outlet to measure the edge of field processes and other relatively larger micro watersheds (17.1 - 125.1 ha) with mixed land uses [22]. Management, precipitation, runoff, air temperature, and sediment data have been collected continuously on these plots since the 1930s, and runoff nutrient since the early 2000s. The configuration, layout, and description of the experimental plots, geophysical characteristics, and installed hydrological monitoring instrumentation are detailed in [22].
SWAT, APEX, and SWAT+ models were set up using a 10 m × 10 m DEM for watershed delineation. The built-in STATSGO soil database in both models, local weather, and field management data were used for watershed discretization and definition. Flow, soil loss, and nutrient data used for model calibration were downloaded from the STEWARDS database [23]. The models were run using their respective editors (SWAT editor, APEX editor, and SWAT+ editor) to generate the initial set of average monthly predictions.
Both models use relatively similar equations, assumptions, and parameters when simulating water budget components, soil, and nutrient losses. Potential evapotranspiration (ET) was estimated using the Penman-Monteith equation. Though more complex and data-intensive than its alternatives, the Penman-Monteith equation is recommended because of its detailed theoretical base and high accuracy in estimating ET [24]. The modified rational equation was used to estimate peak runoff rates and the curve number method to estimate the runoff depths. The rational method is recommended for use in small drainage areas up to 250 km2 [25] and is thus appropriate for this micro watershed. The curve number method uses the total rainfall volume to predict runoff and is suitable for studies like this where rainfall intensity and duration are not accurately known. For both SWAT and SWAT+, the Modified Universal Soil Loss Equation (MUSLE) was used for simulating soil dislodgment, transportation, and sedimentation processes. For APEX, a variant of the MUSLE (MUSS) adapted for small watersheds with no erosion in channels or streams [16] was used. In both models, the EPIC enrichment ratio method was used for estimating sediment-bound phosphorus losses in the runoff, and the groundwater loading effects of agricultural management systems equation for estimating soluble phosphorus in runoff. Uncalibrated runs were performed using default parameters included in respective model editor packages to simulate the edge of field water yield (Yield), soil loss (Sed), and mineral phosphorus (MinP) from the plot.
2.2. Sensitivity Analysis
APEX, SWAT, and SWAT+ are comprehensive process-based models that employ a large set of parameters during the simulation of landscape processes. Sensitivity analysis helps in identifying parameters that have significant impacts on model outputs in complex simulation models such as these by determining how model outputs react to changes in particular input parameter values [26]. The sensitivity of Yield, Sed, and MinP to a long list of parameters (Table 1) was evaluated by undertaking global sensitivity analyses for APEX and SWAT models and local sensitivity analysis for SWAT+. In global sensitivity analysis methods,
![]()
Table 1. Parameters used in sensitivity analysis.
all parameters are simultaneously varied whereas parameters are adjusted singularly, one at a time in local sensitivity analyses. The algorithms included in the standalone APEX-CUTE [27] and SWAT-CUP [28] allow for global sensitivity analysis. To screen and identify the most sensitive parameters in SWAT+, the values of each parameter were changed manually one at a time within the SWAT+ editor interface, while keeping all other parameters constant. The parameters considered during sensitivity analysis include those recommended in [14] and [27] for APEX, in [7] and [29] for SWAT and SWAT+, and those identified as affecting the water balance, soil loss, and phosphorous cycle.
2.3. Calibration and Validation.
Models are an interpretation of reality and are valid only if they represent the “real world” correctly. Calibration and validation of watershed models are necessary steps required to ensure models can make sufficiently accurate predictions of reality. Calibration of APEX and SWAT was done using APEX-CUTE and SWAT-CUP, respectively. During calibration, both programs follow an optimization procedure involving the modification of input files with candidate solutions, calculating, and evaluating model outputs, and iteratively repeating the process until the user-stipulated evaluations are completed. SWAT+ was calibrated manually. Only the top five most influential parameters for each variable identified by the sensitivity analyses were used for model calibration. For all three models, the first 4 years (1998-2001) were excluded from the results since they were used as a warm-up period. A calibration period of five years (2002-2006), when reliable values of water yield, soil loss, and mineral phosphorus loss were recorded, was used for calibration at monthly time steps. The set of parameter solutions that generated the best objective functions was then used during a validation period of three years (2007-2009).
2.4. Performance Evaluation
To calibrate and validate models and for comparison purposes, quantitative information is required to measure model performance. To achieve this, statistical indices are often used as objective functions to determine the quality and reliability of the predictions when compared to observed values. [30] reviewed several statistical evaluation techniques and highly recommended the use of the Nash-Sutcliffe efficiency (NSE), and percent bias (PBIAS) as indicators of model performance. In addition to the above indices, the Coefficient of Determination (R2) has been used in several model evaluation studies [31] [32]. This study used the above three numeric indices for performance evaluation.
The NSE shows the relative magnitude of the variance between the simulated and measured data whereas R2 indicates the degree of collinearity between simulated and measured data. NSE ranges from −∞ to 1 and R2 from 0 to 1. For both statistics, the desired optimal value is 1.0. The PBIAS indicates the average tendency of the simulated data to be larger or smaller than the measured data. The optimal value of PBIAS is 0.0, with low magnitude values indicating an accurate model simulation. A good reference for these indices, detailing the steps for calculating these indices and the equations used in [30]. The rating criterion for satisfactory performance evaluation varies amongst different studies. [31] used NSE > 0.3 and R2 > 0.5 to assess satisfactory performance for discharge and nutrient loss while [32] [33] used an R2 > 0.5 and NSE > 0.4. [30] recommended an NSE > 0.50 for all variables, a PBIAS ± 25% for streamflow, ±55% for sediment, and ±70% for MinP on a monthly time step. This study adopts the criterion recommended in [30] and an R2 > 0.5 for satisfactory performance assessment. Better performance amongst the models was assessed based on which model’s performance statistics were closest to the optimal value. Additionally, model calibration and performance assessment considered the visual comparison of the overall shape of the time series of simulated data vs the observed data.
3. Results
3.1. Sensitivity Analysis
Table 2 lists the top five parameters that influence water yield, and soil and
![]()
Table 2. The top five most sensitive1 parameters used for calibration of SWAT, SWAT+, and APEX models.
1The number of asterisks depicts the degree of sensitivity. The more asterisks, the higher the sensitivity.
mineral phosphorus loss prediction by APEX, SWAT, and SWAT+ models. Results of the sensitivity analysis showed that, in both models, water yield prediction is more impacted by parameters that influence the generation of runoff. In APEX, the peak runoff rate—rainfall energy adjustment factor (APM) was the most influential parameter whereas water yield prediction was most sensitive to the curve number value (CN2) in SWAT and SWAT+. The CN2 parameter indicates the runoff potential of a hydrologic soil cover complex whereas the APM parameter is used to fine-tune the energy factor associated with runoff-rainfall events. In APEX, the erosion-control-practice factor (PEC) and the soluble phosphorus runoff exponent (PARM30) were the most influential parameters driving soil and phosphorous loss, respectively. These parameters do not impact water yield, at least significantly. The PEC factor represents the effectiveness of erosion control measures in the APEX model. However, for SWAT and SWAT+, most of the parameters that drive water yield estimation were also the same factors that significantly influenced soil and nutrient loss prediction.
3.2. Performance of the APEX Model
Predicted edge of field water yield, soil, and mineral phosphorus loss quantities for the 2002-2006 period by the uncalibrated APEX model were all significantly higher than observed values. Calculated NSE values were also unsatisfactory for all variables. Simulated values, particularly for soil loss, contained large outliers, making the NSE value particularly high. NSE is sensitive to extreme values [30]. The model overestimated soil loss more than any other variable and its performance in predicting soil loss was worse than the predictions by SWAT and SWAT+. Calibration improved model performance, delivering near-optimal performance indices for the three variables, particularly water yield and soil loss. Indices of model efficiency (NSE) and collinearity (R2) of simulated data with the observed values for all variables were close to the optimal values (Table 3)
![]()
Table 3. Performance of SWAT, SWAT+, and APEX models in simulating water yield, soil, and mineral phosphorus losses.
for the calibration period. Performance indices were also satisfactory during the validation period.
3.3. SWAT Performance
The uncalibrated SWAT model performed better than the rest of the models in predicting all variables. Correlation and model efficiency were unexpectedly high, especially for water yield and soil loss prediction (Table 3). The predicted soil loss values were consistently lower than the observed values, but the PBIAS value was below the 55% threshold. All other indices save for NSE were within the acceptable range for phosphorus loss estimation. Based on the NSE threshold used in this study, the performance of the SWAT in simulating phosphorous loss was unacceptable, although it would have been acceptable if thresholds used in [32] were adopted. After calibration, the model predicted all variables satisfactorily (Table 3). Its performance matched that of the APEX model. Both models performed well in simulating flow, even though the CN method was used in SWAT and the Green and Ampt (GA) equation in APEX. Performance indices when the CN method was used in APEX were not as good as those generated when the GA method was used. This improved performance of the CN method in SWAT but not in APEX was also observed in [3]. The difference in performance may have to do with the calibration processes rather than the models themselves. In SWAT, the CN2 value can readily be adjusted during calibration. However, in the APEX model, the CN value is not directly adjusted. It is other parameters that influence the CN value that can be adjusted. The predicted soil loss by SWAT was higher than that predicted by APEX, although water yield values were higher for APEX. The higher soil loss values could be due to the relatively lesser deposition of sediments when the SWAT model is used. In SWAT, pollutant yields are merely summed and added directly to the stream whereas, in APEX, pollutants are routed across the landscape, from high-elevation subareas through other subareas to the stream or outlet point. The SWAT model also matched the performance of the APEX model in predicting phosphorus loss, as evidenced by the nearly equal values of R2 and NSE (Table 3).
3.4. Performance of SWAT+
The uncalibrated SWAT+ model, just like SWAT predicted water yield satisfactory, but unsatisfactorily predicted soil and mineral phosphorous loss. The model overpredicted all three variables (Table 3), although the overprediction was lower than that of the uncalibrated APEX model. The major cause for these over-predictions was determined (during calibration) to be due to default values for landcover and SURLAG. By default, the landcover practice was set to a straight row crop providing a good cover condition grown across the slope. Changing this categorization to a straight row crop providing a good cover grown in a terraced and contoured improved performance significantly. This change affects the curve number and manning value which all influence the amount of runoff generated. Also, values of SURLAG, by default, are set to 4.0 in the model. Lower values for this parameter ensure that more potential runoff is retained within the field per day [15], thus reducing runoff, water-induced soil, and mineral phosphorus loss from the watershed. Even though calibration was done manually, the performance of the SWAT+ model matched that of APEX and SWAT models which were subjected to rigorous automatic calibration.
4. Discussion
Mean monthly yields simulated by the SWAT and SWAT+ model were 0.00028 m3/s and 0.00034 m3/s respectively when rounded down to the nearest ten-thousandths value. Despite the small difference between the simulated values, the difference in the calculated PBIAS is substantial (21% and 6% for SWAT and SWAT+ respectively). Computing PBIAS based on values rounded down to the thousandth decimal would show zero PBIAS for both models, which would be misleading. Similarly, processing and recording of measured variables, when the measured values are small can introduce significant errors since computation errors such as those due to rounding down can be carried forward. However, a difference of even 100 m3/s may be insignificant when dealing with discharges from large watersheds. PBIAS calculation when quantities of variables being analyzed are small values is subject to computational errors, and in such cases, the PBIAS index can be misleading.
Uncalibrated runs using the default settings were unsatisfactory for all variables by all models, save for yield and soil loss prediction for SWAT, and yield prediction for the SWAT+ model. Whereas in literature, some studies have used uncalibrated models [13] [34], the results of this study show that such a practice should not be encouraged except when the uncalibrated model has been evaluated and found to predict variables of interest satisfactorily. In data-scarce regions, the SWAT model may be the most appropriate since it can predict water yield and soil loss with minimal or no calibration.
5. Conclusions
Both the basin-wide older version of SWAT, the small watershed APEX, and the new restructured version of SWAT (SWAT+) models use relatively similar equations, assumptions, and parameters when simulating water budget components, soil, and nutrient losses but also have a few but significant differences, for instance on how they spatially conceptualize the routing and flow of runoff and water quality loadings. The performance of the three models in simulating the edge of field water quantity and quality processes for a 6.6 ha agricultural plot was evaluated to determine how the differences amongst the models affect performance at field scale levels. The uncalibrated version of SWAT was able to simulate hydrology and soil loss satisfactorily. The uncalibrated APEX model failed to predict any of the variables satisfactorily whereas the SWAT+ model simulated hydrology but failed to predict water quality variables.
Model calibration significantly improved the performance of the three models, delivering near-optimal performance indicators for hydrology, during both calibration and validation periods. Performance in simulating soil and mineral phosphorus loss by the calibrated models was also relatively high. Notwithstanding the near-equal performance by all the models, the calibrated APEX model performed slightly better in simulating water quality variables than other models. Performance indicators for both variables were generally better than those reported in the literature when the models were used at larger spatial scales. Performance evaluation based on PBIAS values for the edge-of-field processes can be misleading since the index is highly susceptible to computational errors when evaluating variables with generally small values.