|
Bellocchi, G., Rivington, M., Matthews, K., & Acutis, M. (2015). Deliberative processes for comprehensive evaluation of agroecological models. A review. Agron. Sust. Developm., 35(2), 589–605.
Abstract: The use of biophysical models in agroecology has increased in the last few decades for two main reasons: the need to formalize empirical knowledge and the need to disseminate model-based decision support for decision makers (such as farmers, advisors, and policy makers). The first has encouraged the development and use of mathematical models to enhance the efficiency of field research through extrapolation beyond the limits of site, season, and management. The second reflects the increasing need (by scientists, managers, and the public) for simulation experimentation to explore options and consequences, for example, future resource use efficiency (i.e., management in sustainable intensification), impacts of and adaptation to climate change, understanding market and policy responses to shocks initiated at a biophysical level under increasing demand, and limited supply capacity. Production concerns thus dominate most model applications, but there is a notable growing emphasis on environmental, economic, and policy dimensions. Identifying effective methods of assessing model quality and performance has become a challenging but vital imperative, considering the variety of factors influencing model outputs. Understanding the requirements of stakeholders, in respect of model use, logically implies the need for their inclusion in model evaluation methods. We reviewed the use of metrics of model evaluation, with a particular emphasis on the involvement of stakeholders to expand horizons beyond conventional structured, numeric analyses. Two major topics are discussed: (1) the importance of deliberative processes for model evaluation, and (2) the role computer-aided techniques may play to integrate deliberative processes into the evaluation of agroecological models. We point out that (i) the evaluation of agroecological models can be improved through stakeholder follow-up, which is a key for the acceptability of model realizations in practice, (ii) model credibility depends not only on the outcomes of well-structured, numerically based evaluation, but also on less tangible factors that may need to be addressed using complementary deliberative processes, (iii) comprehensive evaluation of simulation models can be achieved by integrating the expectations of stakeholders via a weighting system of preferences and perception, (iv) questionnaire-based surveys can help understand the challenges posed by the deliberative process, and (v) a benefit can be obtained if model evaluation is conceived in a decisional perspective and evaluation techniques are developed at the same pace with which the models themselves are created and improved. Scientific knowledge hubs are also recognized as critical pillars to advance good modeling practice in relation to model evaluation (including access to dedicated software tools), an activity which is frequently neglected in the context of time-limited framework programs.
|
|
|
Conradt, T., Wechsung, F., & Bronstert, A. (2013). Three perceptions of the evapotranspiration landscape: comparing spatial patterns from a distributed hydrological model, remotely sensed surface temperatures, and sub-basin water balances. Hydrol. Earth System Sci., 17(7), 2947–2966.
Abstract: A problem encountered by many distributed hydrological modelling studies is high simulation errors at interior gauges when the model is only globally calibrated at the outlet. We simulated river runoff in the Elbe River basin in central Europe (148 268 km(2)) with the semi-distributed eco-hydrological model SWIM (Soil and Water Integrated Model). While global parameter optimisation led to Nash-Sutcliffe efficiencies of 0.9 at the main outlet gauge, comparisons with measured runoff series at interior points revealed large deviations. Therefore, we compared three different strategies for deriving sub-basin evapotranspiration: (1) modelled by SWIM without any spatial calibration, (2) derived from remotely sensed surface temperatures, and (3) calculated from long-term precipitation and discharge data. The results show certain consistencies between the modelled and the remote sensing based evapotranspiration rates, but there seems to be no correlation between remote sensing and water balance based estimations. Subsequent analyses for single sub-basins identify amongst others input weather data and systematic error amplification in inter-gauge discharge calculations as sources of uncertainty. The results encourage careful utilisation of different data sources for enhancements in distributed hydrological modelling.
|
|
|
Francone, C., Cassardo, C., Richiardone, R., & Confalonieri, R. (2012). Sensitivity Analysis and Investigation of the Behaviour of the UTOPIA Land-Surface Process Model: A Case Study for Vineyards in Northern Italy. Boundary-Layer Meteorology, 144(3), 419–430.
Abstract: We used sensitivity-analysis techniques to investigate the behaviour of the land-surface model UTOPIA while simulating the micrometeorology of a typical northern Italy vineyard (Vitis vinifera L.) under average climatic conditions. Sensitivity-analysis experiments were performed by sampling the vegetation parameter hyperspace using the Morris method and quantifying the parameter relevance across a wide range of soil conditions. This method was used since it proved its suitability for models with high computational time or with a large number of parameters, in a variety of studies performed on different types of biophysical models. The impact of input variability was estimated on reference model variables selected among energy (e.g. net radiation, sensible and latent heat fluxes) and hydrological (e.g. soilmoisture, surface runoff, drainage) budget components. Maximum vegetation cover and maximum leaf area index were ranked as the most relevant parameters, with sensitivity indices exceeding the remaining parameters by about one order of magnitude. Soil variability had a high impact on the relevance of most of the vegetation parameters: coefficients of variation calculated on the sensitivity indices estimated for the different soils often exceeded 100 %. The only exceptions were represented by maximum vegetation cover and maximum leaf area index, which showed a low variability in sensitivity indices while changing soil type, and confirmed their key role in affecting model results.
|
|
|
Sanna, M., Bellocchi, G., Fumagalli, M., & Acutis, M. (2015). A new method for analysing the interrelationship between performance indicators with an application to agrometeorological models. Env. Model. Softw., 73, 286–304.
Abstract: The use of a variety of metrics is advocated to assess model performance but correlated metrics may convey the same information, thus leading to redundancy. Starting from this assumption, a method was developed for selecting, from among a collection of performance indicators, one or more subsets providing the same information as the entire set. The method, based on the definition of “stable correlation”, was applied to 23 performance indicators of agrometeorological models, calculated on large sets of simulated and observed data of four agronomic and meteorological variables: above-ground biomass, leaf area index, hourly air relative humidity and daily solar radiation. Two subsets were determined: {Squared Bias, Root Mean Squared Relative Error, Coefficient of Determination, Pattern Index, Modified Modelling Efficiency}, {Persistence Model Efficiency, Root Mean Squared Relative Error, Coefficient of Determination, Pattern Index}. The method needs corroboration but is statistically founded and can support the implementation of standardized evaluation tools. (C) 2015 Elsevier Ltd. All rights reserved.
|
|