|
Bellocchi, G., Rivington, M., Matthews, K., & Acutis, M. (2015). Deliberative processes for comprehensive evaluation of agroecological models. A review. Agron. Sust. Developm., 35(2), 589–605.
Abstract: The use of biophysical models in agroecology has increased in the last few decades for two main reasons: the need to formalize empirical knowledge and the need to disseminate model-based decision support for decision makers (such as farmers, advisors, and policy makers). The first has encouraged the development and use of mathematical models to enhance the efficiency of field research through extrapolation beyond the limits of site, season, and management. The second reflects the increasing need (by scientists, managers, and the public) for simulation experimentation to explore options and consequences, for example, future resource use efficiency (i.e., management in sustainable intensification), impacts of and adaptation to climate change, understanding market and policy responses to shocks initiated at a biophysical level under increasing demand, and limited supply capacity. Production concerns thus dominate most model applications, but there is a notable growing emphasis on environmental, economic, and policy dimensions. Identifying effective methods of assessing model quality and performance has become a challenging but vital imperative, considering the variety of factors influencing model outputs. Understanding the requirements of stakeholders, in respect of model use, logically implies the need for their inclusion in model evaluation methods. We reviewed the use of metrics of model evaluation, with a particular emphasis on the involvement of stakeholders to expand horizons beyond conventional structured, numeric analyses. Two major topics are discussed: (1) the importance of deliberative processes for model evaluation, and (2) the role computer-aided techniques may play to integrate deliberative processes into the evaluation of agroecological models. We point out that (i) the evaluation of agroecological models can be improved through stakeholder follow-up, which is a key for the acceptability of model realizations in practice, (ii) model credibility depends not only on the outcomes of well-structured, numerically based evaluation, but also on less tangible factors that may need to be addressed using complementary deliberative processes, (iii) comprehensive evaluation of simulation models can be achieved by integrating the expectations of stakeholders via a weighting system of preferences and perception, (iv) questionnaire-based surveys can help understand the challenges posed by the deliberative process, and (v) a benefit can be obtained if model evaluation is conceived in a decisional perspective and evaluation techniques are developed at the same pace with which the models themselves are created and improved. Scientific knowledge hubs are also recognized as critical pillars to advance good modeling practice in relation to model evaluation (including access to dedicated software tools), an activity which is frequently neglected in the context of time-limited framework programs.
|
|
|
Ma, S., Lardy, R., Graux, A. - I., Ben Touhami, H., Klumpp, K., Martin, R., et al. (2015). Regional-scale analysis of carbon and water cycles on managed grassland systems. Env. Model. Softw., 72, 356–371.
Abstract: Predicting regional and global carbon (C) and water dynamics on grasslands has become of major interest, as grasslands are one of the most widespread vegetation types worldwide, providing a number of ecosystem services (such as forage production and C storage). The present study is a contribution to a regional-scale analysis of the C and water cycles on managed grasslands. The mechanistic biogeochemical model PaSim (Pasture Simulation model) was evaluated at 12 grassland sites in Europe. A new parameterization was obtained on a common set of eco-physiological parameters, which represented an improvement of previous parameterization schemes (essentially obtained via calibration at specific sites). We found that C and water fluxes estimated with the parameter set are in good agreement with observations. The model with the new parameters estimated that European grassland are a sink of C with 213 g C m(-2) yr(-1), which is close to the observed net ecosystem exchange (NEE) flux of the studied sites (185 g C m(-2) yr(-1) on average). The estimated yearly average gross primary productivity (GPP) and ecosystem respiration (RECO) for all of the study sites are 1220 and 1006 g C m(-2) yr(-1), respectively, in agreement with observed average GPP (1230 g C m(-2) yr(-1)) and RECO (1046 g C m(-2) yr(-1)). For both variables aggregated on a weekly basis, the root mean square error (RMSE) was similar to 5-16 g C week(-1) across the study sites, while the goodness of fit (R-2) was similar to 0.4-0.9. For evapotranspiration (ET), the average value of simulated ET (415 mmyr(-1)) for all sites and years is close to the average value of the observed ET (451 mm yr(-1)) by flux towers (on a weekly basis, RMSE similar to 2-8 mm week(-1); R-2 = 0.3-0.9). However, further model development is needed to better represent soil water dynamics under dry conditions and soil temperature in winter. A quantification of the uncertainties introduced by spatially generalized parameter values in C and water exchange estimates is also necessary. In addition, some uncertainties in the input management data call for the need to improve the quality of the observational system.
|
|
|
Molina-Herrera, S., Haas, E., Grote, R., Kiese, R., Klatt, S., Kraus, D., et al. (2017). Importance of soil NO emissions for the total atmospheric NOX budget of Saxony, Germany. Atm. Environ., 152, 61–76.
Abstract: Soils are a significant source for the secondary greenhouse gas NO and assumed to be a significant source of tropospheric NOx in rural areas. Here we tested the LandscapeDNDC model for its capability to simulate magnitudes and dynamics of soil NO emissions for 22 sites differing in land use (arable, grassland and forest) and edaphic as well as climatic conditions. Overall, LandscapeDNDC simulated mean soil NO emissions agreed well with observations (r(2) = 0.82). However, simulated day to day variations of NO did only agree weakly with high temporal resolution measurements, though agreement between simulations and measurements significantly increased if data were aggregated to weekly, monthly and seasonal time scales. The model reproduced NO emissions from high and low emitting sites, and responded to fertilization (mineral and organic) events with pulse emissions. After evaluation, we linked the LandscapeDNDC model to a GIS database holding spatially explicit data on climate, land use, soil and management to quantify the contribution of soil biogenic NO emissions to the total NOx budget for the State of Saxony, Germany. Our calculations show that soils of both agricultural and forest systems are significant sources and contribute to about 8% (uncertainty range: 6 -13%) to the total annual tropospheric NO, budget for Saxony. However, the contributions of soil NO emission to total tropospheric NO, showed a high spatial variability and in some rural regions such as the Ore Mts., simulated soil NO emissions were by far more important than anthropogenic sources. (C) 2016 Elsevier Ltd. All rights reserved.
|
|
|
Sanna, M., Bellocchi, G., Fumagalli, M., & Acutis, M. (2015). A new method for analysing the interrelationship between performance indicators with an application to agrometeorological models. Env. Model. Softw., 73, 286–304.
Abstract: The use of a variety of metrics is advocated to assess model performance but correlated metrics may convey the same information, thus leading to redundancy. Starting from this assumption, a method was developed for selecting, from among a collection of performance indicators, one or more subsets providing the same information as the entire set. The method, based on the definition of “stable correlation”, was applied to 23 performance indicators of agrometeorological models, calculated on large sets of simulated and observed data of four agronomic and meteorological variables: above-ground biomass, leaf area index, hourly air relative humidity and daily solar radiation. Two subsets were determined: {Squared Bias, Root Mean Squared Relative Error, Coefficient of Determination, Pattern Index, Modified Modelling Efficiency}, {Persistence Model Efficiency, Root Mean Squared Relative Error, Coefficient of Determination, Pattern Index}. The method needs corroboration but is statistically founded and can support the implementation of standardized evaluation tools. (C) 2015 Elsevier Ltd. All rights reserved.
|
|
|
Yin, X. G., Kersebaum, K. C., Kollas, C., Manevski, K., Baby, S., Beaudoin, N., et al. (2017). Performance of process-based models for simulation of grain N in crop rotations across Europe. Agric. Syst., 154, 63–77.
Abstract: The accurate estimation of crop grain nitrogen (N; N in grain yield) is crucial for optimizing agricultural N management, especially in crop rotations. In the present study, 12 process-based models were applied to simulate the grain N of i) seven crops in rotations, ii) across various pedo-climatic and agro-management conditions in Europe, under both continuous simulation and single year simulation, and for iv) two calibration levels, namely minimal and detailed calibration. Generally, the results showed that the accuracy of the simulations in predicting grain N increased under detailed calibration. The models performed better in predicting the grain N of winter wheat (Triticum aestivum L.), winter barley (Hordewn vulgare L.) and spring barley (Hordeum vulgare L.) compared to spring oat (Avena saliva L.), winter rye (Secale cereale L.), pea (Piswn sativum L.) and winter oilseed rape (Brassica napus L.). These differences are linked to the intensity of parameterization with better parameterized crops showing lower prediction errors. The model performance was influenced by N fertilization and irrigation treatments, and a majority of the predictions were more accurate under low N and rainfed treatments. Moreover, the multi-model mean provided better predictions of grain N compared to any individual model. In regard to the Individual models, DAISY, FASSET, HERMES, MONICA and STICS are suitable for predicting grain N of the main crops in typical European crop rotations, which all performed well in both continuous simulation and single year simulation. Our results show that both the model initialization and the cover crop effects in crop rotations should be considered in order to achieve good performance of continuous simulation. Furthermore, the choice of either continuous simulation or single year simulation should be guided by the simulation objectives (e.g. grain yield, grain N content or N dynamics), the crop sequence (inclusion of legumes) and treatments (rate and type of N fertilizer) included in crop rotations and the model formalism.
|
|