Abstract: This deliverable focuses on the development of methods for model evaluation in order to have unambiguous indications derived from the use of several evaluation metrics. The information about model quality is aggregated into a single indicator using a fuzzy expert system that can be applied to a wide range of model estimates where suitable test data are available. This is a cross-cutting activity between CropM (C1.4) and LiveM (L2.2). No Label

Abstract: Crop growth simulation models can differ greatly in their treatment of key processes and hence in their response to environmental conditions. Here, we used an ensemble of 26 process-based wheat models applied at sites across a European transect to compare their sensitivity to changes in temperature (−2 to +9°C) and precipitation (−50 to +50%). Model results were analysed by plotting them as impact response surfaces (IRSs), classifying the IRS patterns of individual model simulations, describing these classes and analysing factors that may explain the major differences in model responses.   The model ensemble was used to simulate yields of winter and spring wheat at sites in Finland, Germany and Spain. Results were plotted as IRSs that show changes in yields relative to the baseline with respect to temperature and precipitation. IRSs of 30-year means and selected extreme years were classified using two approaches describing their pattern.   The expert diagnostic approach (EDA) combines two aspects of IRS patterns: location of the maximum yield (nine classes, Figure 1) and strength of the yield response with respect to climate (four classes), resulting in a total of 36 combined classes defined using criteria pre-specified by experts. The statistical diagnostic approach (SDA) groups IRSs by comparing their pattern and magnitude, without attempting to interpret these features. It applies a hierarchical clustering method, grouping response patterns using a distance metric that combines the spatial correlation and Euclidian distance between IRS pairs. The two approaches were used to investigate whether different patterns of yield response could be related to different properties of the crop models, specifically their genealogy, calibration and process description.   Although no single model property across a large model ensemble was found to explain the integrated yield response to temperature and precipitation perturbations, the application of the EDA and SDA approaches revealed their capability to distinguish: (i) stronger yield responses to precipitation for winter wheat than spring wheat; (ii) differing strengths of response to climate changes for years with anomalous weather conditions compared to period-average conditions; (iii) the influence of site conditions on yield patterns; (iv) similarities in IRS patterns among models with related genealogy; (v) similarities in IRS patterns for models with simpler process descriptions of root growth and water uptake compared to those with more complex descriptions; and (vi) a closer correspondence of IRS patterns in models using partitioning schemes to represent yield formation than in those using a harvest index.   Such results can inform future crop modelling studies that seek to exploit the diversity of multi-model ensembles, by distinguishing ensemble members that span a wide range of responses as well as those that display implausible behaviour or strong mutual similarities.   The full manuscript of this study is currently under revision (Fronzek et al. 2017).

Abstract: ARMOSA is a dynamic simulation model which was developed to simulate crop growth and development, water and nitrogen dynamics under different pedoclimatic conditions and cropping systems in the arable land. The model is meant to be a tool for the evaluation of the impact of different crop management practices on soil nitrogen and carbon cycles and groundwater nitrate pollution. A large data set collected over three to six years from six monitoring sites in Lombardia plain was used to calibrate and validate the model parameters. Measured meteorological data, soil chemical and physical characterizations, crop-related data of different cropping systems allowed for a proper parameterization. Fit indexes showed the reliability of the model in adequately predicting crop-related variables, such as above ground biomass (RRMSE=11.18, EF=0.94, r=0.97), Leaf Area Index maximum value (RRMSE=8.24, EF=0.37, r=0.72), harvest index (RRMSE=19.4, EF=0.32, r=0.74), and crop N uptake (RRMSE=20.25, EF=0.69, r=0.85). Using two different one-year data set from each monitoring site, the model was calibrated and validated, getting to encouraging results: RRMSE=6.28, EF=0.52, r=0.68 for soil water content at different depths, and RRMSE=34.89, EF=0.59, r=0.75 for soil NO3-N content along soil profile. The simulated N leaching was in full agreement with measured data (RRMSE=26.62, EF=0.88, r=0.98).