Abstract: The most common way to evaluate simulation models is to quantify the agreement between observations and simulations via statistical metrics such as the root mean squared error and the linear regression coefficient of determination. It is agreed that the aggregation of metrics of different nature intro integrated indicators offers a valuable way to assess models. Expanded notions of model evaluation that have recently emerged, based on the trade-off between properties of the model and agreement between predictions and actual data under contrasting conditions, integrate sensitivity analysis measures and information criteria for model selection, as well as concepts of model robustness, and point to expert judgments to explore the importance of different metrics. As a FACCE MACSUR CropM-LiveM action, a composite indicator (MQIm: Model Quality Indicator for multi-site assessment) was elaborated, by a group of specialists, on metrics commonly used to evaluate crop models (with extension to grassland models) while also integrating aspects of model complexity and stability of performances. The indicator, based on fuzzy bounds applied to a set of weighed metrics, was first revised by a broader group of modellers and then assessed via questionnaire survey of scientists and end-users. We document a crop model evaluation in Europe and assess to what extent the MQIm reflects the main components of model quality and supports inferences about model performances. No Label

Abstract: Two independently developed simulation models – the grassland-specific PaSim and the biome-generic Biome-BGC MuSo (BBGC MuSo) – linking climate, soil, vegetation and management to ecosystem biogeochemical cycles were compared in a simulation of carbon (C) and water fluxes. The results were assessed against eddy-covariance flux data from five observational grassland sites representing a range of conditions in Europe: Grillenburg in Germany, Laqueuille in France with both extensive and intensive management, Monte Bondone in Italy and Oensingen in Switzerland. Model comparison (after calibration) gave substantial agreement, the performances being marginal to acceptable for weekly-aggregated gross primary production and ecosystem respiration (R-2 similar to 0.66 – 0.91), weekly evapotranspiration (R-2 similar to 0.78 – 0.94), soil water content in the topsoil (R-2 similar to 0.1 -0.7) and soil temperature (R-2 similar to 0.88 – 0.96). The bias was limited to the range -13 to 9 g C m(-2) week(-1) for C fluxes (-11 to 8 g C m(-2) week(-1) in case of BBGC MuSo, and -13 to 9 g C m(-2) week(-1) in case of PaSim) and -4 to 6 mm week for water fluxes (with BBGC MuSo providing somewhat higher estimates than PaSim), but some higher relative root mean square errors indicate low accuracy for prediction, especially for net ecosystem exchange The sensitivity of simulated outputs to changes in atmospheric carbon dioxide concentration ([CO2]), temperature and precipitation indicate, with certain agreement between the two models, that C outcomes are dominated by [CO2] and temperature gradients, and are less due to precipitation. ET rates decrease with increasing [CO2] in PaSim (consistent with experimental knowledge), while lack of appropriate stomatal response could be a limit in BBGC MuSo responsiveness. Results of the study indicate that some of the errors might be related to the improper representation of soil water content and soil temperature. Improvement is needed in the model representations of soil processes (especially soil water balance) that strongly influence the biogeochemical cycles of managed and unmanaged grasslands. (C) 2015 Elsevier B.V. All rights reserved.

Abstract: • We simulate biomass, soil water content (SWC) and temperature (ST) in grasslands. • We compare nine models to the multi-model median (MMM) at nine sites. • With model calibration, we obtain satisfactory estimates of ST, less of SWC and biomass. • We observe discrepancies across models in the simulation of grassland processes. • We improve performance with multi-model approach. This study presents results from a major grassland model intercomparison exercise, and highlights the main challenges faced in the implementation of a multi-model ensemble prediction system in grasslands. Nine, independently developed simulation models linking climate, soil, vegetation and management to grassland biogeochemical cycles and production were compared in a simulation of soil water content (SWC) and soil temperature (ST) in the topsoil, and of biomass production. The results were assessed against SWC and ST data from five observational grassland sites representing a range of conditions – Grillenburg in Germany, Laqueuille in France with both extensive and intensive management, Monte Bondone in Italy and Oensingen in Switzerland – and against yield measurements from the same sites and other experimental grassland sites in Europe and Israel. We present a comparison of model estimates from individual models to the multi-model ensemble (represented by multi-model median: MMM). With calibration (seven out of nine models), the performances were acceptable for weekly-aggregated ST (R² > 0.7 with individual models and >0.8–0.9 with MMM), but less satisfactory with SWC (R² < 0.6 with individual models and < ∼ 0.5 with MMM) and biomass (R² < ∼0.3 with both individual models and MMM). With individual models, maximum biases of about −5 °C for ST, −0.3 m3 m−3 for SWC and 360 g DM m−2 for yield, as well as negative modelling efficiencies and some high relative root mean square errors indicate low model performance, especially for biomass. We also found substantial discrepancies across different models, indicating considerable uncertainties regarding the simulation of grassland processes. The multi-model approach allowed for improved performance, but further progress is strongly needed in the way models represent processes in managed grassland systems.