Abstract: Working with ensembles of crop models is a recent but important development in crop modeling which promises to lead to better uncertainty estimates for model projections and predictions, better predictions using the ensemble mean or median, and closer collaboration within the modeling community. There are numerous open questions about the best way to create and analyze such ensembles. Much can be learned from the field of climate modeling, given its much longer experience with ensembles. We draw on that experience to identify questions and make propositions that should help make ensemble modeling with crop models more rigorous and informative. The propositions include defining criteria for acceptance of models in a crop MME, exploring criteria for evaluating the degree of relatedness of models in a MME, studying the effect of number of models in the ensemble, development of a statistical model of model sampling, creation of a repository for MME results, studies of possible differential weighting of models in an ensemble, creation of single model ensembles based on sampling from the uncertainty distribution of parameter values or inputs specifically oriented toward uncertainty estimation, the creation of super ensembles that sample more than one source of uncertainty, the analysis of super ensemble results to obtain information on total uncertainty and the separate contributions of different sources of uncertainty and finally further investigation of the use of the multi-model mean or median as a predictor.

Abstract: While crop models are widely used to assess the change in crop productivity with climate change, their skill in assessing irrigation water demand or the risk of crop failure in large area impact assessments is relatively unknown. The objective of this study is to investigate which aspects of modeling crop water use (reference crop evapotranspiration (ET0), soil water extraction, soil evaporation, soil water balance and root growth) contributes most to the variability in estimates of maize crop water use and the risk of crop failure, and demonstrate the resulting uncertainty in a climate change impact study for Europe. The SIMPLACE crop modeling framework was used to couple the LINTUL5 crop model in factorial combinations of 2-3 different approaches for simulating the 5 aspects of crop water use, resulting in 51 modeling approaches. Using experiments in France and New Zeland, analysis of total sensitivity revealed that ET0 explained the most variability in both irrigated maize water use and rainfed grain yield levels, with soil evaporation also imporatant in the French experiment. In the European impact study, net irrigation requirement differed by 36% between the Penman and Hargreaves ET0 methods in the baseline period. Average EU grain yields were similar between models, but differences approached 1-2 tonnes in parts of France and Southern Europe. EU wide esimates of crop failure in the historical period ranged between 5.4 years for Priestley-Taylor to every 7.9 years for the Penman ET0 methods. While the uncertainty in absolute values between models was significant, estimates of relative changes were similar between models, confirming the utility of crop models in assessing climate change impacts. If ET0 estimates in crop models can be improved, through the use of appropriate methods, uncertainty in irrigation water demand as well as in yield estimates under drought can be reduced.

Abstract: When evaluating the performances of simulation models, the perception of the quality of the outputs may depend on the statistics used to compare simulated and observed data. In order to have a comprehensive understanding of model performance, the use of a variety of metrics is generally advocated. However, since they may be correlated, the use of two or more metrics may convey the same information, leading to redundancy. This study intends to investigate the interrelationship between evaluation metrics, with the aim of identifying the most useful set of indicators, for assessing simulation performance. Our focus is on agro-ecological modelling. Twenty-three performance indicators were selected to compare simulated and observed data of four agronomic and meteorological variables: above-ground biomass, leaf area index, hourly air relative humidity and daily solar radiation. Indicators were calculated on large data sets, collected to effectively apply correlation analysis techniques. For each variable, the interrelationship between each pair of indicators was evaluated, by computing the Spearman’s rank correlation coefficient. A definition of “stable correlation” was proposed, based on the test of heterogeneity, allowing to assess whether two or more correlation coefficients are equal. An optimal subset of indicators was identified, striking a balance between number of indicators, amount of provided information and information redundancy. They are: Index of Agreement, Squared Bias, Root Mean Squared Relative Error, Pattern Index, Persistence Model Efficiency and Spearman’s Correlation Coefficient. The present study was carried out in the context of CropM-LiveM cross-cutting activities of MACSUR knowledge hub.