Abstract: Working with ensembles of crop models is a recent but important development in crop modeling which promises to lead to better uncertainty estimates for model projections and predictions, better predictions using the ensemble mean or median, and closer collaboration within the modeling community. There are numerous open questions about the best way to create and analyze such ensembles. Much can be learned from the field of climate modeling, given its much longer experience with ensembles. We draw on that experience to identify questions and make propositions that should help make ensemble modeling with crop models more rigorous and informative. The propositions include defining criteria for acceptance of models in a crop MME, exploring criteria for evaluating the degree of relatedness of models in a MME, studying the effect of number of models in the ensemble, development of a statistical model of model sampling, creation of a repository for MME results, studies of possible differential weighting of models in an ensemble, creation of single model ensembles based on sampling from the uncertainty distribution of parameter values or inputs specifically oriented toward uncertainty estimation, the creation of super ensembles that sample more than one source of uncertainty, the analysis of super ensemble results to obtain information on total uncertainty and the separate contributions of different sources of uncertainty and finally further investigation of the use of the multi-model mean or median as a predictor.

Abstract: Rice models have been widely used in simulating and predicting rice phenology in contrasting climate zones, however the uncertainties from model structure (different equations or models) and/or model parameters were rarely investigated. Here, five rice phenological models/modules (Le., CERES-Rice, ORYZA2000, RCM, Beta Model and SIMRIW) were applied to simulate rice phenology at 23 experimental stations from 1992 to 2009 in two major rice cultivation regions of China: the northeastern China and the southwestern China. To investigate the uncertainties from model biophysical parameters, each model was run with randomly perturbed 50 sets of parameters. The results showed that the median of ensemble simulations were better than the simulation by most models. Models couldn’t simulate well in some specific years despite of parameters optimization, suggesting model structure limit model performance in some cases. The models adopting accumulative thermal time function (e.g., CERES-Rice and ORYZA2000) had better performance in the southwestern China, in contrast, those adopting exponential function (e.g., Beta model and RCM model) had better performance in the northeastern China. In northeastern China, the contribution of model structure and model parameters to model total variance was, respectively, about 55.90% and 44.10% in simulating heading date, and about 75.43% and 24.57% in simulating maturity date. In the southwestern China, the contribution of model structure and model parameters to model total variance was, respectively, about 79.97% and 27.03% in simulating heading date, about 92.15% and 7.85% in simulating maturity date. Uncertainty from model structure was the most relevant source. The results highlight that the temperature response functions of rice development rate under extreme climate conditions should be improved based on environment-controlled experimental data.

Abstract: The use of a variety of metrics is advocated to assess model performance but correlated metrics may convey the same information, thus leading to redundancy. Starting from this assumption, a method was developed for selecting, from among a collection of performance indicators, one or more subsets providing the same information as the entire set. The method, based on the definition of “stable correlation”, was applied to 23 performance indicators of agrometeorological models, calculated on large sets of simulated and observed data of four agronomic and meteorological variables: above-ground biomass, leaf area index, hourly air relative humidity and daily solar radiation. Two subsets were determined: {Squared Bias, Root Mean Squared Relative Error, Coefficient of Determination, Pattern Index, Modified Modelling Efficiency}, {Persistence Model Efficiency, Root Mean Squared Relative Error, Coefficient of Determination, Pattern Index}. The method needs corroboration but is statistically founded and can support the implementation of standardized evaluation tools. (C) 2015 Elsevier Ltd. All rights reserved.