Wallach, D., Thorburn, P., Asseng, S., Challinor, A. J., Ewert, F., Jones, J. W., et al. (2016). Estimating model prediction error: Should you treat predictions as fixed or random. Env. Model. Softw., 84, 529–539.
Abstract: Crop models are important tools for impact assessment of climate change, as well as for exploring management options under current climate. It is essential to evaluate the uncertainty associated with predictions of these models. We compare two criteria of prediction error; MSEPfixed, which evaluates mean squared error of prediction for a model with fixed structure, parameters and inputs, and MSEPuncertain(X), which evaluates mean squared error averaged over the distributions of model structure, inputs and parameters. Comparison of model outputs with data can be used to estimate the former. The latter has a squared bias term, which can be estimated using hindcasts, and a model variance term, which can be estimated from a simulation experiment. The separate contributions to MSEPuncertain(X) can be estimated using a random effects ANOVA. It is argued that MSEPuncertain(X) is the more informative uncertainty criterion, because it is specific to each prediction situation.
|
Ewert, F., Rötter, R. P., Bindi, M., Webber, H., Trnka, M., Kersebaum, K. C., et al. (2015). Crop modelling for integrated assessment of risk to food production from climate change. Env. Model. Softw., 72, 287–303.
Abstract: The complexity of risks posed by climate change and possible adaptations for crop production has called for integrated assessment and modelling (IAM) approaches linking biophysical and economic models. This paper attempts to provide an overview of the present state of crop modelling to assess climate change risks to food production and to which extent crop models comply with IAM demands. Considerable progress has been made in modelling effects of climate variables, where crop models best satisfy IAM demands. Demands are partly satisfied for simulating commonly required assessment variables. However, progress on the number of simulated crops, uncertainty propagation related to model parameters and structure, adaptations and scaling are less advanced and lagging behind IAM demands. The limitations are considered substantial and apply to a different extent to all crop models. Overcoming these limitations will require joint efforts, and consideration of novel modelling approaches.
|
Sanna, M., Bellocchi, G., Fumagalli, M., & Acutis, M. (2015). A new method for analysing the interrelationship between performance indicators with an application to agrometeorological models. Env. Model. Softw., 73, 286–304.
Abstract: The use of a variety of metrics is advocated to assess model performance but correlated metrics may convey the same information, thus leading to redundancy. Starting from this assumption, a method was developed for selecting, from among a collection of performance indicators, one or more subsets providing the same information as the entire set. The method, based on the definition of “stable correlation”, was applied to 23 performance indicators of agrometeorological models, calculated on large sets of simulated and observed data of four agronomic and meteorological variables: above-ground biomass, leaf area index, hourly air relative humidity and daily solar radiation. Two subsets were determined: {Squared Bias, Root Mean Squared Relative Error, Coefficient of Determination, Pattern Index, Modified Modelling Efficiency}, {Persistence Model Efficiency, Root Mean Squared Relative Error, Coefficient of Determination, Pattern Index}. The method needs corroboration but is statistically founded and can support the implementation of standardized evaluation tools. (C) 2015 Elsevier Ltd. All rights reserved.
|
Ruiz-Ramos, M., Rodriguez, A., Dosio, A., Goodess, C. M., Harpham, C., Minguez, M. I., et al. (2016). Comparing correction methods of RCM outputs for improving crop impact projections in the Iberian Peninsula for 21st century. Clim. Change, 134(1-2), 283–297.
Abstract: Assessment of climate change impacts on crops in regions of complex orography such as the Iberian Peninsula (IP) requires climate model output which is able to describe accurately the observed climate. The high resolution of output provided by Regional Climate Models (RCMs) is expected to be a suitable tool to describe regional and local climatic features, although their simulation results may still present biases. For these reasons, we compared several post-processing methods to correct or reduce the biases of RCM simulations from the ENSEMBLES project for the IP. The bias-corrected datasets were also evaluated in terms of their applicability and consequences in improving the results of a crop model to simulate maize growth and development at two IP locations, using this crop as a reference for summer cropping systems in the region. The use of bias-corrected climate runs improved crop phenology and yield simulation overall and reduced the inter-model variability and thus the uncertainty. The number of observational stations underlying each reference observational dataset used to correct the bias affected the correction performance. Although no single technique showed to be the best one, some methods proved to be more adequate for small initial biases, while others were useful when initial biases were so large as to prevent data application for impact studies. An initial evaluation of the climate data, the bias correction/reduction method and the consequences for impact assessment would be needed to design the most robust, reduced uncertainty ensemble for a specific combination of location, crop, and crop management.
|
Refsgaard, J. C., Madsen, H., Andréassian, V., Arnbjerg-Nielsen, K., Davidson, T. A., Drews, M., et al. (2014). A framework for testing the ability of models to project climate change and its impacts. Clim. Change, 122(1-2), 271–282.
Abstract: Models used for climate change impact projections are typically not tested for simulation beyond current climate conditions. Since we have no data truly reflecting future conditions, a key challenge in this respect is to rigorously test models using proxies of future conditions. This paper presents a validation framework and guiding principles applicable across earth science disciplines for testing the capability of models to project future climate change and its impacts. Model test schemes comprising split-sample tests, differential split-sample tests and proxy site tests are discussed in relation to their application for projections by use of single models, ensemble modelling and space-time-substitution and in relation to use of different data from historical time series, paleo data and controlled experiments. We recommend that differential-split sample tests should be performed with best available proxy data in order to build further confidence in model projections.
|
