Wallach, D., & Rivington, M. (2014). A framework for assessing the uncertainty in crop model predictions (Vol. 3).
Abstract: It is of major importance in modeling to understand and quantify the uncertainty in model predictions, both in order to know how much confidence to have in those predictions, and as a first step toward model improvement. Here we show that there are basically three different approaches to evaluating uncertainty, and we explain the advantages and drawbacks of each. This is a necessary first step toward developing protocols for evaluation of uncertainty and so obtaining a clearer picture of the reliability of crop models. No Label
|
Wallach, D., & Rivington, M. (2015). Standardised methods and protocols based on current best practices to conduct sensitivity analysis (Vol. 6).
Abstract: The purpose of this report is to propose a general procedure for sensitivity analysis when used to evaluate system sensitivity to climate change, including uncertainty information. While sensitivity analysis has been largely used to evaluate how uncertainties in inputs or parameters propagate through the model and manifest themselves in uncertainties in model outputs, there is much less experience with sensitivity analysis as a tool for studying how sensitive a system is to changes in inputs. This report should help make clear the differences between these two uses of sensitivity analysis, and provide guidance as to the procedure for using sensitivity analysis for evaluating system sensitivity to climate change. No Label
|
Wallach, D., & Rivington, M. (2015). Identification and quantification of differences between models (Vol. 6).
Abstract: A major goal of crop model inter-comparison is model improvement, and an important intermediate step toward that goal is understanding in some detail how models differ, and the consequences of those differences. This report is intended as a first attempt at describing possible techniques for relating differences between model outputs to specific aspects of the models. No Label
|
|
Wallach, D., & Rivington, M. (2014). A framework structure to integrate improved methods for uncertainty evaluation, and protocols for methods application (Vol. 3).
|
Wallach, D., Thorburn, P., Asseng, S., Challinor, A. J., Ewert, F., Jones, J. W., et al. (2016). Estimating model prediction error: Should you treat predictions as fixed or random. Env. Model. Softw., 84, 529–539.
Abstract: Crop models are important tools for impact assessment of climate change, as well as for exploring management options under current climate. It is essential to evaluate the uncertainty associated with predictions of these models. We compare two criteria of prediction error; MSEPfixed, which evaluates mean squared error of prediction for a model with fixed structure, parameters and inputs, and MSEPuncertain(X), which evaluates mean squared error averaged over the distributions of model structure, inputs and parameters. Comparison of model outputs with data can be used to estimate the former. The latter has a squared bias term, which can be estimated using hindcasts, and a model variance term, which can be estimated from a simulation experiment. The separate contributions to MSEPuncertain(X) can be estimated using a random effects ANOVA. It is argued that MSEPuncertain(X) is the more informative uncertainty criterion, because it is specific to each prediction situation.
|
