| Records |
Author  |
Wallach, D.; Thorburn, P.; Asseng, S.; Challinor, A.J.; Ewert, F.; Jones, J.W.; Rötter, R.; Ruane, A. |
| Title |
Estimating model prediction error: Should you treat predictions as fixed or random |
Type |
Journal Article |
| Year |
2016 |
Publication |
Environmental Modelling & Software |
Abbreviated Journal |
Env. Model. Softw. |
| Volume |
84 |
Issue |
|
Pages |
529-539 |
| Keywords |
Crop model; Uncertainty; Prediction error; Parameter uncertainty; Input uncertainty; Model structure uncertainty |
| Abstract |
Crop models are important tools for impact assessment of climate change, as well as for exploring management options under current climate. It is essential to evaluate the uncertainty associated with predictions of these models. We compare two criteria of prediction error; MSEPfixed, which evaluates mean squared error of prediction for a model with fixed structure, parameters and inputs, and MSEPuncertain(X), which evaluates mean squared error averaged over the distributions of model structure, inputs and parameters. Comparison of model outputs with data can be used to estimate the former. The latter has a squared bias term, which can be estimated using hindcasts, and a model variance term, which can be estimated from a simulation experiment. The separate contributions to MSEPuncertain(X) can be estimated using a random effects ANOVA. It is argued that MSEPuncertain(X) is the more informative uncertainty criterion, because it is specific to each prediction situation. |
| Address |
|
| Corporate Author |
|
Thesis |
|
| Publisher |
|
Place of Publication |
|
Editor |
|
| Language |
English |
Summary Language |
|
Original Title |
|
| Series Editor |
|
Series Title |
|
Abbreviated Series Title |
|
| Series Volume |
|
Series Issue |
|
Edition |
|
| ISSN |
1364-8152 |
ISBN |
|
Medium |
Article |
| Area |
|
Expedition |
|
Conference |
|
| Notes |
CropM, ft_macsur |
Approved |
no |
| Call Number |
MA @ admin @ |
Serial |
4773 |
| Permanent link to this record |
| |
|
| |
| Author |
Wallach, D.; Thorburn, P.; Asseng, S.; Challinor, A.J.; Ewert, F.; Jones, J.W.; Rötter, R.; Ruane, A. |
| Title |
Overview paper on comprehensive framework for assessment of error and uncertainty in crop model predictions |
Type |
Report |
| Year |
2016 |
Publication |
FACCE MACSUR Reports |
Abbreviated Journal |
|
| Volume |
8 |
Issue |
|
Pages |
C4.1-D |
| Keywords |
MACSUR_ACK; CropM |
| Abstract |
Crop models are important tools for impact assessment of climate change, as well as for exploring management options under current climate. It is essential to evaluate the uncertainty associated with predictions of these models. Several ways of quantifying prediction uncertainty have been explored in the literature, but there have been no studies of how the different approaches are related to one another, and how they are related to some overall measure of prediction uncertainty. Here we show that all the different approaches can be related to two different viewpoints about the model; either the model is treated as a fixed predictor with some average error, or the model can be treated as a random variable with uncertainty in one or more of model structure, model inputs and model parameters. We discuss the differences, and show how mean squared error of prediction can be estimated in both cases. The results can be used to put uncertainty estimates into a more general framework and to relate different uncertainty estimates to one another and to overall prediction uncertainty. This should lead to a better understanding of crop model prediction uncertainty and the underlying causes of that uncertainty. This study was published as (Wallach et al. 2016) |
| Address |
|
| Corporate Author |
|
Thesis |
|
| Publisher |
|
Place of Publication |
|
Editor |
|
| Language |
|
Summary Language |
|
Original Title |
|
| Series Editor |
|
Series Title |
|
Abbreviated Series Title |
|
| Series Volume |
|
Series Issue |
|
Edition |
|
| ISSN |
|
ISBN |
|
Medium |
|
| Area |
|
Expedition |
|
Conference |
|
| Notes |
|
Approved |
no |
| Call Number |
MA @ office @ |
Serial |
2954 |
| Permanent link to this record |
| |
|
| |
| Author |
Wallach, D.; Rivington, M. |
| Title |
Development of a common set of methods and protocols for assessing and communicating uncertainties |
Type |
Report |
| Year |
2013 |
Publication |
FACCE MACSUR Reports |
Abbreviated Journal |
|
| Volume |
2 |
Issue |
|
Pages |
D-C4.1.1 |
| Keywords |
|
| Abstract |
This reports sets out an outline approach to create definitions of uncertainty and how it might be classified. This is not a prescriptive approach rather it should be seen as a starting point from which further development can be made by consensus with CropM partners and across MACSUR Themes. We propose both a numerical quantification of uncertainty and text based classification scheme. The rational is to be able to both establish the terms and definitions in quantifying the impact of uncertainty on model estimates and have a scheme to enable identification of connectivity between types and sources of uncertainty. The aim is to establish a common set of terms and structure within which they operate that can be used to guide work within CropM. No Label |
| Address |
|
| Corporate Author |
|
Thesis |
|
| Publisher |
|
Place of Publication |
|
Editor |
|
| Language |
|
Summary Language |
|
Original Title |
|
| Series Editor |
|
Series Title |
|
Abbreviated Series Title |
|
| Series Volume |
|
Series Issue |
|
Edition |
|
| ISSN |
|
ISBN |
|
Medium |
|
| Area |
|
Expedition |
|
Conference |
|
| Notes |
|
Approved |
no |
| Call Number |
MA @ admin @ |
Serial |
2241 |
| Permanent link to this record |
| |
|
| |
| Author |
Wallach, D.; Rivington, M. |
| Title |
A framework for assessing the uncertainty in crop model predictions |
Type |
Report |
| Year |
2014 |
Publication |
FACCE MACSUR Reports |
Abbreviated Journal |
|
| Volume |
3 |
Issue |
|
Pages |
D-C4.1.2 |
| Keywords |
|
| Abstract |
It is of major importance in modeling to understand and quantify the uncertainty in model predictions, both in order to know how much confidence to have in those predictions, and as a first step toward model improvement. Here we show that there are basically three different approaches to evaluating uncertainty, and we explain the advantages and drawbacks of each. This is a necessary first step toward developing protocols for evaluation of uncertainty and so obtaining a clearer picture of the reliability of crop models. No Label |
| Address |
|
| Corporate Author |
|
Thesis |
|
| Publisher |
|
Place of Publication |
|
Editor |
|
| Language |
|
Summary Language |
|
Original Title |
|
| Series Editor |
|
Series Title |
|
Abbreviated Series Title |
|
| Series Volume |
|
Series Issue |
|
Edition |
|
| ISSN |
|
ISBN |
|
Medium |
|
| Area |
|
Expedition |
|
Conference |
|
| Notes |
|
Approved |
no |
| Call Number |
MA @ admin @ |
Serial |
2231 |
| Permanent link to this record |
| |
|
| |
| Author |
Wallach, D.; Rivington, M. |
| Title |
Standardised methods and protocols based on current best practices to conduct sensitivity analysis |
Type |
Report |
| Year |
2015 |
Publication |
FACCE MACSUR Reports |
Abbreviated Journal |
|
| Volume |
6 |
Issue |
|
Pages |
D-C4.2.1 |
| Keywords |
|
| Abstract |
The purpose of this report is to propose a general procedure for sensitivity analysis when used to evaluate system sensitivity to climate change, including uncertainty information. While sensitivity analysis has been largely used to evaluate how uncertainties in inputs or parameters propagate through the model and manifest themselves in uncertainties in model outputs, there is much less experience with sensitivity analysis as a tool for studying how sensitive a system is to changes in inputs. This report should help make clear the differences between these two uses of sensitivity analysis, and provide guidance as to the procedure for using sensitivity analysis for evaluating system sensitivity to climate change. No Label |
| Address |
|
| Corporate Author |
|
Thesis |
|
| Publisher |
|
Place of Publication |
|
Editor |
|
| Language |
|
Summary Language |
|
Original Title |
|
| Series Editor |
|
Series Title |
|
Abbreviated Series Title |
|
| Series Volume |
|
Series Issue |
|
Edition |
|
| ISSN |
|
ISBN |
|
Medium |
|
| Area |
|
Expedition |
|
Conference |
|
| Notes |
|
Approved |
no |
| Call Number |
MA @ admin @ |
Serial |
2100 |
| Permanent link to this record |
