| |
Records |
Links |
|
Author |
Sanna, M.; Bellocchi, G.; Fumagalli, M.; Acutis, M. |
|
| |
Title |
A new method for analysing the interrelationship between performance indicators with an application to agrometeorological models |
Type |
Journal Article |
| |
Year |
2015 |
Publication |
Environmental Modelling & Software |
Abbreviated Journal |
Env. Model. Softw. |
|
| |
Volume |
73 |
Issue |
|
Pages |
286-304 |
|
| |
Keywords |
model evaluation; performance indicators; stable correlation; solar-radiation; simulation-model; environmental-models; statistical-methods; crop nitrogen; validation; rice; uncertainty; calibration; software |
|
| |
Abstract  |
The use of a variety of metrics is advocated to assess model performance but correlated metrics may convey the same information, thus leading to redundancy. Starting from this assumption, a method was developed for selecting, from among a collection of performance indicators, one or more subsets providing the same information as the entire set. The method, based on the definition of “stable correlation”, was applied to 23 performance indicators of agrometeorological models, calculated on large sets of simulated and observed data of four agronomic and meteorological variables: above-ground biomass, leaf area index, hourly air relative humidity and daily solar radiation. Two subsets were determined: {Squared Bias, Root Mean Squared Relative Error, Coefficient of Determination, Pattern Index, Modified Modelling Efficiency}, {Persistence Model Efficiency, Root Mean Squared Relative Error, Coefficient of Determination, Pattern Index}. The method needs corroboration but is statistically founded and can support the implementation of standardized evaluation tools. (C) 2015 Elsevier Ltd. All rights reserved. |
|
| |
Address |
|
|
| |
Corporate Author |
|
Thesis |
|
|
| |
Publisher |
|
Place of Publication |
|
Editor |
|
|
| |
Language |
English |
Summary Language |
|
Original Title |
|
|
| |
Series Editor |
|
Series Title |
|
Abbreviated Series Title |
|
|
| |
Series Volume |
|
Series Issue |
|
Edition |
|
|
| |
ISSN |
1364-8152 |
ISBN |
|
Medium |
Article |
|
| |
Area |
|
Expedition |
|
Conference |
|
|
| |
Notes |
CropM LiveM, ftnotmacsur |
Approved |
no |
|
| |
Call Number |
MA @ admin @ |
Serial |
4503 |
|
| Permanent link to this record |
| |
|
| |
|
Author |
Bellocchi, G.; Rivington, M.; Matthews, K.; Acutis, M. |
|
| |
Title |
Deliberative processes for comprehensive evaluation of agroecological models. A review |
Type |
Journal Article |
| |
Year |
2015 |
Publication |
Agronomy for Sustainable Development |
Abbreviated Journal |
Agron. Sust. Developm. |
|
| |
Volume |
35 |
Issue |
2 |
Pages |
589-605 |
|
| |
Keywords |
component-oriented programing; deliberative approach; modeling; model evaluation; multiple metrics; stakeholders; decision-support-systems; environmental-models; performance evaluation; groundwater models; farming systems; climate-change; irene-dll; simulation; validation; integration |
|
| |
Abstract |
The use of biophysical models in agroecology has increased in the last few decades for two main reasons: the need to formalize empirical knowledge and the need to disseminate model-based decision support for decision makers (such as farmers, advisors, and policy makers). The first has encouraged the development and use of mathematical models to enhance the efficiency of field research through extrapolation beyond the limits of site, season, and management. The second reflects the increasing need (by scientists, managers, and the public) for simulation experimentation to explore options and consequences, for example, future resource use efficiency (i.e., management in sustainable intensification), impacts of and adaptation to climate change, understanding market and policy responses to shocks initiated at a biophysical level under increasing demand, and limited supply capacity. Production concerns thus dominate most model applications, but there is a notable growing emphasis on environmental, economic, and policy dimensions. Identifying effective methods of assessing model quality and performance has become a challenging but vital imperative, considering the variety of factors influencing model outputs. Understanding the requirements of stakeholders, in respect of model use, logically implies the need for their inclusion in model evaluation methods. We reviewed the use of metrics of model evaluation, with a particular emphasis on the involvement of stakeholders to expand horizons beyond conventional structured, numeric analyses. Two major topics are discussed: (1) the importance of deliberative processes for model evaluation, and (2) the role computer-aided techniques may play to integrate deliberative processes into the evaluation of agroecological models. We point out that (i) the evaluation of agroecological models can be improved through stakeholder follow-up, which is a key for the acceptability of model realizations in practice, (ii) model credibility depends not only on the outcomes of well-structured, numerically based evaluation, but also on less tangible factors that may need to be addressed using complementary deliberative processes, (iii) comprehensive evaluation of simulation models can be achieved by integrating the expectations of stakeholders via a weighting system of preferences and perception, (iv) questionnaire-based surveys can help understand the challenges posed by the deliberative process, and (v) a benefit can be obtained if model evaluation is conceived in a decisional perspective and evaluation techniques are developed at the same pace with which the models themselves are created and improved. Scientific knowledge hubs are also recognized as critical pillars to advance good modeling practice in relation to model evaluation (including access to dedicated software tools), an activity which is frequently neglected in the context of time-limited framework programs. |
|
| |
Address |
|
|
| |
Corporate Author |
|
Thesis |
|
|
| |
Publisher |
|
Place of Publication |
|
Editor |
|
|
| |
Language |
English |
Summary Language |
|
Original Title |
|
|
| |
Series Editor |
|
Series Title |
|
Abbreviated Series Title |
|
|
| |
Series Volume |
|
Series Issue |
|
Edition |
|
|
| |
ISSN |
1774-0746 1773-0155 |
ISBN |
|
Medium |
Review |
|
| |
Area |
|
Expedition |
|
Conference |
|
|
| |
Notes |
CropM, LiveM, ft_macsur |
Approved |
no |
|
| |
Call Number |
MA @ admin @ |
Serial |
4551 |
|
| Permanent link to this record |
| |
|
| |
|
Author |
Bellocchi, G.; Rivington, M.; Acutis, M. |
|
| |
Title |
Protocol for model evaluation |
Type |
Report |
| |
Year |
2014 |
Publication |
FACCE MACSUR Reports |
Abbreviated Journal |
|
|
| |
Volume |
3 |
Issue |
|
Pages |
D-L2.2/D |
|
| |
Keywords |
|
|
| |
Abstract |
This deliverable focuses on the development of methods for model evaluation in order to have unambiguous indications derived from the use of several evaluation metrics. The information about model quality is aggregated into a single indicator using a fuzzy expert system that can be applied to a wide range of model estimates where suitable test data are available. This is a cross-cutting activity between CropM (C1.4) and LiveM (L2.2). No Label |
|
| |
Address |
|
|
| |
Corporate Author |
|
Thesis |
|
|
| |
Publisher |
|
Place of Publication |
|
Editor |
|
|
| |
Language |
|
Summary Language |
|
Original Title |
|
|
| |
Series Editor |
|
Series Title |
|
Abbreviated Series Title |
|
|
| |
Series Volume |
|
Series Issue |
|
Edition |
|
|
| |
ISSN |
|
ISBN |
|
Medium |
|
|
| |
Area |
|
Expedition |
|
Conference |
|
|
| |
Notes |
|
Approved |
no |
|
| |
Call Number |
MA @ admin @ |
Serial |
2229 |
|
| Permanent link to this record |
| |
|
| |
|
Author |
Pirttioja, N.; Carter, T.R.; Fronzek, S.; Bindi, M.; Hoffmann, H.; Palosuo, T.; Ruiz-Ramos, M.; Tao, F.; Trnka, M.; Acutis, M.; Asseng, S.; Baranowski, P.; Basso, B.; Bodin, P.; Buis, S.; Cammarano, D.; Deligios, P.; Destain, M.F.; Dumont, B.; Ewert, F.; Ferrise, R.; François, L.; Gaiser, T.; Hlavinka, P.; Jacquemin, I.; Kersebaum, K.C.; Kollas, C.; Krzyszczak, J.; Lorite, I.J.; Minet, J.; Minguez, M.I.; Montesino-San Martin, M.; Moriondo, M.; Müller, C.; Nendel, C.; Öztürk, I.; Perego, A.; Rodríguez, A.; Ruane, A.C.; Ruget, F.; Sanna, M.; Semenov, M.A.; Slawinski, C.; Stratonovitch, P.; Supit, I.; Waha, K.; Wang, E.; Wu, L.; Zhao, Z.; Rötter, R.P. |
|
| |
Title |
Temperature and precipitation effects on wheat yield across a European transect: a crop model ensemble analysis using impact response surfaces |
Type |
Journal Article |
| |
Year |
2015 |
Publication |
Climate Research |
Abbreviated Journal |
Clim. Res. |
|
| |
Volume |
65 |
Issue |
|
Pages |
87-105 |
|
| |
Keywords |
climate; crop model; impact response surface; IRS; sensitivity analysis; wheat; yield; climate-change impacts; uncertainty; 21st-century; projections; simulation; growth; region |
|
| |
Abstract |
This study explored the utility of the impact response surface (IRS) approach for investigating model ensemble crop yield responses under a large range of changes in climate. IRSs of spring and winter wheat Triticum aestivum yields were constructed from a 26-member ensemble of process-based crop simulation models for sites in Finland, Germany and Spain across a latitudinal transect. The sensitivity of modelled yield to systematic increments of changes in temperature (-2 to +9°C) and precipitation (-50 to +50%) was tested by modifying values of baseline (1981 to 2010) daily weather, with CO2 concentration fixed at 360 ppm. The IRS approach offers an effective method of portraying model behaviour under changing climate as well as advantages for analysing, comparing and presenting results from multi-model ensemble simulations. Though individual model behaviour occasionally departed markedly from the average, ensemble median responses across sites and crop varieties indicated that yields decline with higher temperatures and decreased precipitation and increase with higher precipitation. Across the uncertainty ranges defined for the IRSs, yields were more sensitive to temperature than precipitation changes at the Finnish site while sensitivities were mixed at the German and Spanish sites. Precipitation effects diminished under higher temperature changes. While the bivariate and multi-model characteristics of the analysis impose some limits to interpretation, the IRS approach nonetheless provides additional insights into sensitivities to inter-model and inter-annual variability. Taken together, these sensitivities may help to pinpoint processes such as heat stress, vernalisation or drought effects requiring refinement in future model development. |
|
| |
Address |
|
|
| |
Corporate Author |
|
Thesis |
|
|
| |
Publisher |
|
Place of Publication |
|
Editor |
|
|
| |
Language |
English |
Summary Language |
|
Original Title |
|
|
| |
Series Editor |
|
Series Title |
|
Abbreviated Series Title |
|
|
| |
Series Volume |
|
Series Issue |
|
Edition |
|
|
| |
ISSN |
0936-577x 1616-1572 |
ISBN |
|
Medium |
Article |
|
| |
Area |
|
Expedition |
|
Conference |
|
|
| |
Notes |
CropM, ft_macsur |
Approved |
no |
|
| |
Call Number |
MA @ admin @ |
Serial |
4662 |
|
| Permanent link to this record |
| |
|
| |
|
Author |
Sanna, M.; Acutis, M.; Bellocchi, G. |
|
| |
Title |
Interrelationship between evaluation metrics to assess agro-ecological models |
Type |
Report |
| |
Year |
2014 |
Publication |
FACCE MACSUR Reports |
Abbreviated Journal |
|
|
| |
Volume |
3 |
Issue |
|
Pages |
Sp3-5 |
|
| |
Keywords |
|
|
| |
Abstract |
When evaluating the performances of simulation models, the perception of the quality of the outputs may depend on the statistics used to compare simulated and observed data. In order to have a comprehensive understanding of model performance, the use of a variety of metrics is generally advocated. However, since they may be correlated, the use of two or more metrics may convey the same information, leading to redundancy. This study intends to investigate the interrelationship between evaluation metrics, with the aim of identifying the most useful set of indicators, for assessing simulation performance. Our focus is on agro-ecological modelling. Twenty-three performance indicators were selected to compare simulated and observed data of four agronomic and meteorological variables: above-ground biomass, leaf area index, hourly air relative humidity and daily solar radiation. Indicators were calculated on large data sets, collected to effectively apply correlation analysis techniques. For each variable, the interrelationship between each pair of indicators was evaluated, by computing the Spearman’s rank correlation coefficient. A definition of “stable correlation” was proposed, based on the test of heterogeneity, allowing to assess whether two or more correlation coefficients are equal. An optimal subset of indicators was identified, striking a balance between number of indicators, amount of provided information and information redundancy. They are: Index of Agreement, Squared Bias, Root Mean Squared Relative Error, Pattern Index, Persistence Model Efficiency and Spearman’s Correlation Coefficient. The present study was carried out in the context of CropM-LiveM cross-cutting activities of MACSUR knowledge hub. No Label |
|
| |
Address |
|
|
| |
Corporate Author |
|
Thesis |
|
|
| |
Publisher |
|
Place of Publication |
|
Editor |
|
|
| |
Language |
|
Summary Language |
|
Original Title |
|
|
| |
Series Editor |
|
Series Title |
|
Abbreviated Series Title |
|
|
| |
Series Volume |
|
Series Issue |
|
Edition |
|
|
| |
ISSN |
|
ISBN |
|
Medium |
|
|
| |
Area |
|
Expedition |
|
Conference |
|
|
| |
Notes |
|
Approved |
no |
|
| |
Call Number |
MA @ admin @ |
Serial |
2222 |
|
| Permanent link to this record |
