Bellocchi, G., Rivington, M., & Acutis, M. (2014). Deliberative processes for comprehensive evaluation of agro-ecological models. FACCE MACSUR Mid-term Scientific Conference, 3(S) Sassari, Italy.
Abstract: Biophysical models are acknowledged for examining interactions of agro-ecological systems and fostering communication between scientists, managers and the public. As the role of models grows in importance, there is an increase in the need to assess their quality and performance (Bellocchi et al., 2010). However, the heterogeneity of factors influencing model outputs makes it difficult a full assessment of model features. Where models are used with or for stakeholders then model credibility depends not only on the outcomes of well-structured statistical evaluation but also less tangible factors may need to be addressed using complementary deliberative processes. To expand our horizons in the evaluation of crop and grassland models, approaches have been reviewed with emphasis on using combined metrics. Comprehensive evaluation of simulation models was developed to integrate expectations of stakeholders via a weighting system where lower and upper fuzzy bounds are applied to a set of evaluation metrics. A questionnaire-based survey helped understanding the multi-faceted knowledge and experience required and the substantial challenges posed by the deliberative process. MACSUR knowledge hub holds potential to advance in good modelling practice in relation with model evaluation (including access to appropriate software tools), an activity which is frequently neglected in the context of time-limited projects.
|
Sanna, M., Acutis, M., & Bellocchi, G. (2014). Interrelationship between evaluation metrics to assess agro-ecological models. FACCE MACSUR Mid-term Scientific Conference, 3(S) Sassari, Italy.
Abstract: When evaluating the performances of simulation models, the perception of the quality of the outputs may depend on the statistics used to compare simulated and observed data. In order to have a comprehensive understanding of model performance, the use of a variety of metrics is generally advocated. However, since they may be correlated, the use of two or more metrics may convey the same information, leading to redundancy. This study intends to investigate the interrelationship between evaluation metrics, with the aim of identifying the most useful set of indicators, for assessing simulation performance. Our focus is on agro-ecological modelling. Twenty-three performance indicators were selected to compare simulated and observed data of four agronomic and meteorological variables: above-ground biomass, leaf area index, hourly air relative humidity and daily solar radiation. Indicators were calculated on large data sets, collected to effectively apply correlation analysis techniques. For each variable, the interrelationship between each pair of indicators was evaluated, by computing the Spearman’s rank correlation coefficient. A definition of “stable correlation” was proposed, based on the test of heterogeneity, allowing to assess whether two or more correlation coefficients are equal. An optimal subset of indicators was identified, striking a balance between number of indicators, amount of provided information and information redundancy. They are: Index of Agreement, Squared Bias, Root Mean Squared Relative Error, Pattern Index, Persistence Model Efficiency and Spearman’s Correlation Coefficient. The present study was carried out in the context of CropM-LiveM cross-cutting activities of MACSUR knowledge hub.
|
Bellocchi, G., Martin, R., Shtiliyanova, A., Ben Touhami, H., & Carrère, P. (2014). Vul’Clim – Climate change vulnerability studies in the region Auvergne (France). FACCE MACSUR Mid-term Scientific Conference, 3(S) Sassari, Italy.
Abstract: The region Auvergne (France) is a major livestock territory in Europe (beef and dairy cattle with permanent grasslands), with a place in climate change regional studies assisting policy makers and actors in identifying adaptation and mitigation measures. Vul’Clim is a research grant (Bourse Recherche Filière) of the region Auvergne (February 2014-September 2015) to develop model-based vulnerability analysis approaches for a detailed assessment of climate change impacts at regional scale. Its main goal is the creation of a computer-aided platform for vulnerability assessment of grasslands, in interaction with stakeholders from a cluster of eco-enterprises. A modelling engine provided by the mechanistic, biogeochemical model PaSim (Pasture Simulation model) is the core of the platform. An action studies the changes of scales by varying the granularity of the data available at a given scale (e.g. climate data supplied by global scenarios) to let them being exploited at another scale (e.g. high-resolution pixels). Another action is to develop an assessment framework linking modelling tools to entry data and outputs, including a variety of components: data-entry manager at different spatial resolutions; automatic computation of indicators; gap-filling and data quality check; simulation kernel with the model(s) used; device to represent results as maps and integrated indicators.
|
|
Ma, S., Acutis, M., Barcza, Z., Ben, T., H., Doro, L., Hidy, D., et al. (2014). The grassland model intercomparison of the MACSUR (Modelling European Agriculture with Climate Change for Food Security) European knowledge hub..
|
|
Kipling, P., Saetnan, E., Scollan, D., Bartley, D., Bellocchi, G., Hutchings, J., et al. (2014). Modelling livestock and grassland systems under climate change..
|
