Abstract: unless local adaptation can ameliorate these impacts. Ensembles of crop simulation models can be useful tools for assessing if proposed adaptation options are capable of achieving target yields, whilst also quantifying the share of uncertainty in the simulated crop impact resulting from the crop models themselves. Although some studies have analysed the influence of ensemble size on model outcomes, the effect of ensemble composition has not yet been properly appraised. Moreover, results and derived recommendations typically rely on averaged ensemble simulation results without accounting sufficiently for the spread of model outcomes. Therefore, we developed an Ensemble Outcome Agreement (EOA) index, which analyses the effect of changes in composition and size of a multi-model ensemble (MME) to evaluate the level of agreement between MME outcomes with respect to a given hypothesis (e.g. that adaptation measures result in positive crop responses). We analysed the recommendations of a previous study performed with an ensemble of 17 crop models and testing 54 adaptation options for rainfed winter wheat (Triticum aestivwn L.) at Lleida (NE Spain) under perturbed conditions of temperature, precipitation and atmospheric CO2 concentration. Our results confirmed that most adaptations recommended in the previous study have a positive effect. However, we also showed that some options did not remain recommendable in specific conditions if different ensembles were considered. Using EOA, we were able to identify the adaptation options for which there is high confidence in their effectiveness at enhancing yields, even under severe climate perturbations. These include substituting spring wheat for winter wheat combined with earlier sowing dates and standard or longer duration cultivars, or introducing supplementary irrigation, the latter increasing EOA values in all cases. There is low confidence in recovering yields to baseline levels, although this target could be attained for some adaptation options under moderate climate perturbations. Recommendations derived from such robust results may provide crucial information for stakeholders seeking to implement adaptation measures.

Abstract: The use of biophysical models in agroecology has increased in the last few decades for two main reasons: the need to formalize empirical knowledge and the need to disseminate model-based decision support for decision makers (such as farmers, advisors, and policy makers). The first has encouraged the development and use of mathematical models to enhance the efficiency of field research through extrapolation beyond the limits of site, season, and management. The second reflects the increasing need (by scientists, managers, and the public) for simulation experimentation to explore options and consequences, for example, future resource use efficiency (i.e., management in sustainable intensification), impacts of and adaptation to climate change, understanding market and policy responses to shocks initiated at a biophysical level under increasing demand, and limited supply capacity. Production concerns thus dominate most model applications, but there is a notable growing emphasis on environmental, economic, and policy dimensions. Identifying effective methods of assessing model quality and performance has become a challenging but vital imperative, considering the variety of factors influencing model outputs. Understanding the requirements of stakeholders, in respect of model use, logically implies the need for their inclusion in model evaluation methods. We reviewed the use of metrics of model evaluation, with a particular emphasis on the involvement of stakeholders to expand horizons beyond conventional structured, numeric analyses. Two major topics are discussed: (1) the importance of deliberative processes for model evaluation, and (2) the role computer-aided techniques may play to integrate deliberative processes into the evaluation of agroecological models. We point out that (i) the evaluation of agroecological models can be improved through stakeholder follow-up, which is a key for the acceptability of model realizations in practice, (ii) model credibility depends not only on the outcomes of well-structured, numerically based evaluation, but also on less tangible factors that may need to be addressed using complementary deliberative processes, (iii) comprehensive evaluation of simulation models can be achieved by integrating the expectations of stakeholders via a weighting system of preferences and perception, (iv) questionnaire-based surveys can help understand the challenges posed by the deliberative process, and (v) a benefit can be obtained if model evaluation is conceived in a decisional perspective and evaluation techniques are developed at the same pace with which the models themselves are created and improved. Scientific knowledge hubs are also recognized as critical pillars to advance good modeling practice in relation to model evaluation (including access to dedicated software tools), an activity which is frequently neglected in the context of time-limited framework programs.