Abstract: Working with ensembles of crop models is a recent but important development in crop modeling which promises to lead to better uncertainty estimates for model projections and predictions, better predictions using the ensemble mean or median, and closer collaboration within the modeling community. There are numerous open questions about the best way to create and analyze such ensembles. Much can be learned from the field of climate modeling, given its much longer experience with ensembles. We draw on that experience to identify questions and make propositions that should help make ensemble modeling with crop models more rigorous and informative. The propositions include defining criteria for acceptance of models in a crop MME, exploring criteria for evaluating the degree of relatedness of models in a MME, studying the effect of number of models in the ensemble, development of a statistical model of model sampling, creation of a repository for MME results, studies of possible differential weighting of models in an ensemble, creation of single model ensembles based on sampling from the uncertainty distribution of parameter values or inputs specifically oriented toward uncertainty estimation, the creation of super ensembles that sample more than one source of uncertainty, the analysis of super ensemble results to obtain information on total uncertainty and the separate contributions of different sources of uncertainty and finally further investigation of the use of the multi-model mean or median as a predictor.

Abstract: The accurate estimation of crop grain nitrogen (N; N in grain yield) is crucial for optimizing agricultural N management, especially in crop rotations. In the present study, 12 process-based models were applied to simulate the grain N of i) seven crops in rotations, ii) across various pedo-climatic and agro-management conditions in Europe, under both continuous simulation and single year simulation, and for iv) two calibration levels, namely minimal and detailed calibration. Generally, the results showed that the accuracy of the simulations in predicting grain N increased under detailed calibration. The models performed better in predicting the grain N of winter wheat (Triticum aestivum L.), winter barley (Hordewn vulgare L.) and spring barley (Hordeum vulgare L.) compared to spring oat (Avena saliva L.), winter rye (Secale cereale L.), pea (Piswn sativum L.) and winter oilseed rape (Brassica napus L.). These differences are linked to the intensity of parameterization with better parameterized crops showing lower prediction errors. The model performance was influenced by N fertilization and irrigation treatments, and a majority of the predictions were more accurate under low N and rainfed treatments. Moreover, the multi-model mean provided better predictions of grain N compared to any individual model. In regard to the Individual models, DAISY, FASSET, HERMES, MONICA and STICS are suitable for predicting grain N of the main crops in typical European crop rotations, which all performed well in both continuous simulation and single year simulation. Our results show that both the model initialization and the cover crop effects in crop rotations should be considered in order to achieve good performance of continuous simulation. Furthermore, the choice of either continuous simulation or single year simulation should be guided by the simulation objectives (e.g. grain yield, grain N content or N dynamics), the crop sequence (inclusion of legumes) and treatments (rate and type of N fertilizer) included in crop rotations and the model formalism.

Abstract: Highlights
• 14 criteria for use of crop models in assessments of impacts, adaptation and risk • Working with stakeholders to identify timing of risks is key to risk assessments. • Multiple methods needed to critically assess the use of climate model output • Increasing transparency and inter-comparability needed in risk assessments
Abstract
Crop models are used for an increasingly broad range of applications, with a commensurate proliferation of methods. Careful framing of research questions and development of targeted and appropriate methods are therefore increasingly important. In conjunction with the other authors in this special issue, we have developed a set of criteria for use of crop models in assessments of impacts, adaptation and risk. Our analysis drew on the other papers in this special issue, and on our experience in the UK Climate Change Risk Assessment 2017 and the MACSUR, AgMIP and ISIMIP projects. The criteria were used to assess how improvements could be made to the framing of climate change risks, and to outline the good practice and new developments that are needed to improve risk assessment. Key areas of good practice include: i. the development, running and documentation of crop models, with attention given to issues of spatial scale and complexity; ii. the methods used to form crop-climate ensembles, which can be based on model skill and/or spread; iii. the methods used to assess adaptation, which need broadening to account for technological development and to reflect the full range options available. The analysis highlights the limitations of focussing only on projections of future impacts and adaptation options using pre-determined time slices. Whilst this long-standing approach may remain an essential component of risk assessments, we identify three further key components: 1. Working with stakeholders to identify the timing of risks. What are the key vulnerabilities of food systems and what does crop-climate modelling tell us about when those systems are at risk? 2. Use of multiple methods that critically assess the use of climate model output and avoid any presumption that analyses should begin and end with gridded output. 3. Increasing transparency and inter-comparability in risk assessments. Whilst studies frequently produce ranges that quantify uncertainty, the assumptions underlying these ranges are not always clear. We suggest that the contingency of results upon assumptions is made explicit via a common uncertainty reporting format; and/or that studies are assessed against a set of criteria, such as those presented in this paper.