Sieber, S., Amjath-Babu, T. S., McIntosh, B. S., Tscherning, K., Müller, K., Helming, K., et al. (2013). Evaluating the characteristics of a non-standardised Model Requirements Analysis (MRA) for the development of policy impact assessment tools. Env. Model. Softw., 49, 53–63. Abstract: The aim of this paper is to provide a critical analysis of the strengths and weaknesses of a non-standardised Model Requirements Analysis (MRA) used for the purpose of developing the Sustainability Impact Assessment Tool (SIAT). By ‘non-standardised’ we mean not strictly following a published MRA method. The underlying question we are interested in addressing is how non-standardised methods, often employed in research driven projects, compare to defined methods with more standardised structure, with regards their ability to capture model requirements effectively, and with regards their overall usability. Through describing and critically assessing the specific features of the non-standardised MRA employed, the ambition of this paper is to provide insights useful for impact assessment tool (IAT) development. Specifically, the paper will (i) characterise kinds of user requirements relevant to the functionality and design of IATs; (ii) highlight the strengths and weaknesses of non-standardised MRA for user requirements capture, analysis and reflection in the context of IAT; (iii) critically reflect on the process and outcomes of having used a non-standardised MRA in comparison with other more standardised approaches. To accomplish these aims, we first review methods available for IAT development before describing the SIAT development process, including the MRA employed. Major strengths and weaknesses of the MRA method are then discussed in terms of user identification and characterisation, organisational characterisation and embedding, and ability to capture design options for ensuring usability and usefulness. A detailed assessment on the structural differences of MRA with two advanced approaches (Integrated DSS design and goal directed design) and their role in performance of the MRA tool is used to critique the approach employed. The results show that MRA is able to bring thematic integration, establish system performance and technical thresholds as well as detailing quality and transparency guidelines. Nevertheless the discussion points out to a number of deficiencies in application – (i) a need to more effectively characterise potential users, and; (ii) a need to better foster communication among the distinguished roles in the development process. If addressed these deficiencies, SIAT non-standardised MRA could have brought out better outcomes in terms of tool usability and usefulness, and improved embedding of the tool into conditions of targeted end-users. (C) 2013 Elsevier Ltd. All rights reserved. Keywords: impact assessment tools; iat; siat; sustainability; model requirements analysis; user requirement analysis; support; systems; design; methodology; management; decision; science Permanent link \| Save citation: RTF PDF LaTeX Markdown \| Export record: BibTeX Endnote ISI RIS Atom XML MODS XML ODF XML Word XML
Podhora, A., Helming, K., Adenäuer, L., Heckelei, T., Kautto, P., Reidsma, P., et al. (2013). The policy-relevancy of impact assessment tools: Evaluating nine years of European research funding. Environmental Science & Policy, 31, 85–95. Abstract: Since 2002, the European Commission has employed the instrument of ex-ante impact assessments (IA) to help focus its policy-making process on implementing sustainable development. Scientific tools should play an essential role of providing the evidence base to assess the impacts of alternative policy options. To identify the contribution of research funding for IA tool development, this paper analysed the variety of IA tools designed in projects funded by European Framework Programmes (FPs) 6 and 7. The paper is based on project information available on the European Cordis website, individual project websites and a verification of the results by the project coordinators. We analysed the projects from the interests of IA practitioners as tool users (European policy and impact areas addressed by the tools, jurisdictional application levels and tool categories). Out of the 7.781 projects funded in FP6 and FP7, 203 could be identified that designed tools for the IA process. Nearly half of them applied to environmental, agricultural and transport policy areas. Within these areas, the tools primarily addressed environmental impact areas, less economic and least social impact areas. The IA tools focused on European policies. Models represented the largest tool category, whereas approximately half of the tools could not be clearly categorized. Concerning our analysis criteria, the tool descriptions available on the internet were often unclear and thus may limit the application potential of the tools because of a mismatch of technical terms and categorisation criteria between tool providers and tool users. Future IA tools require a joint political and scientific typology and a narrowing of the gaps, e.g., with view to multi-jurisdictional application and a clear reference to the steps of the IA process. (C) 2013 Elsevier Ltd. All rights reserved. Keywords: impact assessment tools; policy appraisal; science policy interface; sustainable development; european commission; affecting land-use; of-the-art; integrated assessment; sustainable development; agricultural systems; analytical framework; union; part Permanent link \| Save citation: RTF PDF LaTeX Markdown \| Export record: BibTeX Endnote ISI RIS Atom XML MODS XML ODF XML Word XML

Abstract: The aim of this paper is to provide a critical analysis of the strengths and weaknesses of a non-standardised Model Requirements Analysis (MRA) used for the purpose of developing the Sustainability Impact Assessment Tool (SIAT). By ‘non-standardised’ we mean not strictly following a published MRA method. The underlying question we are interested in addressing is how non-standardised methods, often employed in research driven projects, compare to defined methods with more standardised structure, with regards their ability to capture model requirements effectively, and with regards their overall usability. Through describing and critically assessing the specific features of the non-standardised MRA employed, the ambition of this paper is to provide insights useful for impact assessment tool (IAT) development. Specifically, the paper will (i) characterise kinds of user requirements relevant to the functionality and design of IATs; (ii) highlight the strengths and weaknesses of non-standardised MRA for user requirements capture, analysis and reflection in the context of IAT; (iii) critically reflect on the process and outcomes of having used a non-standardised MRA in comparison with other more standardised approaches. To accomplish these aims, we first review methods available for IAT development before describing the SIAT development process, including the MRA employed. Major strengths and weaknesses of the MRA method are then discussed in terms of user identification and characterisation, organisational characterisation and embedding, and ability to capture design options for ensuring usability and usefulness. A detailed assessment on the structural differences of MRA with two advanced approaches (Integrated DSS design and goal directed design) and their role in performance of the MRA tool is used to critique the approach employed. The results show that MRA is able to bring thematic integration, establish system performance and technical thresholds as well as detailing quality and transparency guidelines. Nevertheless the discussion points out to a number of deficiencies in application – (i) a need to more effectively characterise potential users, and; (ii) a need to better foster communication among the distinguished roles in the development process. If addressed these deficiencies, SIAT non-standardised MRA could have brought out better outcomes in terms of tool usability and usefulness, and improved embedding of the tool into conditions of targeted end-users. (C) 2013 Elsevier Ltd. All rights reserved.

Keywords: impact assessment tools; iat; siat; sustainability; model requirements analysis; user requirement analysis; support; systems; design; methodology; management; decision; science

| Save citation: RTF PDF LaTeX Markdown

| Export record: BibTeX Endnote ISI RIS Atom XML MODS XML ODF XML Word XML

Abstract: Since 2002, the European Commission has employed the instrument of ex-ante impact assessments (IA) to help focus its policy-making process on implementing sustainable development. Scientific tools should play an essential role of providing the evidence base to assess the impacts of alternative policy options. To identify the contribution of research funding for IA tool development, this paper analysed the variety of IA tools designed in projects funded by European Framework Programmes (FPs) 6 and 7. The paper is based on project information available on the European Cordis website, individual project websites and a verification of the results by the project coordinators. We analysed the projects from the interests of IA practitioners as tool users (European policy and impact areas addressed by the tools, jurisdictional application levels and tool categories). Out of the 7.781 projects funded in FP6 and FP7, 203 could be identified that designed tools for the IA process. Nearly half of them applied to environmental, agricultural and transport policy areas. Within these areas, the tools primarily addressed environmental impact areas, less economic and least social impact areas. The IA tools focused on European policies. Models represented the largest tool category, whereas approximately half of the tools could not be clearly categorized. Concerning our analysis criteria, the tool descriptions available on the internet were often unclear and thus may limit the application potential of the tools because of a mismatch of technical terms and categorisation criteria between tool providers and tool users. Future IA tools require a joint political and scientific typology and a narrowing of the gaps, e.g., with view to multi-jurisdictional application and a clear reference to the steps of the IA process. (C) 2013 Elsevier Ltd. All rights reserved.