Zukunftsforum Energiewende
20. November 2018
Comprehensive representation of models for energy system analyses: Insights from the Energy Modelling Platform for Europe (EMP-E) 2017 (Müller et al. 2018)
20. November 2018Transparency, reproducibility, and quality of energy system analyses – A process to improve scientific work (Hülk et al. 2018)
Ludwig Hülk, Berit Müller, Martin Glauer, Elisa Förster, Birgit Schachler
Paper, eingereicht und veröffentlicht in Energy Strategy Reviews, Volume 22, November 2018
Abstract
In this article we apply the transparency checklist methodology by Cao et al. to a case study to evaluate the degree of transparency in energy system modelling. We use a recent case study conducted by the Reiner Lemoine Institut that analyses heat and electricity supply in a German region. An analysis of the questions of the transparency checklist indicates that different issues are addressed: transparency, reproducibility and quality. The completed checklist with answers regarding the case study and its supplementing materials shows that a large majority of the transparency (93%) and reproducibility criteria (73%) could be satisfied. However, only a smaller proportion (47%) of the questions categorised as quality criteria could be answered satisfyingly. A total of 31 out of 45 checklist questions are answered (69%). More than half (53%) of all questions were answered in the introduced framework, model and scenario fact sheets on the OpenEnergyPlatform. The gaps in answering the checklist (e.g., documenting assumptions, uncertainties, and validations) can be closed by an enhancement of the fact sheets and additional tools like the presented scenario log. We conclude that supplementing a final report or study with the presented fact sheets improves good scientific practice significantly while we identified existing weaknesses.
In this article we apply the transparency checklist methodology by Cao et al. to a case study to evaluate the degree of transparency in energy system modelling. We use a recent case study conducted by the Reiner Lemoine Institut that analyses heat and electricity supply in a German region. An analysis of the questions of the transparency checklist indicates that different issues are addressed: transparency, reproducibility and quality. The completed checklist with answers regarding the case study and its supplementing materials shows that a large majority of the transparency (93%) and reproducibility criteria (73%) could be satisfied. However, only a smaller proportion (47%) of the questions categorised as quality criteria could be answered satisfyingly. A total of 31 out of 45 checklist questions are answered (69%). More than half (53%) of all questions were answered in the introduced framework, model and scenario fact sheets on the OpenEnergyPlatform. The gaps in answering the checklist (e.g., documenting assumptions, uncertainties, and validations) can be closed by an enhancement of the fact sheets and additional tools like the presented scenario log. We conclude that supplementing a final report or study with the presented fact sheets improves good scientific practice significantly while we identified existing weaknesses.