RRR: Reliability, Replicability, Reproducibility for Climate Models
Aidan Heerdegen1 1Access-nri, Action, ACT, Austrlia
Abstract
Climate models are an imperfect translation of extremely complex scientific understanding into computer code. Imperfect because many assumptions are made to make the problems tractable. This is made more complex because a climate model is a number of separate models of different realms of the earth system, which run independently while exchanging information at their boundaries.
In addition these models are run at large high performance computing (HPC) centres, on exotic hardware utilising specially tuned software, which adds another layer of complexity.
As if that wasn’t bad enough, there is also the software engineering complexity involved with building multiple separate models and their multiple complex dependencies.
What if rather than being an additional layer of complexity that made it difficult to reliably build models, replicate results and reproduce science, software engineering could make this easier rather than harder? Removing complexity instead of increasing it?
At ACCESS-NRI we use spack, a build-from-source package manager that targets HPC, to make building climate models easier, reliable and reproducible.
We perform continuous integration testing of build reproducibility, model replicability, and scientific reproducibility. This eliminates a source of complexity and uncertainty: the model is guaranteed to produce the same results from the same code, or modified code, when those changes should not alter answers. Scientists can be confident that any variation in their climate model outputs is due to factors under their control, rather than changes in software dependencies, or the tools they use to build their model.
Biography
Aidan is the model release team lead for ACCESS-NRI. He is passionate about making software work for people. In previous roles he provided computational support to climate researchers in two ARC Centres of Excellence, and prior to that was a physical chemist working in a software role supporting X-ray scattering research.
https://orcid.org/0000-0002-4481-4896