Marchegiani D1, Basevi J1
1ACCESS-NRI, Canberra, Australia
Biography:
Davide is a Senior Research Software Engineer at ACCESS-NRI, contributing to both the Atmosphere and Model Release teams.
His work centres on building robust infrastructure for scientific software delivery, including CI/CD pipelines for Python package deployment, software environment management on HPC systems, and documentation deployment workflows. He also contributes directly to the development of Python packages and technical documentation across the organisation.
He holds a PhD in Atmospheric Sciences from Monash University and brings hands-on experience in data analysis, climate modelling, and high-performance-computing, giving him a grounding in both the scientific needs and the engineering solutions that serve them.
Outside work, Davide loves singing, playing piano and guitar, cooking, and socialising with friends.
Abstract:
Managing software environments on High-Performance Computing (HPC) systems presents continuous operational challenges. Maintaining consistent, reproducible environments across multiple facilities often burdens support teams, challenges filesystem efficiency through high inode usage, and creates friction for researchers who depend on reliable, up-to-date software stacks.
To address this, we developed a CI/CD-driven framework that automates the full lifecycle – build, test and deployment – of containerised software environments on HPC systems. Environments are packaged as Singularity/Apptainer container images, reducing inode consumption from thousands to a handful of files per environment while guaranteeing reproducibility across facilities. A pipeline triggered on every pull request deploys candidates to a staging area for validation. The same workflow promotes approved environments to production across multiple HPC systems, with staging and production logic controlled through build-time environment variables and target HPC machines managed via GitHub Environments, minimising code duplication and simplifying multi-site operations.
End users interact with deployed environments through the familiar "module load" command, with the underlying infrastructure remaining entirely hidden. In addition, dedicated development environments allow contributors to install packages directly from a Git branch or commit, enabling testing within a production-like environment without waiting for a formal release.
This presentation provides a practical, adaptable blueprint for implementing a reproducible environment delivery pipeline on HPC infrastructure, demonstrating how researchers can actively propose new environments to be automatically built, validated and deployed. It will also show how full traceability, from a "module load" invocation to the exact specification and build history of every deployed version, can reduce operational overhead and increase researcher confidence in their software stacks.
