ASKAPsoft/Yandasoft deployment in Pawsey’s Setonix for ASKAP project
Paulus Lahur1, Pascal Elahi2, Eric Bastholm2 1CSIRO, Clayton, VIC, Australia2CSIRO, Kensington, WA, Australia
Abstract
The Australian Square Kilometre Array Pathfinder (ASKAP) is a radio telescope array located at Murchison Radio-astronomy Observatory (MRO) in Western Australia. It is part of Australia Telescope National Facility (ATNF), which is operated and managed by the Space and Astronomy (S&A) business unit of CSIRO. Data from the dishes is ingested and processed using ASKAPsoft, which is a suite of software developed by our computing team. A set of functionalities that are not ASKAP-specific are released as an open source software called Yandasoft. Both are designed to run on high-performance computers.
ASKAP relies heavily on computing facilities in Pawsey Centre. Recently we have transitioned to a new supercomputer called Setonix, where container technology is a major method for software deployment. Docker is being used to create the container image, which is then converted into Singularity (or Apptainer) container for deployment on the target machine.
Using container has proved to be advantageous for end-users, as it dramatically simplifies the process of acquiring and using ASKAPsoft/Yandasoft. However, for the development and deployment process, the story is rather complex. To push for higher hardware performance, newer underlying libraries need to be used, which impacts the software layers above them. Consequently, we found ourselves having to navigate through significant changes regarding the method to construct containers. By adding the use of Spack to help build the base layer of the containers, we have found a working solution.
In this presentation, we would like to share the method and the lessons learned along the way.
Biography
I have been working with CSIRO for eight years as Research Software Engineer. My work involves various aspects of the development and deployment of scientific software.
http://orcid.org/0009-0000-3960-714X