Delivering Software Solutions to Astronomy Researchers

Prof. Jarrod Hurley1, Dr Jenni Harrison2, Dr Rory Smith4Dr Greg Poole3

1Swinburne University of Technology, Melbourne, Australia,

 2Pawsey Supercomputing Centre, Perth, Australia,

3Swinburne University of Technology, Melbourne, Australia,

4Monash University, Melbourne, Australia



Astronomy Data and Computing Services (ADACS) was established in early 2017 by Astronomy Australia Limited (AAL) to empower the national astronomy community to maximize the scientific return from their data and eResearch infrastructure. ADACS is delivered through a partnership that has been created between Swinburne University, Curtin University and the Pawsey Supercomputing Centre – comprising Melbourne-based and Perth-based nodes.

A key element of ADACS is to provide professional software development and data management services to astronomy researchers. By developing partnerships between researchers and computational scientists – adding professional software engineering, project management, system analysis and design methodologies to projects – we aim to address the quality and performance benchmarks that can be lacking in platforms developed by researcher-only teams. Targeted eResearch fields include big-data analysis and processing, optimization of workflows for high-performance computing, parallel computing techniques, data-sharing and dissemination, large-scale visualization and construction of real-time data analysis platforms. The long-term goal is to provide a central hub for resources in these areas that can be accessed by astronomers – embedding the service within projects to develop nationally significant software pipelines and data platforms.

The proposed session is aligned with the Research Software Engineering stream and specifically the Software as a Service and Development Methods, Tools & Practices themes.

Astronomers apply for ADACS services through a merit-based allocation scheme that is operated twice per year, first submitting an expression of interest, then working with an ADACS member to develop the technical requirements for their project and finally submitting a full proposal which is considered by an independent Time Allocation Committee drawn from the national astronomy community by AAL. Proposals are ranked on merit and matched against the developer weeks (and expertise) available (generally the equivalent of 2-3 full-time developers per year). To date the scheme has focused on short to mid-range projects with an over-subscription rate of 300-500%. This clearly demonstrates a need for such services and a lack of provision in the past. Projects have ranged from developing graphics programming unit (GPU) algorithms for speeding up gravitational wave analysis to enhancing the user interface and back-end for citizen science projects.

A key aspect for the success of these projects is a constructive working relationship between the computational scientists and the researcher (the sponsor) and how this operates within a project management framework. Such a framework can be foreign environment to a researcher, e.g. working in sprints, providing user-stories upfront, so there is a need to be adaptable on both sides. In this session we aim to explore this relationship, providing ADACS case studies from both perspectives, with the aim of sharing our experiences to date and starting discussions with others who have worked through similar experiences. We are also interested in discussions around how we ensure the long-term sustainability of such schemes, how the ongoing needs of delivered projects should be managed and how we can prioritise larger-scale technically challenging projects while still meeting the needs of the general research community.

The proposed BoF session is intended to be 40 minutes in duration.

The proposed format of the session is a mixture of presentations, contributed talks and facilitated discussion as follows:

  • Introduction to ADACS and the methodology behind the delivery of software as a service to researchers within the national astronomy community [10 mins];
  • Case study of an ADACS development project from the developer perspective, focusing on project management, techniques applied and delivered outcomes [10 mins];
  • Case study of an ADACS development project from the researcher perspective, focusing on the science goals, project interaction experience and application of the delivered outcome [10 mins];
  • Facilitated discussion on the developer-researcher relationship when delivering software as a service to a research community, including best practice and lessons learnt to date [10 mins].

The case study from a researcher perspective will be sourced from the pool of completed ADACS projects as a contributed talk. The introduction, case study from a developer perspective and discussion will be provided/convened by the listed convenors/presenters who are all ADACS members.

The targeted audience for the session includes researchers and technical staff with an interest in bringing professional software development practices into the methodology of the scientific research community. The audience need not be astronomy specific. In fact, a primary goal for the facilitated discussion is to initiate conversations aimed at translating support services across scientific domains, promoting collaboration and skill sharing between like-minded entities.


An example of a success story for an already completed ADACS project can be found here:

Below are two examples of ADACS projects currently under development. Both are on track for completion by end of June 2018.

  1. An Automated Data Reduction Pipeline for AAO Data Central

Lead Researcher – Simon O’Toole (Australian Astronomical Observatory: AAO)

Development – ADACS Perth node

Summary: This project will create a data reduction pipeline using python and django to manage CLI functions of an application 2DFDR using a restful API. The API will be accessible by the AAO team in addition to their partners.

  1. GPU Acceleration of Gravitational Wave Signal Models

Lead Researcher – Rory Smith (Monash)

Development – ADACS Swinburne node

Summary: This project will develop a CUDA-based GPU implementation of highly parallelizable gravitational-wave signal models to alleviate the computational bottleneck in Laser Interferometer Gravitational wave Observatory (LIGO) parameter estimation codes. Parameter estimation is an optimal tool for gravitational-wave signal detection but the current high cost prohibits use as a search pipeline.


Professor Jarrod Hurley has led the supercomputing program at Swinburne for the past decade and is the manager of the NCRIS-funded OzSTAR national facility. Hurley obtained his PhD in astrophysics at the University of Cambridge (UK) before research positions as a Hubble Fellow at the American Museum of Natural History (USA) and at Monash University. Hurley has a strong research background in computational astrophysics, specialising in realistic N-body simulations of star cluster evolution. He was a founding member of the Astronomy Supercomputing Time Allocation Committee (ASTAC), a former steering committee member for the Australia National Institute for Theoretical Astrophysics (ANITA), a member of the Astronomy eResearch Advisory Committee (AeRAC) for Astronomy Australia Limited (AAL) and manager of the Swinburne node of the Astronomy Data and Computing Services (ADACS) initiative. Hurley is passionate about creating a HPC environment that readily adopts new technology and enables researchers to meet their research goals.

Recent Comments