The Biocommons Nextflow Tower Service: A user-friendly and infrastructure-flexible platform to run and monitor Nextflow pipelines

Dr Nigel Ward1, Dr Ziad Al Bkhetan1, Dr Johan R. Gustafsson1, Audrey Stott2, Uwe Winter1, Dr Matthew Downton3, Lisa Phippard1, Dr Steven Manos1

1Australian BioCommons, , , 2Pawsey Supercomputing Centre, , , 3National Computational Infrastructure, ,

In 2022 the Australian Biocommons conducted a national survey of diverse research communities on bioinformatics workflows. The responses indicated Nextflow was the most popular language for developing and reusing workflows. Subsequent consultations revealed a collective interest in deploying a shared platform to launch, manage, and monitor Nextflow pipelines across various cloud and HPC infrastructures.

In response, the Australian Biocommons is deploying and operating a Nextflow Tower service as part of the ARDC Bring Your Own Data (BYOD) platforms project. The service provides a web-accessible interface for individuals and communities to conduct their analyses utilising shared nextflow pipelines. Various compute platforms can be utilised – including commercial cloud, the ARDC cloud, NCI, Pawsey, and institutional on-prem cloud and HPC infrastructures.

A collaboration was established with Seqera (the company behind Tower) and a year-long Australia-wide, research-use licence of Tower was procured, with the Nextflow Tower platform deployed at Pawsey and made available to BioCommons partners. The service integrates with NCI, Pawsey and AWS compute infrastructures. Several community-prioritised and commonly used pipelines have been made available through the platform and the list is growing.

Research communities can utilise the Australian Biocommons Nextflow Tower service to directly execute commonly used pipelines (growing list) on various compute infrastructures seamlessly with little set-up and configuration. The service accelerates and facilitates configuration and execution of sophisticated analyses and minimises the efforts and time spent on preparing supporting compute environments.


Biography:

Dr Nigel Ward has a 25 year history providing information technology advice to science and education communities. In his role of Associate Director (Platforms) at the Australian BioCommons he works with multiple BioCommons partner organisations to define the shape of and direct the establishment and operations of key national bioinformatics platforms, including: Galaxy Australia, the Australian Apollo Service, the Australian fgenesh++ service, and the BPA data portal.

Categories