Building Bunya: An Evergreen and Bespoke Approach to High-Performance Computing Clusters
Sarah Walters, Ashley Wright, Owen Powell 1The University Of Queensland, St Lucia, Queensland, Australia
Abstract
High-Performance Computing (HPC) clusters are complex systems comprising various hardware and software components. Typically, these systems are acquired through a single large purchase and implemented using pre-configured software packages like OpenHPC. However, this conventional approach hampers individual component upgrades and presents budgetary challenges due to substantial upfront costs.
At the University of Queensland (UQ) Research Computing Centre (RCC), we have embraced an evergreen approach to HPC clusters. Our bespoke cluster, Bunya, leverages commodity components, allowing incremental additions of diverse system configurations over time. Along with a regular funding cycle of smaller annual investments, research groups are encouraged to contribute by purchasing additional hardware, which seamlessly integrates into Bunya. This collaborative model ensures a constant influx of cutting-edge technologies, including upcoming GPUs, empowering researchers to leverage the latest architectures.
To address researchers’ evolving needs, we employ modern configuration management techniques and leverage EasyBuild for streamlined software deployment. These practices are complemented by rigorous testing suites that evaluate software before deployment. This enables us to respond rapidly to changing requirements, encourage end-user contributions to package builds, and accommodate a wide range of hardware and software configurations on the Bunya HPC cluster.
In our presentation, we will delve into the advantages of our bespoke design, highlighting the flexibility, control, and adaptability it offers. Additionally, we will discuss the software stack we employ to support the cluster.
Through our approach, we have accelerated the deployment of new hardware and software, providing timely solutions to meet the ever-evolving requirements of our researchers.
Biography
Sarah Walters is a Research Computing Systems Engineer at the University of Queensland (UQ) with a strong background in Unix and Linux systems administration and programming. With experience spanning various positions within UQ and elsewhere, Sarah brings a wealth of knowledge and expertise to her role at the Research Computing Centre. She is passionate about working on projects that make a difference, particularly in improving researcher outcomes.