Jake Caroll1
1The University of Queensland
Biography:
Jake Carroll is the Director of the Research Computing Centre (RCC) at The University of Queensland. He leads a team that designs and operates UQ’s high-performance computing, AI and research data storage services, including the Bunya supercomputer. His work spans HPC and GPU infrastructure, large-scale data storage, and the institutional strategy that determines how universities invest in research computing.
Jake has more than 20 years of experience across the Research and Higher Education sector, Health, Government and Defence. He is a PhD candidate in experimental computer science, holds an MBA from The University of Queensland and is an elected member of the Global Storage Board of Advisory Council.
Jake’s focus is to bring the absolute best technologies to research to change society and improve life, humanity, and the planet. In his spare time, he is a runner, SCUBA diver, and avid traveller.
Abstract:
There is a recent but obvious realisation that HPC is AI and AI is HPC. Research computing is shifting from being a known set of codes, simulations and parameter sweeping techniques into something else. It is now AI, models, generative capability, learned surfaces in the loop and augmented mixed precision solvers in the workflow, alongside all the old favourites. The pursuit? Efficiency, decreased time to discovery and the quest for scientific progress. In this short talk, UQ will exemplify what this looks like in the real world and at the coalface of research by showcasing some of the amazing work taking place at UQ.
UQ’s research computing challenges are vast. There is an expectation that the UQ Research Computing Centre supports a broad research community with ever more diverse workloads in everything from the life sciences, physics, climate, avionics, aerospace, sustainable energy and architecture, through to the humanities, arts and social sciences. UQ’s scale and variety of research means this is not just one HPC or AI use case.
UQ uses an evergreen strategy for its high-performance and supercomputing infrastructure and more broadly, its entire digital research infrastructure estate. This means it can make purchasing decisions on a yearly basis that keep its systems efficient, give researchers cutting-edge technology and allows it to move with the current ebb and flow of global technology trends. UQ’s researchers benefit from the very best technology as near to release day as possible – and UQ’s sustainability and risk is managed carefully.
UQ’s Bunya Phase 4.0 and Phase 5.0 are based on AMDs EPYC CPUs and AMD Instinct GPUs, using high-density HBM3e memory and a sustainable end-to-end direct-to-chip liquid cooled environment. UQ’s choices are research led. High GPU memory density was one of the highest rated “asks” of the co-design process for Bunya Phase 4.0. Much of this comes from the needs of UQ’s sophisticated frontier-class model researchers, who demand access to the very newest and very largest open weight models on the first day of release.
ROCm was a journey for UQ. At the culmination and summit of that journey, ROCm as a software base gives UQ researchers an open pathway for GPU-accelerated supercomputing and AI workloads in a colourfully heterogenous environment that includes AMD Instinct MI210, AMD Instinct MI300x and AMD Instinct MI355x, all running side by side. In reality, it means users no longer care about the colour of their silicon – they can simply plan, build, schedule and execute at scale.
UQ respects its researchers’ right to have a choice. UQ’s strategy is about matching the right architecture to the right workload – avoiding narrow single-system platform dependencies. Every research workload is unique, and each one has a value that may one day be societally impactful. Each must be treated with the right technology to underpin it.
UQ is conscious of the sustainability and environmental impact that data centre assets at this scale place on the world, much less, the operating realities of our sector. To this end, UQ has a long-term view on balanced power delivery, liquid cooling technologies, data centre footprint and long-term research operations.
The lesson for our community is that the convergency of HPC and AI is here to stay, the large memory footprint accelerators are now the currency of AI-enabled research and that flexibility (such as the need for everything from FP4 all the way to FP64 in the same package) are critical for the success of a heterogeneous HPC facility to deliver for the needs of modern users.
UQ is a real example of an institution that decided to do things differently. It did not listen to the market first. It listened to the reason it exists: its users, researchers and stakeholders. With the interests of its scientific integrity, performance and societal impact at the top of its priority list, it uses AMD’s EPYC CPUs and Instinct GPUs in this generation of its evergreen supercomputing programs of works to deliver for its most valuable commodity – its people.
