Next Generation Research Cloud Architectures

Adrian Torrie1

1XENON Systems, Springvale, Australia

Biography:

Adrian is a skilled solutions architect with broad experience building and maintaining data systems from the ground up.

He has broad skills across all IT domains, with specific depth in automating data analytics/machine learning/artificial intelligence solutions in hybrid environments, along with the necessary skills for training others in this specialty.

Adrian's last role had the primary focus of enabling self-service, via Platform Engineering, for Data Scientists and Data Engineers, allowing rapid iteration of models and their release.

He excels in delivering secure solutions for large scale, high performance computational requirements using modern automation approaches.

Abstract:

The diverse computing environments for research, from local clusters, edge devices, virtual desktops, to public clouds, presents both significant opportunities and integration challenges for modern research. This presentation introduces a next-generation research cloud architecture designed to create a cohesive and powerful platform across these disparate landscapes. We leverage Kubernetes as a foundational "Data Centre Operating System," providing a consistent operational API, and crucially, cluster mesh technology is employed to seamlessly interconnect heterogeneous clusters, forming a unified and secure research infrastructure. This architecture streamlines Kubernetes operations and lifecycle management, and automates network provisioning for services. The result is a highly automated and robust platform that simplifies complex distributed computing.

Attendees will learn how this unified approach can enhance collaboration, improve resource utilisation, accelerate research workflows, and enable new possibilities for computationally intensive science. We will discuss the design principles, key technological enablers, and the practical benefits this architecture offers to researchers. This approach aims to provide a scalable and adaptable foundation for future eResearch endeavours across a unified computational environment.

 

Categories