Hybrid HPC: can we finally manage our data easily and consistently between on-prem and out in the cloud?

Jakub Szarlat1

1XENON Systems, Springvale, Australia

Biography:

Jakub Szarlat is an experienced Senior Solutions Architect with broad experience in HPC, AI, data storage and management, scientific applications, software stacks and workflows.

With a degree in Software Engineering, Jakub has a unique ability of bridging distinct systems to work as a total solution.

Jakub is an expert in architecting, deploying, and managing traditional and virtualised HPC clusters, large distributed data storage and archive solutions, and cloud computing environments for a variety of use cases ranging from Microscopy VDI to Bioinformatics and other GPU workloads. In addition he is supporting researchers compiling, debugging, deploying, and running their applications.

His exposure to large scientific data sets brings a world of expertise in understanding their implication on today's infrastructure and data management.

More recently at XENON, Jakub has been utilising container technologies to provide more customisable and stable HPC deployments both on-prem and in the cloud.

Abstract:

Public cloud use in a hybrid HPC environment remains a challenge: compute is easy, but data access comes with complications. Storage vendors have implemented various approaches to the problem of bursting data to the cloud. Some use global namespaces to stretch the filesystem others use snapshots and others shared repositories. In this talk we explore these different approaches: their strengths, weaknesses and scenarios where they best apply.

 

 

 

Categories