Mr. Mitchell Hargreaves1, Doctor Geoff Duniam1, Doctor Slava Kitaeff1
1Monash University, Australia
Biography:
Mitchell Hargreaves is experienced with training and deploying deep learning models as well as building data engineering pipelines. He is passionate about reducing barriers of entry and making these tools more available for all. He is the primary developer and system administrator for MLeRP.
Geoff has over twenty-five years of experience as a DBA, Data and Systems Architect, developing, designing and performance-tuning database systems for commercial clients in the resources, mining and commercial sectors. After returning to Australia after 11 years working in the UK, Geoff undertook an M-Phil and then a PhD in large-scale data design and analysis, focussing on the data analysis requirements for the Square Kilometer Array at the International Centre for Radio Astronomy Research in Perth. Geoff's main interests are data design and modelling, effective data management and orchestration, and high-performance computing, with specific interests in parallel programming and deep learning.
Abstract:
Current in-house solutions have been pivotal in Monash University's data storage and transfer infrastructure; however, the data management requirements have outgrown currently used systems, particularly in their ability to provide data reliable ingest from the instruments generating large datasets and large volumes of data, ability to easily move the data between the storage and computing facilities, to automate the data processing pipelines, and easily share and collaborate on data.
We’ve been investigating various data orchestration solutions and deployed Globus as a pilot solution of choice. We report on our journey in trialling Globus. We will share our experience, challenges, and will discuss the deployed solutions across the Monash network and Monash eResearch infrastructure.