Data Mover for High-throughput Research Instrument Platforms

Koula Tsiaplias, Mr Jared Winton1

1The University Of Melbourne, University Of Melbourne, Australia

Growing instrument numbers and simulation resolution are driving a deluge in research data, and the demand to share data has also increased dramatically. The need for a capable transfer mechanism was highlighted in an ‘Instrument Needs Analysis’ (2019) undertaken at The University of Melbourne. This identified opportunity to improve and centralise the management of data, in turn bringing consistency and coherence to how data is collected, stored, and dispatched.

Representative instrument platforms from across the University were engaged as key project partners, driving requirements for the development of a new capability.  Partners included University’s Cyber Security and the program of work was embedded in the 5-year Peta-scale Campus Initiative, resulting in a matrix of platform data handling maturity and a map towards improved data handling capability.

A new data dispatch framework and software tool called the Data Mover was developed, extending the University RDM platform (Mediaflux) for which there is already long-term commitment and support. The new capability provides an integrated pipeline for instrument operators and users to move large datasets to central storage and thence to different IT services and facilities (e.g., Research Cloud and high-performance computing clusters) both inside and outside of the University. Since 15/Dec/2021, 3,200 data sets from 61 instruments have been handled via this framework.

The new framework provides instrument operators and researchers with a streamlined, secure, and supported data handling process. In addition, it also provides the University with a better understanding of the digital outputs from instrument platforms.


Biography:

Koula is a Service Delivery Consultant within Research Computing Services at the University of Melbourne, focusing on managing projects associated with the delivery of Research Data Storage services.

Over the past 15 years, Koula has worked in a variety of IT roles in the Higher Education and Energy sectors.

In prior years Koula was an associate board member of VIC ICT for Women and the Deputy Director of the Grad Girls program offered by VIC ICT for Women which received the Victorian Innovation Minister’s Diversity Award in 2018.

Categories