Dr. Mark Endrei1, Dr. Nishanthi Dasanayaka1, Dr. Rubbiya Ali2, Mr. Tom Mason2, Mr. Jake Carroll1, Prof. Roger Wepf2
1Research Computing Centre, The University of Queensland, Australia, 2Centre for Microscopy and Microanalysis, The University of Queensland, Australia
Biography:
Mark Endrei is a senior research software developer at the Research Computing Centre, The University of Queensland, Australia. He also has more than 20 years of experience in IT industry, working with large corporations both nationally and internationally. He has a PhD from The University of Queensland and a Bachelor of Engineering Degree (H1) in Computer Systems Engineering from RMIT University.
Nishanthi Dasanayaka completed her PhD in Computer Science from Queensland University of Technology. She has specialized in system architecture design and algorithm development for time-sensitive road safety applications. She is currently working at the Research Computing Center, the University of Queensland. Her professional interests include incorporating cutting-edge web technology into the development of web apps to assist UQ researchers.
Abstract:
Pitschi (Particle Imaging depoT using Storage CacHing Infrastructure) is a state-of-the-art structured repository designed for managing large electron and correlative microscopy datasets at the Centre for Microscopy and Microanalysis (CMM) at UQ. Developed under the ARDC Australian Characterisation Commons at Scale project, Pitschi is built on the open-source Clowder data management framework¹. It adheres to FAIR data principles and fully integrates with UQ's instrument booking system and storage infrastructure.
Pitschi provides end-to-end data management, encompassing raw data capture, storage, ingestion, and indexing, with automatic metadata extraction from compatible file formats, thereby improving searchability and discovery capabilities. The ingested data are accessible across various platforms, including HPCs, personal computers, and processing platforms, with seamless data transport managed by the Metropolitan Data Caching Infrastructure (MeDICI).
Since its deployment in 2022, Pitschi has been rolled out across all CMM laboratories and is operational on 40 scientific acquisition beam-lines. Enhancements include expanded data synchronization capabilities to support a broader range of instruments and onboarding for commercial users and new third-party laboratories.
Significant successes include widespread user appreciation for the system's "drop and forget" data transfer capability. However, challenges remain, particularly regarding the underuse of advanced search and the complexities of user support across multiple sites.
We are planning substantial infrastructure and application upgrades to enhance Pitschi's capabilities, aiming to improve user experience, expand functionality, and ensure robust support across all sites. This update highlights our commitment to advancing data management in microscopy, driving greater efficiencies, and supporting the research community with reliable, cutting-edge tools.