Mr Jay Van Schyndel1
1Monash University eResearch Centre, Clayton, Australia
The area of Electron Microscopy (EM) is data processing intensive. There is a capability gap nationally that the ‘EM Data Processing Portal’ is addressing. The portal is focussing on supporting the bioscience and materials science communities by running the software CryoSPARC and LiberTEM. The ‘EM Data Processing Portal’ will be accessible to all researchers in the country and is deployed via the Nectar partners, Monash University and Queensland Cyber Infrastructure Foundation (QCIF). Funding for the hardware was provided by the Australian Research Data Commons (ARDC).
The challenge is to deploy CryoSPARC, which is a web-server based product, in a high performance computing environment in two separate physical locations with minimal administration.
Deployment of this service has been achieved by combining our web portal for running desktops on Slurm but adapted to running CryoSPARC and LiberTEM, with Openstack and Ansible to setup the two clusters. The Australian Access Federation (AAF) is employed for user authentication to create the service. Globus is the tool of choice for data movement.
In this talk we will dive into the challenges of delivering the service. Details of how to make CryoSPARC run on a cluster, splitting up NVIDIA A100 GPUs into smaller GPUs, using Gitlab pipelines to confirm your ansible scripts work, and finally user authentication.
Biography:
Jay van Schyndel, Andreas Hamacher and Chris Hines are members of the MASSIVE team based at the Monash University eResearch Centre. Jay is involved in supporting the Big Data communities on MASSIVE with a focus on Cryo Electron Microscopy. Andreas Hamacher is a team leader and involved with daily operations of MASSIVE, Chris Hines is the expert on Strudel Web.