Mr Jafaruddin Lie1
1Monash University, Clayton, Australia, email@example.com
Singularity  is a container solution that is developed specifically with high performance computing in mind. Singularity makes it possible to install programs that will not run on the host operating system by creating a container of the operating system that will run the application. An example on how we use it in MASSIVE M3 is the installation of the package Caffe , the Python-based deep learning framework. MASSIVE M3 primarily runs on CentOS 7 operating system.
Most of the library dependencies needed by Caffe are not available on CentOS because the libraries provided by the operating system are behind the latest. It is possible, of course, to manually compile and install all these dependencies as modules and compile Caffe this way, however, this is very time consuming. The manual process took us one working week to find, configure, compile, and test properly while the Singularity build takes us one day. We did not find any significant performance difference between the natively compiled Caffe and the one installed in Singularity container.
Another advantage of Singularity is that the containers are reproducible. The definition files used to build the containers can be shared, which will make building applications faster and standardized across research teams. Overall, we find that using Singularity to install application enables us to deliver the applications to our users faster, enables us to reliably reproduce and upgrade the applications in the containers, and makes sure that we never have to compile any application ever again. It keeps us sane.
Jafar spends 14 years working in the field of system administration and network security before moving to Monash. He is now happily working with the people in Monash eResearch HPC team, occasionally referencing Pokemon Go in the conversation around the office.