Albert Savary
1NeSI, , New Zealand
Background
One of the core services of NeSI (New Zealand eScience Infrastructure) is to provide compute resources to New Zealand researchers.
These HPC (High Performance Computing) resources are composed of thousands of CPUs.
To use them efficiently, the code must be parallelized.
Several technologies are available to do this and GNU Parallel is one of them.
Method
A description of the GNU Parallel command.
With the right set up, it is very easy to parallelize a shell loop.
Need to ensure the right export of the environment variables necessary for the rest of the code.
Results
Some examples on a NeSI’s HPC system.
Some tips on how to set it up the right way.
The application can run faster by using several tasks in parallel.
Conclusion
This is an elegant option to create a multi-process program and run it on multiple CPUs to increase the speed of the program.
Biography:
Works for NeSI in the applications support team.