Scalable Research Data Visualisation Workflows: From Web to HPC

Owen L Kaluza¹, Andreas Hamacher², Toan D Nguyen³, Daniel Waghorn⁴, David G Barnes⁵, C Paul Bonnington⁶

¹ *owen.kaluza@monash.edu,

² *andreas.hamacher@monash.edu,

³ *toan.nguyen@monash.edu

⁴ *daniel.waghorn@monash.edu,

⁵ *david.g.barnes@monash.edu,

⁶ *paul.bonnington@monash.edu

*Monash University, Clayton, Victoria, Australia

MOTIVATION

The challenge of providing access to high-end visualisation, compute and storage resources has many aspects, including the difficulties of easing new users into accessing their capabilities and providing ways to seamlessly mesh new technological approaches with research workflows. When offering specialised cluster based visualisation hardware, such as the CAVE2 [1], that cannot run standard desktop applications, there is a need to provide researchers with a way to use them with ease and familiarity.

We attempt to provide researchers with access to high end immersive visualisation resources in a manner that scales in multiple dimensions:

• From low-power portable devices to high-end cluster computing
• From local data storage and processing to cloud based infrastructure
• From researchers with simple visualisation demands and limited relevant technology experience to advanced custom vis workflows

METHODS AND DISCUSSION

We provide a set of modular components driven from a web portal based graphical interface, “previs”, which can be used to provide a pre-programmed visualisation output at first visit, but which can later become part of a much more advanced programmable pipeline.

Data sources include integration with Monash research cloud storage sources, data stored on MASSIVE [2] and sourced from various instruments, and the ability to upload local files or source data from other URLs. In this way, researchers who wish to try out visualisation facilities, or show their data to visitors can do so quickly and easily. For example, an artist can upload a laser scanned point cloud dataset for exploration from the web with no involvement from our technical staff. If there is a potential for analysis and exploration of their data within immersive or higher-powered compute environments, then further work is required; the same portal allows a breakout to higher level programmable tools.

The graphical output at each level of the hierarchy uses the same rendering techniques, scaled to best utilise the available hardware. Cluster-based rendering techniques are used on high-performance computing (HPC) facilities and data reduction and re-sampling is used to make viewing feasible on mobile devices and VR headsets. WebGL allows this capability to be scaled to mobile and desktop browsers, even for volume rendering [3], and large point clouds [4], with the potential for use in WebVR as the standard matures.

The underlying process is scriptable in python, allowing access to the visualisation pipeline for advanced users, augmented with the multitude of other libraries available in a python environment. Use of IPython notebooks [5] extends this capability to the browser, with built in visualisation tools designed for interactive analysis in IPython/Jupyter.

Further advanced visualisation techniques we are developing can be integrated into the workflow, such as utilising comparative visualisation on large cluster display systems with “encube” [6].

Figure 1: The previs workflow.                                    Figure 2: Working with a volume dataset

CONCLUSIONS: PUBLICATION OF RESULTS AND FUTURE WORK

Our framework makes it much simpler to get data into our high-end facilities, but also generates visualisations that can be used on desktop and web devices. These tools also provide the researcher with additional capabilities that can be built into their existing workflows and potentially used for publication and outreach.

The ability to generate customisable visualisations that can be taken away and viewed on portable devices helps consolidate standard desktop workflows with the exploration of data in immersive facilities and the final goal of communicating research via publication, driving us towards further development of these aspects.

3D PDF [7] and WebGL [8] visualisations have made some inroads into being accepted as a publication medium. Cloud/web server based visualisations and notebooks [9] have also been published on a non-permanent basis. For larger data, however, it becomes necessary to run cloud based visualisations on graphics processor (GPU) powered hardware. Barriers to the use of these tools exist, which may come down if standardised GPU powered cloud services become ubiquitous enough. For example, if a publication could include a Docker [10] container based visualisation with certainty that the ability to launch and view remote data and visualisation elements will exist for the long term.

REFERENCES

1. Febretti. A. et al., CAVE2: a hybrid reality environment for immersive simulation and information analysis. Proceedings Volume 8649, The Engineering Reality of Virtual Reality 2013.
2. Goscinski. W.J. et al., The multi-modal Australian ScienceS Imaging and Visualization Environment (MASSIVE) high performance computing infrastructure: applications in neuroscience and neuroinformatics research. Frontiers in Neuroinformatics., 27 March 2014
3. Congote. J. et al., Interactive visualization of volumetric data with WebGL in real-time. Web3D ’11 Proceedings of the 16th International Conference on 3D Web Technology, pages 137-146
4. Schuetz. M. Potree: Rendering Large Point Clouds in Web Browsers. Diploma Thesis, Vienna University of Technology, 19 Sep 2016
5. Perez. F. Granger. B. E., IPython: A System for Interactive Scientific Computing. Computing in Science & Engineering, Volume: 9, Issue: 3, May-June 2007.
6. Vohl. D. et al., Large-scale comparative visualisation of sets of multidimensional data, PeerJ, 10 Oct 2016
7. Barnes, D.G., Fluke, C.J. et al., Incorporating interactive three-dimensional graphics in astronomy research papers. New Astronomy, 2008. Volume 13, Issue 6: p. 599-605.
8. Quayle. M.R. et al., An interactive three dimensional approach to anatomical description—the jaw musculature of the Australian laughing kookaburra (Dacelo novaeguineae). PeerJ, 1 May 2014.
9. Shen. H. Interactive Notebooks: Sharing the code, Nature Toolbox, 5 Nov 2014. Available from: https://www.nature.com/news/interactive-notebooks-sharing-the-code-1.16261, accessed 7 Jun 2018.
10. Cito. J. et al., Using Docker Containers to Improve Reproducibility in Software and Web Engineering Research, International Conference on Web Engineering 2016, Lecture Notes in Computer Science, vol 9671. Springer, Cham

Biography:

Andreas is a software developer with a history in scientific visualisation applications and research. After earning his bachelor degree in scientific programming he was employed at RWTH Aachen University, Aachen, in the Virtual Reality Group which also operates a CAVE system. There, Andreas worked across disciplines, helping researchers apply visualisation and VR methods to their fields.

Owen is a research software developer with a computer science and graphics programming background, who has been working with researchers at Monash since 2009, initially developing cross-platform and web-based 3D visualisation software for simulation data and later involved in parallel GPU-based implementations of imaging algorithms.

In his current role as a visualisation software specialist he develops tools for processing, viewing and interacting with scientific datasets in high-resolution immersive VR environments and exploring new scientific and creative applications of the CAVE2 facility at the Monash Immersive Visualisation Platform.