Dr Rui Yang1
1National Computational Infrastructure, Canberra, Australia
Biography:
Rui Yang is a Senior HPC Specialist at the National Computational Infrastructure (NCI), with expertise in data science, AI/ML, and high-performance computing. He specialises in optimising the performance of scientific models across climate, weather, and geoscience domains, and has extensive experience in benchmarking, profiling, and tuning workflows on large-scale HPC systems. Rui has led efforts to deploy and evaluate cutting-edge AI/ML models on HPC platforms, manage ML-ready datasets, and integrate AI techniques into scientific workflows to support domain-specific research and analysis.
https://orcid.org/0009-0001-9698-5929
Abstract:
The increasing volume and complexity of high-resolution climate and weather datasets pose significant challenges in data discovery, access, and visualization. Local applications often offer powerful data processing capabilities but may lack interactivity and accessibility. Conversely, web-based platforms provide user-friendly interfaces but restrict users to predefined datasets with limited analytical flexibility. This divide highlights the need for a solution that combines the strengths of both approaches.
We present ClimViewer, a Python-based, scalable application for interactive climate and weather data analysis. ClimViewer integrates the Intake Earth System Model (Intake-ESM) framework to provide efficient, metadata-driven access to a wide range of datasets, including user-defined collections. By leveraging Xarray and Dask, it supports parallel processing of large-scale data and dynamic resource allocation, enabling fast image overlay rendering, statistical analysis, and animated visualizations.
ClimViewer features a web-based interface built with Voilà, turning Jupyter notebooks into interactive dashboards that require no coding experience. Users can explore 2D, 3D, and 4D datasets, apply temporal or vertical slicing, and generate location-specific time/level series plots directly through a browser.
The tool is integrated with the National Computational Infrastructure (NCI) Intake-ESM catalog, allowing seamless access to major datasets such as CMIP, ERA5, WeatherBench etc., while also supporting custom catalog files for user-specific datasets. This flexibility enhances discoverability, promotes wider data use, and streamlines research workflows.
By bridging the gap between local compute performance and interactive web accessibility, ClimViewer enables scalable, intuitive exploration and analysis of complex climate and weather data.