Mr. Michael Pegios1, Richard Laugesen2, Urooj Khan2, Alex Cornish3
1Bureau of Meteorology, Melbourne, Australia, 2Bureau of Meteorology, Canberra, Australia, 3Bureau of Meteorology, Adelaide, Australia
Biography:
Michael Pegios is a Scientific Software Developer currently working at the Bureau of Meteorology. He has completed a master's degree in information technology from the University of New South Wales specialising in data science and artificial intelligence. He formerly developed and deployed DeepSliceWeb, a deep learning model aligning histological sections of the mouse brain to the cloud as a web application. His experience in Python data processing has been beneficial to the eReefs project, a multi-agency collaboration aimed at providing information on the Great Barrier Reef.
Abstract:
The eReefs project is a multi-agency collaboration aimed at providing situational and retrospective information on the Great Barrier Reef (GBR). The Bureau of Meteorology have developed water quantity and quality models for the 459,000² km GBR catchment area. A distributed hydrological model known as Grid-to-Grid was used for water quantity modelling, generating hourly 1 km grid outputs spanning 459,000 grids. For water quality modelling, multivariate regression statistical models have been developed. The outputs of these river models are used as input for hydrodynamic and biogeochemical ocean models. These provide simulations of the marine environment and contribute to the preparation of the Reef Report Card which captures the health of the Reef.
The Bureau developed two systems using these models, one operated manually to generate long historical simulations using the National Computational Infrastructure, and another operating daily to generate near real time simulations of water quantity and quality. Developing and deploying these models presented many challenges including data handling, ensuring data consistency and availability, managing costs, and providing operational resilience. We have adopted Amazon Web Services to address these challenges, implementing a system using Managed Workflow for Apache Airflow (MWAA). MWAA defines workflows through tasks that call the elastic container service, executing python container images containing the models processing code. These services were created using Terraform, which provide infrastructure as code.
This is the first such large scale water model deployed to the cloud in the Bureau, enabling further insights on catchment based threats to the health of the GBR.