Mr Uwe Rosebrock1, Mr Simon Pigot1
1Csiro, Hobart, Australia, firstname.lastname@example.org
The Australian Wave Energy Atlas (AWavEA) portal provides access to a 32-year hind-cast of wave data from the Australian region at an hourly temporal resolution. In its entirety, it consists of nearly 20 TB of data of which a subset – 5 TB, is used to provide real-time time-series analysis. In average a web-user’s expectations to wait for a asynchronous query is in the order of 10’s of seconds. With some simple measures the supporting data was prepared to allow analysis processes to return results covering over 300,000 records of data in under 10 sec. This is an improvement of 3 magnitudes to the standard layout of the data.
With increasing amount of data available and the cross-disciplinary use, it is no longer feasible to simply copy or query large data holding remotely. Query processes are necessary in front of data, an example is the NCI data-cube. We like to present simple measure which allow improved access and make incorporation in spatial portals feasible.
Uwe Rosebrock is a Senior Software Engineer at CSIRO Oceans and Atmosphere in Hobart. He has extensive experience in large data processing, software design as well as project and defect management. Uwe leads a team of software engineers, who led the development of the ARENA-CSIRO Australian Wave Energy Atlas, and its integration into AREMI, and also developed CSIRO’s relocatable modelling system and the DIVE visualisation packages as part of the CSIRO/BoM/Navy BlueLINK program.