Mr Benjamin Leighton1, Hao Tang, Kimberley Opie, Dr Jonathan Yu, Dr Glenn Newnham, Alessio Arena
1Csiro, ,
There are many ways to deliver large scale geospatial datasets and options continue to evolve as cloud block storage technologies become commonplace. Current technologies can be complex which increases costs incurred in software development, configuration, management of production deployments, load balancing, authentication, and security.
We present CogBoard – a minimalist approach to efficient visualization of large geospatial data stored in the cloud. CogBoard securely delivers COGs (Cloud Optimized GeoTIFFs ) directly from AWS (Amazon Web Services) S3 to browsers. Web browsers render COG data client-side using the OpenLayers Javascript Library. Security is provided by a lightweight AWS lambda function that brokers authentication between the browser and AWS using KeyCloak for user management. Authentication generates pre-signed S3 URLs which are secure and maintain most of the advantages of direct S3 access: excellent reliability, scalability, high performance, and compatibility with existing client-side libraries.
We demonstrate CogBoard in action as a means of securely delivering a working data product, a national slope dataset, as part of the National Bushfire Intelligence Capability.
Biography:
Ben is a CSIRO data engineer and data scientist working across a bunch of science domains, data, and compute infrastructures. He works designing and building solutions on both cloud and on premise HPC systems and regularly finds himself struggling with large geospatial and temporal datasets, workflows, provenance, and scientific models.