Advancing HPC and Data collaborations in Australasia

Dr Jenni Harrison1,2, Mr Mark Gray1,2, Dr Daniel Grimwood1,2, Dr Georgina Rae3, Mr Nick Jones3, Mr Allan Williams4, Mr Peter Elford5

1Pawsey Supercomputing Centre, Kensington, Australia,

2CSIRO, Kensington, Australia,

3New Zealand eScience Infrastructure, Auckland, New Zealand,

4The National Collaborative Infrastructure, Canberra, Australia,

5AARNet, Canberra, Australia



Working cooperatively in an interdisciplinary team is essential to solve complex challenges faced within academic and industry research sectors1.  There are obvious benefits to working as part of a team such as diversity, a wide variety of knowledge, skills and strengths, greater access to resources and funding, a range of facilities and infrastructure and an expanded network of contacts, just to mention a few.  As a consequence, often national collaborations are funded by Governments for various reasons including policy ambitions or to encourage knowledge creation (often in science or engineering) to facilitate economic growth2.

An International Perspective

In the last twenty years, in the eScience / eResearch / HPC world, globally there has been an increase in collaborative working between institutions and across countries, especially in Europe and the US.  For example, in 2015, Nordic eScience and eInfrastructure plan 2.03 was published.  This report was an update to the highly successful Nordic eScience and eInfrastructure plan, published in 2008, written and developed by the Nordic eScience Initiative (eNORIA) group4 which is a collaboration between Norway, Finland, Sweden, Denmark and Iceland.  The key areas highlighted the Nordic eScience and eInfrastructure plan 2.03 include:

  • Creating a Higher Education Arena for eScience
  • Enabling eScience as an Omnipresent Tool in Research
  • Knowledge Creation through Cross-Border Sharing of eInfrastructure

The Partnership for Advanced Computing in Europe (PRACE) was established to enable scientific and engineering research to “enhance European competitiveness for the benefit of society”5. PRACE operatives as a cooperative offering European researchers access to world class computing and data resources as well as services.  More generally, the PRACE partnership is also striving to reduce its impact on the environment by driving and demanding energy efficiency in HPC.  With obvious parallels with the ambitions of the eNORIA group, PRACE has four main interest areas which include:

  • Education and training
  • HPC Access
  • Research Infrastructure
  • HPC Market Surveillance

In the US, the Big Data Innovation Hubs (established by the National Science Foundation) have been set up across the country to “accelerate partnerships among people in business, academia, and government who apply data science and analytics to help solve regional and national challenges6”.  By nature, these Hubs are large in size.  For example, the South Big Data Hub serves 16 States and has more than 500 members (including universities, corporations, foundations, and cities).  Again the goal of the Big Data Innovation hub is similar to the collaborations in Europe.  For example, the Hubs set out to:

  • Build data science capacity for education and workforce development
  • Facilitate data sharing and shared cyber infrastructure and services.
  • Accelerate public-private partnerships that break barriers between industry, academia, and government.
  • Grow R&D communities that connect data scientists with domain scientists and practitioners.

The Australasian Context
There are no similar collaborative science partnerships existing within Australasia, although Australia (through AARNet) has a world-class national and international computer network.  Australia in particular is challenged by geographic scale, with Perth, WA being the most isolated capital on the planet.  At the same time, Boolardy WA is the location of two of the Square Kilometre Array (SKA)7 Pathfinder projects, with the SKA being one of the world’s largest science (radio astronomy) and engineering projects.

The two of the SKA precursors, Australian SKA Pathfinder (ASKAP) and the Murchison Widefield Array (MWA) are located in Boolardy WA, with the data processing engine located in Perth.  However, when operational, it is estimated that the SKA will generate a massive quantity of data, ~ 3 TB transmitted every second to data processing engine.  This sheer size of the data generated, its processing, analysis and storage makes this a global problem.  There are many other science and engineering projects in Australasia which could considerably benefit from the efforts of collaborations, for example, in Climate Modelling, Biosciences, Health Sciences, Geosciences and informatics.


Being part of a successful and ongoing partnership is often complex.  Collaboration is highly regarded as an effective means to tackle challenges (especially in science and engineering).  However, achieving a successful ongoing collaboration is often hard to achieve and sustain with failure widespread8.  Most HPC Centres serve the academic community at least to some degree.  Correspondingly some of the pressures faced by higher education, are also faced by HPC providers.  For example, there is often pressure (internally and externally) to respond to economic, political and / or social need9. Often Centres have multiple stakeholders with competing, significant demands in an environment of uncertain funding, particularly from the Government Sector. In the context of Australasia, there are also “local” issues, such as an expanse of geography, time zone differences, and language and cultural differences.   How then can we create an opportunity for a successful collaborative partnership in Australasia that services the needs of local / national customers and offers the benefits of being effectively connected internationally with the rest of Australasia?


The Authors seek to establish an effective collaborative partnership from Singapore via Australia to New Zealand, making this proposed partnership one of great opportunity as well as diversity and significant distance (~8000Kms).   If successful this partnership of HPC Centres would be one of the largest (geographically) anywhere in the world, spanning multiple countries and time zones.  Some of the key areas that the partnership would initially focus on include supporting skills development and gaining access to knowledge and skills not available locally.  We would expect to build on the very successful partnerships in computer networking that AARNet has undertaken, including its trans-Pacific network capacity, which has benefited New Zealand, its world-class national networks, which span from Cairns to Boolardy (at multiples of 100Gbps), and more recently Project Indigo, collaborating with Telstra, Singtel and others to provide terabit-scale capacity between Perth and Singapore (and Sydney), and the Japan-Guam-Australia cable in conjunction with Google and others.  AARNet’s CloudStor “Data Hub” could also provide a platform for bringing researchers wishing to share data together, both the few with very large datasets, and the many with smaller datasets.


The BoF session will be 60 minutes in duration.  At this BoF the Authors will examine some of the challenges and opportunities that such a partnership presents. The format of the session is a mixture of presentations, facilitated discussion as follows:

  • Understanding the background of international collaborations [10 mins];
  • The opportunities presented by an Australasian Collaborative Network [10 mins];
  • The challenges of developing and sustaining international collaborations from an Australasian perspective [10 mins];
  • Facilitated discussion regarding Australasian Collaborative Network would bring, including best practice [25 mins];
  • Wrap up [5 mins].

The presentations will be provided/convened by the listed presenters.

The targeted audience for the session includes anyone with an interest in how collaborations between organisations across countries can be used to facilitate the sharing of education, eResearch tools or infrastructure to support the research community. A primary goal for the BoF facilitated discussion is to initiate conversations aimed at promoting collaboration, therefore this BoF is not discipline specific.


  1. Neumann, P., et al., Interdisciplinary teamwork in HPC education: Challenges, concepts, and outcomes. Journal of Parallel and Distributed Computing (2017) Vol 105, pp83-91.
  2. Wagner, C. S., International collaboration in science and technology: promises and pitfalls. Science and Technology Policy for Development, Dialogues at the Interface’ by Louk Box and Rutger Engelhard (eds) (2006) Anthem Press London UK. See: s77l634iqvoni0t6vk67 [accessed June 2018]
  3. Nordic eScience and eInfrastructure plan 2.0: eScience and eInfrastructure in an international context. Available from [accessed 8th June 2018]
  4. eNORIA Group [accessed 8th June 2018]
  5. PRACE – The Scientific Case for High Performance Computing in Europe 2012-2020 [accessed 8th June 2018]
  6. Big Southern Data Innovation Hub, [accessed 8th June 2018]
  7. The Square Kilometer Array [accessed 8th June 2018]
  8. Marek, L., I., Brock, D-J., P. & Savla, J., Evaluating Collaboration for Effectiveness: Conceptualization and Measurement. American Journal of Evaluation (2014) pp1-19.
  9. Daniel, B., Big Data and analytics in higher education: opportunities and challenges. Journal of Educational Technology (2015) Vol 46, No. 5 pp904-920.


I migrated to WA in 2011, to direct the Data Team at the Pawsey Supercomputing Centre, a national facility providing researchers access to leading computation and data resources. I now lead strategic projects and engagement at Pawsey. In my present role, my responsibilities include directing strategic projects, developing strategy for Pawsey as well as leading and sustaining strategic partnerships with key stakeholders.

I am presently engaged in the development of a cooperative network of centres in Asia Pacific to increase diversity, form new collaborative projects, support new training initiatives, solve technical challenges and highlight new opportunities.

Recent Comments