Introduction to and Demonstration of Containers in the ARDC Nectar Research Cloud

Conveners: Dr Glenn Moloney5, Wilfred Brimblecombe1

Presenters: Andy Botting2, Sam Morrison3, Jake Yip4

1Australian Research Data Commons (ARDC) (built from ANDS, Nectar, RDS), wilfred@nectar.org.au
2Australian Research Data Commons (ARDC) (built from ANDS, Nectar, RDS), andrew.botting@unimelb.edu.au
3Australian Research Data Commons (ARDC) (built from ANDS, Nectar, RDS), sam.morrison@unimelb.edu.au
4Australian Research Data Commons (ARDC) (built from ANDS, Nectar, RDS), jake.yip@unimelb.edu.au
5Australian Research Data Commons (ARDC) (built from ANDS, Nectar, RDS), glenn.moloney@nectar.org.au

GENERAL INFORMATION

  • Half day workshop
  • Include a hands-on component
  • Maximum of 40 people

DESCRIPTION

Containers provide a solution to the problem of how to get software to run reliably when moved from one computing environment to another.  This workshop provides an introduction to this popular technology by briefly going over container concepts and then demonstrating containers in use on the ARDC Nectar Research Cloud.

The workshop will have a “hands on” component so please bring you laptop.

The following topics/activities will be covered:

  1. Introduction to container concepts and products
  2. Using Docker and Kubernetes technologies on the Research Cloud
  3. Exercise using a simple tool kit that can be deployed and experimented with at the workshop and may be extended for used after the workshop
  4. Seek feedback from participants on “tuning” the container offering on the Research Cloud to meet their needs.

This workshop will provide you with a useful introduction to Container technology and help ARDC determine the container offerings that may be supported on the ARDC Nectar Research Cloud.

WHO SHOULD ATTEND

Target Audience – researchers who may benefit from a lightweight easy to use Container service or are looking for an introduction into the area.  We are assuming current sophisticated heavy users of container technology already have set up their environments and will continue to want to do so.

eResearch staff who are interested in learning about container technologies and how they can be used on the Nectar Research Cloud or across multiple cloud services.

WHAT TO BRING

Bring your laptop.  Required prerequisite knowledge – Moderate to advanced understanding of Unix and cloud environments.  If you are not from an Australian or New Zealand University you will need an AAF account to gain access for the hands-on component.


BIOGRAPHIES

Andy Botting – Senior Engineer at the Australian Research Data Commons (ARDC), Nectar Research Cloud.  I’m a cloud-native Systems Engineer with a background in Linux, HPC.  Specialities: Linux, Android, Puppet, OpenStack and AWS.

Wilfred Brimblecombe – ICT Manager at the Australian Research Data Commons (ARDC), Nectar Research Cloud, is an IT management veteran with over 20 years of leadership experience across various organisations.

Sam Morrison – Senior Engineer at the Australian Research Data Commons (ARDC), Nectar Research Cloud.  Specialties: Linux system administration, Python/Django web programming, Security, Openstack cloud technologies.

Jake Yip – DevOps Engineer at Australian Research Data Commons (ARDC), Nectar Research Cloud. Specialities: Puppet, OpenStack, Networks, DevOps and Security.

 

ARDC Nectar Research Cloud 101 for Beginners

Conveners: Dr Glenn Moloney5, Wilfred Brimblecombe1
Presenters: Andy Botting2, Sam Morrison3, Jake Yip4

1Australian Research Data Commons (ARDC) (built from ANDS, Nectar, RDS), wilfred@nectar.org.au
2Australian Research Data Commons (ARDC) (built from ANDS, Nectar, RDS), andrew.botting@unimelb.edu.au
3Australian Research Data Commons (ARDC) (built from ANDS, Nectar, RDS), sam.morrison@unimelb.edu.au
4Australian Research Data Commons (ARDC) (built from ANDS, Nectar, RDS), jake.yip@unimelb.edu.au
5Australian Research Data Commons (ARDC) (built from ANDS, Nectar, RDS), glenn.moloney@nectar.org.au

GENERAL INFORMATION

  • Half day workshop
  • Includes a hands-on component
  • Maximum of 30 people

DESCRIPTION

This workshop will provide an introduction to using the Australian Research Data Commons (ARDC) Nectar Research Cloud.  It  is for people who have no or limited experience in setting up and using Virtual Machines (VMs) in any cloud environment.

The workshop will have a “hands on” component so please bring your laptop.

The following topics/activities will be covered:

  1. What is the ARDC Nectar Research Cloud and how does it compare with commercial offerings (at technical usage level only – will not go into cost comparisons etc.) 20 minutes
  1. What is the Research Cloud good for? And what it is not good for? 10 minutes
  1. Access to the Research Cloud – How to get some of the Nectar Research Cloud. 20 minutes
    1. How to request access
    2. How to request resources
    3. How to request support to help you deploy and use your VM’s
  1. Exercise – Spin up a VM and install some software on it using Nectar Research Cloud. 120 minutes

WHO SHOULD ATTEND

Researchers or research support people who have perhaps heard of cloud computing or of the Nectar Research Cloud and would like an introduction.

WHAT TO BRING

Bring your laptop.  Required prerequisite knowledge – Moderate understanding of Unix or any other software development environment.  If you are not from an Australian or New Zealand University you will need an AAF account to gain access for the hands-on component.


BIOGRAPHIES

Andy Botting – Senior Engineer at the Australian Research Data Commons (ARDC), Nectar Research Cloud.  I’m a cloud-native Systems Engineer with a background in Linux, HPC.  Specialities: Linux, Android, Puppet, OpenStack and AWS.

Wilfred Brimblecombe – ICT Manager at the Australian Research Data Commons (ARDC), Nectar Research Cloud, is an IT management veteran with over 20 years of leadership experience across various organisations.

Sam Morrison – Senior Engineer at the Australian Research Data Commons (ARDC), Nectar Research Cloud.  Specialties: Linux system administration, Python/Django web programming, Security, Openstack cloud technologies.

Jake Yip – DevOps Engineer at Australian Research Data Commons (ARDC), Nectar Research Cloud. Specialities: Puppet, OpenStack, Networks, DevOps and Security.

 

Introducing ReDBox 2: A Hands On Exploration of ReDBox 2 and the Provisioner for Institutions

Gavin Kennedy1, Dr Peter Sefton2, Andrew Brazzatti1, Moises Sacal Bonequi2, Michael Lynch2

1Queensland Cyber Infrastructure Foundation, Brisbane, Australia, gavin.kennedy@qcif.edu.au
2University of Technology Sydney, Sydney, Australia, Peter.Sefton@uts.edu.au

DESCRIPTION

This is a hands-on workshop for institutions to preview the new ReDBox 2 Research Data Management platform. ReDBox is the most widely used research data management platform in Australian Universities, but with its focus on managing and publishing the metadata for data collections, it has never reached its potential as an end to end solution. QCIF and UTS have collaborated to develop ReDBox 2, a comprehensive platform to support the research data life cycle. It provides an integrated data management planning capability, it allows users to provision and manage research services such as storage infrastructure and it then supports the ingest of data packages using the DataCrate standard, allowing ReDBox to publish the data alongside the metadata. In developing ReDBox 2, we have focussed on making an easy to configure web application using Sails.JS, a modern javascript framework.

This workshop will be delivered in three parts:

Part 1 is an introduction to and discussion of ReDBox and the Provisioner (30 Minutes):

  1. Overview of ReDBox
  • The Mint Namespace Authority
  • Data Management Planning tools
  • Workspaces and Provisioning
  • Metadata and data harvesting and curation
  • DataCrate for data packaging
  • Publication workflows

PART 2 is a demonstration and hands-on exploration of RedBoX for institutions (120 Minutes):

  • Details of the technology stack (nodeJS, Sails, MongoDB)
  • How to install the platform
  • Loading data into Mint
  • Creating records, including DMPs, data records and publication records.
  • How to configure forms and workflows
  • How to integrate with services to create new workflows

PART 3 is a discussion on community involvement, timelines and future developments (30 minutes)

WHO SHOULD ATTEND

This workshop will be of interest to repository managers, data librarians and technical staff, as it will describe how the architecture of the platform supports the research data lifecycle. It does not assume extensive technical knowledge, so we encourage both developers and administrators to attend.

WHAT TO BRING

Attendees will need to bring a laptop with a web browser installed. During the workshop we will install additional software such as SSH/Putty and WinSCP or Cyberduck.


BIOGRAPHIES

Dr Peter Sefton is the Manager, eResearch Support at the University of Technology, Sydney (UTS). Before that he was in a similar role at the university of Western Sydney (UWS). At UTS Peter leads a team working with key stakeholders to implement university-wide eResearch infrastructure, including an institutional data repository, as well as collaborating widely with research communities at the institution on specific research challenges.

Gavin Kennedy is an IT research and solutions expert and is the head of Data Innovation Services at the Queensland Cyber Infrastructure Foundation (QCIF).  Gavin leads the QCIF based development team responsible for ReDBox, the popular research data management and publishing platform. Gavin is a passionate advocate for Open Source platforms to support open research and the FAIR data principles. Gavin has over 30 years IT experience in organisations as diverse as CSIRO, General Electric and British Telecom.

 

DevOps in eResearch

Mr Sven Dowideit1

1CSIRO, Brisbane, Australia

DESCRIPTION

# DevOp learnings

This workshop contains short talks, and discussion on specific tools, techniques, and experiences delivering tools and services to users.

There will be 6 themes, allowing about 30mins for some short talks/demos, and then group discussion:

  • Security as a service
  • Continuous integration and delivery
  • Monitoring, alerting, and logging
  • Moving to the cloud
  • Containerization, orchestration, kubernetes, CaaS systems
  • Keeping your research service running for the next 10 years

WHO SHOULD ATTEND

Anyone that uses a computer to do research related work – predominantly on Linux systems.

WHAT TO BRING

Laptops are useful, but not compulsory


BIOGRAPHY

Sven Dowideit is doing DevSecOps in O&A CSIRO, and previously spent 5 years working in the application container startup space, having lead both the Boot2Docker project and the RancherOS container Linux project.

 

Galaxy architecture and deployment experiences: a case study in how to build complex analysis systems for data-focussed science

Mr Simon Gladman1, Mr Derek Benson4, Dr Jeff Christiansen2, Dr Gareth Price3, A/Prof. Andrew Lonie1

1Melbourne Bioinformatics, University of Melbourne, Melbourne, Australia, simon.gladman@unimelb.edu.au
2Queensland Cyber Infrastructure Foundation, Brisbane, Australia, j.christiansen@uq.edu.au
3Queensland Facility for Advanced Bioinformatics, Brisbane, Australia, g.price@qfab.org
4Research Computing Centre, University of Queensland, Brisbane, Australia, d.benson.imb.uq.edu.au
5Melbourne Bioinformatics, University of Melbourne, Melbourne, Australia, alonie@unimelb.edu.au

GENERAL INFORMATION

  • Half day Workshop
  • includes a hands-on component
  • Up to 20 attendees

DESCRIPTION

Galaxy (https://galaxyproject.org) is a widely used, highly capable bioinformatics analysis platform. It provides users with a large library of analysis and visualization tools, reference datasets, interfaces to global databases, and evolving workflow capabilities that provide provenance and reproducibility. Users build complex analysis jobs in a highly accessible interface, which are then deployed via a scheduler to underlying computational resources. Galaxy has a relatively sophisticated approach to managing user jobs to compute resources and can, for instance, be configured to schedule jobs to disparate HPC and/or cloud resources depending on the job characteristics.

In this workshop we will explore the architecture of Galaxy Australia (http://usegalaxy.org.au), understanding how it is architected to deploy jobs from a common front end to compute resources in Queensland and Victoria. Jobs have access to a common multi-hundred-terabyte reference dataset collection that is intelligently mirrored in real time from the US-based  Galaxy Main (http://usegalaxy.org) using the CernVM file system (https://cernvm.cern.ch/portal/filesystem). We will explore the technologies, cover our experiences of how they work in practice, and discuss the ambitions of a global Galaxy infrastructure network that can leverage the efforts of a global community to maintain and support critical data and software resources.

OUTLINE OF WORKSHOP CONTENT:

  1. Overview of Galaxy. Technical overview of the componentry of Galaxy as a software platform and as a workflow generation and deployment system30 minutes
  1. Galaxy Australia architecture. Overview of the Galaxy Australia archtictural and deployment model.30 minutes
  1. Underlying technologies. Detailed exploration of the job distribution and data sharing technologies being used for Galaxy Australia.90 minutes
  1. Galaxy ‘World’ – roadmap discussion. How can multiple instances of Galaxy make use of complex, high maintenance resources including a tool library which is dependency-free and growing global reference datasets, whilst appearing as a seamless experience to non-expert users?30 minutes

WHO SHOULD ATTEND

Research infrastructure staff  interested in complex, distributed software systems and cutting edge technologies for job and data distribution.

WHAT TO BRING

A laptop, no special software required. We hope to demonstrate some of the technologies being used in Galaxy.


BIOGRAPHY

Andrew Lonie is Director of the Melbourne Bioinformatics, Director of the EMBL Australia Bioinformatics Resource (EMBL-ABR: http://embl-abr.org.au), and an associate professor at the Faculty of Medicine, Dentistry and Health Sciences at the University of Melbourne, where he coordinates the MSc (Bioinformatics). Andrew directs a group of bioinformaticians, computational biologists and HPC specialists within the Melbourne Bioinformatics and EMBL-ABR to collaborate with and support life sciences researchers in a variety of research projects across Australia.

 

Getting onboard with CloudStor

Mr Chris Myers1

1AARNet, Carlton South, Australia, Chris Myers@aarnet.edu.au

DESCRIPTION

Cloudstor is an AARNet developed and supported service that enables AARNet customers and the wider community to quickly and securely sync, share and store files using the high-speed AARNet network.

We will present and demonstrate Cloudstors ability to support and interact with your scientific workflows to help improve outcome delivery, tools to help institutions support Cloudstor users and the provisioning of services. And how Cloudstor works and onboarding.

1. Session 1

a. service overview

i. How can Cloudstor support your current and future research data storage requirements.

b. Roadmap

i. New feature to be released over the next 12 months

c. What features are available

i. Cloudstor Filesender

1. sending files

2. receiving files

3. file encryption

ii. Cloudstor Owncloud

1. Synch and share desktop client

2. WEBDAV connections

3. MOBILE client

4. Sharing files with other Cloudstor users

5. Sharing files with external parties

6. Receiving files from external parties

iii. Rocket use

iv. Cloudstor S3 gateway

1. Sending and receiving

2. 3rdparty integration

2. Session 2

a. Tenant Portal

i. Provision institutes

1. Group drives

2. Collaborators

3. Users

ii. Support clients

iii. Monitor Usage

iv. Generate reporting

3. Session 3

a. Cloudstor and Workflow

i. Workflow discovery session

ii. Solution discussion

iii. Best practice overview

b. Onboarding

i. How we can help

ii. Support channels

iii. Support workflow

iv. What you need to know

v. Discussion on what we need to do to improve your experiences

WHO SHOULD ATTEND

Anyone who uses Cloudstor, Research support staff, IT support staff and researchers.

WHAT TO BRING

Laptop of any flavour.


BIOGRAPHY

Chris Myers is a data solutions specialist at AARNet helping support our members and clients who work in data intensive research environment and equip them with the tools and knowledge required to leverage the network, compute and storage investments and enable collaborative activities to accelerate research outcomes.

Chris is the product manager for Cloudstor

Introduction to an Integrated Jupyter Notebook

Ingrid Mason1, Frankie Stevens1, Ghulam Murtaza2

1AARNet, Sydney, Australia, ingrid.mason@aarnet.edu.au, frankie.stevens@aarnet.edu.au
2Intersect Australia, Sydney, Australia, Ghulam.murtaza@intersect.org.au

DESCRIPTION

The Introduction to Jupyter Notebook workshop will cover:

  1. What sort of tool a Jupyter notebook is
  • Where the Integrated Jupyter notebook (as a researcher tool) fits into different researchers’ toolkits
  • What advantages there are of using the notebook in the cloud versus on the desktop
  • Basic Python programming using a Jupyter notebook

WORKSHOP OUTLINE:

The workshop is ~3 hours (including breaks):

  1. What is a Jupyter notebook and how does it function. 15 minutes
  2. Where does a Jupyter notebook fit in different researcher toolkits. 15 minutes
  3. Why use Jupyter notebook on a desktop and why use it in the cloud. 20 minutes
  4. Basic Python programming using a Jupyter notebook. 75 minutes

WHO SHOULD ATTEND

The workshop is targeted at data librarians, research support, and eResearch professionals interested in what the Jupyter notebook does, how it works, and how it can be used to train researchers in basic programming skills.  Workshop participants will want to know where the Jupyter notebook fits into different researchers’ toolkits (along with Excel, SPSS, Stata, RebExr, RStudio, or MatLab). This workshop will include basic programming commands using the basics of the Python programming language.

WHAT TO ORGANISE

Workshop preparation:

  • Come with a laptop and with a CloudStor account setup

GENERAL INFORMATION

Workshop breakdown:

  • Workshop is half-day and includes a hands-on component
  • Up to 40 attendees with no special seating or table requirements

BIOGRAPHIES

Ingrid, Frankie and Ghulam are eResearch specialists with extensive experience in researcher engagement, training, and have expertise in research data and technologies across STEM and HASS research areas.

Digital Science and International Cooperation: new opportunities for digitally enabled AU-EU ICT research collaboration

Dr Erich Prem1, Jonathan Arthur2

1Eutema, Vienna, Austria,
2Intersect, Sydney, Australia

The massive trend towards the internationalization of research is changing both the global and local landscapes of research (Adams 2014, OECD 2008, Wagner et al. 2015). An expanding set of digital tools and an improving and globally accessible digital infrastructure facilitate international cooperation. Digitization has not only changed business practices and many aspects of everyday life; it is also affecting scientific processes. Scientists in all disciplines have quickly adopted new digital means and formats used for discussing scientific theories, disseminating results of their work, acquiring funding and collaborating across institute boundaries (Anderson et al. 2005, Mackenzie Owen 2007).

This increased digitization of science is also changing the relation of science and society – and science policy. The boundaries of what is to be considered a national research policy is much less clearer today than it was only a decade ago. Policy makers interested in sustaining national competitiveness are faced with large multinational players – in academia and industry – who opportunistically source research and innovation globally. On the other hand, digitization of research is opening science not just for scientists, but also for a global public. This not only enables access to scientific publications, but also goes much further in making available data, software, even infrastructure together with instructional videos and tutorials. It has been argued that this is a transformative step for scientific knowledge from being a club good to a truly public good (Prem 2015). eResearch and the digitization of science therefore need to be put in the context of international collaboration in research and beyond.

The European Commission has recently proposed to start negotiations of a free trade agreement with Australia. This may provide new opportunities including those in accessing e-infrastructure, databases, satellite data sources, and more generally for cooperation with European researchers in academia and industry. This workshop is organized by EPIC – Europe’s ICT research and innovation partnership with Australia with the aim to discuss the following questions:

  1. What is the role of eResearch, e-Infrastructures and digital science in international research cooperation? Do we have the right tools or are we missing important components?
  • What are the Australian interests in Europe’s eResearch and digital science facilities and how could European researchers benefit from Australian eResearch resources?
  • What will be necessary next steps in improving EU/AU cooperation in digital research, especially in the area of ICT? What is needed from Europe’s and Australian research policy makers?
  • What are the new and most promising innovation opportunities arising from strengthened partnership between Europe and Australia?

WORKSHOP FORMAT

This half-day workshop aims to define the necessary next steps to devise a roadmap for improved European-Australian collaboration in ICT research and innovation. Australian and European experts will discuss current challenges and forthcoming opportunities in 1-2 panels with the workshop participants. Contributions are welcome and should be sent to one of the coordinators by August 31.

The workshop will be held in close interaction with participants to collect a broad range of inputs. Panel experts will provide short input statements and discuss contributions from the participants.

WHO SHOULD ATTEND?

  • researchers in digital science, e-Infrastructure and eResearch
  • ICT researchers interested in international cooperation
  • stakeholders interested in furthering Australia’s research cooperation with the European Union
  • research policy makers and agencies supporting eResearch and international research cooperation

WORKSHOP COORDINATORS

REFERENCES

560. Adams (2013) The fourth age of research. In: Nature, Vol. 497, pp. 557-560.

Anderson, Cokie; Bremholm, Tony; Hemminger, Bradley; Brown, Cecelia; Vaughan, K.T. (2005), The impact of digitization of scientific information on the scholarly communication of scientists. Proceedings of the American Society for Information Science and Technology, Vol. 40 (1).

Mackenzie Owen, John (2007), The Scientific Article in the Age of Digitization. Springer, Dordrecht, NL.OECD (2008) The internationalisation of business R&D. Evidence, impacts, and implications. OECD.

Erich Prem, ICT and science 2.0: technology-mediated trends and characteristics of new scientific practices. Proc. 15th International Conference on Knowledge Technologies and Data-Driven Business, ACM, 2015.

C.S. Wagner, H.W. Park, L. Leydesdorff (2015) The continuing growth of global cooperation networks in research: a conundrum for national governments. In: PLoS ONE 10(7): e0131816. https://doi.org/10.1371/journal.pone.0131816


BIOGRAPHY

Dr. Erich Prem is chief RTI strategy advisor and CEO of eutema GmbH. He is a regular coordinator of international research projects, evaluator of RTD projects for the European Commission and an experienced programme manager of funding programmes.

Erich Prem is a certified managerial economist and works scientifically in artificial intelligence, research politics, innovation research and epistemology. He published more than 70 scientific papers and was a guest researcher at the Massachusetts Institute of Technology. He received his Dr. phil. (epistemology) from thte University of Vienna, his Dr.tech. from TU Vienna where he also completed his master in computer science (Dipl.Ing). He was a lecturer at TU Vienna’s Informatics Innovation Center. He received his MBA in General Management from Donau University.

 

About the conference

eResearch Australasia provides opportunities for delegates to engage, connect, and share their ideas and exemplars concerning new information centric research capabilities, and how information and communication technologies help researchers to collaborate, collect, manage, share, process, analyse, store, find, understand and re-use information.

Conference Managers

Please contact the team at Conference Design with any questions regarding the conference.

© 2018 - 2019 Conference Design Pty Ltd