Deployment Pipelines in eResearch: challenges and successes

Mr Nick Rossow1,  Heidi Perrett2Mr Jan Hettenhausen3

1Griffith University, Nathan, Australia


In the current research environment application development has become a critical element in research workflows. Many countries like Australia are investing heavily in not only hardware infrastructure but also the application platforms required to undertake research. Similarly, within institutions as research teams become large and data flows increase the need for software applications is increasing with the requisite level of software engineering practices.

Software Engineering practises have undergone a significant evolution over time with Agile methodologies currently being the dominant approach in many industries, including eResearch. One of the newer models under the umbrella of Agile approaches is continuous delivery, in which software is developed with the idea of being releasable at any time. In order to achieve this, continuous delivery heavily relies on automating testing and building and deploying the code in a pipeline-style process. Some of the benefits, in particular in the context of developing applications in close collaboration with researchers, are that projects can evolve more quickly whilst maintaining a high-quality standard of the product. These benefits also persist after the product is handed over to support as the automation provides a mechanism for confidence in software and hardware updates and patches.

At Griffith University, eResearch Services provide specialist IT support for researchers across all Griffith Schools and Research Centres. Our activities are split between software development for niche research needs, providing services such as high performance computing, hosting and maintaining broadly applicable software applications such as research storage and data collection tools, alongside programming workshops and media productions for research projects.

By implementing a continuous deployment pipeline our goal was to be able to quickly and efficiently deploy new applications, patches to existing applications and be able to quickly scale resources available to the project team. It was our experience that the technical debt required to add an additional developer resource to the project far outweighed any possible benefit this would achieve in a reasonable timeframe. By implementing a Continuous Delivery model, we have been able to successfully add a new developer to a project and have productive input in a matter of hours instead of days.

In our presentation, we will discuss the challenges and successes we faced over the past 18 months as we moved towards a Continuous Delivery model for the way we delivered the development of solutions for niche research needs. To highlight some of the challenges and successes we will look at the past 18 months from 3 distinct vantage points; that of the Project Manager, the Developer and that of the Technical Lead. We will talk about the main drivers for our desire to change and delve into some of the key benefits we have already seen and expect to see in the near future.

This presentation will be of interest to anyone that has an interest in improving internal processes, or those that are involved in the development process; from end users to project managers and developers.


Nick Rossow is the acting manager of the Consultancy and Development eResearch group at Griffith University. Nick has worked at Griffith University for 11 years and in the eResearch team for the last 7 years with substantial contributions to research projects in various sectors of Research such as Health, Criminology, Humanities and Social Sciences.

Recently Nick has taken on the challenges of management of the eResearch Consultancy and Development team being involved in all areas of research projects and governance.

Recent Comments