Prof. James Smithies1, Mr Glen Berman1, Dr Karaitiana Taiuru2
1Australian National University, Canberra, Australia, 2Taiuru & Associates Ltd, Wellington, Aotearoa New Zealand
Biography:
James Smithies is Professor of Digital Humanities & Director of the HASS Digital Research Hub at The Australian National University. Before ANU he was Professor of Digital Humanities in the Department of Digital Humanities at King’s College London, founding director of King's Digital Lab and deputy director of King's eResearch. Prior to working at King's James worked as a senior lecturer at the University of Canterbury in Aotearoa/New Zealand, helping develop the UC CEISMIC Canterbury Earthquakes Digital Archive. He has also worked in the government and commercial IT sectors in the United Kingdom and New Zealand, as a technical writer, editor, business analyst, and project manager.
Abstract:
This paper presents a set of principles designed to guide the implementation of Large Language Models (LLMs) and Artificial Intelligence (AI) technologies in research and public sector organisations. The principles are an output of the AI as Infrastructure (aiinfra.anu.edu.au) project, informed by a combination of technical development and focus group activities. Notably, the principles combine Research Software Test Engineering (RSTE) with Responsible AI, balancing technical assessment with organizational, legal, and cultural considerations. By establishing baselines for the longitudinal assessment of Large Language Models (LLMs) and fostering environments conducive to learning, through technical experimentation and dialogue, our approach supports the development of organizational knowledge and capabilities over decades rather than the most recent hype cycle. As a case study, we examine Aotearoa New Zealand’s Public Service AI Framework, which exemplifies the need for guiding principles that are grounded in real-world practice, capacity building, and eventual operationalization. The framework’s inclusion of Māori data sovereignty principles, shaped by the foundational Treaty of Waitangi, highlights Aotearoa’s unique approach to AI. This approach holds significant relevance for research and cultural institutions both within and beyond Aotearoa, particularly those with commitments to Indigenous and First Nations communities. By focusing on practical engagement while respecting cultural and ethical considerations, this paper outlines a pathway for integrating LLMs and AI responsibly within the research and GLAM sectors and highlights the pressing need to increase the maturity of our RSTE methods.