Junran Lei
1The Australian National University, Australia
Biography:
Junran Lei is a research software engineer from the HASS Digital Research Hub at the Australian National University. She has designed systems for collection management, research tools, metadata, language, geospatial mapping, and mobile applications. Her work supports the preservation and sharing of cultural heritage, analysis of historical reading patterns, and discovery of fiction in historical newspapers. She has also developed a semantic web system for museums and contributed to a UNESCO project on open-source archival preservation. Her current work involves scalable qualitative annotation using large language models and deep learning for OCR and language model training. Her outputs span software, publications, and international collaboration.
Abstract:
This lightning talk demonstrates the technical analysis of large language models (LLMs) for AI-supported qualitative annotation across diverse computing environments. We are using frameworks such as Hugging Face Transformers and Ollama to run inference workflows on high-performance computers like the NVIDIA DGX Station (V100 GPUs) and later scale them to larger models hosted on platforms such as OpenAI, Amazon Web Services (AWS) Bedrock, the ARDC Nectar Research Cloud (Nectar), and the National Computational Infrastructure (NCI). In our approach, we utilise existing Python pipelines and develop new ones for data processing, prompt engineering, and structured evaluation using ground truth datasets. Evaluation concentrates on model accuracy, reproducibility, and computational efficiency on both local and cloud infrastructures. We view the optional fine-tuning as a way to enhance the model's alignment with specific tasks in the domain. This presentation will give the practical experience on how to choose the right models and the infrastructure needed for scalable, reproducible, and effective integration of LLMs into the qualitative research workflow.