Mr Kabir Manandhar Shrestha1, Dr Laura Hardefeldt1, Dr Daniel Russo-Batterham1, Dr Noel Faux1
1University Of Melbourne, Parkville, Australia
Biography:
Software developer and Data scientist with solid understanding of the programming paradigm and deep interests in extracting knowledge from data and using machine learning techniques to solve real world problems.
Abstract:
The accurate classification of veterinary clinical notes is crucial for effective disease and treatment surveillance. This study investigates the application of Large Language Models (LLMs), specifically the Meta Llama model, to automate the categorisation of veterinary records through a structured chain-of-thought prompting approach.
Our method processes each clinical note to generate the top three most relevant categories, accompanied by reasoning and associated probabilities. This chain-of-thought prompting technique aligns the model's output with the interpretive needs of veterinary professionals, providing a complementary tool to assist in classification without replacing clinical judgment. The poster will illustrate this pipeline and show how AI aligns with clinical decision-making.
Our findings demonstrate that Meta Llama, when guided by tailored prompts, can effectively categorise complex veterinary notes across various domains. The model achieves 40% accuracy on the first prediction, increasing to 74% for the top three predictions across 44 categories. This performance has been deemed acceptable after careful assessment, highlighting the potential of this approach to support veterinary tasks.
Looking ahead, we envision this LLM-powered system to benefit various veterinary tasks, including:
– Efficient categorisation of patient records for disease surveillance
– Identification of rare or atypical conditions through contextual note analysis
– Facilitation of antimicrobial stewardship programs by classifying antimicrobial usage patterns
– Optimization of clinical decision-making by surfacing relevant historical cases
By highlighting the capabilities of LLMs in handling veterinary note classification, this poster will demonstrate the application of AI tools for standardising veterinary notes to improve disease and medication surveillance.