Christopher Jenkins1
1Weka, , Australia
Biography:
Christopher is the Field CTO for AI at WEKA, working within the Office of the CTO to help organisations run AI at production scale. His focus is memory for AI: extending GPU memory with token-scale KV cache to transform the throughput and economics of large model inference, and taking that work from the lab into real-world deployments. He works hands-on with governments and AI builders on sovereign AI infrastructure, and speaks internationally on agent memory and agentic analytics. Christopher has published and presented on hybrid cloud adoption and Edge-AI for IoT data challenges, holds a master’s degree in Cloud and Virtualisation from Charles Sturt University, and is an AWS Solutions Architect Associate.
Abstract:
Artificial intelligence is rapidly becoming a foundational tool for research across science, engineering, medicine, and the humanities. While most infrastructure planning has focused on model training and traditional high-performance computing (HPC), a new class of workloads is emerging: agentic AI. Unlike conventional inference, agentic systems perform iterative reasoning, invoke multiple models and tools, retrieve large volumes of contextual data, and execute complex workflows on behalf of researchers.
These characteristics create infrastructure demands that differ significantly from both HPC simulations and AI training environments. Traditional performance metrics such as FLOPS, training throughput, and GPU utilization provide only a partial view of system effectiveness for agentic workloads.
This paper presents an opensource framework for benchmarking agentic AI workloads in research environments. We examine workload characteristics including context retrieval, memory utilization, concurrency, latency sensitivity, and end-to-end task completion. Through representative agentic workflows, we identify emerging bottlenecks across compute, memory, storage, and networking infrastructure, and explore how these factors impact researcher productivity and system efficiency.
The framework is informed by testing and operational observations from large-scale AI cloud and service-provider environments, including deployments supporting research and enterprise AI workloads at CoreWeave, Oracle Cloud Infrastructure (OCI), Nebius, and Firmus Technologies. These examples provide practical insight into how agentic workloads behave at scale and where conventional infrastructure assumptions begin to break down.
The paper concludes with recommendations for research institutions evaluating infrastructure investments to support the next generation of AI-enabled discovery. As agentic AI moves from experimentation into production research workflows, understanding these new performance dynamics will become increasingly important for future eResearch platforms.
Github for current tools: https://github.com/callanjfox/kv-cache-tester
