Activity-Based Costing Framework for Total Cost of Ownership Analysis of LLM Services

Authors’ abstract

The proliferation of Large Language Models (LLMs) requires robust financial planning, yet traditional software cost models cannot capture the unique economics of generative AI. This paper presents a formal Total Cost of Ownership (TCO) framework grounded in Activity-Based Costing (ABC). We deliver a dual-layered accounting architecture: a mathematically tractable, provably convex engine for technical infrastructure costs, alongside a comprehensive, modular taxonomy blueprint for human operational, governance, and organizational cost pools. The framework systematically incorporates LLM-specific cost drivers, including token consumption, Retrieval-Augmented Generation (RAG) operations, and agentic inference steps. To demonstrate practical applicability, the framework is subjected to a multi-faceted empirical validation programme: the technical infrastructure submodel is validated on a 12-month production RAG chatbot deployment, yielding a predictive formula that links high-level business metrics directly to infrastructure expenses with sub-5% forecasting error, and is further exercised on a second institutional deployment through a synthetically augmented dataset of 95,150 user sessions. Complementary cross-case scenario simulations—spanning regulated, high-scale API, and autonomous agentic regimes—together with hybrid local–cloud serving and retrieval-versus-fine-tuning comparisons, illustrate the framework’s structural generalizability across distinct governance, throughput, and multi-step execution regimes. By integrating time-varying vendor prices, stochastic uncertainty, and hybrid deployment extensions, this work provides a rigorous, transparent decision-support tool for the strategic financial management of LLM-based services. INDEX TERMS Activity-based costing (ABC), large language model (LLM), total cost of ownership (TCO). I. INTRODUCTION The emergence of Large Language Models (LLMs) has precipitated a transformative shift across diverse sectors, with organizations rapidly adopting these technologies for applications ranging from automated customer service [1] and intelligent document processing [2] to code genera-tion [3]. However, this widespread adoption has revealed a critical gap in financial planning and cost management frame-works, as traditional software cost estimation approaches prove inadequate for the unique economic characteristics of LLM-based services [4]. The associate editor coordinating the review of this manuscript and approving it for publication was Mohamed M. A. Moustafa. Traditional software cost estimation models, such as COCOMO II [5] and Function Point Analysis [6], were designed for conventional software development paradigms and fail to capture the distinctive cost drivers inherent in LLM services. These models cannot account for token-based pricing structures, where costs scale directly with input and output volume [7], nor do they address the computa-tional complexity of semantic processing operations such as embedding generation, vector similarity searches, and retrieval-augmented generation (RAG) workflows. Further-more, the dynamic nature of LLM service consumption, characterized by highly variable workloads and pay-per-use pricing models, creates cost patterns that differ fundamen-tally from traditional software licensing and infrastructure costs [8]. Current approaches to LLM cost management remain fragmented and insufficient for comprehensive financial planning. Existing research has focused primarily on iso-lated aspects such as inference optimization [9], model compression for cost reduction [10], and energy efficiency improvements [11]. While valuable, these works do not provide systematic frameworks for Total Cost of Ownership (TCO) analysis that encompass the entire LLM service lifecycle, from development and deployment to ongoing operations and maintenance [12]. Activity-Based Costing (ABC) methodologies, originally developed for manufacturing environments [13], have proven highly effective in complex service contexts where traditional cost accounting fails to capture resource consumption patterns accurately [14]. The evolution to Time-Driven ABC (TDABC) further enhanced the methodology’s applicability to service industries by enabling more granular activity analysis and resource allocation [15]. Recent applications of ABC in cloud computing environments [16] and IT service operations [17] demonstrate the methodology’s adaptability to technology-intensive domains. The inherent granularity of ABC frameworks makes them particularly suitable for LLM cost modeling, where diverse resource types—including GPU compute hours, token processing, storage, and specialized operations like vector database queries—must be accurately mapped to specific activities throughout the service lifecycle. Despite the proven effectiveness of ABC methodologies in analogous domains, their systematic application to LLM services remains largely unexplored. This gap is particularly critical given the rapid expansion of LLM adoption in enter-prise environments, where accurate cost forecasting and bud-get allocation are essential for strategic decision-making [18]. Organizations require robust, transparent methodologies to evaluate the financial implications of LLM integration, compare alternative architectural approaches, and optimize resource allocation across the LLMOps pipeline [19]. This research addresses these critical gaps by developing and validating a comprehensive Activity-Based Costing framework specifically designed for Total Cost of Ownership analysis of LLM services. Our approach systematically decomposes LLM service costs across the entire operational lifecycle, from initial data preparation and model integration through ongoing inference, maintenance, governance, and incident handling activities. The framework incorporates LLM-specific cost drivers, including token consumption patterns, embedding generation workflows, vector database operations, and retrieval-augmented generation processes, while also explicitly modeling personnel-driven review, compliance overhead, organizational adoption effort, and expected model-failure-related losses [12], [20], [21], [22], [23]. Mathematical rigor is maintained through a formal cost model and convexity analysis. The practical utility of our framework is demonstrated through a deliberately multi-faceted validation programme rather than a single illustration. Its empirical anchor is a 12-month production case study, the SzAbI RAG-based chatbot deployed at the University of Pannonia, which translates high-level business metrics—such as knowledge base size, user session frequency, and system utilization patterns—into predictive cost formulas that enable systematic financial planning and architectural trade-off analysis. This longitudinal baseline is complemented by five additional analyses that probe the framework from distinct angles: (i) a second institutional deployment, the KFI chatbot, evaluated on a synthetically augmented dataset of 95,150 user sessions; (ii) cross-case scenario simulations covering regulated-enterprise, high-scale commercial API, and autonomous agentic regimes; (iii) an illustrative hybrid local–cloud serv-ing scenario with explicit break-even analysis; (iv) a retrieval-versus-fine-tuning architectural comparison under varying knowledge volatility; and (v) a quantitative accuracy bench-mark against a FinOps-style manual estimation baseline. Our work thereby bridges the gap between operational LLM deployment and strategic financial management, providing practitioners with actionable tools for cost-effective LLM service delivery. The primary contributions of this research, each explicitly positioned against the most recent literature in Section II, are: • A formal ABC-based TCO framework for LLM services that delivers a dual-layered accounting architecture: a mathematically tractable engine for technical infras-tructure costs, alongside a comprehensive, modular taxonomy blueprint for human operational, governance, adoption, and expected incident cost pools. • A comprehensive taxonomy of LLM-specific cost drivers grounded in established LLMOps practices, providing practitioners with a systematic checklist for identi-fying and quantifying both infrastructure-driven and personnel-driven cost components across the service lifecycle. • A scalable operational cost model that distinguishes between one-time setup costs, linearly recurring tech-nical costs, cumulatively growing storage costs, and organization-specific overhead terms, enabling accurate long-term financial forecasting under both static and time-varying vendor pricing conditions, as well as stochastic and hybrid deployment scenarios. • A multi-faceted empirical validation and cross-case sce-nario comparison, anchoring the technical infrastructure submodel (CTech) in a 12-month real-world production RAG chatbot deployment, extending it to a second institutional deployment evaluated on a synthetically augmented dataset of 95,150 sessions, and utilizing representative enterprise-scale scenario simulations— including regulated, high-throughput, hybrid local– cloud, and agentic execution regimes—to demonstrate the framework’s structural generalizability. Beyond the itemized contributions, the specific novelty of this work relative to the most recent literature deserves explicit statement. Emerging FinOps guidance for generative AI [22], [24] offers practitioner-oriented cost taxonomies but lacks a formally analyzable model; RAGOps and agentic-RAG frameworks [25], [26] systematize operational pipelines without quantifying their cost structure; recent work on agentic token consumption [27], [28] combines task-level empirical measurement and limited pre-execution prediction with a broader token-economics survey, but does not embed these drivers in a lifecycle-wide accounting model; and operationalizations of AI-governance regimes such as the EU AI Act and ISO/IEC 42001 [29], [30] specify compliance activities without pricing them. To the best of our knowledge, the proposed framework is the first to unify these four strands—token-level technical costing, RAG and agentic operations, FinOps-style financial planning, and regula-tory compliance effort—within a single, provably convex, activity-based TCO model that is empirically anchored in production deployment data. The remainder of this paper is organized as follows: Section III presents our comprehensive Activity-Based Costing framework for LLM services, establishing mathe-matical foundations, analytical properties, and positioning our approach relative to existing cost modeling frameworks. Section IV demonstrates practical application through the chatbot case study, including strategic architectural decision analysis, empirical cost model derivation, and validation of predictive cost formulas. Finally, Section V summarizes key findings and outlines directions for future research. II. RELATED WORK The forecasting and management of costs for IT services have evolved from traditional software estimation models to sophisticated frameworks tailored for cloud architectures. However, the paradigm shift introduced by Large Language Models (LLMs)—characterized by token-based consump-tion, probabilistic output lengths, and distinct lifecycle phases (e.g., RAG vs. Fine-tuning)—presents economic challenges not fully addressed by existing literature. This section reviews the evolution of cost modeling from foundational estimation techniques and Activity-Based Costing (ABC) to contemporary LLM economics, identifying the specific research gaps this framework addresses. A. TRADITIONAL SOFTWARE COST ESTIMATION AND TCO For decades, software cost estimation was dominated by parametric models designed to predict development effort based on code complexity. The Constructive Cost Model II (COCOMO II), developed by Boehm et al., remains the de facto standard in this domain [5]. COCOMO II estimates effort and schedule based on the size of the software (typically Source Lines of Code or Function Points) and a set of cost drivers such as personnel capability and platform difficulty. While robust for traditional software engineering, these models are ill-suited for LLM-based services where costs are driven less by lines of code written and more by inference volume, context window usage, and vector database operations. Parallel to development estimation, Mieritz and Kirwin established the industry standard for Total Cost of Ownership (TCO) analysis within the IT sector [20]. Their framework emphasizes a holistic view of costs across enterprise bound-aries over time, distinguishing between direct budgeted costs (hardware, software, labor) and indirect costs (downtime, operations). Crucially, they argued that TCO must account for the entire lifecycle of IT assets, a principle that remains vital for LLM services where post-deployment operational costs (OpEx) frequently exceed initial development investments (CapEx). B. ACTIVITY-BASED COSTING IN IT SERVICES To address the distortions inherent in traditional cost accounting—where overhead is often allocated arbitrarily based on direct labor—Cooper and Kaplan introduced Activity-Based Costing (ABC) [14]. ABC traces resource consumption to specific activities before allocating them to products or services, providing the granularity necessary for complex environments. Kaplan and Anderson later refined this into Time-Driven Activity-Based Costing (TDABC) to simplify implementa-tion in service industries. Adeoti and Valverde successfully applied TDABC to IT service operations, utilizing time equations to estimate capacity cost rates [17]. Their work demonstrated that overhead due to staff activities and service variations could be mathematically modeled to support cost control. However, while these frameworks provide the structural basis for accurate allocation, they predate the generative AI era and do not account for non-temporal cost drivers specific to LLMs, such as token density, embedding dimensions, or retrieval latency. C. ECONOMICS OF LARGE LANGUAGE MODELS The emergence of LLMs has spurred new research into the specific economic implications of deploying generative AI. Aryan et al. characterized the challenges of LLM adoption through the ‘‘GCE Trifecta’’ (Generalization, Cost-optimality, and Evaluation) [21]. They analyzed the ‘‘build vs. buy’’ dilemma, highlighting hidden costs such as prompt engineering, compliance management, and the infrastructure overhead required for large context windows. Their work underscores that cost optimality is often orthogonal to model generalization, necessitating careful strategic planning. More recently, Sharma et al. provided a comparative TCO analysis specifically for domain-adapted LLMs in the chip design industry [31]. By comparing a fine-tuned 70B parameter model against general-purpose commercial APIs, they demonstrated that domain-specific models could reduce TCO by over 90% at scale. Their model explicitly accounts for training costs (GPU hours) and inference costs (query volume), providing a linear cost estimation approach relevant to high-volume industrial applications. However, their analysis is primarily restricted to the economic trade-off between model training and inference compute. It does not offer a generalized framework for the complete service lifecycle, leaving out critical auxiliary cost drivers such as vector database operations, embedding generation for RAG pipelines, and cumulative log storage, which are essential for holistic financial planning. Concurrent efforts by the FinOps Foundation have begun to standardize cost estimation for AI workloads, categorizing costs into compute, storage, and networking dimensions [22], [32]. These industry guidelines focus on forecasting cloud spend but often lack the formal mathematical rigor required for optimization and sensitivity analysis. The 2024–2026 wave of research has substantially deep-ened each of the strands this framework builds on. On the financial-operations side, FinOps working-group guidance now specifically targets generative-AI usage optimization, cost–performance trade-offs, and cost visibility under vari-able pricing [24], underscoring the practical demand for formally grounded cost engines. In the systems community, comprehensive surveys of efficient LLM inference serving catalogue the instance-level and cluster-level mechanisms— scheduling, batching, caching, and disaggregation—that affect throughput, resource utilization, and serving cost [33]. On the workload side, recent empirical studies quantify how autonomous agents consume tokens: Bai et al. analyze token consumption across eight frontier models in agentic coding tasks, showing that accumulated input context dominates cost and that pre-execution self-prediction provides only weak-to-moderate signals and systematically underestimates actual usage [27], while Chen et al. provide a dual computing–economics perspective on token economics for LLM agents, conceptualizing tokens as factors of production, media of exchange, and units of account [28]. Agentic retrieval-augmented generation surveys systematize opera-tional patterns—planning, tool use, and reflection loops— whose repeated retrieval, tool-use, and model-execution steps our agentic extension prices explicitly [26]. Finally, on the governance side, Buscemi et al. decompose the high-risk obligations of the EU AI Act into 48 operational sub-requirements linked to 66 verification activities [29], providing exactly the kind of activity-level structure that a cost model must be able to price. None of these strands, however, offers an integrated accounting framework that connects them; this is the gap the present work addresses. The operational balancing of these deployment regimes is further illuminated by comprehensive multi-model bench-marking studies. Meram et al. evaluated the performance of ten distinct large language models in specialized domains, demonstrating a systematic inverse trade-off between semantic diversity and contextual relevance across different model sizes and prompt designs [34]. Similarly, Nasa and Das conducted multi-stage natural language processing evaluations of advanced language models across diverse functional tasks—including technical explanations and open-ended reasoning—identifying that variations in grammatical correctness and vocabulary richness directly impact downstream task efficiency [35]. To exploit these differential performance patterns, recent architectures have shifted toward hybrid implementations that fuse unstructured textual metrics with quantitative features. For instance, Hussain et al. engineered a synergistic LLM-Transformer framework that conceptualizes the language model as a structured signal generator, utilizing a dynamic gating fusion mechanism to weight text-derived sentiment signals according to token-classification confidence scores [36]. These advancements highlight that multi-model deployments and hybrid execution layers introduce distinct operational dependencies, reinforcing the necessity of a unified account-ing framework that structurally decomposes both linguistic drivers and infrastructure allocation. D. RESEARCH GAP Despite these advancements, a critical gap remains in the literature. Traditional models like COCOMO II [5] fail to capture the token-based economics of GenAI. General-purpose TCO models [17], [20] lack the taxonomy to capture LLM-specific drivers like Retrieval-Augmented Generation (RAG) lookups or embedding storage growth. Conversely, recent LLM-specific studies [21], [31] focus primarily on inference strategies or high-level strategic trade-offs without providing a generalized, operational cost model that covers the full lifecycle from data preparation to log management. The most recent 2024–2026 literature sharpens rather than closes this gap: FinOps guidance for generative AI remains practitioner-oriented and informal [24], serving-efficiency surveys optimize cost drivers without providing an account-ing structure [33], empirical and survey work on agentic token economics measures task-level spend, tests limited pre-execution prediction, and synthesizes broader cost models without lifecycle-wide activity attribution [27], [28], and AI-governance operationalizations enumerate compliance activities without costing them [29]. The framework proposed in this paper bridges these domains by integrating the granular activity analysis of TDABC [17] with the specific cost drivers of LLMs identified in recent literature [31], formulated as a provably convex model suitable for systematic financial planning. III. ACTIVITY-BASED TCO FRAMEWORK FOR LLM SERVICES AND COMPARATIVE ANALYSIS A. FRAMEWORK OVERVIEW AND OBJECTIVES We develop a formal Total Cost of Ownership (TCO) framework for LLM-based services that systematically decomposes costs across the entire service lifecycle using Activity-Based Costing (ABC) principles. The framework addresses two critical requirements: (1) generalizability across different LLM architectures and deployment scenar-ios, and (2) analytical tractability for sensitivity analysis and cost forecasting. Our approach models the TCO of an LLM service over a planning horizon H by decomposing it into activity cost pools and measurable cost drivers. This yields a model that is mathematically rigorous, provably convex under standard assumptions, and directly applicable to real-world cost management scenarios. For readability, the framework is introduced in layers. First, we define a generic ABC ledger in which each activity has a measurable driver, a resource-intensity coefficient, and a unit price. Second, we specialize this ledger to the technical LLM service costs that can be validated directly from telemetry, such as tokens, hosting time, database operations, and storage. Third, we add optional organizational and architectural extensions, including governance, risk, agentic execution, and hybrid local–cloud serving. The main distinction is between transaction-driven technical costs and personnel-driven organizational costs. The technical submodel is governed by observable system interactions, such as tokens and compute seconds, and is therefore empirically validated in the case study. The human operational, governance, adoption, and risk layers provide a parameterized enterprise-planning blueprint; their exact numerical calibration remains context-dependent and varies with institutional maturity and regulatory environment. B. FORMAL MATHEMATICAL MODEL

Paper

Authors

P. Tombor, J. Abonyi

Publication date

2026

Read the original paper

Transcript

A transcript has not been published for this episode yet.

Licensing

Source license

Creative Commons Attribution 4.0 International

Recording license

CC-BY-4.0

The authors and publisher do not sponsor or endorse this recording.