SAP: Stay in Control of AI Costs—Reduce Token Spending, Maintain Innovation Capacity

Generative AI has made the leap from the experimental stage to becoming an indispensable part of the infrastructure in a very short time. It has become an integral part of employees’ daily work and is being integrated more deeply into team processes and applications with each passing quarter. This growth brings with it a new category of costs for which finance was simply not prepared: AI token consumption. 

When AI writes code and controls agents, token consumption can exceed originally budgeted costs. Finance leaders are thus faced with the challenge of managing these expenses in a disciplined manner without hindering the use of AI. To strike the right balance, tokens must be treated as a corporate resource, and the results achieved with them must justify the underlying costs. 

At SAP, we’ve addressed this very issue in practice and summarized what we’ve learned while creating, reviewing, and refining our approach to managing these costs. 

You Can’t Manage What You Can’t See 

Most finance leaders would not manage a major cost category based on a single budget line item. Yet this is exactly how companies often approach AI in the beginning. Total spending is an important metric. However, it says nothing about which teams, workloads, or usage patterns lie behind this spending, or whether the specific use of AI actually makes sense. 

Transparency regarding these aspects is only possible when sales, development, finance, and product teams work closely together. Finance handles the cost forecasts and ensures accountability. The other departments provide the operational context that makes the numbers meaningful. 

We were able to improve our financial models with each new planning cycle because we had access to more operational details regarding usage patterns. For a new cost category characterized by dynamic developments, the following therefore applies: You don’t need precise data before taking the next steps; to start with, it’s enough to have sufficient transparency to make better decisions. The model can then be refined as soon as patterns emerge. 

Clear Responsibilities   

Our most important insight was less financial than conceptual in nature: Transparency alone is not enough—someone must also be responsible for overseeing AI usage. 

A centralized AI budget facilitates initial experiments but relieves those who consume tokens of any financial responsibility. The more closely AI is integrated into business processes, the more this model reaches its limits. 

However, this should not be taken as an argument for immediately billing every query to a language model with accounting precision. Rather, the goal is to manage AI consumption in the same way as other corporate resources, such as software, external services, or personnel. Decision-makers need a clear picture of what their teams are consuming and what results are to be achieved with AI. 

At SAP, allocating token costs to business units has led to a new approach—shifting the focus from “How much are we spending?” to “What are we getting in return?” and “Is further investment worthwhile here?” These questions enable a more productive debate.  

Token spending is not merely a line item on the technology side of the balance sheet; rather, it represents a strategic and operational investment decision. The goal is to manage AI spending in a targeted manner, rather than simply keeping costs as low as possible. 

Don’t Just Look at Cost Per Token 

High AI costs attract attention. However, if the focus is exclusively on cost reductions, this can lead to the wrong decisions being made. 

A better guide is the ratio between consumption and business value. An AI tool that noticeably speeds up software development, relieves employees of tedious routine tasks, or improves customer service incurs real token costs. Restricting AI use simply because the budget for it seems high could cost a company more in lost value than it saves. 

We speak from experience: After introducing AI tools for developers at SAP, we saw a mid-double-digit percentage increase in the review and implementation of changes to the source code—a clear sign that development work was progressing faster. The associated token consumption had thus paid off. 

Finance therefore needs a dual perspective: cost metrics on the one hand, and benefit metrics on the other. Those who fail to consider these two aspects in context run the risk of letting cost savings get in the way of value creation. 

Focusing on Unnecessary Token Consumption Rather Than AI Usage in General 

As the use of AI increases, control mechanisms become necessary—specifically in the form of targeted strategies rather than blanket measures. When we examined usage patterns in detail, we identified three fundamental causes of disproportionately high spending: a focus on power users and automated agents, an inappropriate choice of models, and uncontrolled use of tools. 

These are the areas where guidelines make sense. Control mechanisms are valuable when they focus on preventable expenses rather than curtailing AI usage in general. At SAP, we rely on three approaches: Limiting tokens to prevent excessive consumption; the intelligent use of models to better align performance and costs with the specific task; and a more targeted use of tools to eliminate redundancies and focus investments on areas where their use is justified. These measures helped keep the financial risk in the range of hundreds of millions without slowing down the continued adoption of AI. 

The principle is simple: reduce token consumption where it is unnecessary and maintain it where it is worthwhile. 

From Cost Control to Increasing Business Value  

AI progress does not follow corporate planning cycles. The possibilities, use cases, and economic conditions surrounding AI are changing every day. And the associated governance model must keep pace with this change.   

That is why we must remain proactive. The next step is to integrate all these practices into regular planning and reporting, define responsibilities more clearly, and optimize cost allocation. We also need forecasting capabilities to identify cost trends even before they become apparent. 

Even companies that manage all of this well must still bear AI costs; however, they have the transparency and clear lines of responsibility to identify where AI use enables value creation and where it does not. This enables them to manage investments accordingly.   

– – – – –

Further Links

👉 www.sap.com   

Photo: unsplash