![](\"https:\/\/cdn.pblzr.de\/dacbb27c-1270-4041-b681-e2b95f06f8a1\/2026\/09\/criticalmanufacturing-logo-400x300_1_TEXT.jpg\"){kind=logo}
<\/p>\n<\/p>\n
Since agent-based AI in manufacturing is increasingly not just advising but also taking action, the question is no longer whether it is intelligent enough. Rather, the question is whether anyone can explain what it just did.<\/p>\n
Yes, I know that sounds scary to many people, but even if you\u2019re still resistant to this idea today, you\u2019ll soon be letting AI take action on your production line.<\/p>\n
Take a moment to imagine what that might look like: releasing a batch, adjusting a setpoint, routing material for rework, or putting a batch on hold. The technology is ready, and the pressure to deploy it is undeniable.<\/p>\n
The problem is: Some time after that decision, someone with decision-making authority will ask why the system acted that way. It could be a regulatory agency during an inspection. Or an auditor reconstructing an incident. It could be your own executive management following a committee incident. Or a customer\u2019s quality team before they sign the contract.<\/p>\n
And in that situation, \u201cthe model was very reliable\u201d is not an answer. It\u2019s the beginning of a very bad day.<\/p>\n
The question that determines whether you can delegate a decision to AI isn\u2019t how intelligent the system is. It\u2019s whether the system can explain itself to someone who holds you accountable.<\/p>\n
And as it turns out, explainability isn\u2019t something you add to AI after the fact. It\u2019s a property of what the AI is built on.<\/p>\n
If this seems hypothetical or far-fetched to you, think again. There was an initial warning letter from the FDA citing the use of AI in pharmaceutical manufacturing. Investigators found that the company had not performed process validation prior to distributing its drugs, as required by 21 CFR 211.100. The company\u2019s response stated, verbatim, that it had been unaware of this requirement because the AI agent it used \u201cnever informed [them] that this was required.\u201d<\/p>\n
Note that the FDA\u2019s position was not that AI had been used. Rather, the issue was that AI results must be reviewed and approved by a person responsible for CGMP compliance, and that the use of AI to assist in the creation of documents does not eliminate the obligation to verify their accuracy and compliance. The agent\u2019s silence is no justification.<\/p>\n
There are two ways an AI system can answer the question \u201cWhy did you do that?\u201d<\/p>\n
The first is to generate a plausible justification after the fact. If the reasoning was not recorded anywhere, the model reconstructs a narrative that fits the result.<\/p>\n
The second is to trace the decision back through the structure that produced it. The system took this action because a specific rule applied to this step in this version of the recipe, for this product, based on the state of the material and the quality result recorded at that time.<\/p>\n
This is an explanation because the reasoning was captured before the decision was made. You\u2019re not asking the AI to justify itself after the fact; you\u2019re following a path that already exists.<\/p>\n
To understand why this distinction is important, set the AI aside for a moment and consider a single number in manufacturing.<\/p>\n
A measurement value on its own means nothing. It is just noise until it can be placed in context: which product it describes, which piece of equipment generated it, which recipe version was running, which operator performed the work and whether their certification was up to date, which material batch was used, and which order linked all of this together.<\/p>\n
When you add these connections, the number becomes a fact. Without them, it\u2019s just a value without meaning.
This context must be compiled in one of two places:<\/b><\/p>\n\u2013 Late<\/h4>\n
Events end up in separate locations, and when a question is asked, the system immediately re-links everything. This can work, but you pay the price for this linking with every question, and a system that reconstructs relationships it did not witness firsthand can only guess.
Sometimes the guess is wrong, and no one can tell.<\/p>\n
The moment the step is completed, the execution system already knows the product, the equipment, the recipe version, the operator, the batch, and the order. Not because it looked them up, but because it enforced them a moment earlier. It verified the certification before approving the step. It confirmed that the equipment was qualified and linked the batch to the order. The context was the process itself.<\/p>\n
AI can only explain a decision if the reasoning was captured at the very moment it took place\u2014and that is precisely what the MES does by enforcing rules (certifications, equipment qualification, batch linking)\u2014rather than reconstructing them after the fact.<\/p>\n
Interestingly, this is not a new standard invented specifically for AI. It is the standard by which more heavily regulated manufacturing environments already operate.<\/p>\n
The principles of data integrity require that records be traceable and captured at the time of the event. Regulatory authorities also expect electronic records to reconstruct the event, not merely summarize it.<\/p>\n
The reason for this is simple: if you cannot trace a decision back to the moment it was made, you can hardly defend it. AI does not change this requirement; it increases the pressure, because the entity seeking trust is no longer a person.<\/p>\n
The reason a decision can be traced at all is that a well-designed MES does more than just record what happened. Whether explicitly or not\u2014every production step generates a graph of relationships. The material, the batch, the equipment, the operator, and the recipe step become nodes, while the processes between them form the edges.<\/p>\n
MES traceability captures these relationships in real time, thereby creating a coherent record of how production actually unfolded.<\/p>\n
It is this graph that provides an explanation. When an auditor asks \u201cwhy,\u201d the answer is a traceable path through the graph: from the decision back to the rule that triggered it, the step it controlled, as well as the material, the quality status, and the person involved.<\/p>\n
The explanation does not have to be reconstructed on demand. It already exists, since the system captured these relationships during factory operations.<\/p>\n
To be fair to the alternative: A contextualization layer built on top of the execution can reconstruct much of this retrospectively. In many plant- and process-oriented environments, this works well enough to be useful.<\/p>\n
However, the reconstruction is limited by what the execution system actually recorded. It can roughly trace the lineage, but it cannot restore meaning that was never recorded. This gap between \u201croughly\u201d and \u201cverifiable\u201d is greatest where it matters most.<\/p>\n
In highly variant, regulated, discrete manufacturing, where genealogy, deviations, and rework are not edge cases but are fundamental to operational processes.<\/p>\n
That is why you should be cautious of vendors who claim to have solved the problem of \u201cexplainability.\u201d Two different things are being sold under this term.<\/p>\n
One is the understanding of how a model reasons internally\u2014a problem that has not yet been solved by experts and for which there isn\u2019t even a standardized method of evaluation.<\/p>\n
The other is an audit trail that shows which inputs led to a decision\u2014and that is exactly what most explainability tools actually offer.<\/p>\n
That\u2019s why the foundation is so important when it comes to agentic AI. Without the right foundation, AI cannot provide reliable answers.<\/p>\n
Certain topics come up again and again when we discuss decisions made by agent-based AI. These are important, even if they aren\u2019t necessarily related to the topic of explainability.<\/p>\n
Determinism. This is very important, even if it merely prevents the use of generative AI in critical runtime execution functions. In regulated environments or risk-averse organizations, determinism\u2014where identical input conditions always lead to the same output decisions\u2014is a must.<\/p>\n
In these cases, generative AI belongs in the design phase, not the execution phase. Whether through vibe coding or, more intelligently, through vibe metadating (AI-based configurations utilized by the platform)\u2014companies can accelerate their business rules and processes; however, only deterministic behavior makes it into runtime execution.<\/p>\n
Safety barriers. These are also crucial, but they are not enough. With safety barriers, you define the space in which the AI operates, and the boundaries themselves are rules that must be explainable. However, they do not explain the decisions made within those boundaries.<\/p>\n
Human oversight. \u201cHuman-in-the-loop\u201d is constantly cited as the answer to all of this. On its own, however, this is not enough. If a human approves a decision that they cannot question, that is not oversight. It is liability with a signature.<\/p>\n
So, back to explainability and decisions. None of the solutions mentioned relieves you of the most difficult decision. Even a system that can explain everything perfectly won\u2019t tell you how much trust is sufficient to act. This threshold\u2014the line between support and autonomy in a given decision\u2014is a judgment for which someone with accountability must take responsibility. The platform can make the threshold visible, verifiable, and customizable. But it cannot set it for you.<\/p>\n
That is precisely why this is no longer an IT issue, but a matter of leadership. A leader can decide how much trust to place in the system. But no leader can defend a decision that the system cannot explain.<\/p>\n
In regulated environments, the direction is already clear: The European Commission\u2019s draft Annex 22 explicitly draws the line: Generative AI and large language models are not covered and should not be used in critical GMP applications; and where AI is used in non-critical applications, qualified personnel remain responsible, within the framework of a \u201chuman-in-the-loop\u201d approach, for ensuring that the results are appropriate.<\/p>\n
The principle underlying the draft is that intelligence does not transfer responsibility. So before the debate about how autonomous your AI should be even begins, there is a preliminary question\u2014one that is more architectural than strategic in nature. When your AI takes action and someone with decision-making authority asks for the reason, does your factory respond with an explanation or with a predefined procedure?<\/p>\n
You cannot delegate a decision that your system cannot explain. And your system can only explain what it was originally designed to do.<\/p>\n
– – – – –<\/p>\n