Software

adesso: When AI Itself Becomes the Attacker

October 16, 2026. Within just a few weeks, two of the leading AI providers had to admit that their own models had infiltrated real-world systems. At OpenAI and Hugging Face, a combination of AI models autonomously chained together vulnerabilities, exploited an unknown zero-day vulnerability, and—without any insight into the source code—progressed to the point of executing external code on Hugging Face’s production infrastructure. Shortly thereafter, Anthropic reported a similar incident: During test runs, three Claude models unexpectedly infiltrated the systems of three real companies after a misconfiguration connected the isolated test environment to the internet. This was only noticed during a retrospective analysis of over 141,000 runs. In one case, a model even wrote malware and made it publicly available. For adesso, as part of its proactive security strategy, this comes as no surprise—rather, it is a confirmation. Now it is simply documented and real—and explicitly not a problem specific to a single provider, but rather the new normal.

Stock image: Artificial Intelligence (AI) / Photo by Growtika on Unsplash

Nothing New—Just Much Faster

The attack patterns aren’t new. What’s new is the speed. AI is changing the economics of attacks: What used to require specialized knowledge and take weeks now runs automatically in minutes—so beginner-friendly that you no longer need advanced skills. This creates a dangerous asymmetry. With AI, attackers find vulnerabilities at breakneck speed, including zero-day vulnerabilities without patches. According to the BSI, it takes an average of 73 days for a patch to become available—any organization that holds a “patch day” every 14 days is structurally too slow. And the attacker needs only one open door; they automate tens of thousands of attempts per hour.

The problem often runs deeper: Many organizations lack the basics—a clear picture of their own applications and processes. I like to compare security to a house: The best front door is useless if the back door is open or the key is hidden under the doormat. Security only works holistically—the chain is only as strong as its weakest link.

Sovereignty Means Being Able to Act

The debate over digital sovereignty often revolves around server locations. But the core issue is something else: having the ability to act—and you can only do that if you’re prepared. It starts with two steps. First, transparency: Where do I stand, and where are my biggest gaps? Second, prioritization: Because resources are limited, I focus on what’s critical—specifically the areas where an attacker could exploit a chain of vulnerabilities. The good news: AI also helps the defender find these critical points faster.

AttackAI: Playing with the Same Cards

This is exactly where AttackAI comes in. If attackers use AI as a weapon, we must harness the same efficiency for our own attacks. We’ve equipped our Red Team to launch targeted attacks on systems, infrastructure, and data centers using AI models—with the same, or in some cases better, resources than the attackers. This doesn’t guarantee security yet, but it provides an honest snapshot: Where are the doors and windows left open? This serves as a roadmap for defense.

As a trained software developer, I know that vulnerabilities often originate directly in the source code. That’s why our preferred approach is white-box testing, including source code analysis. The Hugging Face incident is instructive in this regard: The models managed to execute code even without access to the source code. If an attacker can achieve this from the outside—how much more thorough can we be if we consciously examine our own code first?

Control makes all the difference

And yes: What went wrong in these evaluations is, at its core, our own doing. That’s exactly why we take this risk very seriously. The difference between a test run gone wrong and responsible offensive security lies not in the tool, but in the control over it.

What happened there—an AI exceeding its limits, affecting unrelated real-world systems, or spreading malware—is ruled out in an AttackAI operation thanks to our approach. We attack only contractually authorized, clearly defined customer systems; there is no such thing as a fictional target that accidentally becomes real. Humans are in control, the agents carry out the tasks—we monitor the attacks instead of letting them run unsupervised. We operate in a highly regulated environment; the results are documented in a controlled report. And data protection, especially for critical infrastructure and the healthcare sector: A real attacker doesn’t care about it, but we do—confidential information never leaves your own network and is stored on specially protected hardware.

Act now—but do it right

With AI, we’ve entered a new generation of cybersecurity. Attacks will increase dramatically—becoming easier for novices to carry out while also being more efficient than ever before. This isn’t an issue where you can bury your head in the sand; you must act now, while we’re still taking proactive steps rather than merely reacting. However: AttackAI offers an initial, sharp insight, not the complete solution. Nothing can replace a comprehensive security infrastructure, including pre-planned emergency playbooks.

Don’t take AI-powered attacks lightly. The right approach is to think like the attacker before the attacker does—and to remain capable of taking action. That’s exactly why we built AttackAI.

Sources:

OpenAI & Hugging Face on the incident (July 21, 2026): openai.com/de-DE/index/hugging-face-model-evaluation-security-incident
heise online on the OpenAI/Hugging Face announcement: heise.de/news/KI-Modelle-von-OpenAI-haben-Cyberangriff-auf-Hugging-Face-autonom-ausgefuehrt-11372797.html
Anthropic on three incidents in cybersecurity evaluations: anthropic.com – “Investigating three real-world incidents in our cybersecurity evaluations”
tagesschau.de on the Anthropic incident: tagesschau.de/wirtschaft/digitales/anthropic-ki-angriff-unternehmen-100.html

– – – – –