Prompt injection and the security of AI models and agents: attacks that hijack chatbots and AI agents through hidden instructions, jailbreaks, data poisoning and the defences against them, from security researchers, arXiv papers, OWASP and the news, plus any story about these attacks.
Summary
109 news articles and 796 research items in this section, back to 20 November 2025. Browse the news ↓Research ↓
3 stories from 3 publishers in the last 24 hours, 10 from 4 this week.
AI agents retrained on a bigger pile of prompt injection attacks came out easier to hijack in three of four tests run by researchers at the University of Waterloo, Zhejiang University and KTH Royal…
Prompt injection has moved from laboratory curiosity to litigated misconduct and a half-billion-dollar acquisition market, as researchers conclude that in-prompt defences cannot be trusted and enforcement must sit outside the model.
The month's most visible episode was a Brazilian court fining lawyers around $16,000 for hiding a prompt injection in a filing, a case that drew unusually heavy discussion and marks the first widely reported judicial sanction for the technique 1819. Commercially, CrowdStrike agreed to pay roughly $495m for Fiddler AI and Dreadnode, and Palo Alto Networks is reported to be negotiating a $500m purchase of Clutch, signalling that incumbents regard agent security as a category worth buying rather than building 141617. Threat reports describe jailbroken or purpose-built agents running autonomous intrusions, including a two-week campaign against a European telecom attributed to a DeepSeek-based agent 726. On the research side, a cluster of papers argues that prompt injection is structural, that human red-teaming is approaching saturation, and that deterministic policy enforcement, intent tracking and taint analysis outside the model are the only defensible path 39404143.
Key developments
Brazilian court sanctions lawyers for hidden injection. A Brazilian court fined lawyers roughly $16,000 after the court's own AI flagged hidden text in a filing demanding superficial legal scrutiny, according to the linked reports [18][19]. The story reached 348 points and 220 comments on Hacker News, far more than any other item in the period, indicating broad public recognition that prompt injection is now a legal and ethical issue, not merely a technical one [19]. It sits alongside a separate write-up on using 'lawyerspeak' to bypass agent guardrails, suggesting legal register is both a weapon and a venue [3]. 18193
Incumbents buy into agent security at scale. CrowdStrike said it will acquire Fiddler AI (agent observability) and Dreadnode (automated AI security testing) for a reported combined $495m, adding both to its Falcon platform [16][17]. TechCrunch separately reports Palo Alto Networks in talks to buy Clutch for about $500m, a company last valued at $85m according to a single source, so that figure should be treated as unconfirmed [14]. Obsidian Security's new Runtime Policy Decision Engine, which vets agent actions before execution, and OpenAI's engagement of startup Hacktron for testing show the same demand from both vendors and model labs [4][13]. 141617413
Autonomous and jailbroken agents attacking real targets. Malbear Labs reported that a jailbroken DeepSeek agent spent two weeks attacking a European telecom, and Symantec separately uncovered an agent designed to run a full cyber-attack without human intervention [7][26]. Check Point disclosed 'Haiou', a Codex jailbreak bundle that rewrites an agent's trusted instructions to inject adverts for a third-party AI gateway, a phenomenon it calls prompt adware [15]. Researchers are responding with 'shibboleth traps', honeypots designed to fingerprint the model and harness behind an agentic attack from telemetry [37]. 7261537
Research consensus: treat the model as untrusted. Several papers converge on deterministic enforcement outside the model: reference monitors with taint tracking and human approval [40], mechanised noninterference-style definitions with release policies [41], intent graphs auditing tool calls for least privilege [43], and intent-aware protection that asks whether an agent remains faithful to its deployed purpose [33]. CapLite questions whether small-model agents can even be trusted to write the security artefacts that schemes such as CaMeL and Progent rely on [31]. This matches Obsidian's commercial pre-execution policy approach [4]. 4041433331
Attacks scale while red-teaming saturates. One paper states that static payload attacks were saturated by mid-2025 and that human red-teaming may be nearing saturation in autumn 2026, proposing adaptive agent swarms to keep pressure on defences [39]. Others show that resizing or re-rendering an image can reveal instructions invisible at one resolution [32], that benign requests can escalate into unsafe actions across multi-agent sessions via poisoned shared state [44], and that the execution harness, not just the backend model, determines web-agent resilience [38]. Together these widen the attack surface beyond text and beyond any single model. 39324438
Injection risk crosses into the physical world. DeepMind released Gemini Robotics 2.5 together with an open safety benchmark named ASIMOV-jailbreak [5], while a separate paper examines coding agents that control real robot arms zero-shot and the safety risks that arise at the digital-physical boundary [42]. The combination means a successful injection could translate into physical action rather than a leaked secret, raising the stakes for the enforcement work above. 542
Themes
Detection products proliferate, traction is thin. A steady stream of Show HN posts offers filters, judges and token-level gates, including formal-logic detectors, PDF injection judges, positional scanners and 'System 1' decision models, yet almost all registered one to three points and no comments [2][11][12][20][25][28][29][30]. The contrast with the heavily cited enforcement research and nine-figure acquisitions suggests the market is moving past phrase-matching toward runtime control and observability [40][16]. 211122025
Governance and workforce catching up. ISC2 launched a vendor-neutral Cyber-AI Workforce Coalition and is recruiting 3,000 volunteers for a pilot of its first AI security certification, expected in early 2027 [21][22]. An IANS report finds head-of-AI-security roles surging in large organisations while accountability for AI governance remains unsettled, echoing a gateway governance write-up asking 'who governs the governor' [27][23]. 212227238
Supply chain and tooling as attack vectors. Prompt adware hidden in a jailbreak toolkit, stored injections through API gateways and new research on AI supply-chain exposure all point to compromise upstream of the deployed agent [15][23][6]. Open datasets of prompt-injection CVEs with a monthly threat index indicate the community is beginning to track these as conventional vulnerabilities [10]. 1523610
What to watch
Whether Palo Alto Networks confirms the reported $500m Clutch acquisition, which would make two near-identical bets by major incumbents within a week and validate the agent-security category [14][16][17]. 141617
Further court or regulatory sanctions following the Brazilian case, which would establish prompt injection as recognised professional misconduct beyond one jurisdiction [18][19]. 1819
Independent validation of the telecom and Symantec agent-attack reports and adoption of fingerprinting techniques, since both are single-source findings so far [7][26][37]. 72637
Take-up of the ASIMOV-jailbreak benchmark and the ISC2 certification pilot as indicators of whether safety evaluation and skills are being standardised rather than left to vendors [5][22]. 522
Written by Claude Fable 5.1 (Anthropic), based on 45 recent headlines and excerpts in this section, 3d ago. AI analysis can misread or miss things: the numbers link to the original reporting, which is the authority.
Research 796
Journal articles, academic blogs, think-tank reports and papers, kept apart from the news below.
Camera-equipped glasses can convert an examination page into a query to a vision-language model (VLM). Device detection addresses possession; a complementary question is whether an examination document can reduce the usefulness of an unauthorized capture. We present SEOYUN, an application-specific framework for refusal-inducing exam documents, with optical and encoder-level interventions as extensions. We use reverse jailbreaking to describe the direction of the objective…
Large language model (LLM) agents routinely interact with untrusted external data, exposing them to indirect prompt injection (IPI) attacks that hijack agents into executing attacker-chosen tasks. To uncover these vulnerabilities at scale, automated red-teaming methods often leverage reinforcement learning (RL) to generate attacks that achieve high attack success rates (ASR). Model developers use the attacks discovered during internal red-teaming to harden agents through…
Prompt injection is the dominant unsolved security risk for LLM agents, and the community now agrees that robustness must be measured against adaptive attackers rather than fixed benchmark injections. Many adaptive red teamers exist, and most are built to maximize attack success rate (ASR). Yet ASR counts how often a target breaks, not in how many different ways. A red teamer can achieve a high ASR by reusing a few techniques that work, leaving the others untested and absent…
Multimodal language models increasingly answer questions over external documents, exposing them to indirect prompt injection: commands planted in untrusted content to hijack the response. Rendering untrusted content as page images lowers attack success on average, leaving open which injected operations survive rendering, at what utility cost, and whether failed image attacks reflect unread commands or commands read but not followed. We pair text and page-image versions of…
Control-flow integrity (CFI) confines a program to a control-flow graph fixed ahead of time. Procedural agent skills move this control flow into prompt text, and a large language model (LLM) agent decides at runtime whether each step runs in its reasoning, a tool call, or a script. The procedure's control skeleton survives, but its enforceability depends on where each step runs. Many violations, such as overwriting a file before validation, use only operations the skill…
Local differential privacy (LDP) has become a crucial paradigm for privacy-preserving graph analytics, enabling graph analytics tasks such as subgraph counting without requiring a trusted curator. However, while LDP provides strong privacy guarantees, the integrity and robustness of LDP subgraph counting protocols in the presence of malicious participants remain poorly understood. In particular, since data are randomized locally before collection, it is inherently difficult…
AI agents retrained on a bigger pile of prompt injection attacks came out easier to hijack in three of four tests run by researchers at the University of Waterloo, Zhejiang University and KTH Royal Institute of Technology. The automated red-teaming tool that found more working attacks produced the weaker defense. Prompt injection purple-teaming workflow (Source: Research paper) Someone hides instructions in an email, a web page or a tool’s output, and the AI agent that ……
The easiest way to fool someone is to give them something they already expect. A job offer. A message about a service outage. A familiar app. Nothing looks strange at first. And that's the problem. This week's stories show how ordinary things are being used to steal data, spread malware, and get people to hand over access. There's another shift worth watching. AI is making some attacks easier
Canonical is letting Ubuntu Pro customers install security patches written by AI agents before its engineers finish reviewing and testing them. The patches come from Surge, a system Canonical built to fix known vulnerabilities in Ubuntu’s userspace software, meaning everything outside the operating system’s kernel. They ship through a new opt-in channel called the Edge archive. If you leave Edge switched off, nothing changes in how your updates are checked: Ubuntu’s regular…
Apple on Monday introduced new controls that allow Mac users to give more granular access to data they give to apps, including those that use artificial intelligence (AI) agents. The changes, currently previewed in macOS 27.3 beta 2, relate to increased transparency and more options for two of its APIs: Device Control and Data Access, and Full Disk Access. "Before granting an app such
In this Help Net Security interview, Peter Stöckli, Security Researcher at GitHub, explains how the team uses AI agents to audit software for vulnerabilities. The discussion covers why the work started with authentication and authorization bugs, how prompts are tested and trimmed for each model, and what the agent produces during threat modeling. Stöckli also describes how every report is checked by a person before it reaches maintainers, how false positives are filtered…
Seed deals for cybersecurity startups dipped slightly in the third quarter of 2026, the second decline in a row, while the wider U.S. venture market had one of its best quarters since 2021. Investment firm DataTribe tracked the split using PitchBook data on U.S. seed through Series E rounds, and sorted the cyber seed deals into categories. Across the market, capital is pooling in fewer, larger rounds, and the bar for the round after seed … More →
Obsidian Security has announced its Runtime Policy Decision Engine, giving security teams centralized control over AI agent actions before they run. Building on Obsidian’s extensive AI agent discovery and governance coverage, the engine uses context about who is behind an agent, its access and the third-party applications involved to allow or block actions, or require human approval where supported. Context from your apps, identity systems and your own labels feeds one…
Two weeks ago, I attended the [un]prompted AI security conference in San Francisco. It was an excellent event: three days of very dense talks about all aspects of AI security. (My talk was about integrous AI, and I will post it when it is online.) AI security can be divided into three broad topics. It’s important for researchers to understand which one they are working on. And it’s important for us as a community to work on them all. They are: security of AI, security with…
0sec Labs has open-sourced 0, an AI security harness for code review, pentesting, reverse engineering and fixes. It runs locally through a browser console, a headless CLI and an MCP server, with your own model provider.
Cybersecurity researchers have disclosed details of an OpenAI Codex jailbreak bundle that's designed to modify an artificial intelligence (AI) agent's trusted instructions to insert advertisements for a third-party AI gateway. The phenomenon has been codenamed prompt adware by Check Point Research. Called Haiou (meaning Seagull in Chinese), the jailbreak modifies OpenAI Codex's configuration
CrowdStrike said it will buy Fiddler AI, which makes AI agent observability software, and Dreadnode, which helps companies automate AI security testing.
ISC2’s new Cyber-AI Workforce Coalition serves as a “vendor-neutral convener” to map the future of AI security careers, ISC2 CEO Scott Beale told Infosecurity