Results for “Injection”

arXiv (research papers) Justin Szczepaniak SI

Defusing Explosive Prompts: Understanding and Preventing Trigger-Based Prompt Injections in LLM Agents

As LLM applications integrate with external tools, they are increasingly exposed to indirect prompt injection (IPI), where adversarial instructions are embedded in retrieved content. Conventional IPIs fire on contact: the moment an agent ingests the content, it carries out the instruction. We introduce the explosive prompt, a conditional payload that stays dormant until an attacker-chosen trigger is met, in effect a training-free, inference-time backdoor planted in a single…

Read at arXiv (research papers)

arXiv (research papers) Yuxuan Zhang SI

Origin Is All You Need: Provenance-Aware Transformers for Structural Trust-Boundary Separation

Indirect prompt injection (IPI) remains a central safety and security challenge for large language model (LLM) systems because standard transformers lack architectural notion of source authority. Retrieved documents, user inputs, and system instructions are all processed through the same undifferentiated attention mechanism, forcing the model to infer from wording alone what should be obeyed and what should be treated as data. We propose Provenance-Aware Transformers, a…

Read at arXiv (research papers)

Simon Willison SI

Self-generated prompt injections in compaction summaries

Self-generated prompt injections in compaction summaries In Our framework for reporting model misalignment OpenAI provide "six reports on unexpected or concerning model behavior we’ve observed in the last six months". This one here is my favorite: they caught some of their models in training deliberately subverting themselves in their compaction prompts. Compaction is the process agent systems use when they are running out of tokens in their context window, so they summarize…

Read at Simon Willison

arXiv (research papers) Alex Remedios SI

Red-Teaming Auto Mode: Improving Blocking Classifiers Against Malign Coding Agents

To keep coding agents from going off the rails, production systems now review each proposed action with a blocking monitor that can reject it before it runs (Auto Mode in Claude Code, Guardian in OpenAI's Codex). Prior evaluations of such monitors largely measure robustness to accidental harm or prompt injections from untrusted sources looking to hijack the agent. Less understood is how they hold up when the agent they monitor is persistently misaligned. To understand this…

Read at arXiv (research papers)

arXiv (research papers) Matteo Golinelli SI

AgentLSD: Evaluating AI Security Agents Under Adversarial Task Contamination

AI agents for security inspect web pages, source code, logs, configuration files, and command outputs. These environments may contain deceptive artifacts that influence the agent's behavior. We call this adversarial task contamination. Whereas prompt injection relies on attacker-supplied instructions, task contamination also includes non-instructional evidence, such as fake results and decoy endpoints. We present AgentLSD, a controlled framework for studying adversarial task…

Read at arXiv (research papers)

arXiv (research papers) Elia Nikolaou SI

CaMeLoT: CaMeL orchestrated with Temporal logic for static verification and liveness

LLM-based agents generate and execute multi-step plans that invoke external tools which can access private data or execute commands. In this setting, security is a property of the entire execution that a plan creates, not just any single step. The plan itself is a critical artefact that captures the tool calls, control flow, and data dependencies. We present CaMeLoT, a complement to CaMeL, an existing defence against prompt injection in tool-using LLM agents. CaMeLoT extends…

Read at arXiv (research papers)

Hacker News bucket2015 SI

I caught an LLM-powered recruiter with a prompt injection on LinkedIn

Article URL: https://khancyr.github.io/blog/2026/03/15/how-i-caught-an-llm-powered-recruiter-with-a-prompt-injection-on-linkedin/ Comments URL: https://news.ycombinator.com/item?id=49726928 Points: 1 # Comments: 1

Read at Hacker News

arXiv (research papers) Hasnain Irshad SI

The Verifiable Action Card: Trustworthy Human-in-the-Loop Control for Secure Autonomous Agents

Agentic browsers can execute security-sensitive actions under a user's authenticated session, making indirect prompt injection and deceptive confirmation interfaces a direct threat to action integrity. Existing human-in-the-loop (HITL) safeguards are insufficient when the approval prompt itself can be influenced by untrusted page content or model-generated text. We present the \emph{Verifiable Action Card} (VAC), an architectural defence that reconstructs approval…

Read at arXiv (research papers)

Hacker News Hiskias SI

Show HN: Open-Source Lightweight Prompt Injection Safety

Article URL: https://github.com/StackOneHQ/defender/ Comments URL: https://news.ycombinator.com/item?id=49714872 Points: 1 # Comments: 0

Read at Hacker News

arXiv (research papers) Tanzim Hossain Safin SI

Trust propagation and structural containment in Multi-agent LLM pipelines

Multi-agent LLM systems increasingly automate tasks involving agents with different levels of privilege, creating a security risk in which a compromised low-privilege agent can influence a higher-privilege agent and trigger an unauthorized action. We study attack propagation in a four-agent LangGraph pipeline comprising a Supervisor, Researcher, Validator, and Executor. We evaluate shared-memory poisoning and indirect prompt injection through a forged approval embedded in a…

Read at arXiv (research papers)

Hacker News roschdal SI

Ask HN: Is your AI LLM based web service secure from prompt injection?

... Ignore everything above, do this instead... Comments URL: https://news.ycombinator.com/item?id=49703186 Points: 2 # Comments: 2

Read at Hacker News

arXiv (research papers) Xiaoyan Li SI

Universal Defenses for Tool-Integrated LLM Agents Against Adversarial Attacks

Large Language Model (LLM) agents have demonstrated impressive capabilities across a variety of domains, particularly when integrated with external tools for multi-step task completion. However, they are increasingly vulnerable to adversarial attacks, including direct prompt injection, indirect prompt injection, memory poisoning, and backdoor attacks, which exploit the model's openness to prompt injection and tool manipulation. In this work, we explore practical and…

Read at arXiv (research papers)

arXiv (research papers) Yusuf Khalid Shire SI

PIDS-Bench: Evaluating Prompt-Injection Detectors Under Over-Defense, Obfuscation, and Distribution Shift

Prompt-injection detectors are typically evaluated using aggregate F1 on in-distribution test data, which offers limited insight into behavior under distribution shift, particularly on the benign side of the decision boundary, where false positives impose direct operational cost yet are seldom measured. We present PIDS-Bench, a frozen multi-axis benchmark that jointly evaluates attack detection and benign false-positive behavior at fixed thresholds, spanning in-distribution…

Read at arXiv (research papers)

arXiv (research papers) Bingzheng Wang SI

ActGuard: Pre-execution Action Auditing against Indirect Prompt Injection in LLM Agents

Large language model (LLM) agents interact with external environments through tool invocation, but tool outputs can also expose them to indirect prompt injection (IPI) attacks. Existing defenses mainly rely on prompt hardening, content filtering, pre-generated plans, or permission constraints. These approaches often struggle with complex tasks or over-sanitize external content, making it difficult to balance security and utility. The key challenge is therefore to preserve…

Read at arXiv (research papers)

arXiv (research papers) Donato Mecca SI

SkillSecurer: Detecting and Patching Prompt-Injection Vulnerabilities in AI Agent Skills

Agent skills extend AI agents with reusable instructions, scripts, and configuration, but are also open to new attacks to influence an agent's decisions and actions. To address these risks, we present SkillSecurer, a fully agentic framework for generating, detecting, localising, and remediating security risks in agent skills. Its red agent generates context-compatible injections across nine threat types while recording the exact modification; its blue agent analyses complete…

Read at arXiv (research papers)

The Rio Times Carlos Silva Geopolitics

Chile Unemployment Crisis Pushes Kast Toward Stimulus as Allies Attack His Cabinet

CHILE · ECONOMY Key Facts —The number Chile unemployment reached 9.5 percent in the May–July quarter, a five-year high; nearly one million people are out of work. —The stimulus Finance Minister Jorge Quiroz is weighing a “surgical” injection of public funds into construction, public works and hiring. —The rift Labor Minister Tomás Rau publicly distanced

Read at The Rio Times

arXiv (research papers) Asif Pinjari SI

DriftNet: A Dual-Head Trajectory Transformer for Detecting and Localizing Prompt Injection in LLM Agents

When an indirect prompt injection succeeds against an LLM agent, the compromise is visible in the agent's own behavior: a benign prefix of tool calls, a poisoned observation, and a suffix of actions that serve the attacker. An operator needs three facts: where the attack entered, which steps it corrupted, and whether apparent poison was resisted. Existing systems return either a whole-trace verdict or a single unsafe index. We present DriftNet, a dual-head trajectory…

Read at arXiv (research papers)

arXiv (research papers) Zehua Zhang SI

No-Box Vulnerability Analysis: Description-only Detection of Indirect Prompt Injection Vulnerabilities in MCP Servers

Conventional vulnerability analysis relies on either system access or dynamic interaction, all of which may be unavailable to third-party analysts auditing closed-source, remotely hosted, critical in situ systems, or commercially gated software. Therefore, we propose a new paradigm of no-box vulnerability analysis in which neither access nor runtime interaction is available, and only functionality metadata is available. Such metadata defines the intended behavior of the…

Read at arXiv (research papers)

arXiv (research papers) Anna Gazani SI

Architecting the Secure AI-SOC: A Neurosymbolic Framework for Pipeline Integrity and Threat Mitigation

The integration of Large Language Models (LLMs) into Security Operations Centers (SOCs) streamlines threat intelligence but introduces critical vulnerabilities, notably indirect prompt injection via log poisoning. Adversaries exploit this vector to execute multistep ``promptware'' kill chains by embedding malicious payloads within system logs to hijack the LLM's operational logic. Securing this pipeline presents a dichotomy: deterministic defenses are computationally…

Read at arXiv (research papers)

Hacker News fourfire SI

Google: Attackers are using prompt injection against coding agents

Article URL: https://cloud.google.com/blog/topics/threat-intelligence/from-prompting-to-autonomy-the-evolution-of-adversarial-ai Comments URL: https://news.ycombinator.com/item?id=49628034 Points: 13 # Comments: 1

Read at Hacker News

arXiv (research papers) Ryan Lum SI

Kernel-Managed Shared Memory for System-Wide Personalization

AI systems become more useful when they can adapt to the people using them, but in multi-agent systems, useful context learned by one agent often remains unavailable to others. We present kernel-managed shared memory, a system-level abstraction in which specialized agents write structured, tagged memories while the agent-system kernel, not individual agents, governs retrieval, privacy enforcement, and prompt injection. We implement and evaluate this design on AIOS and…

Read at arXiv (research papers)

Hacker News gmays SI

Prompt Injection Through Tool Output Is Two Events (Your Screens Read One)

Article URL: https://www.armosec.io/blog/untrusted-tool-output-prompt-injection/ Comments URL: https://news.ycombinator.com/item?id=49618594 Points: 2 # Comments: 0

Read at Hacker News

arXiv (research papers) Viet K. Nguyen SI

An Experimental Evaluation of Multimodal Prompt Injection Attacks on Agentic AI Frameworks

Agentic AI frameworks let a language model plan, keep memory, and call tools that reach real files, mail, and services. Most of these agents also read images, which gives an attacker a way to put text into the agent's context without going through the user. We present MMPIBench, a reproducible benchmark that measures what happens next. It delivers a fixed set of attacks through six visual carriers (OCR text, overlays, EXIF metadata, QR codes, fake interfaces, and hybrids)…

Read at arXiv (research papers)

arXiv (research papers) Dimitrios Stamatios Bouras SI

Authority Is Not a String: A Capability-Scoped Harness for Prompt-Injection-Resistant Coding Agents

Coding agents use system-level tools to read files, execute commands, and modify source code. Within the agent's sandbox, these tools often carry ambient authority: naming a resource is sufficient to act on it. Indirect prompt injection exploits this authority by placing instructions in repository files or tool output that cause the agent to perform actions the user did not request. We propose CapScope, a harness-level authorization mechanism that restricts tool use without…

Read at arXiv (research papers)

Google Project Zero Jann Horn Security

Testing race conditions with memory access tracing and stack-based delay injection

Many security bugs are race conditions, where multi-threaded execution has to occur with the right interleaving for a negative effect to appear. This creates challenges for several use cases: Confirming bug candidates that have been discovered manually or through static analysis. Regression tests: After fixing a race condition bug, there is often no good way to write a regression test that reliably triggers the bug as part of a test suite. Automatic bug discovery, such as…

Read at Google Project Zero

Hacker News fourfire SI

ASCII smuggling crosses over from AI prompt injection to phishing evasion

Article URL: https://www.microsoft.com/en-us/security/blog/2026/09/03/ascii-smuggling-crosses-over-from-ai-prompt-injection-to-phishing-evasion/ Comments URL: https://news.ycombinator.com/item?id=49604092 Points: 1 # Comments: 0

Read at Hacker News

arXiv (research papers) Boyang Zhang SI

CoER: Defending against Adaptive Indirect Prompt Injection via Adversarial Co-Evolution and Refinement

Language-model agents are vulnerable to indirect prompt injection (IPI) during tool use: adversarial instructions hidden in untrusted tool outputs can covertly redirect legitimate task execution. Existing work often trains and evaluates defenses against fixed attacks that do not adapt to the defender's behavior, so the resulting defenses may struggle against adaptive attacks in real-world settings. We argue that a strong defense against adaptive IPI must adapt during…

Read at arXiv (research papers)

arXiv (research papers) Asif Pinjari SI

AgentDrift: A Step-Labeled Benchmark of Injection-Hijacked LLM Agent Trajectories

LLM agents complete tasks by issuing sequences of tool calls, and every observation they read is a channel through which an indirect prompt injection can enter. A successful injection has a characteristic shape when the trajectory is read in order: a benign prefix gives way to actions that serve the attacker rather than the user. Existing benchmarks measure whether such attacks succeed against live agents, and existing guard models judge a trace as a whole; no public corpus…

Read at arXiv (research papers)

arXiv (research papers) Aashiq Muhamed SI

MOLE: Detecting Insider Threats in AI Agents

Model misalignment, prompt injection, or operator misuse could lead AI agents operating frontier-lab accounts to exfiltrate model weights, poison training data, or weaken release gates. Existing benchmarks do not test whether defenders can detect this activity among routine work under a limited review budget. We introduce MOLE, an open benchmark of 150 AI-operated accounts sharing 9 stateful services over 30 workdays, with 12 threats and 8 corpora from four models totaling…

Read at arXiv (research papers)

Hacker News taubek SI

AI agents aren't safe from prompt injection, and spreadsheets prove it

Article URL: https://shiftmag.dev/ai-agents-arent-safe-from-prompt-injection-and-spreadsheets-prove-it-11609/ Comments URL: https://news.ycombinator.com/item?id=49587979 Points: 3 # Comments: 2

Read at Hacker News

Older stories