Results for “Agent”

arXiv (research papers) Po-Han Cheng SI

CodeSentinel: A Three-Layer Defense Against Indirect Prompt Injection in Code Contexts

Code large language models increasingly retrieve external code context from repositories, documentation, issue threads, and coding-agent environments, creating an indirect prompt-injection surface where attackers hide instructions in comments, strings, identifiers, or decoy code. We propose CodeSentinel, a three-layer inference-time sanitizer. It uses Tree-sitter to extract high-risk model-facing CST nodes, then combines syntax-guided pre-filtering, CST-guided Dynamic…

Read at arXiv (research papers)

arXiv (research papers) Laxmipriya Ganesh Iyer SI

The Gate Is Only as Honest as Its Contracts: ContractGuard for the Contract Layer of Risk-Aware Causal Gating

Risk-Aware Causal Gating (RACG) defends tool-augmented LLM agents against indirect prompt injection by removing dangerous tools from the agent's visible action space, so that even a fully injection-compliant agent cannot call a tool it cannot see. We make three points. First, this structural guarantee does not eliminate the trust assumption behind safe tool use; it relocates it into the integrity of the tool contracts -- declared preconditions, effects, risk, and…

Read at arXiv (research papers)

arXiv (research papers) Yuchuan Tian SI

SafeClawBench: Separating Semantic, Audit-Evidence, and Sandbox Harm in Tool-Using LLM Agents

Tool-using language-model agents introduce security failures that go beyond unsafe text: they can disclose protected objects, write persistent memory, send messages, modify databases, or trigger harmful code and tool effects. Existing evaluations often collapse these stages into a single attack success rate, making it difficult to tell whether a model merely agreed with an attacker or actually produced observable harm. We introduce SafeClawBench, a staged benchmark for…

Read at arXiv (research papers)

arXiv (research papers) Xuelong Dai SI

MIRAGE: Stealthy Visual Prompt Injection for Vulnerability Detection in Web Agents

Multimodal Large Language Model (MLLM)-based web agents provide practical, high-precision solutions for visual browser automation; however, they inherently expand the attack surface, introducing novel vision-based vulnerabilities. Existing adversarial evaluations targeting these agents frequently rely on permissive threat models and visually conspicuous artifacts. In this paper, we investigate a constrained vulnerability detection setting: a trusted web platform where the…

Read at arXiv (research papers)

arXiv (research papers) Aaditya Pai SI

PARSE: Provenance-Aware Retrieval Sanitization for Professional Domain LLM Agents

Prompt injection defenses evaluated on synthetic benchmarks do not generalize to real enterprise documents, which are longer, denser, and interleave legitimate authority language with factual content. We demonstrate this gap with a benchmark of 122 tasks across five professional domains (financial, legal, medical, scientific, DevOps) built on real retrieved documents -- actual SEC filings, Federal Register rules, PubMed abstracts, arXiv papers, and GitHub postmortems…

Read at arXiv (research papers)

arXiv (research papers) Hankyul Baek SI

An Evaluation of Data Leakage Risks in Tool-Using LLM Agents in Realistic Scenarios

AI agents are increasingly being adopted in enterprise and personal settings with access to emails, databases, documents, and other tools where they can read, update, and disseminate sensitive information. Much of prior research on data leakage risks in agents has focused on adversarial data exfiltration through prompt injections and jailbreaks. However, sensitive information may also be exposed during non-adversarial use, creating leakage risks even when users issue benign…

Read at arXiv (research papers)

arXiv (research papers) Ismail Hossain SI

SkillVetBench: LLM-as-Judge for Multi-Dimensional Security Risk Evaluation in Open-Source LLM Agent Skills

Open-source LLM agent ecosystems are growing rapidly, yet the security of community-contributed skills - modular tool definitions that extend agent capabilities - remains largely unvetted. The gap we fill: existing scanners operate at the code layer and are structurally blind to instruction-layer and multi-agent risk - natural-language directives that hijack an agent, exfiltrate data through encoded side channels, or chain harm across pipelines - so what is needed is a…

Read at arXiv (research papers)

arXiv (research papers) Zixin Rao SI

FragFuse: Bypassing Access Control of Large Language Model Agents via Memory-Based Query Fragmentation and Fusion

Large language model (LLM) agents increasingly rely on long-term memory to support complex task execution, user personalization, and domain adaptation. Meanwhile, emerging access-control mechanisms for LLM agents are being explored to block policy-violating requests and prevent misuse. We reveal a novel attack surface arising from agent memory operations: prohibited content that would trigger access control can be fragmented across interactions, stored in long-term memory in…

Read at arXiv (research papers)

arXiv (research papers) Lipeng He SI

Defending against Adaptive Prompt Injection Attacks via Reasoning-enabled Task Alignment

Indirect prompt injection attacks hijack LLM-based agents by embedding malicious instructions in third-party data that the agent retrieves during task execution. Existing defenses report near-zero attack success rate on static benchmarks, yet recent adaptive evaluations show that these results collapse once the attacker is allowed to optimize against the deployed defense. In this work, we trace this collapse to two failure modes. First, existing defense methods are confined…

Read at arXiv (research papers)

arXiv (research papers) Xinhang Ma SI

AutoDojo: A Generative Benchmark for Evaluating Prompt Injection Defenses in LLM Agents

Indirect prompt injection (IPI) is a major security threat to LLM-powered agents. Thus, a growing body of work has proposed a variety of defensive approaches against IPI which can be grouped into three broad categories: prompt-level, filter-based, and system-level designs. However, commonly used benchmarks for evaluating defense, such as AgentDojo, are \emph{inherently static}, generating a fixed distribution of IPI attacks. Consequently, a defense can score well on them…

Read at arXiv (research papers)

arXiv (research papers) Saeid Jamshidi SI

Security Engineering of OpenClaw: Analyzing Attack Surface Expansion and Trust-Boundary Violations

Agentic large language model (LLM) systems can now execute actions, not only produce text. When model outputs trigger privileged operations such as shell commands, browser automation, or external tool calls, the security problem shifts from alignment alone to system configuration and structural design. We analyze OpenClaw, a self-hosted multi-agent system in which LLM outputs can execute commands and interact with tools and services. We measure compromise probability…

Read at arXiv (research papers)

arXiv (research papers) Yuguang Zhou SI

From Shield to Target: Denial-of-Service Attacks on LLM-Based Agent Guardrails

LLM-based guardrails have emerged as a highly effective defense against prompt injection and jailbreak attacks in autonomous agents. However, we reveal that the very reasoning and task-following capabilities enabling this protection introduce a novel vulnerability: attackers can inject crafted data to trap the guardrail in extended reasoning loops, effectuating a systematic denial-of-service (DoS) attack. To systematically expose this threat, we design a beam-search…

Read at arXiv (research papers)

arXiv (research papers) Youngduk Kim SI

SkillMutator: Benchmarking and Defending Language-and-Code Cross-modal Attacks on LLM Agent Skills

Large language model (LLM) agents increasingly extend their capabilities at runtime by loading Agent Skills, which pair natural-language specifications (SKILL.md) with executable scripts and resources. Because a skill's behavior relies on both natural-language instructions and executable code, assessing its safety requires cross-modal reasoning, creating a new language-and-code attack surface. Attackers can present a benign workflow in SKILL.md while embedding implicit…

Read at arXiv (research papers)

Simon Willison SI

Claude Fable is relentlessly proactive

After two days of experience with Claude Fable 5 I think the best way to describe it is relentlessly proactive. It knows a whole lot of tricks and it will deploy pretty much any of them to get to its goal. I'll illustrate this with an example. I was hacking on Datasette Agent today when I noticed a glitch: a horizontal scrollbar that shouldn't be there in the jump menu chat prompt. I snapped this screenshot: Then I started a fresh claude session in my datasette-agent…

Read at Simon Willison

arXiv (research papers) Zihao Wang SI

Who Pays the Price? Stakeholder-Centric Prompt Injection Benchmarking for Real-world Web Agents

LLM-based web agents are increasingly deployed in real-world settings such as e-commerce, where they interact extensively with untrusted web content while executing actions that carry direct financial consequences. This makes them vulnerable to prompt-injection attacks, in which seemingly benign web content conceals adversarial instructions that manipulate the agent's behavior. Existing security benchmarks adopt an \textit{attack-centric} perspective, focusing on the…

Read at arXiv (research papers)

arXiv (research papers) Haowei Qian SI

Nous: An Attempt to Extract and Inject the Cognition Behind Prediction-Market Behavior

As LLM agents proliferate in prediction markets and collective decision-making, they risk a cognitive monoculture: agents built on shared foundation models produce correlated forecasts, and recent measurement finds frontier-model errors correlated at r ~ 0.77. We ask whether human cognitive diversity can be recovered from behavior and transferred to LLM agents. Nous extracts a structured eight-dimension behavioral profile from real Polymarket trading activity and injects it…

Read at arXiv (research papers)

arXiv (research papers) Pengfei He SI

PI-Hunter: Automated Red-Teaming for Exposing and Localizing Prompt Injections

Large Language Models (LLMs) are rapidly evolving into agentic systems that interact with external tools and environments, introducing new security risks such as indirect prompt injection attacks through untrusted external sources. Existing defenses mainly focus on blocking malicious content at inference time, and current red-teaming methods primarily optimize attack success. As a result, developers have limited visibility into how latent prompt injections emerge and…

Read at arXiv (research papers)

arXiv (research papers) Timothy McAllister SI

Smarter Saboteurs, Better Fixers: Scaling & Security in Linear Multi-Agent Workflows

As LLM-based multi-agent systems (MAS) are deployed in the wild, the resilience of their collaboration structures against adversarial compromise becomes a critical safety concern. Attackers may leverage prompt-injection or jailbreaking to sabotage individual agents within MAS workflows, but the interaction between model scaling and system-level resilience remains poorly understood. This paper investigates how model scale affects the security of linear multi-agent workflows…

Read at arXiv (research papers)

arXiv (research papers) Yuchen Ling SI

Toward Secure LLM Agents: Threat Surfaces, Attacks, Defenses, and Evaluation

Large language model (LLM) agents are rapidly moving from conversational interfaces to software components that plan, invoke tools, maintain memory, and act on external environments. This transition changes the nature of security risk. In agentic settings, failures are no longer limited to unsafe text generation. Untrusted content may redirect control flow, misuse tool privileges, corrupt persistent state, leak sensitive information, or trigger harmful external actions. At…

Read at arXiv (research papers)

arXiv (research papers) David Hofer SI

Assessing Automated Prompt Injection Attacks in Agentic Environments

Indirect prompt injection poses a critical threat to LLM agents that interact with untrusted external data, yet automated attack methods--proven effective for jailbreaking--remain underexplored in realistic agentic settings. We present a comprehensive empirical evaluation of automated prompt injection attacks against LLM agents, adapting both white-box (GCG) and black-box (TAP) methods to the agentic setting within the AgentDojo framework. We evaluate across 80 task pairs…

Read at arXiv (research papers)

arXiv (research papers) Saeid Jamshidi SI

Game-Theoretic Multi-Agent Control for Robust Contextual Reasoning in LLMs

Large Language Models (LLMs) in multi-turn interactions maintain evolving context rather than generating isolated responses, making them vulnerable to prompt-injection and context-poisoning attacks in which locally plausible adversarial fragments gradually distort reasoning trajectories. Existing defenses mainly filter individual outputs and often ignore context evolution across turns, leaving long-horizon reasoning exposed. Although the Model Context Protocol (MCP)…

Read at arXiv (research papers)

arXiv (research papers) Gilad Gressel SI

PRISM: Recovering Instruction Sets from Language Model Activations

As LLMs are deployed as agents, reliable monitoring requires knowing not only what they output, but which instructions are steering their behavior. This is difficult when models infer unintended subgoals, follow contextual cues, or are influenced by prompt injections and hidden objectives. While activation-to-language methods suggest that hidden states can reveal natural-language information, existing approaches are not designed to recover the full set of simultaneous…

Read at arXiv (research papers)

arXiv (research papers) Jianwei Tai SI

Brain-Prompt Injection: A Route-Safety Audit for BCI-LLM Agents

BCI-to-agent pipelines turn decoded neural activity into an authorization channel for tool-use agents, exposing a new attack surface we call \emph{brain-prompt injection}: signal-side perturbations, context-only injections, and adaptive dual-decoder attacks can all change the routed action while EEG-side or text-side monitors remain blind. Route safety in this stack depends on what the audit log can observe, not on decoder accuracy or agreement alone. We define a…

Read at arXiv (research papers)

arXiv (research papers) Jafar Isbarov SI

GitInject: Real-World Prompt Injection Attacks in AI-Powered CI/CD Pipelines

AI agents are increasingly embedded in continuous integration and continuous delivery (CI/CD) workflows to autonomously review pull requests, triage issues, and maintain codebases. We study fully automated configurations in which external contributions invoke agents without prior human approval. Because these agents ingest untrusted content while operating with repository credentials and tools, prompt injection can have software supply chain consequences. We present…

Read at arXiv (research papers)

arXiv (research papers) Harshil Patel SI

VATS: Exploiting Implicit Authority in Error-Path Injection via Systematic Mutation

As the Model Context Protocol (MCP) standardizes tool-calling for autonomous agents, it introduces a critical, unexamined attack surface: the error-handling loop. We hypothesize that tool error messages possess implicit authority, triggering corrective reasoning modes that bypass standard safety heuristics. We introduce VATS (Vulnerability Analysis of Tool Streams), a mutation-driven framework that systematically evolves adversarial payloads across seven structural and…

Read at arXiv (research papers)

arXiv (research papers) Wenbo Guo SI

MalSkillBench: A Runtime-Verified Benchmark of Malicious Agent Skills

AI coding agents such as Claude Code and Gemini CLI increasingly extend themselves with third-party skills: markdown packages bundling natural-language instructions, executable scripts, and tool permissions. Because a skill is at once code and agent-facing instruction, it introduces a supply chain dependency whose risk is neither pure code nor pure prompt. Detection tools have never been measured against verified ground truth spanning this hybrid space, leaving their…

Read at arXiv (research papers)

arXiv (research papers) Shuyu Guo SI

RubricReviewer: From Direct Critique to Objective and Comprehensive Rubric-Driven Peer Review

Peer review at major venues is under unprecedented submission pressure, motivating the use of large language models (LLMs) as review assistants. Existing LLM-based reviewers, however, face two structural limitations. First, they map manuscripts directly to reviews, leaving the underlying rubric implicit and entangling its derivation with the judgement. Second, the prevailing paradigms each capture only half of a good review: training-free agents gather broad evidence but…

Read at arXiv (research papers)

arXiv (research papers) Quanyan Zhu SI

Insurance of Agentic AI

Agentic artificial intelligence (AI) systems are transforming the risk landscape by extending beyond information generation to autonomous planning, tool invocation, decision execution, and persistent modification of digital and physical environments. These capabilities introduce novel exposures that do not fit neatly within traditional insurance categories such as cyber, professional liability, product liability, or directors and officers coverage. This paper examines the…

Read at arXiv (research papers)

arXiv (research papers) Yuanbo Xie SI

What If Prompt Injection Never Left? Rethinking Agent Security through Cross-Session Stored Prompt Injection

Modern agentic systems fundamentally reshape the security boundary of LLMs by introducing persistent system state including memories, filesystems, tools, and other long-lived contextual artifacts that survives across sessions. As external information crosses this boundary and becomes part of persistent agent state, malicious instructions are no longer confined to a single interaction, but can silently persist and influence future executions long after the original attacker…

Read at arXiv (research papers)

arXiv (research papers) Nicholas Saban SI

Domain-Conditioned Safety in Frontier Computer-Using Agents: A 793-Episode Browser Benchmark, a Coding-Domain Cross-Reference, and a Reproducibility Audit of Recent Red-Teaming

Recent computer-using-agent (CUA) red-teaming papers report prompt-injection attack success rates (ASR) of 42-98%, but these headline numbers cluster on retired models and on the most-vulnerable model in each paper's panel. We ask whether those techniques, reproduced as hand-crafted templates, still work against current frontier CUAs. We release CUA-HandCrafted, a public benchmark of 793 episodes spanning 24 multi-step web tasks, 56 attack templates, 8 attack families, and 4…

Read at arXiv (research papers)

Older stories