Results for “Agent”

arXiv (research papers) Xianglin Yang SI

From Approval to Abuse: Privilege Escalation Attacks on Harness-Protected Coding Agents

Coding agents use execution harnesses to constrain the consequences of prompt injection by enforcing permission boundaries. These systems deterministically intercept any out-of-sandbox action and prompt the user for approval. To reduce approval fatigue, users can approve allow-rules proposed by the agent, after which any future action matching the rule is allowed without a prompt. In this work, we show that this mechanism introduces a new risk: untrusted context can…

Read at arXiv (research papers)

Journal articles (OpenAlex) Kyle Harris · Public Relations Inquiry Geopolitics

Public relations AI practices: Neoliberalism, field theory, and professional dispositions within an AI-mediated habitus

Public relations practitioners increasingly turn to artificial intelligence (AI) bots and agentic systems to enhance decision-making, build campaigns, and support stakeholder research. The growth in AI use, however, reflects pressures of neoliberalism influencing everyday practice. Neoliberalism is an ideology that embodies market ideals and a sense of technological inevitability. Within neoliberalism, practitioners may believe they are exercising a degree of autonomy in…

Read at Journal articles (OpenAlex)

arXiv (research papers) Fardin Ahsan Sakib SI

The Model Knows Better: Defending Medical Multi-Agent Systems Against Prompt Injection

Medical multi-agent systems, in which LLM agents discuss a case, are increasingly studied for clinical decision support. Prompt injection, text that an attacker adds to a model's input, can manipulate LLMs into wrong decisions. How it affects these systems, and how to protect them, remains unclear. Across two systems, two model families and three benchmarks, a single injected passage succeeds in up to 61% of cases, and collaboration does not consistently protect against it…

Read at arXiv (research papers)

arXiv (research papers) Thuy Dung Nguyen SI

INFUSE: A Unified Surface Benchmark for Indirect Prompt Injection in Agentic Systems

LLM-based agents increasingly execute long-horizon, end-to-end workflows involving shell commands, code modification, dependency installation, and interaction with remote services. This growing autonomy expands their exposure to indirect prompt injection (IPI), where adversarial instructions embedded in external content are interpreted as legitimate task directives. Existing evaluations, however, often focus on isolated malicious strings, single-step tool calls, or narrowly…

Read at arXiv (research papers)

arXiv (research papers) Chenxu Wang SI

AgentSafeEvo: Adversarial Co-Evolution for LLM Agent Safety

Large language model (LLM) agents accomplish complex tasks through multi-turn interactions with external tools. However, malicious content embedded in user queries, tool descriptions, or environment observations can manipulate their decisions and induce unsafe actions. Adversarial co-evolution has emerged as an effective approach to strengthening LLM safety, but existing work primarily focuses on language-level safety or indirect prompt injection in web tasks. In this paper…

Read at arXiv (research papers)

arXiv (research papers) Min-Young Yu SI

Policy-Verified Self-Training: Learning Agent Compliance from Verifiable Policy Rewards

Runtime policy verifiers can prevent agents from executing disallowed actions, but their decisions are typically used only for enforcement and discarded afterward. We ask whether these decisions can instead become verifiable policy rewards for training the agents they guard. We introduce Policy-Verified Self-Training (PVST), a post-training paradigm in which an agent generates its own interaction trajectories and an executable policy verifier, here a deterministic gate…

Read at arXiv (research papers)

arXiv (research papers) Kshitij Mishra SI

TRACER: Trace-Regularized Training Against Indirect Prompt Injection

Indirect prompt injection uses untrusted retrieved content, tool outputs, or agent messages to override an LLM's intended behavior. We show that defense robustness is channel-conditioned: the same attack succeeds 18% of the time as a follow-up message but 53% when placed in retrieved context. We propose TRACER (Trace-Regularized Adversarial Channel Exposure with Reward-shaping), a channel-aware adversarial training method that samples presentation wrappers and combines an…

Read at arXiv (research papers)

arXiv (research papers) Dominik Seip SI

Towards Safe Autonomy for Computer Use

The safety of LLM-based agents currently rests on alignment training and safety classifiers, neither of which can rule out that a sophisticated indirect prompt injection (IPI) goes through and causes significant harm. Recent work therefore builds agents with information-flow guarantees that rely on structured tool interfaces to interact with the environment. Computer Use Agents (CUAs) act visually on the computer screen, which does not provide a boundary between trusted…

Read at arXiv (research papers)

arXiv (research papers) Kristoffer Hempel SI

Kairos: Transactional Security Enforcement at the System Boundary for LLM Agents

Large language model (LLM) agents are becoming an integral part of the software development lifecycle, and the capabilities granted to them allow an agent to act with the user's authority on any resource it can access. The reach that gives an agent its utility is equally available to an attacker who can steer it. Anything an agent ingests can carry an injected instruction, and since the model does not reliably distinguish instructions from data, indirect prompt injection…

Read at arXiv (research papers)

arXiv (research papers) Kaan Kale SI

Prompt-Injection Probes Learn the Conversation, Not Just the Injection

Linear probes on a model's hidden states are used as cheap monitors for indirect prompt injection: they screen the tool outputs an agent reads and send suspicious ones to a slower second check, so every false alarm has a cost. But a probe learns whatever separates its training examples. The standard evaluation of injection probes can be passed without the model: conversation position and text lengths alone reach 0.91-0.98 AUROC. When each attack is compared with the same…

Read at arXiv (research papers)

arXiv (research papers) Yuetian Chen SI

BrowseSafe++: Benchmarking Prompt Injection Detection Across Web Contexts, Languages, and Browser Views

Browser agents need to make decisions and execute actions on behalf of users while consuming untrusted webpages, creating opportunities for malicious instructions embedded in webpages to redirect legitimate tasks. We introduce BrowseSafe++, which extends BrowseSafe from 14,719 pages to 38,675 pairs of pages. Most of its prompt injections were observed in the wild on production websites. We keep each injection unchanged and rewrite the sensitive parts of the page around it to…

Read at arXiv (research papers)

arXiv (research papers) Arya Fayyazi SI

The Agent Knew: Internal States Forecast Unauthorized Actions and Stop Them, and the Lies That Follow

Language-model agents sometimes act beyond their authorization and then misreport what they did. On CAPE-Bench, matched agent tasks in which one fact decides whether an action is authorized, frontier open-weight agents do both, and one behavior marks the lie: retroactive permission laundering, acting and then asking permission as if the action were pending, precedes a false report in 70% of 1081 fresh informed violations, against 8% otherwise (preregistered odds ratio 7.0)…

Read at arXiv (research papers)

arXiv (research papers) Rishabh Sinha SI

Exposure Is Not Commitment: Double-Stratified Evaluation of Latent Monitors for Tool-Using Agents

Tool-using language agents are vulnerable to indirect prompt injection, and a growing literature probes model internals to catch unsafe actions before they execute, with sharply conflicting results. We argue much of the conflict is an evaluation artifact, and propose a double-stratification protocol: separate injection exposure from behavioral commitment, in both the evaluation and the training population. On the primary Qwen AgentDojo grid, monitors trained on the naive…

Read at arXiv (research papers)

arXiv (research papers) Quanyan Zhu SI

Computational Reflective Equilibrium for Agentic AI: A Rawlsian Alignment Scaffold for Governance

The rapid deployment of agentic AI has introduced complex challenges for the ethics and governance of AI systems. Agents built on large language models act through tools on behalf of people, and each deployment case brings its own instructions, policies, evidence, and stakeholders, so alignment must be formed case by case. To this end, this work proposes agentic reflective equilibrium (ARE), a computational reflective equilibrium framework that serves as an ethical alignment…

Read at arXiv (research papers)

Journal articles (OpenAlex) Livnat Konopny-Decleve · Journal of Borderlands Studies Geopolitics

Solidarity in the Borderlands of Sovereignty – A Potential (Un)Ontopology

This paper introduces two distinct expressions of solidarity of Jewish radical left activists living in Palestinian localities in the West Bank with the Palestinian struggle. The first entails evading Israeli state agents’ surveillance and being politically engaged away from the state's radar. The other entails using privileges as Jews to make power visible, to expose privileges to make them seen and heard. By exploring these two types of solidarity, I claim that the…

Read at Journal articles (OpenAlex)

arXiv (research papers) Michael Cardei SI

Only What You've Signed up For: Secure Workflows with Constrained Diffusion Agents

As agentic AI systems take on increasingly complex and consequential tasks, securing them against prompt injection becomes critical. Existing defenses often rely on detecting or reasoning about injected malicious instructions and can therefore remain vulnerable to adaptive attacks that optimize the injection itself. Diffusion language models (dLLMs), an emerging framework for language generation, offer a new opportunity for security enforcement: by iteratively refining their…

Read at arXiv (research papers)

arXiv (research papers) Josef A. Habdank SI

Can agents whose sole duty is to keep humans alive be hijacked to kill? Delayed multi-turn derived-action trap attacks in simulated high-stakes environments

We ask whether a tool-using agent whose sole duty is to keep humans alive can be induced, by delayed prompt injection, to kill. The keep-alive duty is in force from the first watch, with the attack arriving later, appended to otherwise truthful telemetry the operator is required to read. Experiments are based on a NASA-derived BioSim lunar habitat simulation, where the AI operator controls the life-support systems. The attacker cannot falsify sensors or operate equipment: it…

Read at arXiv (research papers)

Hacker News manveerc SI

Prompt Injection Detection and Defense Tools for Enterprise AI Agents

Article URL: https://www.arcade.dev/blog/prompt-injection-detection-tools/ Comments URL: https://news.ycombinator.com/item?id=50032282 Points: 1 # Comments: 0

Read at Hacker News

arXiv (research papers) Daniel Commey SI

Names Are Not Objects: Object-Bound Confinement for Compromised AI Coding Agents

Coding agents read repositories and web pages that anyone can write, and then act with their user's authority. Prompt-injection defenses make it less likely that the model is steered but do not prevent it, so the system around the model must contain an agent that is already compromised. Many exploits against deployed coding agents share one cause: a check approves a name, such as a path, an argument vector, or a directory, and a later step maps that name to a different…

Read at arXiv (research papers)

Journal articles (OpenAlex) Radhika Govinda, Fiona Mackay · European Journal of Women s Studies Geopolitics

Voices from the early days: Recovering histories of feminist academics as critical actors, weaving a collective biography of gender and sexuality studies at a Scottish University

This article engages with the voices of six pioneering women academics who were instrumental in introducing ‘gender and sexuality studies’ at a Scottish research-intensive higher education institution. The article’s aim is to weave a collective biography of both the origin and development of gender and sexuality studies at the institution and also the emergence of feminists as change agents or ‘critical actors’ on the inside. It begins by exploring these academics’ early…

Read at Journal articles (OpenAlex)

arXiv (research papers) Seyedarmin Azizi SI

One Word Opens the Gate: The Option-Channel Attack on Typed Decision Models as Agent Guardrails

A typed decision model reads a piece of text and returns a probability over caller-defined options, each with a short written definition, generating no text. Recent work places these models in agent systems as guardrails: the component that reads a proposed tool call or incoming message and decides whether to allow it. We evaluate seven open-weight models in that role and report the two error directions separately: a fail-open error allows a prohibited action and is a…

Read at arXiv (research papers)

OWASP GenAI Security Project OWASPGenAIProject Editor Security

GenAI and Agentic AI Exploit Roundup Q3 2026

Coverage period: July 1, 2026 through September 30, 2026 Overview This roundup consolidates selected major AI-related security incidents and exploit disclosures reported during the coverage period. It aligns each entry to the OWASP Top 10 for LLM Applications 2026 and the OWASP Top 10 for Agentic Applications 2026, with published CVE references where available. It

Read at OWASP GenAI Security Project

arXiv (research papers) Cathleen Qiao SI

An Exploration of Attacks on AI Agent Systems

As autonomous AI agents become more integrated into sensitive domains, it is crucial that single and multi-agent AI systems are secured. This paper investigates how prompt injection attacks can exploit both direct user inputs and inter?agent communication to leak sensitive information and trigger cross-agent infections. While our tests mostly highlight key vulnerabilities in single-agent systems, we also extend these attacks to multi-agent frameworks, investigating how…

Read at arXiv (research papers)

arXiv (research papers) Zitong Yao SI

AgentTracer: Tracing Indirect Prompt Injection Attack through Fine-Grained Intention-Execution Alignment

Large language model (LLM) agents interact with external resources to complete complex user tasks, exposing them to indirect prompt injection (IPI), where malicious instructions redirect agents toward attacker-intended tasks. Since IPI is difficult to defend against in real-world environments, post-incident tracing is essential for locating the injection source and reconstructing the attack chain. However, existing tracing methods primarily capture explicit control-flow and…

Read at arXiv (research papers)

arXiv (research papers) Yupu Wang SI

Package Hallucination Attacks on Coding Agents through Prompt Injection in Rule Files

Modern agentic coding frameworks increasingly rely on community-shared rule files (e.g., AGENTS.md or .cursorrules) to guide autonomous code generation, yet the security risks of this pipeline remain underexplored. To bridge this gap, we introduce the package hallucination attack, where an attacker injects malicious prompts into benign rule files to induce coding agents to replace legitimate dependencies with attacker-controlled packages. To obtain effective malicious…

Read at arXiv (research papers)

arXiv (research papers) Pengfei He SI

ASPIRE: Agentic Safety & Prompt Injection Red-teaming Engine

LLM agents retrieve untrusted content and act through tools, creating indirect prompt-injection risks that can cause unauthorized actions or persistent state changes. Existing automated red-teaming largely optimizes payloads for pre-specified scenarios, leaving latent vulnerabilities across the agent's behavior space unexplored. We present ASPIRE, an Agentic Safety & Prompt Injection Red-teaming Engine for open-ended, behavior-level vulnerability discovery. ASPIRE maintains…

Read at arXiv (research papers)

arXiv (research papers) Sarim Hashmi SI

AdvSim2Real : Training Web Agents Against Adaptive Prompt Injection in a Web World Model

Web agents complete user requests by reading and acting on pages that third parties write, so an instruction planted on a page can redirect the agent away from the user's goal. The agent cannot simply ignore the page, because the page also holds the values and controls the task requires. Current defenses fine-tune the agent on injections fixed before training, and attackers that adapt to the trained model bypass them. Adversarial training lets the attacker adapt but keeps…

Read at arXiv (research papers)

arXiv (research papers) Haneen Najjar SI

Surviving the Router: Optimizing Skill Injections for Retrieval and Execution

AI agents increasingly rely on modular third-party "skills" that are dynamically selected by skill routers to execute complex tasks. While recent studies highlight the threat of prompt injections embedded in these skills, existing evaluations often assume settings where the malicious skill is already selected for execution. We show that this assumption can substantially overestimate attack success. In realistic multi-skill environments, injected skills must first compete for…

Read at arXiv (research papers)

arXiv (research papers) Mohamed Dhouib SI

RAISED: Self-Distillation for Robustness to Prompt Injection in LLM Agents

Tool-using language-model agents are vulnerable to indirect prompt injection because they must act on untrusted external content. Existing training-time defenses can reduce attack success rates, but often at the cost of general capabilities. We show that training-based defenses induce substantial drift in the model's output distribution, altering its behavior even in benign settings and providing a potential mechanism for utility degradation. We further identify a failure…

Read at arXiv (research papers)

Older stories