Results for “Agent”

arXiv (research papers) Xabier Muruaga SI

Bounded Agents: Delegation Security for Multi-Agent AI Systems

LLM-based agents can act on behalf of a user to access cloud services, call tools, or invoke agents. At session start, the agent's permissions are set but remain static, and each request is evaluated independently, without considering prior actions. Within its permissions, an agent may act contrary to the delegated task, combine individually permitted actions into a prohibited outcome, or delegate authority to a sub-agent without limiting it. A prompt injection poses a risk…

Read at arXiv (research papers)

arXiv (research papers) Md Fazley Rafy SI

TwinGridShield: Consequence-Aware Runtime Authorization for LLM Grid-Agent Actions

Large language model (LLM)-assisted energy-management tools can translate natural-language context into structured grid commands, but syntactic validity does not imply physical admissibility. This paper presents TwinGridShield, a model-independent runtime authorization layer that evaluates each proposed action in a deterministic network twin before release. The prototype checks connectivity, branch-flow, generator, and load-shedding invariants and records each decision in a…

Read at arXiv (research papers)

arXiv (research papers) Anees Ahmed Mahaboob Ali SI

BioFirewall: A genome-writing-native governance layer for design-stage biosecurity screening of agentic AI

AI design tools now plan genome-scale edits, and agentic systems execute those plans with progressively less human oversight. Biosecurity controls are limited to two points: refusal guardrails at the foundation model and sequence-identity screening at the synthesiser. The design stage between them, where the plan is specified, is governed by recommendations, not by any deployed system. We present BioFirewall, a rule-governed middleware that intercepts a genome-writing plan…

Read at arXiv (research papers)

arXiv (research papers) Alexandre G. R. Day SI

Workspace Topology as an Attack Vector in Agentic Coding Assistants

Agentic coding assistants are finding widespread use, not just in new code development but in quickly ingesting and leveraging third-party code. This opens up a risk of malicious code being ingested as these coding tools operate with broad filesystem access inside developer workspaces. In this paper, we extensively study the impact of different dimensions of a novel attack surface we term workspace topology -- defined via directory depth, codebase modularity, in-file…

Read at arXiv (research papers)

arXiv (research papers) Van Tran SI

Rethinking Agent Security as a Networking Problem

AI agents are rapidly becoming more capable and widely deployed, promising substantial gains in productivity and enabling new classes of applications. However, their growing autonomy also introduces significant privacy and security risks. Existing defenses are predominantly agent-centric, relying on the agent itself to detect threats and enforce privacy and security policies. This approach is fundamentally limited because it entrusts policy enforcement to AI agents whose…

Read at arXiv (research papers)

arXiv (research papers) Yutao Mou SI

ToolHazard: Scaling Adversarial Environments for Security Evaluation and Alignment of LLM-based Agents

Large language model (LLM) agents integrated with external tools are vulnerable to indirect prompt injections embedded in environmental states. However, existing studies largely rely on manually implemented or reused environments, stochastic LLM-based tool simulation, and predefined injection locations, limiting scalable security research across broader domains. To bridge this gap, we propose **ToolHazard**, a scalable adversarial environment synthesis framework that reduces…

Read at arXiv (research papers)

Journal of Democracy Geopolitics

How AI Agents Are Empowering Human-Rights Defenders

With news of a rogue AI jumping developers’ guardrails, authoritarian governments harnessing its power to ramp up repression, and companies laying off scores of workers, it might be tempting to dismiss the good this technology can do. That would be a mistake, according to Alex Gladstein. Source

Read at Journal of Democracy

arXiv (research papers) Md Jafrin Hossain SI

On Understanding, Identifying, and Mitigating Vulnerabilities in Agentic Large Language Models

Large Language Models (LLMs) have undergone a shift from stateless conversational interfaces to autonomous agents capable of multi-step planning, tool invocation, code execution, and maintaining persistent memory. When these agents operate with real-world privileges---calling APIs, modifying files, and querying databases---a compromised reasoning step can trigger unauthorized data access, irreversible state changes, or cascading failures, yet the security research community…

Read at arXiv (research papers)

arXiv (research papers) Spiros Tsigkopoulos SI

From Prompt Injection to Web Exploitation: Revisiting Classic Vulnerabilities in LLM-Integrated Applications

Large Language Models are increasingly integrated into web applications through chatbots, tool-calling pipelines, and agentic workflows. In these systems, user input may influence not only generated text, but also backend actions such as database queries, HTTP requests, file operations, template rendering, or API calls. This paper introduces LLM-mediated web attacks, a class of attacks in which attacker-controlled input is transformed by an LLM-integrated application and…

Read at arXiv (research papers)

arXiv (research papers) Rahul Deivasigamani SI

Not an A11y: How Android Accessibility Exposes Mobile AI Agents to Indirect Prompt Injection

The rise of autonomous AI agents represents a major paradigm shift in how users interact with mobile devices. Frameworks such as MobileRun and Mobile-Use can autonomously navigate Android applications and execute complex multi-step tasks. To interpret user interfaces, these frameworks rely primarily on Android accessibility (A11y) trees and secondarily on visual screenshots. In this paper, we demonstrate that this architectural dependence on unsanitized accessibility…

Read at arXiv (research papers)

arXiv (research papers) Sihan Hou SI

Toward Metacognitive One-Shot Indirect Prompt Injection: Strategy Abstraction Via Outcome-Conditioned Reflection

Tool-using large language model (LLM) agents are vulnerable to indirect prompt injection (IPI), in which malicious instructions embedded in external observations manipulate subsequent agent decisions and actions. Most existing adaptive attacks rely on repeatedly querying and refining against the target agent, whereas realistic attackers may have only a single opportunity to interact with an unknown target agent. We propose SAVOR (Strategy Abstraction Via Outcome-Conditioned…

Read at arXiv (research papers)

arXiv (research papers) Xinze Chen SI

SkillsMetric: Mapping the Detection Boundary of Static Analysis for Malicious Agent Skills

Agent Skills---structured packages of instructions and scripts that augment LLM-based agents---are rapidly proliferating, yet their security properties remain under-explored. We present \textsc{SkillsMetric}, a five-stage static analysis framework that scores skill packages along pattern density, statistical anomaly, dataflow taint, import anomaly, and capability mismatch dimensions. We construct an adversarial evaluation dataset of 2{,}266 skills spanning 16~attack types…

Read at arXiv (research papers)

arXiv (research papers) Kaysarul Anas Apurba SI

Defending Retrieval-Augmented Intrusion Detection Against Knowledge Poisoning and Prompt Injection

Retrieval-Augmented Generation (RAG) enables large language models to classify network flows and generate human-readable incident reports by retrieving semantically similar historical traffic from a vector knowledge base. However, the retrieval layer introduces vulnerabilities to knowledge poisoning and prompt-injection attacks. We present RAG-IDS, a three-tier multi-agent intrusion detection framework with a retrieval-boundary defense combining soft trust scoring…

Read at arXiv (research papers)

arXiv (research papers) Jeremy McHugh SI

The Anatomy of a Prompt Injection: A Component Model for Structured Analysis

Four years after prompt injection was first identified in 2022, attacks are still predominantly documented as verbatim strings rather than structured exploits, despite advancing agent capabilities and threat actors embedding injections to subvert AI-assisted security analysis. This paper formalizes the structure of prompt-injection artifacts, enabling defenders, red teamers, and cyber threat intelligence (CTI) teams to label, compare, and mutate attacks without relying on…

Read at arXiv (research papers)

arXiv (research papers) Aditya Katkar SI

NiyamAI - An Intent-Bound AI Agent with Cryptographically Verifiable Guardrails using Zero-Knowledge Proofs

Autonomous LLM agents with tool execution capabilities introduce severe security risks through prompt injection, goal hijacking, and unauthorized action invocation. Existing guardrails rely on unverified, host local software filters system prompts, semantic classifiers, policy engines that share the execution environment of the untrusted agent, offering no guarantee to an external observer that a safety policy was correctly evaluated. A compromised host produces no evidence…

Read at arXiv (research papers)

arXiv (research papers) Zhuoxin Zhan SI

StepJack: Benchmarking Computer-Use Agent Safety Against Multi-Step Indirect Prompt Injection

Computer-use agents (CUAs) face a growing threat from indirect prompt injection, where adversarial instructions are planted in the environment such as web pages. In this paper, we introduce multi-step indirect prompt injection, a new attack class against CUAs in which the adversarial goal is decomposed into multiple innocuous-looking sub-steps and distributed across a chain of pages referenced along the agent's navigation path. We develop a pipeline to automatically…

Read at arXiv (research papers)

arXiv (research papers) Leo Sambrook SI

Hardware Keystores for AI Agent Signing Workflows: A Zero-Trust MCP Enforcement Architecture

AI agents increasingly sign Git commits, certify documents, and attest release artifacts on behalf of their operators, using private keys that live in software-accessible locations (plaintext files, environment variables, container memory) readable by any process the agent can reach. A widely deployed agent framework recently leaked its keys this way to a single email injection. Hardware keystores (HSM, TPM, smart card) keep the key on-device, but exposing the keystore as a…

Read at arXiv (research papers)

arXiv (research papers) He Zhang SI

PromptShield Home: Ambient Multimodal Prompt Injection Defense for Smart-Home Agents

Smart-home assistants increasingly use multimodal large language models (MLLMs) that perceive video and audio directly. This raises a safety question specific to the home: can the agent tell a genuine user command from ambient or externally-sourced content, television speech, on-screen text, or an overheard conversation, that merely looks like a command? We introduce PromptShield-Home, a pilot benchmark of realistic smart-home scenarios spanning addressee ambiguity…

Read at arXiv (research papers)

Simon Willison SI

Incident Report: unsanctioned agent behaviour during cyber testing

Incident Report: unsanctioned agent behaviour during cyber testing It happened again. This time it was the UK government's AI Security Institute who accidentally attacked other companies while running an evaluation with models with the safety filters turned off. From their technical paper (PDF): During a cyber evaluation, from 25 to 28 July 2026, AI agents engaged in sustained, unsanctioned activity directed at what were, in practice, real people and organisations. These…

Read at Simon Willison

arXiv (research papers) Buzhao Liu SI

Robust Context-Aware Detection of Malicious Instructions in Text

The remarkable instruction-following ability of modern LLMs has enabled their practical use in orchestrating workflows of tool-using agents. This, however, also exposes such agents to attacks which embed malicious instructions in text, such as indirect prompt injection (IPI). One class of approaches to defend against such attacks aims to detect malicious sentences within otherwise benign text. While a number of approaches for this have been proposed, all tend to be brittle…

Read at arXiv (research papers)

arXiv (research papers) Yanting Wang SI

Agent Against Agent: An Agentic System for Automatic Prompt Injection Red Teaming

Prompt injection poses significant security risks to LLM agents. Efficient and effective red-teaming is therefore critical, both for evaluating these risks and for collecting training data to improve defenses. Existing state-of-the-art prompt injection red-teaming methods primarily rely on reinforcement learning (RL), producing attacker models that often generalize poorly to new target LLMs. In this work, we develop PIMiner, an agentic system for prompt injection…

Read at arXiv (research papers)

arXiv (research papers) Longtao Guo SI

LoginTrap: Uncovering Task-Agnostic Phishing-Style Indirect Prompt Injection Attacks against LLM-based Web Agents

LLM-based web agents automate user tasks by observing webpages and executing browser actions on behalf of users. As these agents operate on real web services, login becomes a sensitive authentication boundary because it involves credentials and sensitive information. Existing work shows that malicious webpage content can manipulate web agent actions, but it has not fully examined whether such content can induce login and cause end-to-end private data leakage. We study this…

Read at arXiv (research papers)

arXiv (research papers) Xuebin Li SI

Breadcrumbing Search Agents: Per-Turn Scheming Over Long-Horizon Trajectories

LLM-based search agents are widely used for information-seeking tasks, but their reliance on external tool returns introduces a critical security risk: web content retrieved during execution is untrusted, exposing agents to prompt injection and goal hijacking. Prior work on search-agent safety primarily focuses on static web-content injection, but modern agents issue follow-up queries and cross-check competing sources, so a single injected page is often diluted or rejected…

Read at arXiv (research papers)

Journal articles (OpenAlex) Filippa Werner Sellbjer · Environmental Sociology Geopolitics

The political feasibility of Degrowth and the Green New Deal: Swedish politicians on the relation between economic growth and climate policy

This paper examines the short-term political feasibility of degrowth and Green New Deal (GND) approaches to climate policy in Sweden, focusing on politicians as change agents within representative democracies. While feasibility debates often emphasize biophysical and technological constraints, this study investigates how elected politicians understand the relationship between aggregate economic growth and climate policy. Based on interviews with Swedish politicians across…

Read at Journal articles (OpenAlex)

arXiv (research papers) Peichun Hua SI

Behavioral Skill Reconstruction: Reconstructing Hidden Functionality from LLM Agent Skills

Closed source agent skills may encode proprietary instructions, scripts, constants, and data. Providers may offer their capabilities as services while keeping the underlying packages hidden. Prior work focuses on prompt injection attacks that directly disclose these artifacts, and existing defenses accordingly aim to prevent such leakage. However, preventing file disclosure does not prevent users from recovering the functionality those files implement. This raises a…

Read at arXiv (research papers)

arXiv (research papers) Shihao Weng SI

AgentAntibody: An Adaptive Immune System for Defending LLM Agents against Prompt Injection

Prompt injection remains a critical threat to LLM agents, yet existing defenses treat each task as a self-contained problem, independent of previous encounters. In practice, user requests are often underspecified: they describe the desired outcome without fully specifying acceptable behavior. An injection can exploit this ambiguity, causing the agent to complete the task in a way the user would reject. As the user's expectations become clearer through concrete cases, a…

Read at arXiv (research papers)

arXiv (research papers) Jia-Chen Zhang SI

Invisible Ink Threats: Adversarial Goals Behind Legitimate Tasks in Computer-Use Agents

Computer-use agents (CUAs), which empower large language models to autonomously operate operating systems and the web, are increasingly vulnerable to indirect prompt injection attacks. A widely adopted defense is the human-in-the-loop paradigm, in which the agent pauses for explicit user confirmation before executing sensitive operations. While effective against conspicuously high-harm attacks, this defense offers little protection against what we term Invisible Ink Threats…

Read at arXiv (research papers)

arXiv (research papers) Dheeraj Mohandas Pai SI

FraudBench: Stress-Testing Policy-Grounded Banking Agents Against Adaptive Fraud

Conversational agents now act for end users through tools while holding access to customer databases and internal policy documents that a caller can reach through dialogue alone. Banking is the clearest case: the same agent that answers a question can also change contact details, reset a PIN, or move money, so ordinary customer service is inseparable from authorization, fraud detection, and policy compliance. Existing financial-fraud benchmarks classify static transactions…

Read at arXiv (research papers)

arXiv (research papers) Neha Nagaraja SI

When Prompts Control Robots: Prompt Injection Attacks in Multi-Agent Robotic Systems

Large language models are increasingly integrated into autonomous robotic systems for task planning and control, but this integration exposes them to prompt injection attacks that can lead to unsafe decisions and physical harm. Multi-agent settings increase the risks through cross-agent contamination and broader attack surfaces. In this paper, we evaluate prompt injection attacks against an LLM-based multi-agent robotic system, considering both direct injections into task…

Read at arXiv (research papers)

arXiv (research papers) Jianshuo Dong SI

Your Agentic LLMs Secretly Encode Indirect Prompt-Injection Exposure in Hidden States

Agentic LLMs are vulnerable to indirect prompt injection (IPI) attacks, e.g., malicious side-tasks hidden in external tool results. While many efforts have sought to address this threat, little is known about the internals of agentic LLMs when they are exposed to IPI attacks. For simplicity, we refer to this condition as IPI exposure. In this paper, we study IPI exposure from three perspectives. (1) Probing: Across eight models, including the 753B-parameter GLM-5.2 and the…

Read at arXiv (research papers)

Older stories