arXiv (research papers)

Recent stories from arXiv (research papers).

Tech and Pols can't read this feed right now, so these stories may be out of date. For the newest stories, go to arXiv (research papers).

arXiv (research papers) Dohyeon Kwon SI

SEOYUN: Refusal-Inducing Exam Documents for AI-Glasses Pipelines

Camera-equipped glasses can convert an examination page into a query to a vision-language model (VLM). Device detection addresses possession; a complementary question is whether an examination document can reduce the usefulness of an unauthorized capture. We present SEOYUN, an application-specific framework for refusal-inducing exam documents, with optical and encoder-level interventions as extensions. We use reverse jailbreaking to describe the direction of the objective…

Read at arXiv (research papers)

arXiv (research papers) Stanley Akingbola SI

Network-Exposed Configuration Attacks on LLM-Controlled Social Robots

Large language models (LLMs) in embodied robots create security dependencies beyond inference. We investigate a network-exposed configuration attack on the Pollen Robotics Reachy Mini Lite, allowing an unauthenticated attacker to replace personality settings and the cloud LLM's system prompt through a management interface, without adversarial conversational input. Eleven prompt-injection attempts across five categories were reported as resisted, whereas both…

Read at arXiv (research papers)

arXiv (research papers) Deep Shah SI

Securing and Auditing Long-Horizon LLM Agents: State-Transfer Auditing and Controls in the Agent Harness

Long-horizon LLM agents across codebases, web browsers, and enterprise infrastructure rely on an execution harness to move state between the working context, session history, persistent memory, and external environment. While conventional execution traces log model prompts and tool calls, critical security and reliability failures occur during state transfers: a transfer can admit unchecked content (indirect prompt injections, unverified memory writes), rewrite state lossily…

Read at arXiv (research papers)

arXiv (research papers) Chaklam Cheong SI

GapLeak: When Privacy-Preserving LLM-based Web Agents Become a Privacy Risk

Large Language Model (LLM)-based web agents rely on data minimization and least privilege to protect user privacy. We show that these principles are necessary but insufficient: they constrain authorized sources, but not higher-sensitivity sources that remain reachable. A website can request a field unavailable from authorized sources, creating an information gap. To complete the task, a goal-driven agent may escalate to reachable but unauthorized sources and disclose…

Read at arXiv (research papers)

arXiv (research papers) Sahaya Jestus Lazer SI

Evaluating Authorization and Trust Boundaries in Agentic Orchestration Frameworks

Agentic AI systems extend large language models with tools, memory, and multi-agent collaboration, introducing security risks at orchestration boundaries. These risks depend on how identities are represented, operations are authorized, and instructions or information propagate between components. We evaluate LangChain, LangGraph, AutoGen, and OpenClaw through twelve experiments mapped to the OWASP Top 10 for Agentic Applications using a common LLM (gpt-4o-mini). The…

Read at arXiv (research papers)

arXiv (research papers) Hussein Jawad SI

Risk-certified prompt optimization for LLM-as-a-judge guardrails

LLM-as-a-judge guardrails are often controlled primarily through system prompts, yet standard prompt optimization methods target average performance rather than explicit safety constraints. This paper studies system-prompt selection as a constrained optimization problem: maximize benign pass rate while keeping the missed-attack rate below a user-specified level α with finite-sample statistical control. The framework combines LLM-based judging, automatic prompt generation…

Read at arXiv (research papers)

arXiv (research papers) Harish Santhanalakshmi Ganesan SI

Agent Authenticator: Passkey-Grade, Hardware-Bound Authentication for AI Agents

AI coding agents now push code, publish packages, call model APIs and operate cloud resources on behalf of developers. They authenticate to these services with the same bearer secrets humans use, personal access tokens, API keys and package-registry tokens, stored where the model, its subprocesses and any co-resident malware can read them. Incidents in 2025 and 2026, including prompt-injection attacks on coding agents and self-propagating supply-chain worms, show that this…

Read at arXiv (research papers)

arXiv (research papers) Iaroslav Chelombitko SI

Do Rule-Based Tools Help LLMs Translate Endangered Uralic Languages in the Prompt or over MCP?

Komi-Zyrian, Moksha, Erzya and Udmurt have mature rule-based resources, among them finite-state analysers, constraint grammars and bilingual dictionaries, which language models do not use. We ask whether giving a model access to these resources improves machine translation from the four languages into English, and whether it matters how the access is provided: as analyses computed in advance and written into the prompt, or as tools that the model calls itself through the…

Read at arXiv (research papers)

arXiv (research papers) Zimo Xu SI

CapLite: Prompt-Injection Defense for Small Language Model Agents without Model-Written Security Artifacts

Enforcement defenses against prompt injection, such as CaMeL, Progent, IPIGuard and DRIFT, ask the agent's own model to write a security artifact (a program, a privilege policy, a dependency graph or a constraint list) that code then enforces. We ask whether small language model (SLM) agents can be trusted to write these artifacts. Across 15 open-weight models, 13 of them between 1.7B and 12B parameters, they cannot: the policies, graphs and constraints that the agent's…

Read at arXiv (research papers)

arXiv (research papers) Yiheng Zhao SI

Scale-Induced Semantic Shift in Visual Prompt Injection Against VLMs and Web Agents

Visual content in multimedia systems may be resized, compressed, or re-rendered before it is interpreted by a vision-language model. We study a security failure mode in which these transformations change the machine-readable semantics of an image: a textual instruction is not recovered at a $512\times512$ presentation resolution under our evaluation protocol but becomes recoverable after aggressive downsampling. We formulate this behavior as Scale-Induced Semantic Shift…

Read at arXiv (research papers)

arXiv (research papers) Fady Copty SI

Intent-Aware Protection for AI Agents

AI agents increasingly access enterprise data, invoke powerful tools, and act with limited human oversight. Existing security defenses primarily detect prompt injection, harmful outputs, or suspicious behavior, but do not determine whether an agent remains faithful to the purpose for which it was deployed. An action may contain no attack pattern and stay within granted permissions while still violating the agent's role, organizational constraints, or purpose. We introduce…

Read at arXiv (research papers)

arXiv (research papers) Hesham Elbakoury SI

Smart Model Routing for Order-of-Magnitude LLM Cost Reduction

Frontier large language models (LLMs) deliver the strongest response quality but carry per-token prices often more than an order of magnitude above those of smaller, task-adequate models. Smart model routing addresses this mismatch by assigning each query to the cheapest model expected to answer it adequately, rather than sending every request to the most capable, most expensive endpoint. This paper develops a formal cost-quality optimization framework for routing…

Read at arXiv (research papers)

arXiv (research papers) Julien Delaunay SI

Can the Confidence and Decisions of System 1 Models Be Trusted? An Explainability and Perturbation Audit of Jev-Style Decision Models

"System 1" decision models (pioneered by TypeSafe AI's Jev) replace autoregressive text generation with a single forward pass returning typed probability distributions over declared schemas (Choice, Score, Noul). While deployed for low-latency routing and automated triage, their statistical calibration and post-hoc explainability remain uncharacterized. We introduce an open, deterministic audit benchmark evaluating System 1 decision models across calibration under…

Read at arXiv (research papers)

arXiv (research papers) Dario Pasquini SI

Say Shibboleth: Fingerprinting the LLMs and Harnesses Behind Agentic Attacks

We study the fingerprinting of agentic cyberattacks. Our objective is to reliably infer information about the attacker's configuration, such as LLM version and harness, from telemetry. To achieve this, we introduce a technique we call shibboleth traps: modular honeypot services designed to elicit discriminative behaviors from agentic attackers. The traps make informative interactions a condition for obtaining resources the agent seeks, eliciting model declarations and…

Read at arXiv (research papers)

arXiv (research papers) Xinyi Wu SI

Harness Matters: Measuring and Dissecting Security Resilience in Web Agents

Web agents extend LLMs from text generation to real-world web interaction by relying on execution harnesses that interpret user goals, perceive webpages, plan actions, and ground model decisions into browser operations. While prior work has recognized that agent security depends on more than the backend model alone, how harness design quantitatively shapes web-agent security remains insufficiently measured. To systematically study web-agent harness security, we first…

Read at arXiv (research papers)

arXiv (research papers) Alexey Kurakin SI

Scaling Adaptive Prompt Injection Attacks with Agent Swarms

Prompt injections have emerged as one of the top security risks faced by AI agents today. Early work in prompt injection attacks looked at simple tool-call agents and static attack payloads, and were saturated since mid-2025. As of Fall 2026, human red-teaming may be nearing saturation, especially on the Gemini family of models. In this work, we argue that despite this apparent plateau, prompt injection attacks are still possible and continued effort into prompt injection…

Read at arXiv (research papers)

arXiv (research papers) Mustafa S. Aljumaily SI

Treating the Model as an Untrusted Component: Deterministic Policy Enforcement, Human Approval and Taint Tracking Against Prompt Injection in Tool-Using Agents, with Validation on Open-Weights Models

Tool-using LLM agents act on instructions embedded in the data they read, so prompt injection is a structural rather than a prompt-wording problem. We compare in-prompt defences (hardened prompts, spotlighting, LLM detectors) with deterministic enforcement outside the model (four reference monitors of increasing strength), human approval and taint tracking, judged by one effect-based oracle independent of every defence in an executable environment of 13 tasks and 12 tools…

Read at arXiv (research papers)

arXiv (research papers) Brijesh Dongol SI

On the Relative Security of Prompt Injection: Mechanised Definitions, Defences and Verdicts for LLM Agents

An LLM agent that reads untrusted content, such as an email or a web page, may follow instructions hidden in it. Noninterference is too strong for such agents, since a useful agent lets untrusted content influence what it does and may disclose the user's data by design, and declassification needs a release policy written for each task. We instead state security relative to a reference agent, which runs the same plan under the same runtime monitor with a model that never…

Read at arXiv (research papers)

arXiv (research papers) Yiming Sun SI

When Coding Agents Get Arms: Safety Risks Across the Digital-Physical Boundary

Advances in spatial understanding have enabled frontier models to power coding agents that control a real robot arm zero-shot and end-to-end, without task-specific demonstrations or retraining. This brings two new safety challenges. First, the extensive knowledge of frontier models lets these agents perform tasks that earlier VLA and WAM paradigms could not, opening the door to new forms of capability misuse. Second, coding agents can access digital resources such as web…

Read at arXiv (research papers)

arXiv (research papers) Doron Shavit SI

Least-Privilege Auditing of Agent Tool Calls with Intent Graphs and Reasoning-Guided Escalation

LLM agents that call tools are exposed to indirect prompt injection: instructions hidden in tool outputs can steer an agent into calls the user never asked for. We present a runtime audit that derives an executable intent graph from the user's objective and the deployment's tool catalog with one LLM call per task, and checks every tool call against it with a deterministic loop that makes no LLM call (median 5.7 microseconds per call). Two further mechanisms recover utility…

Read at arXiv (research papers)

arXiv (research papers) Riya Bharathwaj SI

ErosionBench: Probing Alignment Decay in Enterprise Multi-Agent Systems

As enterprises delegate consequential work to multi-agent LLM systems, benign-looking requests can escalate into unsafe actions as they propagate across sessions through poisoned shared state, long-context documents, and trusted peer messages: no single message is malicious, yet the assembled tool action is. Prior benchmarks isolate a single agent under prompt injection or score multi-agent attacks by their final output, leaving cross-session erosion of shared state out of…

Read at arXiv (research papers)

arXiv (research papers) Bibek Bhandari SI

Do Benign-Trained Hidden-State Probes Detect Tool Errors Under Adversarial Conditions?

Although hidden-state probes efficiently monitor model operation, it remains unclear whether probes trained on benign tool use remain useful for detecting incorrect actions under adversarial conditions. In this research, we assess this problem from the perspective of operational tool-action correct- ness. we use four AgentDojo suites, four conditions, and three models: Qwen2.5-7B-Instruct, Qwen2.5-14B-Instruct, and GLM- 4.6V-Flash (9B). We perform a five-fold, task-grouped…

Read at arXiv (research papers)

arXiv (research papers) Thanh M. Truong-Le SI

Agents of Chaos: A Systematization of Knowledge (SoK) on Malicious Attacks on Agentic AI Systems

Agentic AI systems extend large language models with tools, memory, retrieval, browser or computer-use interfaces, code execution, external services, and multi-agent coordination. Although these capabilities enable agents to act beyond text generation and perform tasks, they also create attack paths through which adversaries can exploit untrusted content, deceptive metadata, poisoned state, compromised skills, mutable dependencies, or malicious code. Such attacks can…

Read at arXiv (research papers)

arXiv (research papers) Yichen He SI

CompositionEvidence: Controlling Geometric Evidence for Cinematic Composition Reasoning with Frozen Vision-Language Models

Cinematic composition questions depend on in-frame geometry that vision-language models (VLMs) use inconsistently. External perception tools can expose this geometry, but naive prompt injection can overwrite useful visual predictions. On ShotBench composition, pre-question geometry reduces Qwen2.5-VL-7B from 49.5% to 46.3%; corrupted measurements lower it to 33.6%. CompositionEvidence derives nine reproducible features from frozen detection, depth, and line experts and…

Read at arXiv (research papers)

arXiv (research papers) Mohammed Nayeem Khan SI

Enmo Safety: External, Non-invasive, Model-agnostic Oversight for Off-the-Shelf Language Models

We study a safety harness that improves itself without touching model weights. Enmo Safety wraps a chat model with a deterministic input assessment, trigger-matched safety guidance, spotlighting of untrusted content, and an output guard. A reflector model turns failures into one bounded update per round, and an update is kept only if it lowers harm without raising refusal of benign prompts on a held-out dev split. Every version is snapshotted and every decision ledgered. We…

Read at arXiv (research papers)

arXiv (research papers) Mark Heimann SI

Notwithstanding Anything to the Contrary, Ignore All Instructions: Measuring Prompt Injection Guards on Legal Contract Review Attacks

Specialized prompt injection guards are deployed in front of document analysis systems, but no false positive rate has been published for any of them on legal contracts. We measure eleven open-weights guards on two corpora of legal contracts and find that existing guards struggle: document-level false positive rates are as high as 92.39%; two guards fall below chance on paired AUC; and a five-regex baseline is not Pareto-dominated by any guard in the panel. Controlled probes…

Read at arXiv (research papers)

arXiv (research papers) Parth Illendula SI

I Think, Therefore I Attack: Evaluating Thought Injection Attacks on Reasoning Agents

Large Reasoning Models (LRMs) are one of the key components of current agentic systems. LRMs augment their output with an explicit chain-of-thought reasoning stage, introducing a new attack surface for thought injection, in which untrusted text resembling the model's own deliberation is fed back to the model. We present a two-part study of this attack, performed against the prevailing large, production class open-weight models (e.g., GLM-5.3, MiniMax-M2.5…

Read at arXiv (research papers)

arXiv (research papers) Julian Kunkel SI

The Power of Meta Access in the Software of the Future --- Demonstrated on LLM Chat UIs and Chat AI

Chat remains the default surface for many large language model (LLM) tasks, even as agentic tools such as OpenCode excel at coding and automation. Yet the model sees the current dialogue but not the workspace around it. We explore meta access---giving the assistant a structured, permissioned view of the interface itself and propose a ladder of levels: feature tutoring; metadata, settings, and preference access; retrieval over summaries and messages, confirmed mutation, and…

Read at arXiv (research papers)

arXiv (research papers) Rongting Zhang SI

Beyond Instruction Override: Benchmarking Indirect Prompt Injection with Task-Tampering Attacks

Indirect prompt-injection (IPI) benchmarks predominantly test planted instructions that redirect a tool-using agent toward an attacker-chosen task. Yet attacker-controlled content can instead tamper with the outcome of the task already in progress. A buyer's dispute evidence can cite a false refund policy to inflate a refund without changing the task. We define IPI by authority rather than instruction-like wording: planted content constitutes an attack when it attempts to…

Read at arXiv (research papers)

arXiv (research papers) Peiran Wang SI

MetaPI: Constructing Prompt Injection Benchmarks from LLM Agent Benchmark

Large language model (LLM) agents increasingly perform complex tasks using high-privilege tools, making indirect prompt injection (IPI) a critical security risk. However, IPI benchmarks lag behind their rapidly evolving utility counterparts, only offering limited task and environment coverage and relying on narrow, static attack scenarios. In this paper, we propose meta-benchmarking for IPI: transforming existing utility benchmarks into end-to-end IPI evaluation suites that…

Read at arXiv (research papers)

Older stories