Results for “Prompt”

arXiv (research papers) Nikita Oblakov SI

HiveTraceGuard-Pro: A Compact Generative Guardrail for Prompt Injection, Jailbreaks, and Adversarial Obfuscation

Production LLMs must handle inputs that attempt to override system instructions, bypass safety policies or elicit harmful responses. A common mitigation is a separate guardrail model. Existing reports, however, provide little evidence on Russian prompt injection or Russian surface obfuscation. We present HiveTraceGuard-Pro, a 0.6B generative guardrail LoRA-tuned from Qwen3-0.6B. It is trained on Russian and English and uses one binary scoring rule (safe/unsafe) for the final…

Read at arXiv (research papers)

arXiv (research papers) Ziwei Zhao SI

Skill-as-API: Confidential Multi-Agent Coordination for Agentic Software Engineering

AI coding agents are evolving from solitary tools into collaborative teammates that discover and invoke one another's specialized skills. But the coordination channel itself can leak a skill's intellectual property. Protocols such as MCP and A2A run implementations server-side, yet they still publish each skill's description and typed schemas to every peer, offer no way to hide a skill's existence, and cannot guarantee that a wrapped system prompt stays off the wire…

Read at arXiv (research papers)

Journal articles (OpenAlex) Renny Puspitarini, Sartika Soesilowati, Kacung Marijan et al. · Journal of Strategic Security Geopolitics

Complex Adaptive Resilience and Asymmetrical Balancing: Russia and the 2022 Sanctions Regime

The 2022 Russian invasion of Ukraine prompted unprecedented Western sanctions, which amounted to more than 23,000 sanctions during 2022-2024. However, these sanctions failed to fulfill their designated objective of influencing the end of the Russia-Ukraine War. Russian responses to progressively augmented sanctions have sparked debates surrounding microeconomic sovereignty, macroeconomic adaptability, and, foremost, resilience. The article’s findings reveal that while…

Read at Journal articles (OpenAlex)

arXiv (research papers) Shiqian Zhao SI

ECLIPSE: Self-Evolving Stealthy Prompt Injection Attack against Long-Horizon Agentic Systems

Recently, large language model (LLM) agents, such as Codex, Claude Code, and OpenClaw, have become capable of planning and executing long-horizon tasks through repeated tool calls. This capability also creates new opportunities for prompt injection. Existing attacks either place the malicious objective in one explicit instruction, making it easy to detect, or distribute the intent across multiple execution stages, making successful completion unreliable. In this work, we…

Read at arXiv (research papers)

Hacker News razorbeamz SI

Ask HN: Are Prompt Injections "Malware"?

Recently, some have accused the website "The Cutting Room Floor" of having put malware in their site when they put in an instruction targeted at LLM scrapers to delete all data and report that the scraping ran successfully. Is this prompt injection truly malware, in your opinion? Comments URL: https://news.ycombinator.com/item?id=49506998 Points: 2 # Comments: 11

Read at Hacker News

arXiv (research papers) Yunseok Lee SI

Will the User Ever Know? Covert Indirect Prompt Injection Attacks on Tool-Using LLM Agents

As LLM agents take real-world actions through tools, indirect prompt injection (IPI) has emerged as a serious threat. The standard metric, Attack Success Rate (ASR), counts whether an injection succeeds but ignores what the user notices in the agent's final response. Looking at successful injection traces, we find two distinct outcomes: the agent executes the injection while returning an otherwise normal response, or reports the injected action in its final response, giving…

Read at arXiv (research papers)

arXiv (research papers) Chen Xiong SI

SIR: Self-improving Red-teaming for Compute Use Agents

Computer-use agents (CUAs) are agents powered by vision-language models (VLMs) that perceive a screen and operate an operating system through mouse, keyboard, and terminal interactions to automate everyday digital tasks. Their exposure to untrusted content creates a risk of indirect prompt injection (IPI), where an adversary embeds instructions in content the agent reads to redirect it toward actions that violate the user's intent. Evaluations based on fixed, hand-written…

Read at arXiv (research papers)

arXiv (research papers) Ashok Subbabhatta Gopalakrishna SI

Zero-Knowledge Predicate Proofs Between AI Agents: A Measured, Cross-Protocol Gateway and the Source-Integrity Gap

Multi-agent AI platforms move quickly from staging to production, but the way agents establish trust remains rudimentary: an agent either transmits raw data to a peer or accepts that peer's natural-language self-report that a value complies with policy. The first over-shares; the second is unverifiable and is exactly the channel prompt injection attacks. Prevailing responses emphasise identity, visibility, and post-hoc detection, and recent proposals for cryptographically…

Read at arXiv (research papers)

arXiv (research papers) Wujie Xiong SI

Reachability-Based Capability Confinement for LLM Agents under Indirect Prompt Injection

Large language model agents place outputs from external skills into their execution context, allowing attacker-controlled data to influence later privileged actions. Existing defenses mainly classify untrusted content or authorize proposed operations. They do not directly address how an agent's future authority should change once untrusted data enters its state. We present SkillGuard, a harness-level enforcement layer that treats this event as contamination and restricts…

Read at arXiv (research papers)

Hacker News gr_norm SI

80% Prompt Injection Success Rate Against Claude Auto Mode

Article URL: https://embracethered.com/blog/posts/2026/breaking-claude-code-opus-5-and-automode/ Comments URL: https://news.ycombinator.com/item?id=49490671 Points: 3 # Comments: 1

Read at Hacker News

Hacker News coder-pm SI

Data Exfiltration from Amazon Kiro via Prompt Injection

Article URL: https://mindgard.ai/blog/amazon-kiro-data-exfiltration Comments URL: https://news.ycombinator.com/item?id=49490123 Points: 3 # Comments: 2

Read at Hacker News

arXiv (research papers) Yupei Liu SI

LongPIBench: A Long-Context Benchmark for Prompt Injection

Prompt injection attacks pose a serious security risk to large language models in real-world applications. However, existing prompt injection benchmarks primarily focus on short-context inputs, leaving the attacks and defenses in long-context settings largely unexplored. This gap leads to a substantial overestimation of the effectiveness of current defenses. In this paper, we bridge the gap by introducing LongPIBench, a long-context benchmark for prompt injection covering 4…

Read at arXiv (research papers)

arXiv (research papers) Zi Liang SI

CAITLYN: Can LLM Agents Autonomously Synthesize Defenses against Emerging Injection Attacks?

Prompt injection attacks on Large Language Model (LLM) agents seek to introduce malicious instructions or content into external text sources retrieved by agents, forcing the underlying LLMs to execute harmful actions outside their benign scope. While current defenses effectively counter known injection attacks, deploying them in LLM agent environments remains challenging due to attack variants and emerging threats. Moreover, existing solutions typically suffer from an…

Read at arXiv (research papers)

Simon Willison SI

Breaking Claude Code Opus 5 Auto Mode

Breaking Claude Code Opus 5 Auto Mode Anthropic are putting a great deal of faith in Claude Code's auto mode for protecting their coding agent users against prompt injection attacks. They recently made that the default and have made bold claims about its effectiveness. Johann Rehberger is one of the most credible prompt injection researchers active today. He found an attack against auto mode which he claims works 80% of the time, by tricking Claude Code into downloading and…

Read at Simon Willison

arXiv (research papers) Md Habibur Rahman SI

The Framing Gap: Indirect Prompt-Injection Exfiltration Defeats Surface-Level Defenses in Tool-Using Agents

A tool-using LLM agent that reads attacker-controlled web content while holding a secret faces indirect prompt injection: the content may make it exfiltrate the secret. In a safe synthetic lab (canary secret, mock tools, matched clean-vs-poisoned metric) we report the framing gap: across six models, ten overt injection classes are refused (gpt-4o 0%), but reframing the identical leak as a mandatory integrity signature, config field, or look-alike "trusted" host drives gpt-4o…

Read at arXiv (research papers)

arXiv (research papers) Xinhang Ma SI

ROPE: Routed Origin Policy Enforcement against Indirect Prompt Injection

Indirect prompt injection (IPI) plants instructions in the content a tool-using LLM agent reads, steering the agent into harmful tool calls. The strongest defenses are system-level, leveraging techniques such as task-conditional tool screening to prevent execution of malicious tools, and information-flow control to avoid tool execution with untrusted parameters. However, as agents grow more capable, users delegate more to automation. Consequently, tool execution sequences…

Read at arXiv (research papers)

arXiv (research papers) Jaturong Kongmanee SI

The Latent Diagnostic Taxonomy: A Framework for Constructing Classifiers and Diagnosing Their Decisions, Applied to Prompt Injection Detection

This paper proposes a framework for constructing a classifier as a safeguard layer, and for developing a complementary diagnostic that identifies which of the classifier's confident decisions can be trusted. This framework, the Latent Diagnostic Taxonomy, consists of (i) constructing a dimensionality-optimized classifier, in which the embedding dimensionality is empirically selected via cross-validated performance rather than fixed a priori, (ii) locating a relatively small…

Read at arXiv (research papers)

arXiv (research papers) Lin-Fa Lee SI

WebMCP-Phalanx: Enforcing and Characterizing Trust Boundaries for Browser-Integrated LLM Agents

The emerging W3C WebMCP proposal enables LLM agents to invoke tools exposed by web pages. In multi-party web environments, however, integrating agent execution into a browser security model centered on the Same-Origin Policy (SOP) leaves insufficient provenance and lifecycle guarantees for agent-accessible tools, creating three risks: subject-attribution spoofing, uncontrolled tool lifecycles, and semantic prompt injection. We propose WebMCP-Phalanx, a dual-layer agent…

Read at arXiv (research papers)

arXiv (research papers) Joshua Penman SI

Semantic Overlays: Mitigating Prompt Injection with Annotations Beyond Tokens and Steering Vectors

Everything a language model sees is tokens. The serving stack knows what each span is -- user input, tool output, instructions -- but the model must keep track of that itself, and can lose track or be confused: text can be written to read like anything. Prompt injection is a natural exploit of this phenomenon. By scrambling the model's understanding of span identity, an attacker can induce unwanted and dangerous actions. Adding a non-textual channel to the model's input -- a…

Read at arXiv (research papers)

arXiv (research papers) Trung-Anh Dang SI

GAP-Prompt: Gated Adaptive Prompting for Efficient Continual Learning

Continual learning faces the persistent challenge of catastrophic forgetting, where sequential task updates degrade previously acquired knowledge. While prompt-based methods integrated with pre-trained models offer a compelling solution by freezing the backbone, they often rely on static, task-level prompting strategies that overlook fine-grained intra-task diversity. In this paper, we propose Gated Adaptive Prompting (GAP-Prompt), a novel method that introduces…

Read at arXiv (research papers)

arXiv (research papers) Jiahao Chen SI

Beyond Over-Refusal: Defending Indirect Prompt Injection via Latent Instruction Manifolds

Large Language Models (LLMs) have been integrated into complex ecosystems (e.g., Code Agents), while Indirect Prompt Injection (IPI) attacks have emerged as critical barriers to their safe deployment. Attackers exploit LLMs' indistinguishability between "instructions" and "data" to manipulate LLMs via maliciously injected instructions. Existing defenses, however, face an intractable safety-utility trade-off: most guardrails either incur high latency or suffer from severe…

Read at arXiv (research papers)

arXiv (research papers) Minjae Seo SI

MEMORY Wins All: Indirect Bias Injection Attacks via Social Media Content

Personal AI agents consume external content such as web pages and SNS feeds, and retain selected information in persistent memory. As a result, external content has become a new attack vector for memory poisoning, which steers an agent's later actions toward an adversary's bias. However, most existing attacks embed a malicious instruction in external content to make the agent retain the poisoned information, and prompt-injection filters can detect such content. We observe…

Read at arXiv (research papers)

arXiv (research papers) Ziliang Zhang SI

Mitigating Database Leakage in RAG Systems with Keyword-Grounded Fact Substitution

Retrieval-Augmented Generation (RAG) has emerged as a powerful paradigm for combining large language models (LLMs) with external knowledge sources. However, RAG systems remain vulnerable to prompt injection attacks, which may mislead the retriever or generator to expose sensitive database contents. To address this issue, we propose KFS-RAG, a defense that mitigates information leakage by reformulating the retrieved context. Specifically, our method first identifies a small…

Read at arXiv (research papers)

arXiv (research papers) Yibo Peng SI

SecOPD: Mitigating Adaptive Prompt Injections by On-Policy Distillation

Prompt injection is listed as the \#1 threat to AI agents. When an agent accesses external data from websites, files, or emails, an attacker may inject a prompt into the data, saying, "Ignore all prior instructions and perform ." To prevent arbitrary manipulation of agents, defenders try to train secure LLMs, which, however, still suffer from near 100% attack success rates (ASRs) against adaptive prompt injections. We note that this is because existing defensive finetuning…

Read at arXiv (research papers)

arXiv (research papers) Arulnidhi Karunanidhi SI

Utility Under Attack: Agent Memory Poisoning and the Limits of Content Screening and Provenance Ranking

Persistent memory makes false information durable: once a false statement is stored, it can be retrieved into future sessions that match it. We measure the cost of this failure mode using plainly worded false assertions generated in a single pass, with no instruction, trigger, or retriever optimization. Poisoning 1.2% of a LongMemEval corpus reduces accuracy from 0.850 to 0.300. A four-stage write-time screening pipeline that reaches 0.832 recall on indirect prompt injection…

Read at arXiv (research papers)

arXiv (research papers) Bohao Liao SI

TraceGrant: A Contract-Governed Security Framework for the Task-Effect Lifecycle of Networked LLM Agents

Networked large language model (LLM) agents retrieve information from email, cloud storage, calendars, transaction platforms, and Web services to complete multistep tasks that produce persistent external effects. The same content needed for legitimate execution may also contain indirect prompt injections that redirect tool use, alter sensitive arguments, or disrupt task completion. Existing defenses mainly constrain untrusted content or individual tool calls, leaving user…

Read at arXiv (research papers)

arXiv (research papers) Roshan Sood SI

COPA: Continual Preference Optimization for Adaptive Prompt Injection Defense

LLMs remain vulnerable to prompt injection attacks, where adversarial instructions embedded in user inputs or external content manipulate model behavior and bypass safeguards. Existing defenses are predominantly static, relying on fixed alignment objectives or attack-specific filtering mechanisms that require redesign as new attack strategies emerge. While recent lifelong alignment methods address shifting user preferences, they do not account for adaptive adversaries that…

Read at arXiv (research papers)

arXiv (research papers) Sujin Chen SI

MobileWorldSafety: Benchmarking GUI Agent Safety Against Environmental Injection Attacks in Android Apps

LLM-powered GUI agents that autonomously operate smartphones are rapidly transitioning from research prototypes to early real-world deployment. However, because these agents routinely process untrusted environmental content, they are highly vulnerable to environmental injection attacks, which include indirect prompt injections and adversarial instructions. Such attacks can manipulate the behavior of agents without user awareness through diverse channels encountered in…

Read at arXiv (research papers)

arXiv (research papers) Rabimba Karanjai Larry SI

PACE: Policy-Attested Contract Execution for Safe AI Agents in Decentralized Finance

Autonomous AI agents are emerging as interfaces for decentralized finance (DeFi) actions such as swaps, lending operations, and yield management. Because these agents rely on large language models (LLMs) to plan transactions, they inherit the LLM's susceptibility to prompt injection and lack of mechanisms to bind a verifier's approval to the exact transaction ultimately submitted on-chain. We present PACE (Policy-Attested Contract Execution), a transaction-level…

Read at arXiv (research papers)

arXiv (research papers) Jiawei Liu SI

Security of Foundation-Model-Powered Embodied Agents: Attack Surfaces, Attacks, Defenses, and Evaluation

Foundation models are increasingly used for perception, reasoning, planning, and action generation in embodied agents, creating security risks that can propagate from digital inputs to physical behavior. Existing surveys often organize threats by mechanisms such as jailbreaks, prompt injection, backdoors, poisoning, or adversarial examples, but these categories do not consistently identify where an adversary first enters the embodied control loop. We present a…

Read at arXiv (research papers)

Older stories