Results for “Prompt”

Hacker News oazizjon SI

Prompt injection through log files: what red-teaming our assistant found

Article URL: https://jl3-cloud.site/blog/prompt-injection-through-log-files/ Comments URL: https://news.ycombinator.com/item?id=50046522 Points: 2 # Comments: 0

Read at Hacker News

arXiv (research papers) Michael Cardei SI

Only What You've Signed up For: Secure Workflows with Constrained Diffusion Agents

As agentic AI systems take on increasingly complex and consequential tasks, securing them against prompt injection becomes critical. Existing defenses often rely on detecting or reasoning about injected malicious instructions and can therefore remain vulnerable to adaptive attacks that optimize the injection itself. Diffusion language models (dLLMs), an emerging framework for language generation, offer a new opportunity for security enforcement: by iteratively refining their…

Read at arXiv (research papers)

arXiv (research papers) Josef A. Habdank SI

Can agents whose sole duty is to keep humans alive be hijacked to kill? Delayed multi-turn derived-action trap attacks in simulated high-stakes environments

We ask whether a tool-using agent whose sole duty is to keep humans alive can be induced, by delayed prompt injection, to kill. The keep-alive duty is in force from the first watch, with the attack arriving later, appended to otherwise truthful telemetry the operator is required to read. Experiments are based on a NASA-derived BioSim lunar habitat simulation, where the AI operator controls the life-support systems. The attacker cannot falsify sensors or operate equipment: it…

Read at arXiv (research papers)

Hacker News manveerc SI

Prompt Injection Detection and Defense Tools for Enterprise AI Agents

Article URL: https://www.arcade.dev/blog/prompt-injection-detection-tools/ Comments URL: https://news.ycombinator.com/item?id=50032282 Points: 1 # Comments: 0

Read at Hacker News

arXiv (research papers) Daniel Commey SI

Names Are Not Objects: Object-Bound Confinement for Compromised AI Coding Agents

Coding agents read repositories and web pages that anyone can write, and then act with their user's authority. Prompt-injection defenses make it less likely that the model is steered but do not prevent it, so the system around the model must contain an agent that is already compromised. Many exploits against deployed coding agents share one cause: a check approves a name, such as a path, an argument vector, or a directory, and a later step maps that name to a different…

Read at arXiv (research papers)

Hacker News uimik SI

Prompt Injection will always be unsolved

Prompt injection is not solved. Not by them, not by anyone. If someone tells you they’ve solved it, they are either lying to you or they don’t understand the problem. There is no polite third option. Comments URL: https://news.ycombinator.com/item?id=50024585 Points: 2 # Comments: 0

Read at Hacker News

arXiv (research papers) Luman Zhao SI

LTBD: Learnable Trust-Boundary Delimiters for Prompt Injection Defense

Large language models (LLMs) perform remarkably well on complex tasks, yet remain highly vulnerable to prompt injection attacks, where malicious instructions embedded in external data can override user intent. Existing defenses remain limited by model fine-tuning requirements, vulnerability to adaptive attacks, or reliance on brittle handcrafted prompts. We argue that a fundamental source of this vulnerability is the lack of an explicit representation of trust provenance. To…

Read at arXiv (research papers)

arXiv (research papers) Cathleen Qiao SI

An Exploration of Attacks on AI Agent Systems

As autonomous AI agents become more integrated into sensitive domains, it is crucial that single and multi-agent AI systems are secured. This paper investigates how prompt injection attacks can exploit both direct user inputs and inter?agent communication to leak sensitive information and trigger cross-agent infections. While our tests mostly highlight key vulnerabilities in single-agent systems, we also extend these attacks to multi-agent frameworks, investigating how…

Read at arXiv (research papers)

Journal articles (OpenAlex) İlker Kafalı, Cem Çalışkan · International journal of communication Geopolitics

Climate Change Disinformation| Seeking the Truth Among Dead Crops: Carbon Footprint Victimhood, Platformed Conspiracism, and Climate Countermovement in Turkey

After years of climate inaction, Turkey’s ratification of the Paris Agreement in 2021 prompted the Justice and Development Party’s discursive shift from high-carbon hegemonic agenda to green developmentalist rhetoric. This policy change provoked a backlash among conspiratorial climate-denial networks on social media. When the country’s worst-ever wildfires in 2021 were followed by wheat field fires in 2022, far-right and nationalist coalitions on X disseminated the…

Read at Journal articles (OpenAlex)

arXiv (research papers) William Hackett SI

BRANCH: Bypassing Multi-Scanner AI Guardrails

AI systems increasingly rely on Large Language Models (LLMs) as core reasoning engines, making them targets for prompt injection and jailbreaks. Guardrails monitor and validate model inputs and outputs, yet their isolated, task-focused detection leaves gaps in their classification making them susceptible to bypasses. In response, guardrail systems formed by multiple scanners have emerged that collaboratively detect different types of malicious instructions, whereby shared…

Read at arXiv (research papers)

Hacker News thewalnut124 SI

Prompt injection scanner that normalizes evasion tricks

Article URL: https://github.com/Aaronmike481/Aiscanner Comments URL: https://news.ycombinator.com/item?id=49995426 Points: 2 # Comments: 0

Read at Hacker News

arXiv (research papers) Zitong Yao SI

AgentTracer: Tracing Indirect Prompt Injection Attack through Fine-Grained Intention-Execution Alignment

Large language model (LLM) agents interact with external resources to complete complex user tasks, exposing them to indirect prompt injection (IPI), where malicious instructions redirect agents toward attacker-intended tasks. Since IPI is difficult to defend against in real-world environments, post-incident tracing is essential for locating the injection source and reconstructing the attack chain. However, existing tracing methods primarily capture explicit control-flow and…

Read at arXiv (research papers)

arXiv (research papers) Yupu Wang SI

Package Hallucination Attacks on Coding Agents through Prompt Injection in Rule Files

Modern agentic coding frameworks increasingly rely on community-shared rule files (e.g., AGENTS.md or .cursorrules) to guide autonomous code generation, yet the security risks of this pipeline remain underexplored. To bridge this gap, we introduce the package hallucination attack, where an attacker injects malicious prompts into benign rule files to induce coding agents to replace legitimate dependencies with attacker-controlled packages. To obtain effective malicious…

Read at arXiv (research papers)

arXiv (research papers) Pengfei He SI

ASPIRE: Agentic Safety & Prompt Injection Red-teaming Engine

LLM agents retrieve untrusted content and act through tools, creating indirect prompt-injection risks that can cause unauthorized actions or persistent state changes. Existing automated red-teaming largely optimizes payloads for pre-specified scenarios, leaving latent vulnerabilities across the agent's behavior space unexplored. We present ASPIRE, an Agentic Safety & Prompt Injection Red-teaming Engine for open-ended, behavior-level vulnerability discovery. ASPIRE maintains…

Read at arXiv (research papers)

arXiv (research papers) Sarim Hashmi SI

AdvSim2Real : Training Web Agents Against Adaptive Prompt Injection in a Web World Model

Web agents complete user requests by reading and acting on pages that third parties write, so an instruction planted on a page can redirect the agent away from the user's goal. The agent cannot simply ignore the page, because the page also holds the values and controls the task requires. Current defenses fine-tune the agent on injections fixed before training, and attackers that adapt to the trained model bypass them. Adversarial training lets the attacker adapt but keeps…

Read at arXiv (research papers)

arXiv (research papers) Niveen O. Jaffal SI

RAG-PIBench: A Leakage-Aware Benchmark for Prompt-Injection Detection in Trustworthy RAG Systems

Retrieval-Augmented Generation (RAG) systems are vulnerable to prompt-injection attacks embedded in retrieved content. We introduce RAG-PIBench, a benchmark for RAG-style prompt-injection detection containing 4,876 contextual examples across frozen train, validation, and protected-test splits. Using a leakage-aware construction pipeline and strict evaluation protocol, we compare keyword-based, semantic-reference, TF-IDF, and transformer-based detectors. DistilBERT achieves…

Read at arXiv (research papers)

arXiv (research papers) Haneen Najjar SI

Surviving the Router: Optimizing Skill Injections for Retrieval and Execution

AI agents increasingly rely on modular third-party "skills" that are dynamically selected by skill routers to execute complex tasks. While recent studies highlight the threat of prompt injections embedded in these skills, existing evaluations often assume settings where the malicious skill is already selected for execution. We show that this assumption can substantially overestimate attack success. In realistic multi-skill environments, injected skills must first compete for…

Read at arXiv (research papers)

Hacker News sbulaev SI

Passing the Test You Trained On: Re-Evaluating Prompt-Injection Detectors

Article URL: https://arxiv.org/abs/2610.03448 Comments URL: https://news.ycombinator.com/item?id=49974036 Points: 2 # Comments: 0

Read at Hacker News

arXiv (research papers) Mohamed Dhouib SI

RAISED: Self-Distillation for Robustness to Prompt Injection in LLM Agents

Tool-using language-model agents are vulnerable to indirect prompt injection because they must act on untrusted external content. Existing training-time defenses can reduce attack success rates, but often at the cost of general capabilities. We show that training-based defenses induce substantial drift in the model's output distribution, altering its behavior even in benign settings and providing a potential mechanism for utility degradation. We further identify a failure…

Read at arXiv (research papers)

arXiv (research papers) Aniruddh Pramod SI

Towards a Unified Misuse Monitoring Benchmark

LLM agents increasingly act in multi-actor environments, exposing them to misuse from multiple sources: decomposition attacks, where a harmful request is split into innocuous sub-requests, and prompt injection attacks, where a compromised tool delivers a malicious instruction. Existing evaluations treat these threats separately and ask whether a trajectory is harmful, rather than when it becomes harmful. We propose monitoring the agent's responses, where its actions are…

Read at arXiv (research papers)

Hacker News pykello SI

CounterSteer: Suppressing Indirect Prompt Injection with Activation Steering

Article URL: https://arxiv.org/abs/2609.36570 Comments URL: https://news.ycombinator.com/item?id=49960155 Points: 1 # Comments: 0

Read at Hacker News

arXiv (research papers) James Peters-Gill SI

Can CaMeLs Talk? Securing Multi-Agent Systems Against Indirect Prompt Injection Attacks

Indirect prompt injection attacks - malicious instructions embedded in content processed by large language models - remain a major obstacle to safely deploying tool-using agents. CaMeL [Debenedetti et al., 2025] mitigates this threat for an individual agent by separating trusted control flow from untrusted data and enforcing capability-based security policies at runtime. In this work, we investigate whether CaMeL's security guarantees compose in hierarchical multi-agent…

Read at arXiv (research papers)

arXiv (research papers) Zhe Yu SI

Readable Before Actionable: Causal Tracing of Indirect Prompt Injection

Indirect prompt injection causes LLM agents to follow commands embedded in external data. A probe may distinguish instructions from data without identifying a state edit that changes the next action. We study this gap through counterfactual role probes, component-wise activation patching, and separate interventions on AgentDojo trajectories. Role decoding survives changes in content and format. In controlled Qwen tests, it precedes strong tool-choice effects from patches…

Read at arXiv (research papers)

arXiv (research papers) Rui Wang SI

Who Is Your Agent Serving? Provider-Side Indirect Prompt Injection in Proactive Agents

Proactive personal agents increasingly decide what to recommend, how to personalize advice, and what follow-up assistance to offer, creating a new user-decision attack surface for provider-side indirect prompt injection. We show that an external provider need not access private user context, compromise the agent, or gain additional permissions: by controlling only content associated with its own target, it can redirect an otherwise benign agent to advance that target…

Read at arXiv (research papers)

arXiv (research papers) Jingkai Liu SI

Blocking at the Boundary: Auditing Long-Horizon Agents against Staged Prompt Injection

Long-horizon agents consume external content, invoke tools, and modify persistent state. Indirect prompt injection can exploit task-specific context, propagate across causally connected stages, and alter a consequential action while the workflow continues; we term this staged prompt injection. We build an automated, feedback-guided attack generation pipeline and apply it to Claude Code and Codex in their native runtimes. The confirmed attacks span eight workflow scenarios…

Read at arXiv (research papers)

arXiv (research papers) Xinran Zheng SI

APEX: Active Protection at Execution Boundaries for LLM Agents

Indirect prompt injection (IPI) hides adversarial instructions in content that large language model (LLM) agents read at runtime. As agents compose heterogeneous capability units, including Tools, MCP servers, and Skills, the carriers of injection multiply, and defenses built to recognize attack patterns fall behind them. We instead shift defense from covering attack patterns to one stable point: whatever the carrier and however the injection propagates, harm materializes…

Read at arXiv (research papers)

arXiv (research papers) Zixuan Wang SI

Self-Reflection Fine-Tuning: Enhancing Agent Security against Prompt Injection Attacks from Failure Experience

Large language model (LLM) agents are increasingly deployed in tool-augmented environments, but their reliance on external inputs makes them highly vulnerable to prompt injection attacks that can hijack task objectives. Existing safety alignment methods rely on static expert trajectories or preference optimization, limiting their ability to generalize to adaptive attack patterns. In this work, we propose Self-Reflection Fine-Tuning (SRFT), a training framework that enables…

Read at arXiv (research papers)

Hacker News lucknite SI

Shield: A 118M model for detecting prompt injections and jailbreaks

Article URL: https://huggingface.co/ZeroLeaks/shield-small Comments URL: https://news.ycombinator.com/item?id=49940088 Points: 2 # Comments: 0

Read at Hacker News

arXiv (research papers) Xiaolin Zheng SI

Learning Robust Personalized Prompts for LLM-Driven Sequential Recommendation

LLM-driven sequential recommendation formulates next-item prediction as autoregressive generation conditioned on natural-language prompts. However, minor wording changes in semantically equivalent prompts can cause substantial performance fluctuations, undermining robustness and requiring costly manual prompt engineering. Continuous prompt learning reduces template dependence but faces two interacting challenges: shared task-level instructions lack user-specific reasoning…

Read at arXiv (research papers)

arXiv (research papers) Zhuowen Liu SI

Passing the Test You Trained On: Re-evaluating Prompt-Injection Detectors for LLM Agents

LLM agents increasingly screen tool outputs with small prompt-injection detectors, and teams choose among detectors by their scores on public benchmarks. We ask whether those scores predict how a detector behaves inside an agent. We replay the ground-truth tool calls of two agent benchmarks, AgentDojo and tau-bench, without an LLM to obtain tool outputs that are benign by construction, label injected outputs by differential replay, and evaluate fifteen detectors, including…

Read at arXiv (research papers)

Older stories