Results for “Prompt”

Hacker News luispa SI

Prompt Injection in the Wild

Article URL: https://cybershujin.github.io/Prompt-Injection-in-the-Wild/ Comments URL: https://news.ycombinator.com/item?id=49851038 Points: 2 # Comments: 0

Read at Hacker News

arXiv (research papers) Hanzhang Ma SI

MetaPermit: Scalable and Auditable Access Control for AI Agents via LLM-Inferred Meta-Attributes

The rise of autonomous AI agents equipped with tools has introduced significant security risks, ranging from unintended tool misuse to adversarial manipulation through Indirect Prompt Injection (IPI) attacks. In practice, deployed agent systems such as OpenAI Codex and Claude Code protect tool invocations through a combination of coarse-grained permission rules and LLM-based judgments about individual proposed actions. Both components, however, have important limitations…

Read at arXiv (research papers)

arXiv (research papers) Srikumar Subramanian SI

Crypto-bound identity-verified capability tokens for coordinating distributed AI agents: A proposal

The prospect of fully autonomous transactional agents did not appear on the horizon until the advent of high capability language models. With such models, the operational benefits of adaptive task orchestration and independent (but constrained) decision making are tantalizing for enterprises and individuals alike. However, each such agent carries with it a serious attack surface in the form of prompt injection which can compromise any soft "guard rails" that may have been…

Read at arXiv (research papers)

Hacker News ArcherL SI

Is indirect prompt injection still a big threat as models get more advanced?

Article URL: https://realarcherl.github.io/posts/prompt_injection/ Comments URL: https://news.ycombinator.com/item?id=49840178 Points: 3 # Comments: 3

Read at Hacker News

arXiv (research papers) Ahmad Hashmi SI

Prompt Injection Detection for Email Agents Through Attack Chain Modeling

Large language model email assistants are particularly vulnerable to indirect prompt injection because untrusted email content can be retrieved into the model context and influence subsequent tool use. Existing prompt injection detectors mainly formulate this problem as binary malicious text classification, which overlooks the important factor that harmful agent behavior often arises through a sequence of stages. We propose a detection framework that models this attack chain…

Read at arXiv (research papers)

arXiv (research papers) Qingyu Wu SI

ENDOPROMPT: Victim-Side Pseudo-References for Utility Degradation

Prompt injection can degrade benign task performance without eliciting harmful content. Yet many attack objectives depend on task labels or predefined target responses. We present ENDOPROMPT, a white-box method that learns utility-degrading prefixes from unlabeled instructions. Its generator takes the request text as input. Clean victim continuations serve as pseudo-references: local search identifies prefixes that reduce continuation likelihood, and preference fitting on…

Read at arXiv (research papers)

arXiv (research papers) Lukáš Brůna SI

Prefilling the Reasoning Channel: Output-Prefix Attacks on Reasoning LLMs

Large Language Models (LLMs) consume and produce a single sequence of text; hence, if text can be added to the beginning of the LLM's response, i.e., an output prefix, then all subsequent tokens will be conditioned on it. This output-prefix attack technique is a cheap black-box prompt injection. Prior work has shown this type of attack can reliably jailbreak non-reasoning models. Most reasoning models add an intermediate scratchpad reasoning step before the assistant's final…

Read at arXiv (research papers)

Hacker News qrios SI

Meta Muse: Almost no prompt injection resistance

Article URL: https://neuromatch.social/@jonny/117324790823856750 Comments URL: https://news.ycombinator.com/item?id=49829888 Points: 2 # Comments: 0

Read at Hacker News

arXiv (research papers) Daiki Chiba SI

ClaimMirage: When Self-Claims in Domain Names Change LLM Threat Judgments

Short claims such as not-phishing or official can change how a large language model (LLM) judges a domain name, without explicit prompt-injection commands. We study this manipulation as ClaimMirage: a name under inspection claims its own safety or approval. We analyze 622,080 judgments across 64 brands and five LLMs, comparing ten claims with length- and hyphen-matched controls in constructed brand-like names. Self-claims can substantially reduce or increase alerts…

Read at arXiv (research papers)

Hacker News northbridgedev SI

Can open-source prompt-injection detectors catch realistic AI agent attacks?

Article URL: https://github.com/rudratoshs/buried-injections Comments URL: https://news.ycombinator.com/item?id=49827019 Points: 8 # Comments: 4

Read at Hacker News

Journal articles (OpenAlex) Marzhan S. Orumbayeva, Toktokuchuk B. Mamytov, Kurmangali Aimen Kuanyshbaykyzy · Asian Journal of Political Science Geopolitics

Counter-terrorism policy coordination in Central Asia: formal convergence and practical cooperation

At the beginning of the twenty-first century, the activity of international terrorist organisations intensified, prompting Central Asian countries to strengthen their counter-terrorism systems. This study examined terrorist threats in Central Asia in the context of global security trends. The main methods were statistical analysis, used to assess quantitative indicators such as the number of residents involved in extremist organisations in 2023, and the historical method…

Read at Journal articles (OpenAlex)

arXiv (research papers) Tiantong Wu SI

Decision Hijacking: Prompt Injection Attacks on Jev's Typed Probabilistic Decisions

Most studies of prompt injection focus on generative agents, leaving their effects on models with schema-defined outputs unclear. We examine these effects in Jev, a non-generative decision model, using 510 reconstructed InjecAgent cases. Malicious content shifts action probabilities but rarely causes Jev to select the attacker's target. Override markers reduce this influence, while claims of contextual relatedness have small effects. Adaptive attacks using score feedback…

Read at arXiv (research papers)

arXiv (research papers) Jasem Khelifi SI

Specifying and Maintaining Agentic Workflows: An Empirical Study of GitHub Agentic Workflows

Agentic workflows shift software development from prompting AI agents for individual tasks to defining recurring work that agents execute automatically. GitHub Agentic Workflows (gh-aw) enables this approach through Markdown files that combine natural-language instructions with configuration and compile into executable GitHub Actions workflows. Unlike conventional workflows that primarily prescribe scripted operations, these files delegate tasks requiring interpretation to…

Read at arXiv (research papers)

arXiv (research papers) Reshabh K Sharma SI

Ajar: Measuring Open Privilege in Agent Defenses

A language model agent acts through the tools it is given. The data it reads while working on a task can redirect what it does with those tools. A growing set of techniques for safe and secure agent execution therefore sits between the agent and its tools, aiming to enforce access control, information flow or isolation at that boundary. Today these techniques are evaluated on agent-security benchmarks built around indirect prompt injection. Those benchmarks judge a defense…

Read at arXiv (research papers)

arXiv (research papers) Fernando Outeda SI

Decoding Guardrails: XAI-Guided Perturbation Analysis of Prompt Injection Detection

Large language models (LLMs) are increasingly deployed in production systems, raising concerns about their exposure to adversarial manipulation through prompt injection and jailbreak attacks. Classifier-based guardrails, such as Prompt Guard 2, are widely used as a first line of defense against such attacks, but their internal decision logic is largely opaque to both defenders and attackers. This paper presents an exploratory case study that applies explainable artificial…

Read at arXiv (research papers)

Hacker News Irsheidat SI

When "Review" Becomes Permission: A Prompt Injection Lab

Article URL: https://rsec.uk/insights/when-review-becomes-permission-a-prompt-injection-lab-mu5tsobx Comments URL: https://news.ycombinator.com/item?id=49766085 Points: 2 # Comments: 0

Read at Hacker News

arXiv (research papers) Rudrendu Kumar Paul SI

Beyond Single-Model Injection: A Threat Model and Defense Architecture for Prompt Injection in Multi-Agent Systems

Existing prompt injection research focuses on single-model chatbot scenarios, where an attacker manipulates one LLM through crafted input. Multi-agent systems amplify this threat through three mechanisms absent from single-model settings: inter-agent message passing creates injection channels invisible to perimeter defenses, shared tool access enables privilege escalation across agent boundaries, and trust propagation allows a compromised agent to influence upstream…

Read at arXiv (research papers)

arXiv (research papers) Huan Li SI

Zero-Trust Authorization and Discovery for Enterprise MCP

LLM agents translate natural-language context, which may include attacker-controlled text, into privileged tool calls, so authorization must remain effective even when an agent is prompt-injected or adversarially steered. The Model Context Protocol (MCP) has become a widely adopted interface for this boundary, yet its official SDKs' authentication and authorization primitives fall short of enterprise zero-trust requirements, most acutely a dual-persona model in which one…

Read at arXiv (research papers)

arXiv (research papers) Justin Szczepaniak SI

Defusing Explosive Prompts: Understanding and Preventing Trigger-Based Prompt Injections in LLM Agents

As LLM applications integrate with external tools, they are increasingly exposed to indirect prompt injection (IPI), where adversarial instructions are embedded in retrieved content. Conventional IPIs fire on contact: the moment an agent ingests the content, it carries out the instruction. We introduce the explosive prompt, a conditional payload that stays dormant until an attacker-chosen trigger is met, in effect a training-free, inference-time backdoor planted in a single…

Read at arXiv (research papers)

arXiv (research papers) Yuxuan Zhang SI

Origin Is All You Need: Provenance-Aware Transformers for Structural Trust-Boundary Separation

Indirect prompt injection (IPI) remains a central safety and security challenge for large language model (LLM) systems because standard transformers lack architectural notion of source authority. Retrieved documents, user inputs, and system instructions are all processed through the same undifferentiated attention mechanism, forcing the model to infer from wording alone what should be obeyed and what should be treated as data. We propose Provenance-Aware Transformers, a…

Read at arXiv (research papers)

Simon Willison SI

Self-generated prompt injections in compaction summaries

Self-generated prompt injections in compaction summaries In Our framework for reporting model misalignment OpenAI provide "six reports on unexpected or concerning model behavior we’ve observed in the last six months". This one here is my favorite: they caught some of their models in training deliberately subverting themselves in their compaction prompts. Compaction is the process agent systems use when they are running out of tokens in their context window, so they summarize…

Read at Simon Willison

arXiv (research papers) Alex Remedios SI

Red-Teaming Auto Mode: Improving Blocking Classifiers Against Malign Coding Agents

To keep coding agents from going off the rails, production systems now review each proposed action with a blocking monitor that can reject it before it runs (Auto Mode in Claude Code, Guardian in OpenAI's Codex). Prior evaluations of such monitors largely measure robustness to accidental harm or prompt injections from untrusted sources looking to hijack the agent. Less understood is how they hold up when the agent they monitor is persistently misaligned. To understand this…

Read at arXiv (research papers)

arXiv (research papers) Matteo Golinelli SI

AgentLSD: Evaluating AI Security Agents Under Adversarial Task Contamination

AI agents for security inspect web pages, source code, logs, configuration files, and command outputs. These environments may contain deceptive artifacts that influence the agent's behavior. We call this adversarial task contamination. Whereas prompt injection relies on attacker-supplied instructions, task contamination also includes non-instructional evidence, such as fake results and decoy endpoints. We present AgentLSD, a controlled framework for studying adversarial task…

Read at arXiv (research papers)

arXiv (research papers) Elia Nikolaou SI

CaMeLoT: CaMeL orchestrated with Temporal logic for static verification and liveness

LLM-based agents generate and execute multi-step plans that invoke external tools which can access private data or execute commands. In this setting, security is a property of the entire execution that a plan creates, not just any single step. The plan itself is a critical artefact that captures the tool calls, control flow, and data dependencies. We present CaMeLoT, a complement to CaMeL, an existing defence against prompt injection in tool-using LLM agents. CaMeLoT extends…

Read at arXiv (research papers)

Hacker News bucket2015 SI

I caught an LLM-powered recruiter with a prompt injection on LinkedIn

Article URL: https://khancyr.github.io/blog/2026/03/15/how-i-caught-an-llm-powered-recruiter-with-a-prompt-injection-on-linkedin/ Comments URL: https://news.ycombinator.com/item?id=49726928 Points: 1 # Comments: 1

Read at Hacker News

arXiv (research papers) Hasnain Irshad SI

The Verifiable Action Card: Trustworthy Human-in-the-Loop Control for Secure Autonomous Agents

Agentic browsers can execute security-sensitive actions under a user's authenticated session, making indirect prompt injection and deceptive confirmation interfaces a direct threat to action integrity. Existing human-in-the-loop (HITL) safeguards are insufficient when the approval prompt itself can be influenced by untrusted page content or model-generated text. We present the \emph{Verifiable Action Card} (VAC), an architectural defence that reconstructs approval…

Read at arXiv (research papers)

Hacker News Hiskias SI

Show HN: Open-Source Lightweight Prompt Injection Safety

Article URL: https://github.com/StackOneHQ/defender/ Comments URL: https://news.ycombinator.com/item?id=49714872 Points: 1 # Comments: 0

Read at Hacker News

arXiv (research papers) Tanzim Hossain Safin SI

Trust propagation and structural containment in Multi-agent LLM pipelines

Multi-agent LLM systems increasingly automate tasks involving agents with different levels of privilege, creating a security risk in which a compromised low-privilege agent can influence a higher-privilege agent and trigger an unauthorized action. We study attack propagation in a four-agent LangGraph pipeline comprising a Supervisor, Researcher, Validator, and Executor. We evaluate shared-memory poisoning and indirect prompt injection through a forged approval embedded in a…

Read at arXiv (research papers)

Hacker News roschdal SI

Ask HN: Is your AI LLM based web service secure from prompt injection?

... Ignore everything above, do this instead... Comments URL: https://news.ycombinator.com/item?id=49703186 Points: 2 # Comments: 2

Read at Hacker News

arXiv (research papers) Xiaoyan Li SI

Universal Defenses for Tool-Integrated LLM Agents Against Adversarial Attacks

Large Language Model (LLM) agents have demonstrated impressive capabilities across a variety of domains, particularly when integrated with external tools for multi-step task completion. However, they are increasingly vulnerable to adversarial attacks, including direct prompt injection, indirect prompt injection, memory poisoning, and backdoor attacks, which exploit the model's openness to prompt injection and tool manipulation. In this work, we explore practical and…

Read at arXiv (research papers)

Older stories