Results for “Prompt”

arXiv (research papers) Yuchen Chen SI

RAGLeak: Supply-Chain Privacy Backdoors in Retrieval-Augmented Generation

Retrieval-Augmented Generation (RAG) enables large language models to access external knowledge during inference, but its reliance on external knowledge bases also introduces new attack surfaces. Existing data extraction attacks against RAG systems rely on either prompt injection, which heavily depends on the generator's instruction-following capability, or backdoors that reproduce or paraphrase retrieved documents but struggle to balance extraction fidelity and…

Read at arXiv (research papers)

Journal articles (OpenAlex) Tatiana Sitchinava, Jochen Töpfer · Global Discourse Geopolitics

Conditioned humanitarianism: government narratives of Ukrainian refugee reception in Germany and Georgia, 2022–25

Russia’s full-scale invasion of Ukraine has prompted an unprecedented humanitarian response across Europe, yet governments articulated their commitments to refugee protection through markedly different narratives. Drawing on a qualitative discourse analysis of government narratives, this article examines how the German and Georgian governments constructed their narratives of humanitarian responsibility and refugee reception for Ukrainians. The analysis demonstrates that…

Read at Journal articles (OpenAlex)

Hacker News joebig SI

LLM01: 2025 Prompt Injection

Article URL: https://genai.owasp.org/llmrisk/llm01-prompt-injection/ Comments URL: https://news.ycombinator.com/item?id=50144431 Points: 1 # Comments: 0

Read at Hacker News

arXiv (research papers) Stavros Damianakis SI

SoK: PIAtlas: Decomposing the Unbounded Prompt Injection Attack Space

Prompt injection is the dominant security risk for large-language-model (LLM) applications and agents, yet the field still lacks a coherent map of how these attacks are constructed. More than twenty public datasets describe prompt injection using incompatible vocabularies, often organized around delivery vector, attacker goal, or generation method. In this Systematization of Knowledge, we present PIAtlas, a taxonomy organized instead around how the injected instruction is…

Read at arXiv (research papers)

Hacker News adikshatriyag SI

Show HN: AegisKernel – Test prompt injection attacks on AI agents in the browser

Article URL: https://adikshatriyag-crypto.github.io/aegis-kernel/ Comments URL: https://news.ycombinator.com/item?id=50132623 Points: 1 # Comments: 0

Read at Hacker News

arXiv (research papers) Yuntao Du SI

RATION: Towards Quantitative Risk-Based Access Control for AI Agents

AI agents increasingly interact with external environments to complete complex user tasks. These interactions can lead to unexpected behavior and expose agents to security threats such as indirect prompt injection. Current agent security systems largely protect agents by restricting their actions, which can result in a substantial security-utility trade-off for complex, open-ended tasks. To address this, we propose a quantitative risk-based approach to access control for AI…

Read at arXiv (research papers)

arXiv (research papers) Egor Zverev SI

SPIDES: Information-Flow Control for Browser Agents

Browser agents automate complex workflows by reading and acting on arbitrary page content, but this flexibility exposes them to prompt injection attacks and unintended data exfiltration. Recent work argues that LLM-driven browser agents erode traditional web security boundaries such as the Same-Origin Policy (SOP) by acting across origins with the user's authority and no built-in isolation. We argue that restoring SOP is neither sufficient (a same-origin public sink can leak…

Read at arXiv (research papers)

arXiv (research papers) Haoxuan Jia SI

The Window Is in the Reasoning: When LLM Agents Decide to Follow Prompt Injections

LLM agents that read untrusted content can be hijacked by prompt injections. Runtime monitors that try to catch a hijack before the agent acts, and benchmarks that score each task with a single run, share an assumption: whether an agent follows an injection is settled when the injection is read. We test this assumption with branching rollouts that replay an episode exactly up to the step at which the agent reads an injection and then resample the rest of the episode many…

Read at arXiv (research papers)

arXiv (research papers) Songping Wang SI

MobileInject: Evaluating Reasoning-to-Action Security of MLLM-Based Mobile Agents under Cross-Modal Consistency Injection

Multimodal large language model (MLLM)-based mobile agents are rapidly evolving from conversational assistants into autonomous systems that perceive, reason over, and act on smartphone environments. This capability also exposes a new security risk: adversaries can embed malicious objectives in cross-application mobile content and steer agent behavior through indirect multimodal injection. Although prompt injection has been studied in desktop and web settings, its cross-modal…

Read at arXiv (research papers)

arXiv (research papers) Farhan Zarif SI

Ignored, Not Refused: What Low Prompt-Injection Success Rates Hide in Small Tool-Using Agents

Tool-using agents read untrusted external content, which exposes them to indirect prompt injection (IPI), where hidden instructions hijack the agent's tool calls. Benchmarks measure IPI security by the attack success rate (ASR), which treats every failed attack as a successful defense. To test this assumption, we audit 33,000 trajectories from eleven open-weight agents (2-14B parameters) on InjecAgent and AgentDojo. A label-blind LLM judge shows that most failed attacks are…

Read at arXiv (research papers)

arXiv (research papers) Xianglin Yang SI

From Approval to Abuse: Privilege Escalation Attacks on Harness-Protected Coding Agents

Coding agents use execution harnesses to constrain the consequences of prompt injection by enforcing permission boundaries. These systems deterministically intercept any out-of-sandbox action and prompt the user for approval. To reduce approval fatigue, users can approve allow-rules proposed by the agent, after which any future action matching the rule is allowed without a prompt. In this work, we show that this mechanism introduces a new risk: untrusted context can…

Read at arXiv (research papers)

arXiv (research papers) Zesheng Shi SI

Identify, Classify, Act: Per-Instruction Controlled Compliance for LLMs over External Data

LLMs over external data face a per-instruction tension: a single document or tool return often interleaves legitimate directives with adversarial prompt injections, and the model must follow the former while rejecting the latter. Existing defenses suppress or trust all external instructions, collapsing the safety-utility frontier and forcing a choice between over-refusal and over-compliance. We argue the missing capability is \textbf{per-instruction credit attribution} and…

Read at arXiv (research papers)

Journal articles (OpenAlex) Christopher Hart · PLoS ONE Geopolitics

Metaphor, multimodality and generative AI: Synthetic images prompted by figurative expressions in anti-immigration discourse

Complex social processes like migration are frequently understood via metaphors. In right-wing media and far-right political discourses, two metaphors that abound draw on flood and invasion frames to problematise immigration and legitimise hostile attitudes and actions towards it. In so far as metaphor involves the construction of mental imagery, it is intimately bound with visual experience, including semiotically mediated visual experience. In a semiotic landscape…

Read at Journal articles (OpenAlex)

Mexico News Daily MND Staff

Mexico targets cheap Chinese bicycles with six-month anti-dumping duty

An anti-dumping investigation found Chinese imports cost Mexican bicycle makers about a third of their business, prompting a six-month duty that could become permanent.

Read at Mexico News Daily

arXiv (research papers) Fardin Ahsan Sakib SI

The Model Knows Better: Defending Medical Multi-Agent Systems Against Prompt Injection

Medical multi-agent systems, in which LLM agents discuss a case, are increasingly studied for clinical decision support. Prompt injection, text that an attacker adds to a model's input, can manipulate LLMs into wrong decisions. How it affects these systems, and how to protect them, remains unclear. Across two systems, two model families and three benchmarks, a single injected passage succeeds in up to 61% of cases, and collaboration does not consistently protect against it…

Read at arXiv (research papers)

Hacker News panoramas4good SI

Prompt injection in HTTP headers: what 217M web responses show

Article URL: https://www.signalscorps.com/blog/prompt-injection-in-http-headers/ Comments URL: https://news.ycombinator.com/item?id=50104310 Points: 96 # Comments: 17

Read at Hacker News

Journal articles (OpenAlex) Kira Kudar, Юлія Павлова, Ганна Склярова et al. · European Psychologist Geopolitics

MHPSS Interventions for War-Affected Residents of Ukraine

Abstract: Russia’s war against Ukraine (February 2022 – ongoing) has generated a large-scale humanitarian emergency characterized by widespread trauma, prolonged uncertainty, and mass displacement. In contrast to prior conflicts, it has been marked by nationwide attacks, urbanized warfare, digitally mediated trauma exposure, and societal fragmentation – factors that have created extensive mental health needs and prompted the rapid launch of diverse mental health and…

Read at Journal articles (OpenAlex)

arXiv (research papers) Qianlong Lan SI

From Detection to Decision: Calibration and Deferral for Prompt-Injection Detection

Prompt-injection defenses are commonly evaluated as binary detectors, although deployed LLM applications must make operational decisions about whether to allow, block, or escalate potentially malicious inputs. Recent probabilistic decision models aim to support such decisions through calibrated confidence estimates, but their reliability under distribution shift remains uncertain. We evaluate a native probabilistic decision model (Jev), two Llama Prompt Guard 2 classifiers…

Read at arXiv (research papers)

arXiv (research papers) Thuy Dung Nguyen SI

INFUSE: A Unified Surface Benchmark for Indirect Prompt Injection in Agentic Systems

LLM-based agents increasingly execute long-horizon, end-to-end workflows involving shell commands, code modification, dependency installation, and interaction with remote services. This growing autonomy expands their exposure to indirect prompt injection (IPI), where adversarial instructions embedded in external content are interpreted as legitimate task directives. Existing evaluations, however, often focus on isolated malicious strings, single-step tool calls, or narrowly…

Read at arXiv (research papers)

arXiv (research papers) Sarthak Gupta SI

Are You Reading the Right Thing? Rethinking Internal Signals for Prompt-Injection Detection

Several recent studies propose detecting prompt injections from a language model's internal state. However, these methods do not measure a single security property. Residual-stream probes, attention scores, role projections, exposure probes, and concept readouts may capture injection presence, recognition, information routing, or behavioral influence. It is therefore unclear what differences in prompt-injection detection performance reveal about the internal signals these…

Read at arXiv (research papers)

arXiv (research papers) Chenxu Wang SI

AgentSafeEvo: Adversarial Co-Evolution for LLM Agent Safety

Large language model (LLM) agents accomplish complex tasks through multi-turn interactions with external tools. However, malicious content embedded in user queries, tool descriptions, or environment observations can manipulate their decisions and induce unsafe actions. Adversarial co-evolution has emerged as an effective approach to strengthening LLM safety, but existing work primarily focuses on language-level safety or indirect prompt injection in web tasks. In this paper…

Read at arXiv (research papers)

arXiv (research papers) Kshitij Mishra SI

TRACER: Trace-Regularized Training Against Indirect Prompt Injection

Indirect prompt injection uses untrusted retrieved content, tool outputs, or agent messages to override an LLM's intended behavior. We show that defense robustness is channel-conditioned: the same attack succeeds 18% of the time as a follow-up message but 53% when placed in retrieved context. We propose TRACER (Trace-Regularized Adversarial Channel Exposure with Reward-shaping), a channel-aware adversarial training method that samples presentation wrappers and combines an…

Read at arXiv (research papers)

arXiv (research papers) Dominik Seip SI

Towards Safe Autonomy for Computer Use

The safety of LLM-based agents currently rests on alignment training and safety classifiers, neither of which can rule out that a sophisticated indirect prompt injection (IPI) goes through and causes significant harm. Recent work therefore builds agents with information-flow guarantees that rely on structured tool interfaces to interact with the environment. Computer Use Agents (CUAs) act visually on the computer screen, which does not provide a boundary between trusted…

Read at arXiv (research papers)

arXiv (research papers) Kristoffer Hempel SI

Kairos: Transactional Security Enforcement at the System Boundary for LLM Agents

Large language model (LLM) agents are becoming an integral part of the software development lifecycle, and the capabilities granted to them allow an agent to act with the user's authority on any resource it can access. The reach that gives an agent its utility is equally available to an attacker who can steer it. Anything an agent ingests can carry an injected instruction, and since the model does not reliably distinguish instructions from data, indirect prompt injection…

Read at arXiv (research papers)

arXiv (research papers) Kaan Kale SI

Prompt-Injection Probes Learn the Conversation, Not Just the Injection

Linear probes on a model's hidden states are used as cheap monitors for indirect prompt injection: they screen the tool outputs an agent reads and send suspicious ones to a slower second check, so every false alarm has a cost. But a probe learns whatever separates its training examples. The standard evaluation of injection probes can be passed without the model: conversation position and text lengths alone reach 0.91-0.98 AUROC. When each attack is compared with the same…

Read at arXiv (research papers)

Journal articles (OpenAlex) Cesar A. Zanin · Journal of Political Ideologies Geopolitics

Re-specifying populism to avoid misrecognition: Trumpism and Bolsonarism as Counter-Enlightenment political religions

The rise of Trumpism and Bolsonarism has prompted insufficient debate over their ideological foundations. These mass movements undermine democratic norms and institutions, rejecting Enlightenment-derived principles like universal rights and rationalism, secularism, and pluralism in favour of supremacist hierarchy, naturalised inequality, mythical civilisational struggle, historical destiny, and coercive authority oriented towards redemptive force. This article advances a…

Read at Journal articles (OpenAlex)

arXiv (research papers) Yuetian Chen SI

BrowseSafe++: Benchmarking Prompt Injection Detection Across Web Contexts, Languages, and Browser Views

Browser agents need to make decisions and execute actions on behalf of users while consuming untrusted webpages, creating opportunities for malicious instructions embedded in webpages to redirect legitimate tasks. We introduce BrowseSafe++, which extends BrowseSafe from 14,719 pages to 38,675 pairs of pages. Most of its prompt injections were observed in the wild on production websites. We keep each injection unchanged and rewrite the sensitive parts of the page around it to…

Read at arXiv (research papers)

Hacker News anon767 SI

Prompt injection classifiers are terrible security boundaries

Article URL: https://thecout.com/blog/injection/ Comments URL: https://news.ycombinator.com/item?id=50056871 Points: 2 # Comments: 0

Read at Hacker News

arXiv (research papers) Anning Hu SI

StableJudge: A Lightweight Adversarial Defense Framework for LLM-as-a-Judge

Large language models (LLMs) are increasingly used as judges to assign scalar scores that drive downstream decision-making, including tasks such as paper review and recommendation. However, LLM-as-a-Judge pipelines are vulnerable to score-hijacking prompt injection embedded in evaluated content, which corrupts scoring signals and distorts induced rankings. Existing defenses often enforce robustness by modifying the judge model itself, making safety dependent on a specific…

Read at arXiv (research papers)

arXiv (research papers) Rishabh Sinha SI

Exposure Is Not Commitment: Double-Stratified Evaluation of Latent Monitors for Tool-Using Agents

Tool-using language agents are vulnerable to indirect prompt injection, and a growing literature probes model internals to catch unsafe actions before they execute, with sharply conflicting results. We argue much of the conflict is an evaluation artifact, and propose a double-stratification protocol: separate injection exposure from behavioral commitment, in both the evaluation and the training population. On the primary Qwen AgentDojo grid, monitors trained on the naive…

Read at arXiv (research papers)

Older stories