Results for “attacks”

arXiv (research papers) Mingxiao Liu Hangzhou Dianzi University SI

Piggybacking on Perception: Stealthy Concurrent Audio Prompt Injections against Multimodal LLM Agents

Large Language Model (LLM)-driven multimodal agents are increasingly deployed to execute autonomous tasks via continuous audio interaction. While this paradigm enhances interaction naturalness, it introduces a critical yet under-explored attack surface, as audio inputs inevitably contain environmental noise beyond user control. In this paper, we investigate concurrent audio prompt injection attacks targeting multimodal agents. Distinct from traditional acoustic attacks on…

Read at arXiv (research papers)

Simon Willison SI

AI Worming through Word

AI Worming through Word Neat new prompt injection variant by Håkon Måløy, who found a way to upgrade prompt injection attacks against Microsoft Word to full self-replicating worms: An attacker places hidden instructions in a document that is later used as source material in Copilot for Word. Copilot may interpret those instructions as part of the user’s request, causing it to manipulate the document being drafted or edited. Copilot may then also copy the hidden instructions…

Read at Simon Willison

arXiv (research papers) Eric Wallace SI

GPT-Red: Automated Red Teaming via Self-Play at Scale

We introduce \textbf{GPT-Red}, an automated red-teaming agent that is trained to discover novel prompt injection attacks against frontier LLMs. The goal of this model is to evaluate and improve the robustness of our production systems. To this end, we use it to adversarially train GPT-5.6, our most robust model to prompt injections to date. To create GPT-Red, we design a scalable self-play algorithm where the model is tasked with attacking a diverse population of…

Read at arXiv (research papers)

arXiv (research papers) Arseny Kravchenko SI

APPA: Recoverable Information-Flow Control for Real-World LLM Agents

LLM agents deployed in practical workflows routinely mix private context, untrusted tool and web outputs, and external side effects. While information-flow control (IFC) provides structural defenses against prompt injection, data exfiltration, and confused-deputy attacks, conventional IFC relies on monotone taint tracking that either over-blocks benign operations or permanently strands downstream execution once an agent ingests unvetted data. We present APPA (Agentic…

Read at arXiv (research papers)

Article 19 natashas Geopolitics

Tunisia: Stop gender- and identity-based attacks on women in public spaces

ARTICLE 19 expresses its deep concern over the growing number of smear campaigns, cyber-harassment, hate speech, and judicial proceedings targeting women in Tunisia, particularly those who are visible in public life. For several months now, women who exercise their freedom of expression to voice critical opinions, challenge dominant narratives, or take part in public debate

Read at Article 19

Journal articles (OpenAlex) Thuy Thu Nguyen, Phan Thanh Hoa · Information and Computer Security Geopolitics

Deepfake awareness and privacy protective behaviours: evidence from Vietnamese young faceswap apps users

Purpose Deepfakes are synthetic media (audio recordings, videos or photos) created using artificial intelligence. In recent years, deepfakes have become an increasing concern for internet users; therefore, protecting oneself from cybercrime attacks is vital. The purpose of this study is to use the Technology Acceptance Model and Protection Motivation Theory to build a serial mediation model of the relation between deepfake awareness, perceived security and privacy risks…

Read at Journal articles (OpenAlex)

arXiv (research papers) Yedidel Louck SI

Protocol-Level Attacks on Agentic Commerce Platforms: A Cross-Platform Taxonomy, AIP-Bench, and Unified Defense

Agentic commerce platforms let AI agents autonomously discover services, move payments, and wield user credentials on their users' behalf, and they already handle real money. Their security has so far been studied almost entirely at the level of the AI model, through prompt injection and misalignment. We show that the more consequential risks lie one layer down, in the protocol between agents and commerce services. There, vulnerabilities are structural : exploitation is…

Read at arXiv (research papers)

arXiv (research papers) Or Zion Eliav SI

Know Your Agent: Reconnaissance-Driven Pentesting of AI Agents

Traditional pentesting uses reconnaissance at each step to uncover unseen weaknesses, build stronger attacks, and advance the objective; we argue that AI agents require the same treatment. We formalize agent reconnaissance by modeling the process and identifying the knowledge assets it seeks to extract: what they are, how they are used, and which agent weaknesses they exploit to give adversaries leverage in indirect prompt injection attacks. We instantiate these insights in…

Read at arXiv (research papers)

arXiv (research papers) Zhanhao Hu SI

Twin Agent: Context Residual Compression for Privilege Separated Agents

Large language model (LLM) agents are vulnerable to security risks, such as prompt injection attacks from untrusted context that manipulate downstream reasoning and tool use. Existing secure-by-design approaches mitigate this risk by separating untrusted observations from privileged execution and careful control of information flow, but often degrade utility and require extensive task-specific engineering. We thus propose Twin Agent, a general privilege separation design…

Read at arXiv (research papers)

arXiv (research papers) Akansha Shukla SI

Data Leakage Prevention in Agentic Applications via Preemptive Hardening

Agentic systems integrate LLM driven planning with interfaces to external tools, making data leakage and tool misuse feasible via instruction/data boundary failures and prompt injection attacks. Enforcing required controls consistently is particularly challenging in workflows spanning many codebases and heterogeneous agents. To address this challenge in multi agentic systems, we present a pre-deployment pipeline for scanning, hardening, and validation of agentic…

Read at arXiv (research papers)

The Guardian Cuba Maya Yang Geopolitics

‘Front groups and fellow travelers’: state department attacks US left over Cuba

New report denounces Havana and claims ‘campaign of subversion’ has infiltrated highest levels of US government As part of the Trump administration’s continuing attack on left-leaning groups, the US state department has listed several activists and organizations as “front groups and fellow travelers” for Cuba, including Hasan Piker and the Amazon Labor Union founder Chris Smalls. In a 100-page report released on Monday, the state department denounces Cuba as the “capital of…

Read at The Guardian Cuba

arXiv (research papers) Om Narayan SI

ChainWatch: A Kill Chain-Aligned Sequential Detection Framework for Multi-Step Attacks in MCP-Based AI Agent Systems

The Model Context Protocol (MCP) is an open-source standard that allows AI agents to connect to external tools, databases, and services. While this connectivity enables powerful agent capabilities, it also introduces multi-step attacks that existing per-call defenses cannot reliably detect. Attackers can compose individually benign tool invocations into malicious sequences that evade isolated inspection. This paper presents ChainWatch, a sequential detection framework for…

Read at arXiv (research papers)

arXiv (research papers) Devina Jain SI

Adaptive Adversaries: A Multi-Turn, Multi-LLM Benchmark for LLM Agent Security

LLM-based agents process external content, exposing them to prompt injection and multi-turn manipulation. We present a 21-scenario benchmark for adaptive cross-session attacks against fresh-session LLM defenders: an autonomous LLM attacker observes prior defender responses and pivots across rounds, while each defender response is evaluated as a fresh interaction. A controlled 3 x 3 attacker-defender matrix contains 945 battles. Restricting scoring to the first round yields…

Read at arXiv (research papers)

arXiv (research papers) Yu Cui SI

Refusal is Not Safety! Benchmarking Latent Safety Risks of LLM-Driven Content Humorization

Safety defenses for large language models (LLMs) have been extensively studied, with existing approaches focusing on attack detection and refusal mechanisms. Such fixed-form direct refusal strategies may introduce the risk of prefix injection attacks. Recent work has explored a new direction that leverages humor as an indirect refusal mechanism to mitigate over-refusal in jailbreak scenarios and reduce prefix injection risks. However, this approach implicitly assumes that…

Read at arXiv (research papers)

Article 19 natashas Geopolitics

Kenya: Justice for journalists attacked while covering by-election

ARTICLE 19 Eastern Africa is deeply alarmed by and strongly condemns the violent physical assaults and armed intimidation of journalists covering the Ol Kalou Constituency by-election in Nyandarua County today, 16 July 2026. These coordinated attacks, which also included arbitrary confiscation of media equipment, were reportedly perpetrated by armed, masked men in civilian clothes and

Read at Article 19

arXiv (research papers) Soham Gadgil SI

Bad Memory: Evaluating Prompt Injection Risks from Memory in Agentic Systems

A growing class of agentic systems maintain persistent state across sessions through memory files, behavioral preferences, and knowledge bases. While this makes agents more useful and self-improving, it also creates a new attack surface for prompt injections in which malicious instructions can be embedded within persistent files and influence future behavior. In this work, we study prompt injection attacks in memory-based agentic systems using a sandboxed synthetic…

Read at arXiv (research papers)

arXiv (research papers) Sanket Badhe SI

Agent Skill Security: Threat Models, Attacks, Defenses, and Evaluation

Reusable skills are becoming a fundamental building block of Large Language Model (LLM) agents, enabling capabilities to be packaged, shared, and reused across diverse applications. However, existing security research primarily focuses on prompt injection and runtime execution, leaving security risks throughout the broader skill lifecycle largely unexplored. In this paper, we present SkillSec-Eval, a lifecycle-aware framework for systematically evaluating the security of…

Read at arXiv (research papers)

Simon Willison SI

How I tricked Claude into leaking your deepest, darkest secrets

How I tricked Claude into leaking your deepest, darkest secrets I've been impressed by the way the Claude web_fetch tool is designed to avoid data exfiltration attacks. Ayush Paul found a hole in that design. To recap: regular Claude chat is at risk of lethal trifecta attacks, because it has access to private data (in the form of memories of your past interactions) and has a tool for accessing online content which can both read hostile instructions and exfiltrate data…

Read at Simon Willison

arXiv (research papers) Junhui Wang SI

PVDetector: Detecting Prompt Injection Attacks on Purpose-Specific LLM Agents through Policy-Violation Concept Analysis

Large language models (LLMs) are increasingly deployed as purpose-specific agents to handle domain-specific tasks such as customer service and code generation. These agents are expected to comply with not only generic safety guardrails but also purpose-specific restrictions tailored to their designated roles. Such additional restrictions enlarge the attack surface, particularly to prompt injection (PI) attacks. To defend against such attacks, existing detection methods…

Read at arXiv (research papers)

arXiv (research papers) Nguyen Khoi Vo SI

The Effect of Multi-Lingual and Keyword Adversarial Injection on LLM Relevance Judgment

Large language models (LLMs) are increasingly being used as automated judges for relevance evaluation in information retrieval, yet their robustness to adversarial manipulation remains insufficiently understood, particularly in multilingual settings. In this work, we investigate the impact of cross-lingual prompt injection attacks on LLM-based relevance judgments using TREC Deep Learning collections and two open-weight models under established prompting frameworks. We…

Read at arXiv (research papers)

arXiv (research papers) Aya Spira SI

Beware of Agentic Botnets: Scalable Untargeted Promptware Attacks via Universal and Transferable Adversarial HalluSquatting

The growing adoption of agentic LLM applications has introduced a new threat previously named as promptware. While prior work has established that adversaries can exploit direct channels to LLM applications to apply promptware under weak threat models, many applications do not provide any direct channels that could be exploited for prompt injection beyond the Internet. This raises a question: can attackers exploit LLM applications at scale without any direct channels in…

Read at arXiv (research papers)

arXiv (research papers) Kristina Nikolić SI

Untrusted Content Masking for Web Agents with Security Guarantees

Defenses that provide security guarantees against prompt injection attacks rely on strict isolation between trusted instructions and untrusted data. In text-based environments such as tool-use APIs, this separation arises naturally: agents can reason from interface definitions without ever processing untrusted content. Extending these guarantees to web agents faces a fundamental challenge: to perceive and interact with their environment, web agents must first observe the…

Read at arXiv (research papers)

arXiv (research papers) Woohyuk Choi SI

Agent Data Injection Attacks are Realistic Threats to AI Agents

AI agents act on behalf of user prompts, consuming external data and taking actions based on the agent context. Prior research on AI agent security has primarily focused on indirect prompt injection (IPI). Its most well-studied category is instruction injection, where attacker-controlled untrusted data is interpreted as an instruction. In response, many mitigations have been proposed to prevent instruction injection attacks. In this paper, we introduce a new category of IPI…

Read at arXiv (research papers)

arXiv (research papers) Bogdan Banu SI

Biological Motifs for Agentic Control

The transition of Large Language Models (LLMs) from passive generators to autonomous agents has introduced significant challenges in reliability, security, and state management. Current agentic architectures are often constructed ad-hoc, prone to hallucination cascades, infinite loops, and prompt injection attacks. This paper argues that many of these failure modes can be analyzed using control motifs long studied in systems biology, provided the comparison is made at the…

Read at arXiv (research papers)

arXiv (research papers) Juhee Kim SI

DualView: Preventing Indirect Prompt Injection in Personal AI Agents

Personal AI agents that run on the user's local machine automate daily tasks including web search, email, and file management. Their access to computer resources, including the network, file system, and shell, exposes them to indirect prompt injection (IPI) attacks. Prior Dual LLM defenses block IPI by replacing untrusted data with symbols that the agent can reference but not read. However, they track untrusted data only inside the agent's context, so when the agent saves…

Read at arXiv (research papers)

The Guardian Belarus Stephanie Kirchgaessner Geopolitics

Spyware used against MEP investigating Pegasus abuses, report finds

Researchers say Stelios Kouloglou’s device was compromised after he joined European parliamentary committee NSO Group’s hacking software was repeatedly used against a member of the European parliament while he was conducting an investigation of spyware abuses in Europe, according to a new report. Researchers at the Citizen Lab at the University of Toronto said they could not attribute the attacks against Stelios Kouloglou to any particular government operator of Pegasus…

Read at The Guardian Belarus

Journal articles (OpenAlex) Berkant Akkuş · Frontiers in Political Science Geopolitics

Datafied naval warfare: big data attacks, maritime cyber operations, and the convergence of the law of the sea and international humanitarian law

Introduction Data-driven maritime operations increasingly challenge the legal frameworks governing the use of force at sea. This article examines their legal qualification under International Humanitarian Law, the UN Charter, and UNCLOS, questioning whether existing rules adequately address non-kinetic forms of strategic harm. Methods The study employs doctrinal legal analysis of Article 49 of Additional Protocol I, Article 51 of the UN Charter, the ICJ’s Nicaragua judgment…

Read at Journal articles (OpenAlex)

arXiv (research papers) Josh Hills SI

Distributed Attacks in Persistent-State AI Control

As AI coding agents become more autonomous, they increasingly ship code iteratively, with the codebase persisting across sessions. This persistence creates a new attack surface: a misaligned or prompt-injected agent can distribute attacks across pull requests (PRs) and time its payload for the PR with the best natural cover. To study the resulting dynamics, we introduce Iterative VibeCoding, a setting for AI control, the study of safely deploying capable but potentially…

Read at arXiv (research papers)

arXiv (research papers) Nima Dorzhiev SI

RIPA: Sensory-Vector Prompt Injection Attacks on LLM-Controlled ROS 2 Robots

We present RIPA, the first systematic multi-channel empirical study of prompt injection attacks delivered through the sensory pipeline of a ROS 2-based LLM-controlled robotic system. Across 100 independent runs per injection variant on five LLMs spanning four model families and parameter scales from approximately 4B to approximately 284B (DeepSeek-V4-Flash, Llama-3-8B-Instruct-Lite, Llama-3.3-70B-Instruct-Turbo, Qwen 2.5-7B-Instruct-Turbo, Gemma-3n-E4B), we identify…

Read at arXiv (research papers)

arXiv (research papers) Inderjeet Singh SI

MIRROR: Novelty-Constrained Memory-Guided MCTS Red-Teaming for Agentic RAG

Multimodal agentic retrieval-augmented generation (RAG) systems expand the attack surface beyond prompt injection to include text poisoning, image injection, direct-query attacks, and orchestrator-level tool manipulation. Existing red-teaming approaches are typically surface-specific and often recycle known attack templates; on text-poisoning benchmarks we measure 73-84% exact duplication. We present MIRROR, a unified cross-surface framework that performs memory-guided Monte…

Read at arXiv (research papers)

Older stories