Results for “model”

arXiv (research papers) Yinan Zhong SI

Attention is All You Need to Defend Against Indirect Prompt Injection Attacks in LLMs

Large Language Models (LLMs) have been integrated into many applications (e.g., web agents) to perform more sophisticated tasks. However, LLM-empowered applications are vulnerable to Indirect Prompt Injection (IPI) attacks, where instructions are injected via untrustworthy external data sources. This paper presents Rennervate, a defense framework to detect and prevent IPI attacks. Rennervate leverages attention features to detect the covert injection at a fine-grained token…

Read at arXiv (research papers)

arXiv (research papers) Shiva Gaire SI

Systematization of Knowledge: Security and Safety in the Model Context Protocol Ecosystem

The Model Context Protocol (MCP) has emerged as the de facto standard for connecting Large Language Models (LLMs) to external data and tools, effectively functioning as the "USB-C for Agentic AI." While this decoupling of context and execution solves critical interoperability challenges, it introduces a profound new threat landscape where the boundary between epistemic errors (hallucinations) and security breaches (unauthorized actions) dissolves. This Systematization of…

Read at arXiv (research papers)

arXiv (research papers) Guanquan Shi SI

SoK: Trust-Authorization Mismatch in LLM Agent Interactions

Large Language Models (LLMs) are evolving into autonomous agents capable of executing complex workflows via standardized protocols (e.g., MCP). However, this paradigm shifts control from deterministic code to probabilistic inference, creating a fundamental Trust-Authorization Mismatch: static permissions are structurally decoupled from the agent's fluctuating runtime trustworthiness. In this Systematization of Knowledge (SoK), we survey more than 200 representative papers to…

Read at arXiv (research papers)

arXiv (research papers) Zhibo Liang SI

SIEVE: Selective Integrity Verification and Escalation for Defending LLM Agents against Indirect Prompt Injection

Large Language Models (LLMs) are increasingly used as the core of agentic systems due to their strong reasoning, planning, and tool-use capabilities. By interacting with external environments, LLM agents can execute real-world tasks on behalf of users rather than merely generate text. This expanded capability also amplifies the threat of indirect prompt injection (IPI), where malicious external content can manipulate agent behavior and trigger unauthorized actions, privacy…

Read at arXiv (research papers)

arXiv (research papers) Saeid Jamshidi SI

Semantic Attacks on Tool-Augmented LLMs: Securing the Model Context Protocol Against Descriptor-Level Manipulation

The Model Context Protocol (MCP) enables Large Language Models (LLMs) to interact with external tools via tool descriptors, thereby extending their capabilities for task execution, autonomous decision-making, and multi-agent coordination. Existing MCP deployments treat tool descriptors as trusted metadata, despite their direct integration into the LLM reasoning context. This introduces a previously underexplored semantic attack surface. Current defenses primarily target…

Read at arXiv (research papers)

arXiv (research papers) Weikai Lu SI

ARGUS: Defending Against Multimodal Indirect Prompt Injection via Steering Instruction-Following Behavior

Multimodal Large Language Models (MLLMs) are increasingly vulnerable to multimodal Indirect Prompt Injection (IPI) attacks, which embed malicious instructions in images, videos, or audio to hijack model behavior. Existing defenses, designed primarily for text-only LLMs, are unsuitable for countering these multimodal threats, as they are easily bypassed, modality-dependent, or generalize poorly. Inspired by activation steering researches, we hypothesize that a robust, general…

Read at arXiv (research papers)

arXiv (research papers) M Zeeshan SI

Chameleon: Adaptive Adversarial Agents for Scaling-Based Visual Prompt Injection in Multimodal AI Systems

Multimodal Artificial Intelligence (AI) systems, particularly Vision-Language Models (VLMs), have become integral to critical applications ranging from autonomous decision-making to automated document processing. As these systems scale, they rely heavily on preprocessing pipelines to handle diverse inputs efficiently. However, this dependency on standard preprocessing operations, specifically image downscaling, creates a significant yet often overlooked security…

Read at arXiv (research papers)

arXiv (research papers) Eranga Bandara SI

ASTRIDE: A Security Threat Modeling Platform for Agentic-AI Applications

AI agent-based systems are becoming increasingly integral to modern software architectures, enabling autonomous decision-making, dynamic task execution, and multimodal interactions through large language models (LLMs). However, these systems introduce novel and evolving security challenges, including prompt injection attacks, context poisoning, model manipulation, and opaque agent-to-agent communication, that are not effectively captured by traditional threat modeling…

Read at arXiv (research papers)

arXiv (research papers) Chenlin Xu SI

Boundary-Aware Test-Time Adaptation for Zero-Shot Medical Image Segmentation

Due to the scarcity of annotated data and the substantial computational costs of model, conventional tuning methods in medical image segmentation face critical challenges. Current approaches to adapting pretrained models, including full-parameter and parameter-efficient fine-tuning, still rely heavily on task-specific training on downstream tasks. Therefore, zero-shot segmentation has gained increasing attention, especially with foundation models such as SAM demonstrating…

Read at arXiv (research papers)

Simon Willison SI

Claude 4.5 Opus' Soul Document

Claude 4.5 Opus' Soul Document Richard Weiss managed to get Claude 4.5 Opus to spit out this 14,000 token document which Claude called the "Soul overview". Richard says: While extracting Claude 4.5 Opus' system message on its release date, as one does, I noticed an interesting particularity. I'm used to models, starting with Claude 4, to hallucinate sections in the beginning of their system message, but Claude 4.5 Opus in various cases included a supposed "soul_overview"…

Read at Simon Willison

arXiv (research papers) Omar Farooq Khan Suri SI

Securing Large Language Models (LLMs) from Prompt Injection Attacks

Large Language Models (LLMs) are increasingly being deployed in real-world applications, but their flexibility exposes them to prompt injection attacks. These attacks leverage the model's instruction-following ability to make it perform malicious tasks. Recent work has proposed JATMO, a task-specific fine-tuning approach that trains non-instruction-tuned base models to perform a single function, thereby reducing susceptibility to adversarial instructions. In this study, we…

Read at arXiv (research papers)

arXiv (research papers) Mintong Kang SI

Mitigating Indirect Prompt Injection via Instruction-Following Intent Analysis

Indirect prompt injection attacks (IPIAs), where large language models (LLMs) follow malicious instructions hidden in input data, pose a critical threat to LLM-powered agents. In this paper, we present IntentGuard, a general defense framework based on instruction-following intent analysis. The key insight of IntentGuard is that the decisive factor in IPIAs is not the presence of malicious text, but whether the LLM intends to follow instructions from untrusted data. Building…

Read at arXiv (research papers)

arXiv (research papers) Neemesh Yadav SI

Are LLMs Good Safety Agents or a Propaganda Engine?

Large Language Models (LLMs) are trained to refuse to respond to harmful content. However, systematic analyses of whether this behavior is truly a reflection of its safety policies or an indication of political censorship, that is practiced globally by countries, is lacking. Differentiating between safety influenced refusals or politically motivated censorship is hard and unclear. For this purpose we introduce PSP, a dataset built specifically to probe the refusal behaviors…

Read at arXiv (research papers)

Journal articles (OpenAlex) Jaime Cubides Cárdenas, Christian Esteban Hernández-Alvarado, Jonnathan Jiménez Reina et al. · Frontiers in Political Science Geopolitics

Regional cooperation against transnational organized crime: challenges and opportunities in the Western Hemisphere

Transnational Organized Crime represents a critical threat to the political, economic, and social stability of the Western Hemisphere. This article analyses the dynamics of the criminal organizations that constitute this phenomenon, including their illicit activities such as drug, human, and arms trafficking, as well as their relationship with migratory flows and advanced technologies. Using a qualitative methodology based on the multidimensional security model, it evaluates…

Read at Journal articles (OpenAlex)

arXiv (research papers) Kaiyuan Zhang SI

BrowseSafe: Understanding and Preventing Prompt Injection Within AI Browser Agents

The integration of artificial intelligence (AI) agents into web browsers introduces security challenges that go beyond traditional web application threat models. Prior work has identified prompt injection as a new attack vector for web agents, yet the resulting impact within real-world environments remains insufficiently understood. In this work, we examine the landscape of prompt injection attacks and synthesize a benchmark of attacks embedded in realistic HTML payloads…

Read at arXiv (research papers)

Journal articles (OpenAlex) Zhihan Wei, Gi-Young Chung · Modern Economics & Management Forum Geopolitics

Analysis of the Economic Impact of Dongxing Port on the China-Vietnam Border Region

This study examines the Dongxing Port on the China-Vietnam border, focusing on analyzing the impact of the "border residents' mutual trade + on-site processing" model on regional economic development. Analysis reveals that this model, leveraging policy innovations and the enabling role of digitalization, has significantly increased border residents' income levels. It has also facilitated the formation of distinctive industrial clusters, driving regional economic growth…

Read at Journal articles (OpenAlex)

arXiv (research papers) Steven Peh SI

Prompt Fencing: A Cryptographic Approach to Establishing Security Boundaries in Large Language Model Prompts

Large Language Models (LLMs) remain vulnerable to prompt injection attacks, representing the most significant security threat in production deployments. We present Prompt Fencing, a novel architectural approach that applies cryptographic authentication and data architecture principles to establish explicit security boundaries within LLM prompts. Our approach decorates prompt segments with cryptographically signed metadata including trust ratings and content types, enabling…

Read at arXiv (research papers)

Simon Willison SI

Claude Opus 4.5, and why evaluating new LLMs is increasingly difficult

Anthropic released Claude Opus 4.5 this morning, which they call "best model in the world for coding, agents, and computer use". This is their attempt to retake the crown for best coding model after significant challenges from OpenAI's GPT-5.1-Codex-Max and Google's Gemini 3, both released within the past week! The core characteristics of Opus 4.5 are a 200,000 token context (same as Sonnet), 64,000 token output limit (also the same as Sonnet), and a March 2025 "reliable…

Read at Simon Willison

arXiv (research papers) Yanxi Li SI

Semantics as a Shield: Label Disguise Defense (LDD) against Prompt Injection in LLM Sentiment Classification

Prompt injection can cause large language models (LLMs) to deviate from their intended tasks by following instructions embedded in untrusted input. We investigate whether changing a classifier's output labels improves accuracy under such attacks, using sentiment classification as a case study. Label Disguise Defense (LDD) replaces positive/negative with alternatives such as good/bad or cat/dog, introduced through demonstrations, and maps predictions back to sentiment…

Read at arXiv (research papers)

arXiv (research papers) Qingsong He SI

Z-Space: A Multi-Agent Tool Orchestration Framework for Enterprise-Grade LLM Automation

Large Language Models can break through knowledge and timeliness limitations by invoking external tools within the Model Context Protocol framework to achieve automated execution of complex tasks. However, with the rapid growth of enterprise-scale MCP services, efficiently and accurately matching target functionalities among thousands of heterogeneous tools has become a core challenge restricting system practicality. Existing approaches generally rely on full-prompt…

Read at arXiv (research papers)

arXiv (research papers) Aram Vardanyan SI

Building Browser Agents: Architecture, Security, and Practical Solutions

Browser agents enable autonomous web interaction but face critical reliability and security challenges in production. This paper presents findings from building and operating a production browser agent. The analysis examines where current approaches fail and what prevents safe autonomous operation. The fundamental insight: model capability does not limit agent performance; architectural decisions determine success or failure. Security analysis of real-world incidents reveals…

Read at arXiv (research papers)