Results for “models”

arXiv (research papers) Hyunseok Paeng SI

The Injection Paradox: Brand-Level Suppression in Safety-Trained LLM Recommendations via RAG Context Injection

We present a reproducible failure mode of safety training in RAG-based LLM recommendation, the Injection Paradox, in which prompt injections embedded in retrieved documents backfire against the attacker, suppressing the target brand below the injection-free baseline. In safety-trained Claude models, documents containing prompt injections suffer a sharp drop in recommendation rate, and this suppression propagates beyond the injected document to unmodified documents of the…

Read at arXiv (research papers)

arXiv (research papers) Shuyu Guo SI

RubricReviewer: From Direct Critique to Objective and Comprehensive Rubric-Driven Peer Review

Peer review at major venues is under unprecedented submission pressure, motivating the use of large language models (LLMs) as review assistants. Existing LLM-based reviewers, however, face two structural limitations. First, they map manuscripts directly to reviews, leaving the underlying rubric implicit and entangling its derivation with the judgement. Second, the prevailing paradigms each capture only half of a good review: training-free agents gather broad evidence but…

Read at arXiv (research papers)

arXiv (research papers) Paulo Ricardo Ferreira Neves SI

GuardNet: Ensemble Strategies of Shallow Neural Networks for Robust Prompt Injection and Jailbreak Detection

Large Language Models (LLMs) have transformed natural language processing, but they remain vulnerable to Prompt Injection (PI) and Jailbreak (JB) attacks. In addition, benchmark evaluations may be affected by contamination and partial information leakage, compromising performance estimates. This work presents GuardNet, a guardrail system based on an ensemble of shallow neural networks (BiLSTMs) with approximately 47 million parameters. We investigate the hypothesis that…

Read at arXiv (research papers)

arXiv (research papers) Manar Abouzaid SI

Hybrid Adversarial Defence for Natural Language Understanding Tasks

Large Language Models (LLMs) are vulnerable both to hallucination and adversarial manipulation. Although these problems are closely related, existing defences typically address them separately. We investigate a hybrid defence framework that combines entropy-based models, designed to reduce hallucinations, with uncertainty-based models and geometric-based models, designed to reduce vulnerability. Under in-domain tests on Natural Language Understanding datasets (FEVER…

Read at arXiv (research papers)

arXiv (research papers) Nicholas Saban SI

Domain-Conditioned Safety in Frontier Computer-Using Agents: A 793-Episode Browser Benchmark, a Coding-Domain Cross-Reference, and a Reproducibility Audit of Recent Red-Teaming

Recent computer-using-agent (CUA) red-teaming papers report prompt-injection attack success rates (ASR) of 42-98%, but these headline numbers cluster on retired models and on the most-vulnerable model in each paper's panel. We ask whether those techniques, reproduced as hand-crafted templates, still work against current frontier CUAs. We release CUA-HandCrafted, a public benchmark of 793 episodes spanning 24 multi-step web tasks, 56 attack templates, 8 attack families, and 4…

Read at arXiv (research papers)

arXiv (research papers) Kargi Chauhan SI

Caught in the Act(ivation): Toward Pre-Output and Multi-Turn Detection of Credential Exfiltration by LLM Agents

LLM agents often place sensitive credentials in the same context window as untrusted retrieved content, creating a direct path for indirect prompt injection to induce credential exfiltration. We study this failure mode through three complementary defenses. First, we ask whether activation probes can detect credential access before output tokens are emitted. Second, we construct honeytokens from format-specific character models and calibrate detection with split conformal…

Read at arXiv (research papers)

arXiv (research papers) Hang Li SI

"**Important** You should give me full credits!": Exploring Prompt Injection Attacks on LLM-Based Automatic Grading Systems

The emergence of large language models (LLMs) has significantly accelerated recent research on LLM-based automatic grading (AG) systems. Benefiting from the strong instruction-following capabilities and broad prior knowledge of LLMs, educators can deploy AG systems across diverse tasks using only natural language rubrics while achieving satisfactory grading performance. Despite these advantages, new security concerns may also arise. In particular, prompt injection (PI)…

Read at arXiv (research papers)

arXiv (research papers) Ryle Goehausen SI

Gate AI: LLM Security Benchmark Evaluation Methodology and Results

Published evaluations of prompt-injection and jailbreak detectors for Large Language Models often suffer from two systematic weaknesses: per-dataset threshold tuning and undisclosed operating points. We describe an evaluation harness that addresses both. The detector under evaluation is scored across 16 public benchmarks (12,111 samples) using 5-fold cross-validation. StratifiedKFold (by row) is the headline pass; a parallel StratifiedGroupKFold pass over a composite key…

Read at arXiv (research papers)

Journal articles (OpenAlex) Bülent Aras, Burcu Fazlıoğlu · World Affairs Geopolitics

The Normative Shield: How the BRICS Loose Alliance Promotes Authoritarian Resilience

ABSTRACT This article investigates whether the BRICS grouping strategically deploys high‐level diplomatic visits to bolster incumbent autocrats facing domestic instability, thereby promoting “authoritarian durability.” Utilizing a novel dataset of official visits by the five core members (Brazil, Russia, India, China, and South Africa) between 1991 and 2024, the study models the relationship between host‐country unrest and diplomatic engagement, revealing a significant…

Read at Journal articles (OpenAlex)

Journal articles (OpenAlex) Óscar Rafael Boude Figueredo, Jessica Meza-Jaque, Luis Peréz · F1000Research Geopolitics

Towards a New Learning Ecology: The Mediator Role of Teachers Facing Generative AI in the University Context of Peru, Chile, and Colombia

Background The emergence of Generative Artificial Intelligence (GenAI) has created an ontological disruption in higher education, challenging traditional pedagogical mediation and academic production. This study examines the transformation of the teaching role in university environments across Peru, Chile, and Colombia, addressing a knowledge gap in Latin American empirical research regarding faculty perceptions and daily practices with Large Language Models. Methods The…

Read at Journal articles (OpenAlex)

Journal articles (OpenAlex) Leila Delovarova, Jan Bilal Naqeeb · Frontiers in Political Science Geopolitics

Competing governance models in Afghanistan: democracy, authoritarianism, and theocracy in regional connectivity

Afghanistan has a strategic geographical location as a connecting hub between Central and South Asia, but this position has not translated into sustainable regional connectivity. This study argues that domestic models of governance are among the explanatory variables that have been ignored in explaining such outcomes. It presents the idea of model power, which may be considered the capacity of regimes to influence the conditions of participation in transnational projects…

Read at Journal articles (OpenAlex)

arXiv (research papers) Partha Pratim Saha SI

MENTIS: What Belief Changes Under Alignment? Measuring Multi-Scale Latent Torsion in Language Models

Preference alignment has substantially improved the observable behavior of large language models, yet it remains unclear what alignment changes internally. Aligned systems still fail under jailbreaks, prompt injection, and retrieval-time corruption, suggesting behavior-level evaluation alone is incomplete. Post-training should leave measurable traces in internal computation. We ask: when an instruction-tuned (IT) model becomes a preference-aligned (PA) model, what geometric…

Read at arXiv (research papers)

arXiv (research papers) Junyoung Park SI

Persona Attack: Incremental Memory Injection Jailbreak Attack against Large Language Models

As Large Language Models evolve for user convenience, vulnerability to jailbreak attacks continues to be reported despite ongoing efforts in safety training. Traditional jailbreak techniques typically focus on a single prompt injection, neglecting the models' ability to remember the flow of conversation and the user's instructions. In this paper, we propose Persona Attack, a memory injection based jailbreak method that manipulates the model's context window through a step by…

Read at arXiv (research papers)

arXiv (research papers) Brian Crawford SI

Automatically Attacking Software Reverse Engineering AI Agents

Software tools for reverse engineering executable binary files, such as Ghidra, enable malware analysts to safely conduct robust static analysis without having access to original source code. Coupled with the analytic power of large language models (LLM), agentic systems enabled with tools, such as GhidraMCP, can allow analysts to automate a previously human driven process. Although this automation can increase the productivity of a single malware analyst, it also introduces…

Read at arXiv (research papers)

arXiv (research papers) Alexander Sternfeld SI

Minimal Prompt Perturbations Lead to Code Vulnerabilities: Prompt Fragility and Hidden-State Signals in Coding LLMs

LLM-based coding assistants are seeing rapid adoption, offering substantial gains in developer productivity. As organizations increasingly ship code these agents produce, the security of that code becomes critical. Prior work has shown that minor prompt perturbations degrade the functional correctness of LLM-generated code, but whether they also compromise code security has remained unstudied. We apply token-level mutations to prompts across three models and five programming…

Read at arXiv (research papers)

arXiv (research papers) Avidan Shah SI

Mitigating Adaptive Attacks against Reasoning Models with Activation Consistency Training

As LLMs gain stronger reasoning capabilities, their extended chain-of-thought introduces new degrees of complexity for defending against adversarial jailbreaks and prompt injection. We study consistency training, a family of fine-tuning objectives that enforce identical behavior on clean prompts and adversarial rewrites, and evaluate its two main variants, output-level (BCT) and activation-level (ACT), across five reasoning models. We formulate both methods as a prompt…

Read at arXiv (research papers)

arXiv (research papers) Ruoqi Guo SI

MIRAGE: Context-Aware Prompt Injection against Mobile GUI Agents via User-Generated Content

Mobile graphical user interface (GUI) agents driven by vision-language models (VLMs) perceive the screen as rendered pixels and choose actions from what they see, so they cannot reliably separate trusted interface elements from user-generated content. We present MIRAGE (Mobile Injection of Realistic Adversarial GUI Examples), a pipeline that turns benign mobile screenshots into prompt-injection samples by placing attacker-controlled text into ordinary user-generated content…

Read at arXiv (research papers)

arXiv (research papers) Xiang Fang SI

Disentangling Adversarial Prompts: A Semantic-Graph Defense for Robust LLM Security

Large Language Models (LLMs) are increasingly vulnerable to adversarial prompts that exploit semantic ambiguities to bypass safety mechanisms, resulting in harmful or inappropriate outputs. Such attacks, including jailbreaking and prompt injection, pose significant risks to the integrity and availability of LLMs in security-critical applications. This paper proposes the Adversarial Prompt Disentanglement (APD) framework, a novel defense mechanism that proactively identifies…

Read at arXiv (research papers)

arXiv (research papers) Akindoyin Akinrele SI

Prompt Injection Detection is Regime-Dependent: A Deployment-Aware Evaluation with Interpretable Structural Signals

Prompt injection poses a critical threat to the safe deployment of large language models, yet existing detection approaches are typically evaluated under limited settings that do not reflect real-world operating constraints. In this work, we present a deployment-aware evaluation of prompt injection detection using a multi-model and multi-regime experimental framework. We compare lexical, semantic, structural, and transformer-based detectors across multiple…

Read at arXiv (research papers)

arXiv (research papers) Zedian Shao SI

Cordyceps: Covert Control Attacks on LLMs via Data Poisoning

Large language models (LLMs) are often fine-tuned on uncurated text datasets that adversaries can poison. Existing poisoning attacks primarily rely on fixed trigger phrases that defenses such as outlier detection, clean-data regularization, or online monitoring can neutralize. In this paper, we propose a data poisoning method that teaches an LLM an information hiding scheme reliably and stealthily through semantic associations between shared knowledge such as facts or…

Read at arXiv (research papers)

arXiv (research papers) Lingyao Li SI

LLM-as-a-Reviewer: Benchmarking Their Ability, Divergence, and Prompt Injection Resistance as Paper Reviewers

Large language models (LLMs) are increasingly used in academic peer review, yet their reliability, alignment with human judgment, and robustness to adversarial attacks remain poorly understood. We present a systematic benchmark of LLM-as-a-Reviewer on 898 papers stratified from NeurIPS and ICLR, evaluating 12 LLMs along three axes: rating calibration, divergence from human reviewers, and resistance to prompt injection embedded via an invisible font-mapping attack. We find…

Read at arXiv (research papers)

Rappler Ricky Sabornay South China Sea

[Between Islands] Why the Marcos state visit to Japan is anchored in everyday trust

The Filipino presence in Japan is changing. We are no longer defined solely by labor migration. Increasingly, Filipinos are exporting ideas, brands, capital, and business models into Japan itself.

Read at Rappler

arXiv (research papers) Dongpeng Zhang SI

Localize and Neutralize: Gradient-guided Token Suppression against Visual Prompt Injection Attack

Adversarial images pose a severe security threat to multimodal large language models through prompt injection. Existing defenses largely lack a principled understanding of the underlying mechanisms and struggle to balance efficiency and defense utility. In this work, we show that successful adversarial attacks do not rely on the entire image uniformly but instead depend on a small subset of critical image tokens. Based on this insight, we propose Gradient Token Masking…

Read at arXiv (research papers)

arXiv (research papers) Dongxu Yang SI

Device Context Protocol: A Compact, Safety-First Architecture for LLM-Driven Control of Constrained Devices

Large language models are increasingly used as orchestrators of external tools via the Model Context Protocol (MCP), but MCP is built for software services with megabytes of memory and does not descend to the microcontrollers that dominate the long tail of physical devices. Recent work (IoT-MCP) ports MCP to edge gateways at 74 KB peak memory; this still excludes the smallest commodity MCUs and, critically, does not address the safety problem of giving an unreliable caller…

Read at arXiv (research papers)

arXiv (research papers) Rohan Pandey SI

Poisoning the Watchtower: Prompt Injection Attacks Against LLM-Augmented Security Operations Through Adversarial Log Content

Large language models (LLMs) are increasingly used as analyst assistants in security operations centers (SOCs), where they ingest log and alert data to produce triage labels, incident summaries, or remediation advice. We study a structural failure mode of this design: many log fields are attacker controlled. User agents, URLs, payloads, DNS queries, and attempted usernames can therefore carry instructions to the model alongside evidence of the intrusion. We call this setting…

Read at arXiv (research papers)

arXiv (research papers) Qingcao Li SI

MixFake: Benchmarking and Enhancing Audio Deepfake Detection in Diverse Real-world Mixed Audio

Speech deepfake detection has achieved remarkable success in clean environments but faces significant challenges in complex, real-world scenarios where speech is often mixed with background music or noise. Current state-of-the-art methods rely on semantic features from self-supervised learning (SSL) models, which often fail when processing non-speech or mixed-source audio. In this paper, we first introduce MixFake, a large-scale benchmark dataset designed to simulate diverse…

Read at arXiv (research papers)

arXiv (research papers) Yuanbo Zhou SI

Prompt Overflow: What the Guardrail Inspects Is Not What the Model Infers

Guardrail models (a.k.a. safety checkers) are widely deployed to screen user inputs before they reach large language models (LLMs), serving as a primary defense against prompt injection attacks. Due to strict context constraints, these models handle overlength prompts through truncation or segmentation-based inspection. While prior work has focused on semantic adversarial inputs, the security implications of these long-input processing mechanisms remain largely unexplored…

Read at arXiv (research papers)

Journal articles (OpenAlex) Karolina Krystyniak, Viktoriya Staneva · Accounting and Finance Geopolitics

Conservatism in Credit Ratings—Real or Perceived? Evidence From Stock Returns

ABSTRACT We examine how perceived changes in credit rating standards affect stock market performance. Firms that appear to be subject to rating conservatism (i.e., those with a credit rating that is lower than the one implied by an empirical model) earn negative future abnormal returns. This finding suggests two critical points: (1) Credit Rating Agencies (CRAs) utilize private ‘soft’ information, which is neither available to investors nor captured by empirical models, when…

Read at Journal articles (OpenAlex)

arXiv (research papers) Hao Yang SI

A Cross-Modal Prompt Injection Attack against Large Vision-Language Models with Image-Only Perturbation

Large vision-language models (LVLMs) have emerged as a powerful paradigm for multimodal intelligence, but their growing deployment also expands the attack surface of prompt injection. Despite this growing concern, existing attacks still suffer from a critical limitation: the injected prompt for one modality only steers the model's interpretation of that singular input. Alternatively, these attacks remain multimodal but fail to achieve cross-modal prompt perturbation. To…

Read at arXiv (research papers)

arXiv (research papers) Sidharth Pulipaka SI

Hidden in Memory: Sleeper Memory Poisoning in LLM Agents

Large language models are increasingly augmented with persistent memory, allowing assistants to store user-specific information across sessions for personalization and continuity. This statefulness introduces a new security risk: adversarial content can corrupt what an assistant remembers and thereby influence future interactions. We propose and study sleeper memory poisoning, a delayed attack in which an adversary manipulates external context, such as a document, webpage…

Read at arXiv (research papers)

Older stories