Results for “models”

arXiv (research papers) Xinyi Wu SI

When Bots Take the Bait: Exposing and Mitigating the Emerging Social Engineering Attack in Web Automation Agent

Web agents, powered by large language models (LLMs), are increasingly deployed to automate complex web interactions. The rise of open-source frameworks (e.g., Browser Use, Skyvern-AI) has accelerated adoption, but also broadened the attack surface. While prior research has focused on model threats such as prompt injection and backdoors, the risks of social engineering remain largely unexplored. We present the first systematic study of social engineering attacks against web…

Read at arXiv (research papers)

arXiv (research papers) Shawn Li SI

Defenses Against Prompt Attacks Learn Surface Heuristics

Large language models (LLMs) are increasingly deployed in security-sensitive applications, where they must follow system- or developer-specified instructions that define the intended task behavior, while completing benign user requests. When adversarial instructions appear in user queries or externally retrieved content, models may override intended logic. Recent defenses rely on supervised fine-tuning with benign and malicious labels. Although these methods achieve high…

Read at arXiv (research papers)

arXiv (research papers) Hongyan Chang SI

Overcoming the Retrieval Barrier: Indirect Prompt Injection in the Wild for LLM Systems

Large language models (LLMs) increasingly rely on retrieving information from external corpora. This creates a new attack surface: indirect prompt injection (IPI), where hidden instructions are planted in the corpora and hijack model behavior once retrieved. Previous studies have highlighted this risk but often avoid the hardest step: ensuring that malicious content is actually retrieved. In practice, unoptimized IPI is rarely retrieved under natural queries, which leaves…

Read at arXiv (research papers)

arXiv (research papers) Masahiro Kaneko SI

Paraphrasing Adversarial Attack on LLM-as-a-Reviewer

The use of large language models (LLMs) in peer review systems has attracted growing attention, making it essential to examine their potential vulnerabilities. Prior attacks rely on prompt injection, which alters manuscript content and conflates injection susceptibility with evaluation robustness. We propose the Paraphrasing Adversarial Attack (PAA), a black-box optimization method that searches for paraphrased sequences yielding higher review scores while preserving…

Read at arXiv (research papers)

arXiv (research papers) Junda Lin SI

VIGIL: Defending LLM Agents Against Tool Stream Injection via Verify-Before-Commit

LLM agents operating in open environments face escalating risks from indirect prompt injection, particularly within the tool stream where manipulated metadata and runtime feedback hijack execution flow. Existing defenses encounter a critical dilemma as advanced models prioritize injected rules due to strict alignment while static protection mechanisms sever the feedback loop required for adaptive reasoning. To reconcile this conflict, we propose \textbf{VIGIL}, a framework…

Read at arXiv (research papers)

arXiv (research papers) Zhi Yang SI

FinVault: Benchmarking Financial Agent Safety in Execution-Grounded Environments

Financial agents powered by large language models (LLMs) are increasingly deployed for investment analysis, risk assessment, and automated decision-making, where their abilities to plan, invoke tools, and manipulate mutable state introduce new security risks in high-stakes and highly regulated financial environments. However, existing safety evaluations largely focus on language-model-level content compliance or abstract agent settings, failing to capture execution-grounded…

Read at arXiv (research papers)

arXiv (research papers) Suyash Mishra SI

From Understanding to Engagement: Personalized pharmacy Video Clips via Vision Language Models (VLMs)

Vision Language Models (VLMs) are poised to revolutionize the digital transformation of pharmacyceutical industry by enabling intelligent, scalable, and automated multi-modality content processing. Traditional manual annotation of heterogeneous data modalities (text, images, video, audio, and web links), is prone to inconsistencies, quality degradation, and inefficiencies in content utilization. The sheer volume of long video and audio data further exacerbates these…

Read at arXiv (research papers)

arXiv (research papers) Saad Alqithami SI

Autonomous Agents on Blockchains: Standards, Execution Models, and Trust Boundaries

Advances in large language models have enabled agentic AI systems that can reason, plan, and interact with external tools to execute multi-step workflows, while public blockchains have evolved into a programmable substrate for value transfer, access control, and verifiable state transitions. Their convergence introduces a high-stakes systems challenge: designing standard, interoperable, and secure interfaces that allow agents to observe on-chain state, formulate transaction…

Read at arXiv (research papers)

arXiv (research papers) Alexandre Le Mercier SI

Hidden State Poisoning Attacks against Mamba-based Language Models

State space models (SSMs) like Mamba offer efficient alternatives to Transformer-based language models, with linear time complexity. Yet, their adversarial robustness remains critically unexplored. This paper studies the phenomenon whereby specific short input phrases induce a partial amnesia effect in such models, by irreversibly overwriting information in their hidden states, referred to as a Hidden State Poisoning Attack (HiSPA). Our benchmark RoBench-25 allows evaluating…

Read at arXiv (research papers)

arXiv (research papers) Giuseppe Canale SI

The Silicon Psyche: Anthropomorphic Vulnerabilities in Large Language Models

Large Language Models (LLMs) are rapidly transitioning from conversational assistants to autonomous agents embedded in critical organizational functions, including Security Operations Centers (SOCs), financial systems, and infrastructure management. Current adversarial testing paradigms focus predominantly on technical attack vectors: prompt injection, jailbreaking, and data exfiltration. We argue this focus is catastrophically incomplete. LLMs, trained on vast corpora of…

Read at arXiv (research papers)

Journal articles (OpenAlex) Kawsar Uddin Mahmud · World Affairs Geopolitics

From Silicon Valleys to Superpowers: The US–China Tech Rivalry in the Age of Artificial Intelligence

ABSTRACT This article investigates the growing tech rivalry between the United States and China in artificial intelligence (AI), with a particular focus on recent developments in the domain. It analyzes China's rapid pace of development with different DeepSeek models, despite notable export controls and semiconductor restrictions from the US. The study also compares the strengths and weaknesses of AI in the US and China, and thus discusses how the superpowers compete in this…

Read at Journal articles (OpenAlex)

arXiv (research papers) Panagiotis Theocharopoulos SI

Multilingual Hidden Prompt Injection Attacks on LLM-Based Academic Reviewing

Large language models (LLMs) are increasingly considered for use in high-impact workflows, including academic peer review. However, LLMs are vulnerable to document-level hidden prompt injection attacks. In this work, we construct a dataset of approximately 500 real academic papers accepted to ICML and evaluate the effect of embedding hidden adversarial prompts within these documents. Each paper is injected with semantically equivalent instructions in four different languages…

Read at arXiv (research papers)

arXiv (research papers) Toqeer Ali Syed SI

Toward Trustworthy Agentic AI: A Multimodal Framework for Preventing Prompt Injection Attacks

Powerful autonomous systems, which reason, plan, and converse using and between numerous tools and agents, are made possible by Large Language Models (LLMs), Vision-Language Models (VLMs), and new agentic AI systems, like LangChain and GraphChain. Nevertheless, this agentic environment increases the probability of the occurrence of multimodal prompt injection (PI) attacks, in which concealed or malicious instructions carried in text, pictures, metadata, or agent-to-agent…

Read at arXiv (research papers)

arXiv (research papers) Armstrong Foundjem SI

Multi-Agent Framework for Threat Mitigation and Resilience in AI-Based Systems

Machine learning (ML) underpins foundation models in finance, healthcare, and critical infrastructure, making them targets for data poisoning, model extraction, prompt injection, automated jailbreaking, and preference-guided black-box attacks that exploit model comparisons. Larger models can be more vulnerable to introspection-driven jailbreaks and cross-modal manipulation. Traditional cybersecurity lacks ML-specific threat modeling for foundation, multimodal, and RAG…

Read at arXiv (research papers)

arXiv (research papers) Karolina Korgul SI

It's a TRAP! Task-Redirecting Agent Persuasion Benchmark for Web Agents

Web-based agents powered by large language models are increasingly used for tasks such as email management or professional networking. Their reliance on dynamic web content, however, makes them vulnerable to prompt injection attacks: adversarial instructions hidden in interface elements that persuade the agent to divert from its original task. We introduce the Task-Redirecting Agent Persuasion Benchmark (TRAP), a benchmark for studying how persuasion techniques misguide…

Read at arXiv (research papers)

arXiv (research papers) Kerem Zaman SI

Is Chain-of-Thought Really Not Explainability? Chain-of-Thought Can Be Faithful without Hint Verbalization

Recent work, using the Biasing Features metric, labels a CoT as unfaithful if it omits a prompt-injected hint that affected the prediction. We argue this metric adopts a narrow notion of faithfulness and confuses unfaithfulness with incompleteness, the lossy compression needed to turn distributed transformer computation into a linear natural language narrative. On multi-hop reasoning tasks with instruct-tuned and reasoning models, many CoTs flagged as unfaithful by Biasing…

Read at arXiv (research papers)

arXiv (research papers) Jiayu Hu SI

Look Closer! An Adversarial Parametric Editing Framework for Hallucination Mitigation in VLMs

While Vision-Language Models (VLMs) have garnered increasing attention in the AI community due to their promising practical applications, they exhibit persistent hallucination issues, generating outputs misaligned with visual inputs. Recent studies attribute these hallucinations to VLMs' over-reliance on linguistic priors and insufficient visual feature integration, proposing heuristic decoding calibration strategies to mitigate them. However, the non-trainable nature of…

Read at arXiv (research papers)

Journal articles (OpenAlex) Afees Adebare Salisu · Journal of Economic Studies Geopolitics

Climate risks and green stocks in Vietnam

Purpose Vietnam is among the countries most vulnerable to tropical cyclone risks, and its carbon-intensive production model influences its climate change trajectory. Nevertheless, various initiatives have been undertaken to tap into the country’s green economy potential, and I advance the related literature by exploring the connection between climate risks and green assets in Vietnam. Design/methodology/approach Firstly, by employing the predictive models, I examine the…

Read at Journal articles (OpenAlex)

arXiv (research papers) Yihan Wang SI

AegisAgent: An Autonomous Defense Agent Against Prompt Injection Attacks in LLM-HARs

The integration of Large Language Models (LLMs) into wearable sensing is creating a new class of mobile applications capable of nuanced human activity understanding. However, the reliability of these systems is critically undermined by their vulnerability to prompt injection attacks, where attackers deliberately input deceptive instructions into LLMs. Traditional defenses, based on static filters and rigid rules, are insufficient to address the semantic complexity of these…

Read at arXiv (research papers)

arXiv (research papers) Kanchon Gharami SI

ChatGPT: Excellent Paper! Accept It. Editor: Imposter Found! Review Rejected

Large Language Models (LLMs) like ChatGPT are now widely used in writing and reviewing scientific papers. While this trend accelerates publication growth and reduces human workload, it also introduces serious risks. Papers written or reviewed by LLMs may lack real novelty, contain fabricated or biased results, or mislead downstream research that others depend on. Such issues can damage reputations, waste resources, and even endanger lives when flawed studies influence…

Read at arXiv (research papers)

arXiv (research papers) Jaykumar Kasundra SI

AprielGuard

Safeguarding large language models (LLMs) against unsafe or adversarial behavior is critical as they are increasingly deployed in conversational and agentic settings. Existing moderation tools often treat safety risks (e.g. toxicity, bias) and adversarial threats (e.g. prompt injections, jailbreaks) as separate problems, limiting their robustness and generalizability. We introduce AprielGuard, an 8B parameter safeguard model that unify these dimensions within a single…

Read at arXiv (research papers)

arXiv (research papers) Haotian Deng SI

Rubric-Conditioned LLM Grading: Alignment, Uncertainty, and Robustness

Automated short-answer grading (ASAG) remains a challenging task due to the linguistic variability of student responses and the need for nuanced, rubric-aligned partial credit. While Large Language Models (LLMs) offer a promising solution, their reliability as automated judges in rubric-based settings requires rigorous assessment. In this paper, we systematically evaluate the performance of LLM-judges for rubric-based short-answer grading. We investigate three key aspects…

Read at arXiv (research papers)

OWASP GenAI Security Project Helen Oakley and Dmitry Raidman Security

Evolving AI Transparency: The Journey of the AIBOM Generator and Its New Home at OWASP

Earlier this year, during RSAC 2025, we introduced something the industry had never seen before: an open-source tool capable of generating an AI Software Bill of Materials (AIBOM) for models on Hugging Face. That launch ignited a wave of interest across the security, AI, and software supply chain communities. It confirmed what many of us

Read at OWASP GenAI Security Project

arXiv (research papers) Safwan Shaheer SI

Beyond the Benchmark: Innovative Defenses Against Prompt Injection Attacks

In this fast-evolving area of LLMs, our paper discusses the significant security risk presented by prompt injection attacks. It focuses on small open-sourced models, specifically the LLaMA family of models. We introduce novel defense mechanisms capable of generating automatic defenses and systematically evaluate said generated defenses against a comprehensive set of benchmarked attacks. Thus, we empirically demonstrated the improvement proposed by our approach in mitigating…

Read at arXiv (research papers)

arXiv (research papers) Richard Helder Moulton SI

Quantifying Return on Security Controls in LLM Systems

Although large language models (LLMs) are increasingly used in security-critical workflows, practitioners lack quantitative guidance on which safeguards are worth deploying. This paper introduces a decision-oriented framework and reproducible methodology that together quantify residual risk, convert adversarial probe outcomes into financial risk estimates and return-on-control (RoC) metrics, and enable monetary comparison of layered defenses for LLM-based systems. A…

Read at arXiv (research papers)

arXiv (research papers) Xinyu Huang SI

Trust in LLM-controlled Robotics: a Survey of Security Threats, Defenses and Challenges

The integration of Large Language Models (LLMs) into robotics has revolutionized their ability to interpret complex human commands and execute sophisticated tasks. However, such paradigm shift introduces critical security vulnerabilities stemming from the ''embodiment gap'', a discord between the LLM's abstract reasoning and the physical, context-dependent nature of robotics. While security for text-based LLMs is an active area of research, existing solutions are often…

Read at arXiv (research papers)

arXiv (research papers) Viet K. Nguyen SI

Penetration Testing of Agentic AI: A Comparative Security Analysis Across Models and Frameworks

Agentic AI introduces security vulnerabilities that traditional LLM safeguards fail to address. Although recent work by Unit 42 at Palo Alto Networks demonstrated that ChatGPT-4o successfully executes attacks as an agent that it refuses in chat mode, there is no comparative analysis in multiple models and frameworks. We conducted the first systematic penetration testing and comparative evaluation of agentic AI systems, testing five prominent models (Claude 3.5 Sonnet, Gemini…

Read at arXiv (research papers)

Journal articles (OpenAlex) Jhon Valdiglesias · Frontiers in Political Science Geopolitics

From governance to growth: comparative insights on institutional reforms in China and Latin America

Introduction This study examines the effects of China’s rule-of-law reforms and anti-corruption measures under Xi Jinping on economic performance, using China as a benchmark to compare with seven Latin American countries. It aims to explain how different institutional trajectories shape governance quality and long-term growth. Methods The analysis draws on two decades of panel data and employs fixed and random effects models with interaction terms to investigate how…

Read at Journal articles (OpenAlex)

arXiv (research papers) Najmul Hasan SI

Phishing Email Detection Using Large Language Models

Email phishing is one of the most prevalent and globally consequential vectors of cyber intrusion. As systems increasingly deploy Large Language Models (LLMs) applications, these systems face evolving phishing email threats that exploit their fundamental architectures. Current LLMs require substantial hardening before deployment in email security systems, particularly against coordinated multi-vector attacks that exploit architectural vulnerabilities. This paper proposes…

Read at arXiv (research papers)

arXiv (research papers) Yinan Zhong SI

Attention is All You Need to Defend Against Indirect Prompt Injection Attacks in LLMs

Large Language Models (LLMs) have been integrated into many applications (e.g., web agents) to perform more sophisticated tasks. However, LLM-empowered applications are vulnerable to Indirect Prompt Injection (IPI) attacks, where instructions are injected via untrustworthy external data sources. This paper presents Rennervate, a defense framework to detect and prevent IPI attacks. Rennervate leverages attention features to detect the covert injection at a fine-grained token…

Read at arXiv (research papers)

Older stories