Is indirect prompt injection still a big threat as models get more advanced?
Article URL: https://realarcherl.github.io/posts/prompt_injection/ Comments URL: https://news.ycombinator.com/item?id=49840178 Points: 3 # Comments: 3
Article URL: https://realarcherl.github.io/posts/prompt_injection/ Comments URL: https://news.ycombinator.com/item?id=49840178 Points: 3 # Comments: 3
Large language model email assistants are particularly vulnerable to indirect prompt injection because untrusted email content can be retrieved into the model context and influence subsequent tool use. Existing prompt injection detectors mainly formulate this problem as binary malicious text classification, which overlooks the important factor that harmful agent behavior often arises through a sequence of stages. We propose a detection framework that models this attack chain…
Healthcare startups drive innovation in the healthcare sector, yet they face a complex institutional environment to thrive, especially in developing countries. This study explores how institutional factors influence the creation and implementation of business models by healthtech startups in this context. Drawing on Institutional Theory and based on a multiple case study of 12 healthtech startups in Brazil, data reveal that, during the creation phase of the business model…
Large Language Models (LLMs) consume and produce a single sequence of text; hence, if text can be added to the beginning of the LLM's response, i.e., an output prefix, then all subsequent tokens will be conditioned on it. This output-prefix attack technique is a cheap black-box prompt injection. Prior work has shown this type of attack can reliably jailbreak non-reasoning models. Most reasoning models add an intermediate scratchpad reasoning step before the assistant's final…
Detectors of alignment failures screen deployed language models and score alignment benchmarks. Most are generative judges that spend a decoding pass on every criterion, and classifiers that read token probabilities, such as Llama Guard, still score one fixed label per call. Jev, a model trained with reinforcement learning for calibrated decisions (RLCD), answers many typed questions about one input with calibrated probabilities in a single call. Whether it detects alignment…
Predictive modelling of soil nutrient dynamics is an essential tool for promoting sustainable agricultural practices and environmentally responsible farming methods. The statistical and machine learning techniques used to forecast the availability and dynamics of soil nutrients are summarised in this review. The core frameworks for measuring spatio-temporal nutritional variability are established by traditional statistical approaches such as time-series models autoregressive…
IoT-enabled smart homes require accurate activity classification to enhance safety, healthcare monitoring, and energy efficiency through continuous sensing and intelligent interpretation of human behavior. Common approaches employ wearable or ambient sensors, signal preprocessing, feature extraction, and deep learning models such as CNNs, LSTMs, BiLSTM, and attention-based classifiers. These techniques generally achieve high classification accuracy, robust temporal modeling…
Most studies of prompt injection focus on generative agents, leaving their effects on models with schema-defined outputs unclear. We examine these effects in Jev, a non-generative decision model, using 510 reconstructed InjecAgent cases. Malicious content shifts action probabilities but rarely causes Jev to select the attacker's target. Override markers reduce this influence, while claims of contextual relatedness have small effects. Adaptive attacks using score feedback…
Deploying Large Language Models for runtime operational triage incurs prohibitive latency (>100-500 ms), high VRAM requirements (>4-8 GB), and excessive energy dissipation. Extending Mandelbrot Fractal Neural Synthesis (Dagli et al., 2026), this paper presents the Universal Fractal Natural Language Decision Map, realized via the werr machine-native edge reflex runtime and the production answerr platform (https://answerr.me). Operating entirely without stored weight tensors…
Large language models (LLMs) are increasingly deployed in production systems, raising concerns about their exposure to adversarial manipulation through prompt injection and jailbreak attacks. Classifier-based guardrails, such as Prompt Guard 2, are widely used as a first line of defense against such attacks, but their internal decision logic is largely opaque to both defenders and attackers. This paper presents an exploratory case study that applies explainable artificial…
Agentic AI systems built on large language models can plan over multiple steps, use external tools, retain information in memory, and coordinate with other agents. These capabilities make them more useful than static language models, but they also introduce new security and operational risks. Untrusted content from websites, emails, documents, and databases can enter the same context as system instructions; persistent memory can carry compromised information across sessions…
Self-generated prompt injections in compaction summaries In Our framework for reporting model misalignment OpenAI provide "six reports on unexpected or concerning model behavior we’ve observed in the last six months". This one here is my favorite: they caught some of their models in training deliberately subverting themselves in their compaction prompts. Compaction is the process agent systems use when they are running out of tokens in their context window, so they summarize…
Activation steering modifies LLM behavior at inference time, but identifying where and how strongly to steer remains manual. We introduce Deep Noir, a framework that uses Logit Lens convergence and causal head-level attribution to autonomously discover optimal steering parameters. Across three scales (1B x 3, 2-3B x 2, and 7-9B x 4), our engine achieves 16.7 percentage-point improvement on spam at 1B (standard deviation 4.7; 39 runs), with gains increasing to 21 to 42…
ABSTRACT This study investigates volatility spillovers and minimum‐variance portfolio allocation among Islamic finance, sustainable, and technological innovation markets across four major global turbulences (COVID‐19 pandemic, Russia–Ukraine conflict, 2023 banking crisis, and 2025 US policy shifts) from July 07, 2021, to April 17, 2025. The study employs the Diebold‐Yilmaz and DCC‐GARCH models. Results show that volatility connections (conditional correlations) are generally…
Grounded in Cultural Discourse Studies (CDS), this article presents a comparative, multimodal discourse analysis of contemporary social movements in Ukraine and South Korea. Integrating collective action framing and cultural approaches to protest, the study investigates how language, socio-acoustics, and visual symbols function during intense civic mobilization in both nations. By moving beyond Euro-centric and text-exclusive models of Critical Discourse Analysis (CDA), the…
In response to the lack of high-quality annotated corpora in the current smart home field and the insufficient accuracy and interpretability of existing intent classification models in complex home scenarios, this study proposes an intent classification model system that integrates unsupervised topic discovery and deep learning classification. Firstly, a Chinese smart home corpus containing 120,000 real conversations with a dual-layer annotation structure is constructed, and…
ELT in Islamic Higher Education (IHE) sits at the meeting point of global English demands and faith-based educational aims. Using PRISMA-guided procedures, this systematic literature review synthesizes 43 peer-reviewed studies and organizes findings into four themes: skills interventions, online/technology-mediated ELT, negotiation of English–identity–values, and ethics/integrity. Across themes, outcomes depend not only on methods or tools but also on affect, identity…
ABSTRACT Using the European Quality of Government Index across 183 NUTS‐2 regions (2010–2024), this paper documents that education improves institutional quality in Western Europe but not in postsocialist regions. Oaxaca–Blinder decomposition reveals that 77 percent of the West–East institutional gap reflects different returns to education, not different levels. Triple interaction models identify state capture as the blocking mechanism: where executive corruption is high…
Abstract Background Suicidal ideation (SI) among university students is a growing public health concern. Self-report screening can be limited by concealment and delayed disclosure. We evaluated a leakage-resistant, proof-of-concept pipeline to detect SI from standardized smartphone-recorded speech. Objective This study aimed to extract acoustic markers from brief smartphone-based reading tasks and develop machine learning models for suicide risk prediction in university…
The Taiwan Strait serves as a crucial maritime corridor for both East Asian and global trade. However, ongoing territorial disputes between China and Taiwan present significant challenges to navigational safety and transport efficiency. This study examines the potential of legal dispute resolution mechanisms, particularly arbitration, to mitigate risks associated with maritime transport in the Strait. By referencing international legal frameworks such as UNCLOS and examining…
Publication integrity in LLM systems requires binding approved content to its current authorization context. We examine exact-content binding, authorization freshness and checkpoint recovery in Lightcap's publication enforcement mechanism. On 900 independently human-annotated RAGTruth responses from 150 source tasks, three dated Ministral models and a same-model direct-grounding baseline yield 3,600 assessments. The production response-act checker instantiated with 14B…
Accurate estimation of crop area using classification techniques applied to drone imagery plays an important role in precision agriculture. Traditional machine learning (ML) approaches have been widely used for agricultural image classification; however, advanced deep learning (DL) models generally provide superior feature extraction and classification capability for complex crop patterns. Differentiating various rice establishment methods is essential for precise area…
Cosmopolitanism in international schooling frequently reinforces elite privilege and social stratification rather than serving as a neutral, stable ideal. However, comparative empirical research examining how different international schooling models discursively construct cosmopolitanism alongside national belonging remains limited. This study employs cosmopolitan nationalism as an analytical lens to examine the construction of cosmopolitan identities in the mission…
Personal AI agents built on large language models (LLMs) are increasingly given access to a user's private data and communications in order to provide personalized assistance. This access creates a persistent privacy risk: the agent must decide whether a given sensitive information should be disclosed to a particular party. Existing defenses address this by making the agent's backend LLM more privacy-preserving through stronger system prompts, training, or explicit…
Unless treated and controlled early, weeds considerably impede the growth of crops as they compete over vital factors (resources) like nutrients, water, and light. To overcome this problem, an automated weed species early detection and classification system using machine learning-based image analysis is suggested. The framework integrates classical machine learning algorithms Support Vector Machines (SVM), Random Forests and k-Nearest Neighbors (k-NN), using handcrafted…
The integration of Large Language Models (LLMs) into Security Operations Centers (SOCs) streamlines threat intelligence but introduces critical vulnerabilities, notably indirect prompt injection via log poisoning. Adversaries exploit this vector to execute multistep ``promptware'' kill chains by embedding malicious payloads within system logs to hijack the LLM's operational logic. Securing this pipeline presents a dichotomy: deterministic defenses are computationally…
Introduction This study compares two asymmetrical security models in Eurasia: Europe’s NATO-centered security architecture, which relies on extended nuclear deterrence, and Kazakhstan’s institutionalized strategy of denuclearization. Methods The theoretical framework is grounded primarily in classical realism, especially Morgenthau’s state-centric understanding of power and national interest, while Aron’s work is used to clarify the relationship between military force and…
LLM agents complete tasks by issuing sequences of tool calls, and every observation they read is a channel through which an indirect prompt injection can enter. A successful injection has a characteristic shape when the trajectory is read in order: a benign prefix gives way to actions that serve the attacker rather than the user. Existing benchmarks measure whether such attacks succeed against live agents, and existing guard models judge a trace as a whole; no public corpus…
Model misalignment, prompt injection, or operator misuse could lead AI agents operating frontier-lab accounts to exfiltrate model weights, poison training data, or weaken release gates. Existing benchmarks do not test whether defenders can detect this activity among routine work under a limited review budget. We introduce MOLE, an open benchmark of 150 AI-operated accounts sharing 9 stateful services over 30 workdays, with 12 threats and 8 corpora from four models totaling…
Large language model (LLM) agents can translate natural-language requests into real-world actions, creating safety risks from both indirect prompt injections and directly harmful requests. Existing system-level safety harnesses are typically designed by experts and applied across heterogeneous models and domains, despite substantial differences in model behavior, utility tolerance, and domain-specific safety requirements. We present EvoSafeHarness, a framework for…