Results for “Inference”

arXiv (research papers) Yuzhuo Li SI

Local AI Agents in Power Electronics Simulation: Deployment, Measurements and Costs

Agentic artificial intelligence (AI) is emerging in power electronics, where it drives the simulation loop that engineers used to close largely by hand. In this loop, a converter design, or a study of the network it connects to, is built, run, measured and revised until it satisfies its specification. Such an agent can be served from a cloud API or from an inference device on the premises. For organizations whose designs, customer data or export-controlled material cannot…

Read at arXiv (research papers)

arXiv (research papers) Akshay L Chandra

DiWALI: Diffusion Policy Learning in World Models via Latent Action Inference from Human and Robot Play

Learning robot manipulation is constrained by the availability of action-labeled robot data, for both imitation and world models. Humans can demonstrate skills in seconds, but their demonstrations lack the required robot actions. We introduce DiWALI, a framework that learns and improves diffusion policies within a world model trained on human and robot play. DiWALI infers latent actions from consecutive frames using an inverse dynamics model, enabling world model training…

Read at arXiv (research papers)

arXiv (research papers) Yicheng He

Mechanistic Critics for Sample-Efficient NPU Design-Space Exploration

NPU design-space exploration for LLM inference mainly relies on an expensive simulator that often returns only scalar feedback such as latency and power. Such feedback can rank evaluated configurations, but gives little information for deciding why a configuration is slow or which architecture parameter should be changed next. This limitation is especially severe because prefill and decode workloads can be limited by different resources, including compute throughput and HBM…

Read at arXiv (research papers)

SiliconANGLE AI Jonathan Anthony SI

AI’s hidden energy tax: More than 80% of scale-up traffic is just memory moving around

Memory bottlenecks are emerging as one of the costliest problems in AI infrastructure as inference overtakes training. GPUs sit idle waiting on data, while high-bandwidth memory, or HBM, stays bolted to processors that cannot share it. Photonics is shaping up as one way out. Marvell Technology Inc. has been pairing optics with photonic memory to

Read at SiliconANGLE AI

arXiv (research papers) Jiayou Zhu Security

RECLAIM: Robot-Vision-Informed Adversarial Patch Defense via Embodied Attention Feedback

Vision-language-action policies control embodied agents through a perception-action loop grounded in physical interaction, where adversarial patches can corrupt action predictions and thereby alter subsequent observations. To mitigate these threats, many existing defenses rely on policy adaptation or auxiliary models, but require additional training or inference. We introduce RECLAIM, a training-free, inference-time defense that uses a frozen VLA's own attention response to…

Read at arXiv (research papers)

The New Stack Paul Sawers

“Collaboration over competition”: Hugging Face hands model inference traffic to OpenRouter

Hugging Face and OpenRouter have announced a partnership that promises to make AI model inference faster and more reliable for

Read at The New Stack

arXiv (research papers) Guolong Zheng SI

Semantic Context Handoff with On-Demand Evidence for Long-Horizon Agents

Traditional long-horizon LLM agents often hand off context through full-history prompting, retaining all previous action--observation interactions. As execution proceeds, this history keeps growing, increasing inference cost and potentially degrading accuracy. Existing methods control this growth by preserving compact conclusions while removing earlier observations, but doing so can discard evidence that later becomes relevant. We introduce Semantic Context Handoff (SCH)…

Read at arXiv (research papers)

arXiv (research papers) Boxi Zhou SI

Reducing Inference Cost of Coding Agents with Timeout Cascades

AI coding agents have become one of the most popular applications of large language models. Yet, agentic AI comes at the cost of a tremendous amount of GPU compute resources, which limits its profitability and its viability at scale. In this paper, we investigate whether smaller, cheaper models can cut this cost without sacrificing accuracy. We evaluate 36 popular large and small open-weight models released since 2024 on a software engineering benchmark, and find that recent…

Read at arXiv (research papers)

arXiv (research papers) Robin Geens

Hardware Acceleration of Vision-Language-Action Models with Weight Precision Rising over the Flow Steps

Vision-language-action (VLA) models such as GR00T control robots through an action head: a diffusion transformer that turns noise into a chunk of actions in a few flow-matching steps. Every step re-reads all of the head's weights from DRAM but applies each weight to only about fifty tokens. Inference is therefore memory-bound, and its latency and energy are largely determined by the total number of transferred weight bits. We show that later flow steps attenuate the…

Read at arXiv (research papers)

arXiv (research papers) Keming Fan

ReCIM-HD: A 40 nm On-Device Learning Accelerator with Hyperdimensional Computing via Importance-Based Hybrid Digital-ReRAM CIM

ReCIM-HD is an on-device learning (ODL) accelerator that integrates gradient-free hyperdimensional computing (HDC) with resistive RAM (ReRAM)-based analog compute-in-memory (CIM). ReCIM-HD introduces three key techniques: (1) ultra-low-power HDC-based edge ODL without costly back-propagation operations; (2) non-volatile ReRAM-based storage for class hypervectors and CIM similarity search, enabling energy-efficient inference and power-gating during idle periods without data…

Read at arXiv (research papers)

Analytics India Magazine Supreeth Koundinya SI

Neysa Deploys AMD GPU Cluster in Mumbai for AI Training and Inference

The new deployment brings together AMD Instinct MI350X GPUs, AMD EPYC processors as well as the AMD ROCm software ecosystem.

Read at Analytics India Magazine

arXiv (research papers) Vikash Kumar

Inference-Time Scaling in Agentic Hardware Verification Workflows: A Multi-Strategy Characterization Across Protocol Complexity

Inference-time scaling, allocating more compute during generation rather than pre-training, has demonstrated dramatic quality improvements for LLM reasoning tasks. Its behaviour in structured hardware verification code generation, where outputs must satisfy compiler correctness, protocol semantics, and functional coverage constraints simultaneously, is less understood. We present a systematic, simulator-grounded study of inference-time compute scaling for agentic UVM…

Read at arXiv (research papers)

Inc42 Team Inc42 Startups

Sarvam’s AI Inference Play, Perplexity’s IPO Soon & More

Sarvam AI Fires Up Its Inference Engine Sarvam’s Token Factory is finally live. The new offering will let enterprises access…

Read at Inc42

AWS Machine Learning Blog Vinay Arora SI

Managed NVIDIA Dynamo on Amazon SageMaker HyperPod Inference

Amazon SageMaker HyperPod Inference now supports NVIDIA Dynamo for distributed inference, with multi-node and disaggregated serving, KV-aware routing, and automatic fault recovery. Serve large models across many nodes without building the orchestration layer yourself.

Read at AWS Machine Learning Blog

arXiv (research papers) Shoma Yukawa

EEG2MCU: Sensing-to-Execution Co-Design for Wearable EEG Inference

Wearable EEG inference is constrained not only by model size, but also by the channel-dependent sensing front end that must process high-dimensional inputs before compact downstream computation. We present EEG2MCU, a sensing-to-execution co-design framework that treats electrode count as a hardware design variable through task-aware montage selection. EEG2MCU retains a compact time-frequency representation and generates only the INT8 regions required by each first-layer…

Read at arXiv (research papers)

SiliconANGLE AI Kelly Knight SI

Dell bets liquid-cooled, software-defined storage can keep AI factories fed

AI factories are growing so dense that storage, not just compute, now decides how quickly they turn data into tokens. For Dell Technologies Inc., the answer starts with turning its PowerEdge servers into flexible, software-defined storage nodes that can change roles as workloads shift. The move from training toward inference and agents has put new

Read at SiliconANGLE AI

arXiv (research papers) Sehyeon Lee

Towards Fair Resource Sharing in Agentic LLM Serving

As LLM agents increasingly execute complex, multi-step tasks on shared inference infrastructure, resource contention across the end-to-end LLM serving stack causes severe and uneven slowdowns across agents. Existing agentic serving systems primarily optimize throughput and latency, with comparatively limited attention to fairness among concurrent agents. Although fairness has been studied in conventional LLM serving at the token and request levels, these approaches do not…

Read at arXiv (research papers)

AWS Machine Learning Blog Dmitry Soldatkin SI

Amazon SageMaker AI real-time endpoints now support KV cache aware routing

Amazon SageMaker AI real-time endpoints now support KV cache-aware routing, a new strategy that routes each request to the instance holding the longest cached match for its prompt. Learn how it works, when to use it compared with prefix-aware routing, and how to enable, invoke, and verify it on single model and inference component endpoints.

Read at AWS Machine Learning Blog

arXiv (research papers) Almog Anschel SI

WoRD: Diffusion-Based Posterior Inference for Online Goal Recognition

Goal recognition is the task of inferring an agent's intended goal from its observed behavior. Most existing work focuses on high-level symbolic recognition, abstracting away the continuous motion through which agents reach their goal. Methods that do reason over continuous trajectories rely on an explicit dynamics model that is hard to acquire and rarely transfers across settings. Neither approach explicitly learns how agents move toward each candidate goal, limiting…

Read at arXiv (research papers)

arXiv (research papers) Florian Gunsilius SI

Inference after coding agents

Empirical research is changing rapidly as AI coding agents are becoming increasingly proficient in every step of it, from writing code to manipulating data. We argue that while the potential of coding agents for science is vast, their use in hypothesis testing poses a substantial threat to classical inference, as these agents enable a supercharged form of selective inference. In particular, we argue that coding agents carrying out empirical research introduce a new type of…

Read at arXiv (research papers)

arXiv (research papers) Mridul Mahajan SI

Learning Stratified Policies for Inference-Time Objectives

Inference-time scaling improves the performance of large language models by sampling multiple responses and evaluating their collective utility. However, many existing approaches obtain these samples by repeatedly sampling from a single policy. In this work, we study if the sample budget at inference time can instead be allocated across a population of policies. We introduce stratified policies, where each of $k$ learned policies contributes one response to a $k$-sample…

Read at arXiv (research papers)

arXiv (research papers) Minh Luu SI

BugGen 2.0: Rethinking the Roles of LLMs for Scalable RTL Bug Synthesis

AI-assisted RTL localization and debugging need ground-truth bug datasets with known locations and observable failures, while testbench grading uses injected bugs to measure detection capability. Creating such datasets at scale remains a major EDA challenge. BugGen 1.0 shows that Large Language Models (LLMs) can generate context-aware RTL mutations. Still, it relies on the model for both choosing where to mutate and generating the edit, which increases inference costs and…

Read at arXiv (research papers)

arXiv (research papers) Fangchuan Wang

Making Future-Predictive Representations Actionable

Recent world action models (WAMs) drop future prediction at inference, assuming that representations learned through auxiliary future supervision still benefit action generation. However, our layerwise analysis of Fast-WAM reveals a depth-dependent mismatch: while probes decode future state transitions more accurately from deeper visual layers, the gradients and visual focuses of the future and action objectives increasingly diverge with depth, leaving much of this…

Read at arXiv (research papers)

AWS Machine Learning Blog Shubham Gupta SI

How ArisGlobal scaled generative AI inference on Amazon Bedrock

ArisGlobal runs NavaX, its life-sciences AI engine, on Amazon Bedrock. By caching the shared prompt prefix instead of each per-case document, ArisGlobal cut effective input cost by about 50% and overall monthly inference spend by about 40%, while onboarding more tenants without proportional cost growth.

Read at AWS Machine Learning Blog

TechPowerUp btarunr SI

(PR) AMD Completes Acquisition of Taalas to Advance AI Inference Innovation

AMD today announced the completion of its acquisition of Taalas, a pioneer in specialized AI inference silicon, further strengthening the company's full-stack AI portfolio and long-term inference roadmap. As demand for AI inference accelerates, Taalas brings differentiated technology and world-class engineering expertise designed to deliver breakthrough performance and efficiency for increasingly specialized AI workloads. "AI inference is evolving rapidly, and customers need…

Read at TechPowerUp

arXiv (research papers) Stanley Akingbola SI

Network-Exposed Configuration Attacks on LLM-Controlled Social Robots

Large language models (LLMs) in embodied robots create security dependencies beyond inference. We investigate a network-exposed configuration attack on the Pollen Robotics Reachy Mini Lite, allowing an unauthenticated attacker to replace personality settings and the cloud LLM's system prompt through a management interface, without adversarial conversational input. Eleven prompt-injection attempts across five categories were reported as resisted, whereas both…

Read at arXiv (research papers)

TestingCatalog Erin | AI Agent SI

Hugging Face moves Inference Providers routing to OpenRouter

Hugging Face now routes language model requests on Inference Providers through OpenRouter, with automatic failover, no markup and the same API.

Read at TestingCatalog

arXiv (research papers) Lorenzo Paiola

Task Convergent Flow Matching for Generative Robotics

Successful manipulation requires the manipulated objects to progress toward a desired task outcome. Generative policies can produce plausible robot motions without ensuring this progress, allowing successive action chunks to stall or undo earlier changes to the scene. We introduce a task-centric inference-time refinement framework for flow matching policies that evaluates candidate action chunks through parallel physics-based rollouts and searches for motions satisfying…

Read at arXiv (research papers)

MarkTechPost Asif Razzaq SI

Sarvam Launches Sarvam Token Factory: Bringing India-Hosted Inference to 8 Open Models

Sarvam Token Factory is now open to everyone. It serves Sarvam 105B, GLM 5.3, DeepSeek V4.1 Flash, Kimi K3.1 Pro and Gemma 4 31B from GPU clusters in India through one OpenAI-compatible API.

Read at MarkTechPost

Older stories