Results for “Inference”

AWS Machine Learning Blog Aryaman Kothari SI

Amazon SageMaker Inference: 2026 launches in review, part 2

This second installment reviews 13 more Amazon SageMaker AI inference launches from 2026, spanning configuration recommendations, fitting more models onto each GPU, serving models too large for one instance, cross-Region routing, and post-launch tuning, bringing the year's total to 26.

Read at AWS Machine Learning Blog

AWS Machine Learning Blog Hazim Qudah SI

Amazon SageMaker AI now detects latency drift, idle GPUs, and missed optimizations on your inference endpoints

Amazon SageMaker AI now monitors your generative AI inference endpoints by default and tells you when a change could improve performance or lower cost. It tracks latency, GPU utilization, and serving configuration, then sends a finding through AWS Health with a one-click path to benchmark alternatives in SageMaker Studio before you apply any change.

Read at AWS Machine Learning Blog

SiliconANGLE AI Kelly Knight SI

D-Matrix bets stacked DRAM can solve inference’s memory bandwidth problem

As artificial intelligence workloads shift to inference, memory bandwidth decides whether hardware generates tokens quickly or sits idle. D-Matrix Corp. is answering with Raptor, a second-generation chip stacking 3D DRAM beneath its compute logic. AI infrastructure has become a systems game, and winners combine memory, interconnect and software into cohesive designs, according to Satyam Srivastava

Read at SiliconANGLE AI

AWS Machine Learning Blog Dmitry Soldatkin SI

Keep inference running through regional capacity shortfalls with cross-Region inference routing in Amazon SageMaker AI

Cross-Region inference routing in Amazon SageMaker AI gives your application one invocation path across the endpoints and inference components you already run, in one or more AWS Regions and AWS accounts. The router prefers its own Region and shifts new requests to another destination only when the local one can't serve reliably.

Read at AWS Machine Learning Blog

arXiv (research papers) Dwaipayan Dubey Space

Demographic inference in exoplanet chemistry is shaped by retrieval assumptions

Understanding the chemical diversity of exoplanet atmospheres is crucial for linking atmospheric composition to planetary formation history. Atmospheric C/O can provide clues about where and how planets form, particularly through the accretion of gas and solids across disk snowlines. Since planetary migration can change orbital distances and hence equilibrium temperatures ($T_{eq}$), investigating C/O across different planetary populations may provide insights into their…

Read at arXiv (research papers)

arXiv (research papers) Balint Varga

An Open-Source ROS 2 Framework for Shared Control in Surgical Teleoperation: Integrating the Haply Inverse3 with the da Vinci Research Kit

Surgical teleoperation deprives surgeons of touch, forcing visual inference of tissue forces; the commercial platforms now restoring force feedback remain closed, preventing open research on haptic shared control. This paper provides a system contribution: it presents an open-source, ROS 2-native framework that integrates the compact, three-degree-of-freedom (3-DOF) Haply Inverse3 haptic device with the da~Vinci Research Kit (dVRK). The framework includes a novel mapping…

Read at arXiv (research papers)

arXiv (research papers) Sagi Meir

Imaginary-Time Density Correlations Encode Quantum Entropy and Free Energy

In the classical limit, maximum-entropy inference yields an estimate of the excess entropy from the static structure factor through a single momentum sum. Quantum mechanically this estimate fails, because interactions also change the kinetic energy, while delocalization and exchange add information that equal-time correlations do not contain. We show that imaginary-time density correlations contain this information, and that with them the classical relation carries over to…

Read at arXiv (research papers)

arXiv (research papers) Changtao Zhao

FastRoverVLA: Fast Vision-Language-Action Execution for Off-Road Autonomous Driving

Off-road vision-language-action (VLA) models generate scene-conditioned plans, but frequent trajectory updates can repeatedly invoke expensive multimodal inference. Reusing a prediction reduces this demand while leaving its commands exposed to changes in vehicle motion. FastRoverVLA addresses this mismatch by decoupling visual plan generation from motion-conditioned execution. Its training-free interface maintains decoded plans between visual predictions. A speed-curvature…

Read at arXiv (research papers)

arXiv (research papers) Jiawei Xing Security

ChronoPatch: Black-Box Energy-Latency Attacks on Vision-Language Models

Vision-language models (VLMs) are increasingly deployed as generative inference services, where autoregressive decoding makes response length a key determinant of latency and computational cost. This creates an availability risk: adversarial visual inputs can prolong generation and increase serving overhead. Existing energy-latency attacks typically rely on gradients, logits, or token-level feedback, leaving open whether such risks persist when an attacker observes only the…

Read at arXiv (research papers)

arXiv (research papers) Robin Geens

Kameleo: In-Situ Activation-Precision Selection for Bit-Serial LLM Accelerators

Weight-only quantization keeps the activations of large language models (LLMs) in a higher-precision format, pushing the on-chip inference cost to the activation arithmetic. Bit-serial, variable-length accelerators offer a promising path to reduce this cost by lowering activation precision while preserving model quality. They process only the mantissa bits that each layer needs. The length has to be chosen per layer and operator, and today it is chosen by offline…

Read at arXiv (research papers)

South China Morning Post Associated Press Geopolitics

UN top court says Myanmar committed genocide against Rohingya

In a landmark judgment, top UN court said Myanmar committed genocide against the Rohingya. The court ruled on Thursday that the “only reasonable inference” from evidence of widespread atrocities by Myanmar security forces during a 2017 military campaign is that they had the intent to destroy a “substantial part” of the Rohingya minority. The intent to fully or partially destroy a group is a key element in establishing whether crimes amount to genocide under international…

Read at South China Morning Post

arXiv (research papers) Qifan Wang Security

BlockJack: Cross-Tenant Redirection Fault Attacks on Paged KV Cache Address Translation in LLM Serving

Large language model (LLM) serving frameworks accelerate inference by retaining previously computed states in a key-value (KV) cache. Paged serving frameworks such as vLLM organize this cache through block tables analogous to OS page tables, which map each request's logical KV blocks to physical KV blocks, forming a software-managed address translation layer. Although fault attacks such as Rowhammer have exploited OS page tables to redirect memory accesses, prior fault…

Read at arXiv (research papers)

arXiv (research papers) Youssef ElNahas Tech policy

Verifiable Logic in Medical LLMs: A Critical Review of Agentic Clinical Reasoning Optimization and Neuro-Symbolic Grounding

Clinical large language models are usually judged by whether the final answer is right. Yet a model can reach a fluent, plausible output through invalid reasoning that static benchmarks do not detect, and confident hallucination, causally invalid inference, and spurious demographic shortcuts can pass conventional evaluation. This critical review covers four paradigms in turn: how clinical hallucination arises and why static metrics miss it; the agentic hallucination cascade…

Read at arXiv (research papers)

Help Net Security Industry News Security

Palo Alto Networks to acquire Applied Compute for $4.5 billion

Palo Alto Networks will acquire Applied Compute, a platform for post-training and inference of open-weight AI models. The all-stock transaction is valued at $4.5 billion, subject to adjustments. The acquisition will bring Applied Compute’s platform and world-class AI research team to Palo Alto Networks to help enterprises adopt AI securely, whether they choose open-weight models, closed frontier models, or both. Palo Alto Networks believes the future of enterprise AI is…

Read at Help Net Security

arXiv (research papers) Jinkun Cao

Fast and Accurate Body Estimation in Pixels, World Space and Scenes

We present FAB (Fast and Accurate Body estimation), a human body estimator that jointly improves inference efficiency and body-recovery fidelity. Rather than predicting only parametric mesh-model parameters, FAB directly predicts human skeleton joints and dense surface points. In images, it predicts dense 2D and 3D body points alongside a parametric Momentum Human Rig (MHR) head, then fits SMPL in one learned pass followed by two Gauss-Newton steps. Its ViT-L student, FAB-L…

Read at arXiv (research papers)

arXiv (research papers) Kai Shi

LeCompass: Task-Level Execution through State-Indexed Interfaces

Action-conditioned joint-embedding predictive architectures (JEPAs) offer a foundation for predictive decision-making by modeling the consequences of actions in representation space. Yet evaluating local reachability in latent space does not by itself provide a task-level execution interface: intermediate targets must be generated, local inference has finite budgets, and execution requires spatial feedback and persistent interaction state. We introduce LeCompass, a…

Read at arXiv (research papers)

arXiv (research papers) Zhihao Liu Security

Ask in the Crowd: Differentially Private LLM Inference via Dummy-Augmented Shuffling

Large language models are increasingly deployed as LLM-as-a-Service (LLMaaS), where user queries are sent to remote LLMs for inference, raising severe privacy concerns about the exposure of sensitive user inputs. Differentially private inference mitigates these risks by injecting noise into input embeddings. However, existing methods struggle to balance privacy and utility, as stronger privacy protection often leads to substantial accuracy degradation in LLM inference. In…

Read at arXiv (research papers)

arXiv (research papers) Xu Pan

CauSyncVLA: Causal Latent World Modeling for Asynchronous Robot Control

Vision-Language-Action (VLA) policies must react while visual inference is still running. Overlapping computation and execution reduces waiting, yet leaves perceptual memory stale and successive action chunks disconnected. We present CauSyncVLA, an asynchronous dual-system controller that aligns perceptual-memory prediction and trajectory correction with actual robot execution. Rather than forecasting pixels or physical states, a Causal Latent World Model predicts current…

Read at arXiv (research papers)

arXiv (research papers) Sikha Pentyala Security

GPU-Accelerated Secure Inference for Drug--Target Interaction Prediction

Deep learning models that predict drug--target interactions (DTI) can speed up drug discovery, but the candidate drugs they score are valuable intellectual property that their owners cannot reveal. Secure multiparty computation (MPC) lets two servers compute a prediction on a secret-shared drug, without either server learning the drug. However, secure inference of deep models is slow. Non-linear operations require communication between the servers, and MPC arithmetic does…

Read at arXiv (research papers)

arXiv (research papers) Prabhu Vellaisamy

Splice-Core: Low-Order-Bit-Splicing GEMM for Highly Energy-Efficient AI Inference Accelerators

Splice-Core is a temporal-unary/binary (tub) hybrid general matrix multiplication (GEMM) unit that minimizes low-order accumulation latency, while retaining value-dependent execution for high-order activation bits. We present two novel processing element (PE) variants: (i) Splice-BP utilizing bit planes, and (ii) Splice-LUT, which selects a shared weight multiple in a single cycle. Compared to a binary systolic array, Splice-BP/Splice-LUT achieve 77.4%/76.4% geometric-mean…

Read at arXiv (research papers)

arXiv (research papers) Anh Dao

LoopedWAM: Looped World Action Models for Elastic Inference

World action models (WAMs) couple generative video modeling with robotic action prediction but typically require large video and action transformers, limiting deployment efficiency. We present LoopedWAM, a compact WAM that recurrently applies a shared six-block core in each expert, executing 30 transformer layers while storing only 12 blocks per expert. To account for their asymmetric inference costs, the video and action experts can operate at different depths: visual…

Read at arXiv (research papers)

Analytics India Magazine Kanishk Singh SI

GitHub Now Lets Indian Enterprises Store Code Locally Ahead of DPDP Deadline

Global AI labs and cloud hyperscalers have accelerated their data residency and local inference rollouts in India ahead of full enforcement of the DPDP Rules.

Read at Analytics India Magazine

arXiv (research papers) Zihao Dan Security

ToolPulse: Side-Channel Eavesdropping on Tool-Using LLM Agents

Large language models (LLMs) are increasingly deployed as agents that translate user requests into sequences of local tool invocations. While prior cache-based side-channel attacks target the locally deployed models, they typically require the adversary to share model-related components or hardware resources with the victim's inference process. These assumptions do not apply to the predominant agentic settings, in which commercial LLMs execute on vendor-controlled…

Read at arXiv (research papers)

YourStory Startups

Debate brews over Sarvam hosting Chinese AI models; Grand Theft Auto VI is finally here

Token Factory is an inference platform, which provides the computing infrastructure required to run trained AI models and generate responses.

Read at YourStory

Tech in Asia Aiko Gao Ishida Startups

AWS Lambda adds GPU support for serverless inference

AWS said the feature targets workloads that need GPU acceleration but are often built on self-managed infrastructure.

Read at Tech in Asia

AWS Machine Learning Blog Summer Devlin SI

Exa boosts embedding throughput by 50% to index billions of documents on AWS AI chips

Exa's neural search engine is backed by an in-house vector database, so the unit economics of embedding tens of billions of web documents drive the business. This post shares how Exa optimized embedding inference on AWS Neuron hardware (Inferentia2 and Trainium2) with custom NKI kernels, reaching a 47% gain in performance and cost per million documents.

Read at AWS Machine Learning Blog

AWS Machine Learning Blog Apoorva Chandra SI

How Salesforce is advancing global-scale model serving with cross-Region inference routing in Amazon SageMaker AI

Learn how Salesforce uses the cross-Region inference router in Amazon SageMaker AI to turn fragmented Regional capacity into reliable model serving. Salesforce adds capacity securely across Regions and accounts while keeping its data residency commitments, and sees per request where each call was served and why.

Read at AWS Machine Learning Blog

arXiv (research papers) Shashwat Jaiswal

TripleKV: Predict, Prefetch, Protect -- KV Cache Management for AI Agents

LLM agents alternate between inference and tool execution, repeatedly reusing their growing context across calls. Under high concurrency, their combined key-value (KV) cache working set can reach hundreds of GiB, far exceeding GPU memory capacity. As idle KV cache blocks are evicted to slower storage tiers, returning agents incur expensive retrievals that stall inference. To address this problem, we present \emph{TripleKV}, a three-tier KV cache management system that hides…

Read at arXiv (research papers)

arXiv (research papers) William Fleshman Security

Living off the Local Model: Abusing Browser-Embedded Language Models for Post-Exploitation

Browsers now ship local language models to ordinary endpoints. Google includes Gemini Nano in Chrome to enable productivity features such as summarization, with inference running entirely on the device. These browser-embedded models provide new living-off-the-land opportunities for attackers, each instance a potential post-exploitation engine already installed, signed by the vendor, and invisible to network monitoring. The Gemini model is multimodal, capable of processing…

Read at arXiv (research papers)

arXiv (research papers) Biel Escolà-Rodrigo Space

Validation and Physical Characterization of Seven TESS Exoplanets: A Compact Multi-Planet System and Targets for Atmospheric Spectroscopy

The Transiting Exoplanet Survey Satellite (TESS) continues to identify thousands of transit candidates that require statistical validation to rule out astrophysical false positives before atmospheric characterization can be pursued. We statistically validate and characterize seven TESS exoplanet candidates orbiting five host stars. As a consistency check of our automated vetting and Bayesian inference pipeline, we include TOI-2392.03, a sub-Neptune companion previously…

Read at arXiv (research papers)

Older stories