Robotics: research

Journal articles, academic blogs, think-tank reports and papers in this section, newest first. Back to the news.

arXiv (research papers) Binay Kumar Singh SI

Does the Action Matter? Probing Action-Conditioned Semantic Forecasting as a World Model for Autonomous Driving

World models are increasingly used in autonomous driving to predict how a scene evolves under the ego vehicle's actions, supporting counterfactual reasoning and model-based planning. However, existing evaluations focus on prediction fidelity and planning performance, and rarely verify that the predictions depend on the conditioning action. In this paper, we study action dependence in action-conditioned semantic forecasting, which predicts future semantic segmentation from…

Read at arXiv (research papers)

arXiv (research papers) Akshay L Chandra

DiWALI: Diffusion Policy Learning in World Models via Latent Action Inference from Human and Robot Play

Learning robot manipulation is constrained by the availability of action-labeled robot data, for both imitation and world models. Humans can demonstrate skills in seconds, but their demonstrations lack the required robot actions. We introduce DiWALI, a framework that learns and improves diffusion policies within a world model trained on human and robot play. DiWALI infers latent actions from consecutive frames using an inverse dynamics model, enabling world model training…

Read at arXiv (research papers)

arXiv (research papers) Wenwei Lin

RoboMorpher: Scaling Robot Morphology Representations for Cross-Embodiment Action Prediction

Learning shared action representations is central to cross-embodiment manipulation. However, identical joint values can produce different spatial configurations across robots, limiting representations learned from joint values alone. We introduce RoboMorpher, a morphology-conditioned action tokenizer that jointly reconstructs joint angles and link poses to associate control coordinates with their kinematic effects. MorphGen supplies scalable kinematic supervision by…

Read at arXiv (research papers)

arXiv (research papers) Moaiz Saeed

Planner-Controller Interaction for Ackermann Navigation: A Matched Simulation and Hardware Benchmark

Global planners and path-tracking controllers are often evaluated independently, although their interaction determines how a vehicle follows a planned path. Whether simulation accurately predicts the performance of specific planner-controller combinations on Ackermann-steered hardware remains uncertain. This study evaluated three geometric controllers (Regulated Pure Pursuit, Stanley, and Vector Pursuit) paired with two global planners (Hybrid A* and NavFn) on a 1/10-scale…

Read at arXiv (research papers)

arXiv (research papers) Phuong Tuan Dat

HINT-WAM: Hindsight-Informed Training for World Action Models

World action models (WAMs) have shown promising generalization and data efficiency in robot manipulation by coupling action generation with predictions of future visual dynamics. Translating these capabilities into reliable task execution, however, remains a challenge for existing post-training methods. Offline supervised fine-tuning provides dense supervision on expert trajectories, but its flow-matching action expert is trained only on noised demonstration actions and…

Read at arXiv (research papers)

arXiv (research papers) Randall Balestriero SI

Counterfactual World Models: JEPAs That Reason About "What If"

World models operate in a world they cannot fully observe. When a hidden disturbance, such as a current pushing a boat, changes the outcome of an action, a counterfactual question follows: what would have happened under a different action, subject to the same disturbance? Its answer separates bad decisions from bad luck. We prove that world models trained on factual transitions cannot answer it on their own, whatever their data or capacity. For many physical systems, the…

Read at arXiv (research papers)

arXiv (research papers) Minghao Xu

Hierarchical Admittance-Impedance Control for Mobile Manipulator Load Transfer

This paper presents a hierarchical admittance-impedance controller that combines an inner Cartesian impedance controller, an outer 6D admittance supervisor, and workspace-aware mobile-base assistance for guided load transfer. The method is motivated by assistive tasks such as airport baggage handling, where a robot should support a payload while a human worker guides its motion through small physical forces. Following an interaction-role interpretation of admittance and…

Read at arXiv (research papers)

arXiv (research papers) Emad Aghajanzadeh

DRIFT: Detachable Reasoning for Runtime-Flexible and Efficient End-to-End Driving

The integration of vision-language models (VLMs) into end-to-end (E2E) autonomous driving has demonstrated strong potential by providing scene description and reasoning capabilities. However, VLMs are computationally expensive, and switching language reasoning on and off at runtime without hurting closed-loop performance has not been demonstrated, which hinders practical adoption. To address these issues, we propose DRIFT, an E2E autonomous driving framework that integrates…

Read at arXiv (research papers)

arXiv (research papers) Felix Florian Schwald

Conceptual full-vehicle modeling of a modular all-wheel-drive robotic ADS test platform as a foundation for a digital twin architecture

This paper presents the development of an initial full-vehicle simulation model of the PixKit modular electric test platform in IPG CarMaker. The objective is to create a physics-based and extendable model structure that can serve as a low-level foundation for future digital twin applications. The reference vehicle is a skateboard chassis combining unconventional vehicle characteristics such as four independent wheel-hub motors and rear axle steering capable of high steering…

Read at arXiv (research papers)

arXiv (research papers) Giovanni Corsi

Hydrodynamics of robotic micro-swimmers in confinement

Inspired by seminal work by E.M. Purcell, we discuss the problem of predicting and optimizing the performance of autonomous robots swimming in confinement. Our approach is based on the explicit numerical resolution of the fluid flow induced by the swimmer, and it is validated experimentally on a cm-size prototype composed of a cargo head and a rotating propulsive tail. Using hydrodynamic scaling, we predict the performance of replicas of our prototype scaled down to mm- and…

Read at arXiv (research papers)

arXiv (research papers) Yaping Li

SHAPE: Discrete Supervision and Parallel Continuous Streams for Learning Dexterous Manipulation from Egocentric Human Data

Egocentric human recordings are far cheaper than dexterous robot demonstrations, promising stronger pretraining and direct skill learning without task-specific robot data. Yet humans and robots differ in morphology and kinematics, so human recordings lack the joint positions and actuator commands that supervise robot policies, providing only retargeted end-effector (EEF) actions. This raises two questions: how human data should supervise the policy, and how a dexterous robot…

Read at arXiv (research papers)

arXiv (research papers) Rong Zou

ALIVE: Bring Foundation Features to Life with Events

Visual foundation models provide powerful representations for diverse tasks, but their features become stale as visual content changes between image observations. Refreshing them requires new frames and repeated runs of a computationally expensive image encoder. We introduce ALIVE, an event-based method that incrementally updates native foundation model features between image observations. Given an anchor image, ALIVE uses subsequent events to transport and correct existing…

Read at arXiv (research papers)

arXiv (research papers) Nicolas Kosanovic

TinyARM-X: A Series of Open-Source, Configurable, and Transparent Torque-Controlled Desktop-Scale Robotic Manipulators

Visuomotor policies have demonstrated that imprecise low-cost robots can autonomously complete complex manipulation tasks. However, most inexpensive manipulators rely on compact and rigidly position-controlled servos for actuation, which complicate simulation, operation, and safety. This work introduces TinyARM-X, a framework for easily manufacturable, configurable, and torque-controllable low-cost ( 3 DoF robot within 10 min. Actuator modeling and dynamic compensation…

Read at arXiv (research papers)

arXiv (research papers) Mathias Jensen

OmniFish: A Bioinspired, Modular, Reconfigurable Soft Robotic Fish Enabling Multi-Modal Swimming

In this work, we introduce OmniFish, a bioinspired modular soft robotic fish designed to emulate the full spectrum of body and/or caudal fin (BCF) locomotion modes within a single robotic platform. The design combines morphology averaging across representative fish species with a tendon-driven compliant segmented spine whose effective stiffness can be reconfigured through progressive mechanical jamming. By selectively constraining body segments, OmniFish shifts deformation…

Read at arXiv (research papers)

arXiv (research papers) Fardien Azizi

Single-Controller Teleoperation and Multimodal Dataset Generation for a Mobile Manipulator

Robot learning systems require synchronized, high-quality demonstrations that combine visual, proprioceptive, and action data. However, collecting such datasets with mobile manipulators is often slowed down by fragmented teleoperation interfaces and manual post-processing workflows. This paper presents an integrated framework for teleoperated multimodal data collection on a Clearpath Dingo-O mobile base equipped with a Kinova Gen3 Lite manipulator and an Intel RealSense…

Read at arXiv (research papers)

arXiv (research papers) Jiayou Zhu Security

RECLAIM: Robot-Vision-Informed Adversarial Patch Defense via Embodied Attention Feedback

Vision-language-action policies control embodied agents through a perception-action loop grounded in physical interaction, where adversarial patches can corrupt action predictions and thereby alter subsequent observations. To mitigate these threats, many existing defenses rely on policy adaptation or auxiliary models, but require additional training or inference. We introduce RECLAIM, a training-free, inference-time defense that uses a frozen VLA's own attention response to…

Read at arXiv (research papers)

arXiv (research papers) Jingni Huang

When to Refine: Selective Search for Frozen Visual World Models

Planning with a visual world model requires deciding both which actions to execute and how long to search for them. We introduce Selective CEM Refinement (SCR), a learned rule for deciding whether to continue cross-entropy method (CEM) search with a frozen LeWorldModel. After five it- erations, the gate measures how far the mean first action chunk has moved, how much the mean terminal cost of elite plans has fallen, and how widely their first chunks vary. It returns the…

Read at arXiv (research papers)

arXiv (research papers) Wenhao Li SI

Towards Pose-Robust Few-Shot Vision-Language-Action Models

Fine-tuning a vision-language-action (VLA) model on a few dozen demonstrations is the standard way to put a manipulation policy on a robot, yet the resulting policy silently assumes that the camera and the robot base stay exactly where they were during data collection: a bumped tripod or a base re-docked a few centimetres away makes it reach for the wrong place. We show that both failures are one phenomenon, a change of the camera-to-base extrinsic that the policy has…

Read at arXiv (research papers)

arXiv (research papers) Ghadi Nehme

RoboBuilder: Agentic Robot Co-Design with Real-world Video Feedback in the Loop

Can AI agents design robots that work in the real world? We introduce RoboBuilder, a benchmark in which coding agents co-design a robot's hardware and program from standard parts, motors and a custom 3D-printed part, and every design that passes in simulation is built and tested on a physical course. Simulation alone is not enough: under public part data, none of 15 initial passing designs crossed the real course, so an agent learns the simulator from measurements it…

Read at arXiv (research papers)

arXiv (research papers) Apoorva Vashisth

RePlan-VLA: Progress-Aware Replanning for Long-Horizon Vision-Language-Action Models

Vision-Language-Action (VLA) models have emerged as a promising paradigm for general-purpose robotic manipulation, yet their performance often degrades on long-horizon tasks that require multiple interdependent manipulations. Instead of burdening the VLA policies with long-horizon reasoning, memory, action generation, progress tracking, and error recovery, we argue for leveraging their short-horizon manipulation capabilities within an execution-aware framework. To this end…

Read at arXiv (research papers)

arXiv (research papers) Himanshu Paudel

Estimation-Aware Reachability

Hamilton-Jacobi Reachability computes the states from which a system can be kept safe under worst-case disturbances, but it assumes the state is known exactly, whereas real robots act on noisy estimates whose uncertainty evolves with their motion. We extend Hamilton-Jacobi Reachability to the robot's belief, estimate and uncertainty, treating unexpected measurements as a bounded adversary and enlarging the unsafe set by the current uncertainty, so that certified states are…

Read at arXiv (research papers)

arXiv (research papers) Jiqing Wen

Learning Martial Arts Motion Skills for Animating Opponents in VR Sparring Games

Animating virtual opponents is an important yet challenging task in game development. In martial arts games, opponent motions must not only be physically plausible but also exhibit appropriate technique, timing, transitions, and target-directed intent. Traditional game animations often rely on handcrafted clips, state machine transitions, and procedural adjustments, which can be labor-intensive and may limit the behavioral diversity of non-player characters. This paper…

Read at arXiv (research papers)

arXiv (research papers) Soham Dan

RoboDx: Repair and Diagnosis of Faults in Robot Software

A patch to a robot's navigation software can pass every unit test and still drive the robot off its path. Checking how the repaired robot drives catches such patches. It does not show that the agent found everything that was wrong. We introduce RoboDx, a generator of repair tasks for simulated robot navigation software with known injected faults. A hidden verifier marks a repair as verified only if the robot then drives correctly in hidden scenarios, which the agent never…

Read at arXiv (research papers)

arXiv (research papers) Yunzhen Zhang

TacPilot: Does Touch Need to Be Everywhere?

Tactile feedback helps visual manipulation policies respond to local contact, but changes in tactile observations do not always require corrective motion, and sustained corrections do not always require new tactile features. We introduce TacPilot, a tactile residual framework that separates representation updates from action intervention around a frozen visual policy. TacPilot uses a perception gate to determine when tactile features need refreshing and updates only affected…

Read at arXiv (research papers)

arXiv (research papers) Jinhao Zhang

CoT-WAM: Collaborative Training of Vision-Language Models and World-Action Models via Plan Latents

General robot manipulation requires both semantic competence and physical foresight. World action models (WAMs) acquire an understanding of and predictive capability over the physical world by jointly modeling visual foresight and action distributions. However, existing WAMs still lack semantic competence: they have limited ability to perform open-world scene understanding, semantic reasoning, and long-horizon planning. Although vision-language models (VLMs) provide these…

Read at arXiv (research papers)

arXiv (research papers) Yang Li

Improve How You Improve: Recursive Policy Self-Improvement for Zero-Shot Open-Vocabulary Multi-Object Navigation

To advance the development of embodied general navigation, zero-shot open-vocabulary multi-object navigation has attracted increasing attention, aiming to enable robots to efficiently search multiple objects in unseen environments without training. For this task, the key lies in improving navigation policies from earlier searches to locate subsequent goals more efficiently. Recent methods automate this process through trajectory reflection and rule revision, enabling policy…

Read at arXiv (research papers)

arXiv (research papers) Chenxi Han

ZERITH V2.5: From Predictive Power to Policy Learnability

When humans learn to manipulate an unfamiliar object, they may already be able to anticipate how its state will change while still being unable to control it skillfully. This gap between prediction and control motivates us to study how the priors acquired by video generation models can be used effectively by an action policy. We present ZERITH V2.5, a video-action model (VAM) in which the action policy reads intermediate features of a video generation model, and use it to…

Read at arXiv (research papers)

arXiv (research papers) Xuji Zhang

RecallFlow: Episodic Memory as the Source Distribution of Generative Robot Policies

Diffusion and flow-matching policies have become a standard choice for visuomotor imitation learning, owing to their ability to model multimodal action distributions. In memory-dependent manipulation, the robot must act on information that is no longer visible. Existing methods feed the observation history to the policy as an additional condition, while action generation still starts from Gaussian noise. However, the policy must then pick out the relevant target from a long…

Read at arXiv (research papers)

arXiv (research papers) Yangang Ren

Improving While Deployed: Learning Proactive End-to-End Driving from Imperfect Human Takeover Data

Once deployed on vehicles, end-to-end driving models continuously encounter challenging scenarios, generating valuable human takeover data that capture both model outputs and human behaviors. However, human interventions are often imperfect, as drivers may intervene too late or unnecessarily. Consequently, direct imitation can reinforce late responses or discourage otherwise appropriate maneuvers. To address this challenge, we propose an offline post-training framework that…

Read at arXiv (research papers)

arXiv (research papers) Keke Tang

ExecGap: Exploiting Command-Execution Gap for Patch Attacks on Vision-Language-Action Models

Vision-Language-Action (VLA) models have become a powerful paradigm for robotic manipulation and show strong promise for real-world deployment, making adversarial robustness critical for reliable robot operation. Existing adversarial attacks on VLA-controlled robots mainly optimize prediction-level discrepancies or rollout-level deviations to disrupt manipulation. However, they rarely account for the fact that VLA-predicted commands are not executed directly, but must be…

Read at arXiv (research papers)

Older stories