Enabling Large Model Inference in Dynamic and Heterogeneous Mobile Robots by Incorporating Collaborative Inference and Model Context Protocol
Modern robotic systems increasingly rely on transformer-based AI models to process sensor input and support high-level decision making. However, due to limited GPU resources in each ego robot, it is difficult to run a large model locally. Outsourcing the model inference to cloud providers could address this issue, which however suffers from limitations including unreliable network connection, high latency, and high monetary cost. In this work, we have developed an effective…