Running Cosmos 3 Robot Policies on a Single RTX 4090
How quantization, sampler tuning, and kernel work put NVIDIA's 16B Cosmos 3 robot policy on one 24 GB GPU — and turned the runtime into a parallel rollout engine.
Xiangyu Li
Ph.D. Candidate, AIR, Tsinghua University
I am a 4th year Ph.D. candidate at AIR, THU, working on on-device AI (Ph.D. work overview) and embodied AI. Currently I am exploring efficient deployment of embodied foundation models (VLAs and WAMs), as well as self-evolving physical intelligence.
News
See all →- 2026/08 Cosmos Lite updated to v0.3.0: add RoboLab parallel rollout support and RoboLab-120 benchmark results. Blog Code Models
- 2026/08 OxyGen updated: add suffix-only LoRA training and inference support, plus a LIBERO textual-memory example. Paper Code Page
- 2026/08 Zetta ζ released: an efficient closed-loop embodied harness for self-evolving physical intelligence. Congrats to team! Paper Code Page 机器之心
Selected Papers
See all →OxyGen: Unified KV Cache Management for VLA Inference under Multi-Task Parallelism
ArXiv preprint, 2026
Vec-LUT: Vector Table Lookup for Parallel Ultra-Low-Bit LLM Inference on Edge Devices
ACM MobiSys 2026Best Paper Award Runner-Up, Results Reproduced @AE
Personal LLM Agents: Insights and Survey about the Capability, Efficiency and Security
ArXiv preprint, 2024Survey & Position, “Efficiency” Section Lead
Latest Posts
See all →Making Low-Bit LLMs Actually Fast in Parallel via Vector Table Lookup
Vec-LUT turns repetitive scalar table lookups into contiguous vector reads, accelerating parallel ternary LLM inference on x86 and ARM CPUs by up to 4.2×.
Enhancing GPTQv2 Format Support in vLLM: Analysis and Implementation
An analysis of GPTQv2 format limitations in vLLM and the CUDA kernel changes needed for low-bit asymmetric quantization inference.