Quantization

Deploying and Accelerating Cosmos 3 Robot Policy on RTX 4090s

2026/08/21

How quantization, sampler tuning, kernel work, and rollout topology accelerate NVIDIA's 16B Cosmos 3 robot policies on 24 GB GPUs — from single-request serving to full 1,200-episode evaluations.

Making Low-Bit LLMs Actually Fast in Parallel via Vector Table Lookup

2026/07/11

Vec-LUT turns repetitive scalar table lookups into contiguous vector reads, accelerating parallel ternary LLM inference on x86 and ARM CPUs by up to 4.2×.

Enhancing GPTQv2 Format Support in vLLM: Analysis and Implementation

2025/10/12

An analysis of GPTQv2 format limitations in vLLM and the CUDA kernel changes needed for low-bit asymmetric quantization inference.