Making Low-Bit LLMs Actually Fast in Parallel via Vector Table Lookup
Vec-LUT turns repetitive scalar table lookups into contiguous vector reads, accelerating parallel ternary LLM inference on x86 and ARM CPUs by up to 4.2×.
Xiangyu Li
Ph.D. Candidate, AIR, Tsinghua University
I am a 4th year Ph.D. candidate at AIR, THU, working on efficient inference systems for edge AI (Ph.D. work overview). My current research interests focus on model-system co-design for embodied AI, especially VLA training and deployment.
News
See all →- 2026/07 DWI: Efficient On-Device LLM Decoding Through Dynamic Width Selection on Nested Models accepted to IEEE TMC. Congrats to Weining Chen. Paper
- 2026/06 Vec-LUT received Best Paper Award Runner-Up at ACM MobiSys 2026. PaperBlogCodeSlides
- 2026/06 ActProbe: Action-Space Probe for Early Failure Detection of Generative Robot Policies released. PaperCodePage
Selected Papers
See all →OxyGen: Unified KV Cache Management for VLA Inference under Multi-Task Parallelism
ArXiv preprint, 2026
Vec-LUT: Vector Table Lookup for Parallel Ultra-Low-Bit LLM Inference on Edge Devices
ACM MobiSys 2026Best Paper Award Runner-Up, Results Reproduced @AE
Personal LLM Agents: Insights and Survey about the Capability, Efficiency and Security
ArXiv preprint, 2024Survey & Position, “Efficiency” Section Lead
Latest Posts
See all →Enhancing GPTQv2 Format Support in vLLM: Analysis and Implementation
An analysis of GPTQv2 format limitations in vLLM and the CUDA kernel changes needed for low-bit asymmetric quantization inference.
Vision-Language-Action (VLA) Models: A Review of Recent Progress
Recent VLAs are moving from discrete to continuous control and from single-system to dual-system designs.