1Cat-vLLM

Modern LLM inference engine optimized for NVIDIA Tesla V100 / SM70 GPUs.

Open source

Description

An engineering fork of vLLM specifically optimized to restore high-performance inference to NVIDIA Volta (SM70) hardware. It achieves up to 260 tok/s on Qwen 3.8 models through custom FlashAttention-V100 kernels, 128-bit memory load optimizations, and support for NVFP4/E4M3 quantization schemes.

Descriptions, tags, and model credits may be AI-generated or inferred from public sources and can be incomplete or wrong. Learn more.

Details

Why this is interesting: Impressive engineering feat achieving 3-5x throughput gains on legacy V100 hardware, rivaling modern GPU inference…
Built with: Qwen3.8-Flash-Next GLM-5.3-Flash Qwen3.8-27B
Links: View tweet
Tags: #inference #v100 #cuda #optimization

Discovered: X
First posted: X · Oct 13, 2026
Listed: Added 5d ago
Build: Strong
Models: Strong

1Cat-vLLM

Description

How it grew

1Cat-vLLM

Description

How it grew