NInfer-3090

All projects

Apps & Tools

Fast Qwen3.8-27B inference on one RTX 3090 with MTP3 and ReplaySSM.

byDon-Chad

Open source

NInfer-3090 media is blocked

Allow external media to connect to the provider and play this content.

Description

A specialized C++20/CUDA inference engine designed to run Qwen3.8-27B and Qwen3.6 models efficiently on a single 24GB NVIDIA RTX 3090. It supports OpenAI/Anthropic APIs, speculative decoding (MTP3), ReplaySSM state transactions, and reasoning-effort controls for models with deliberation features. The engine includes native Windows and Linux builds with support for vision-enabled artifacts.

Descriptions, tags, and model credits may be AI-generated or inferred from public sources and can be incomplete or wrong. Learn more.

Details

Why this is interesting: Highly optimized local inference engine achieving impressive throughput for Qwen3.8 on a single consumer RTX 3090.
Attributed to: @Don-Chad
Built with: Qwen3.8-27B Grok Voice Think Fast 1.0 Veo 3.1 Fast Qwen3 14B Qwen3 8B Qwen3 30B-A3B Qwen3.6
Links: View tweet
Tags: #inference #cuda #qwen #gpu-optimization

Discovered: X
First posted: X · Aug 17, 2026
Listed: Added 8d ago
Build: Strong
Models: Strong

NInfer-3090

Description

Timeline

NInfer-3090

Description

Timeline