Apps & Tools
Edge0-145B Q3_K
High-speed local inference for a 145B sparse model on a single consumer GPU
byEdge0
Is this your creation?
PlayableOpen source
Description
An optimized implementation of the Edge0-145B sparse MoE model that achieves 134.3 tok/s decode on a single RTX 5090 by keeping experts in system memory and reading from SSD. The system uses a custom Edge0 engine built from source to handle the 145B parameters with only 10.12B active per token, enabling massive context local runs.
Descriptions, tags, and model credits may be AI-generated or inferred from public sources and can be incomplete or wrong. Learn more.