Big Moe on Edge

Run massive MoE models (up to 284B) on low-RAM phones and PCs via flash-streaming.

Open source

Description

An inference engine built on llama.cpp that enables running Mixture-of-Experts (MoE) models far exceeding available RAM. It works by streaming only the required experts from flash storage (UFS/SSD) just-in-time for each token, allowing a 284B DeepSeek V4 model to run on a 12GB phone.

Descriptions, tags, and model credits may be AI-generated or inferred from public sources and can be incomplete or wrong. Learn more.

Details

Why this is interesting: Successfully runs a massive 284B parameter model on a 12GB smartphone, demonstrating a major breakthrough in local AI…
Attributed to: Raffaele Mancini@raffomancio
Built with: Seedance 2.0
Links: View tweet
Tags: #llm #android #llama-cpp #moe

Discovered: X
First posted: X · Aug 19, 2026
Listed: Added 52d ago
Build: Strong
Models: Strong

Big Moe on Edge

Description

How it grew

Big Moe on Edge

Description

How it grew