Outrageously Small Neural Networks
A 3.65M-parameter MoE model demonstrating emergent reasoning on a single CPU core.
Is this your creation?
Description
A research project exploring the limits of tiny neural networks by training a 3.65M active-parameter block-routed Mixture-of-Experts (MoE) model on a single Intel AMX CPU core. The model demonstrates emergent behaviors like in-context induction and two-digit addition at 6,616 tok/s, proving that reasoning capabilities can appear much earlier in the token budget than expected through effective distillation from large-model curated corpora.
Descriptions, tags, and model credits may be AI-generated or inferred from public sources and can be incomplete or wrong. Learn more.