Description

An experimental language model with only 3.3M active parameters trained end-to-end on a single Intel Emerald Rapids core. It features a custom architecture of chunked sliding-window and log-decay linear attention to achieve extractive QA capabilities and emergent reasoning behaviors usually reserved for much larger models.

Descriptions, tags, and model credits may be AI-generated or inferred from public sources and can be incomplete or wrong. Learn more.