Description

This hardware-software hybrid setup utilizes Tinygrad's 'chestnut' to connect an NVIDIA GPU to an Apple Silicon Mac via USB-C. It leverages the Strata framework for advanced memory tiering across Mac RAM, GPU VRAM, and local storage, achieving 80 tok/s decode and over 1,000 tok/s prefill on a 2-bit quantized model.

Descriptions, tags, and model credits may be AI-generated or inferred from public sources and can be incomplete or wrong. Learn more.