Description

An educational visualization tool that follows an LLM request from client to GPU, exploring infrastructure optimizations like KV caching, continuous batching, and speculative decoding. It provides a detailed look at the internal mechanics of request routing, tokenization, and the prefill/decode phases on the GPU.

Descriptions, tags, and model credits may be AI-generated or inferred from public sources and can be incomplete or wrong. Learn more.