Description

MeshLLM is a distributed system built in Rust that pools GPU resources and memory across multiple machines into a single OpenAI-compatible API. It utilizes 'Skippy stage splits' to divide model layers across machines, enabling inference for models (70B+) that exceed the capacity of any single device.

Descriptions, tags, and model credits may be AI-generated or inferred from public sources and can be incomplete or wrong. Learn more.