Self-hosted AI Cluster

Get the best builds by email. No account needed.

Explore

Apps & Tools

Self-hosted AI Cluster

A distributed LLM inference server built from a cluster of low-power mini-PCs.

byPraveen Kumar Verma

Is this your creation?

Description

A hardware and software project that connects multiple small PCs into a distributed cluster for hosting LLMs. The setup explores routing separate requests to different machines or splitting large models across multiple nodes based on RAM and network bandwidth to maximize inference performance on existing hardware.

Descriptions, tags, and model credits may be AI-generated or inferred from public sources and can be incomplete or wrong. Learn more.

Get the best builds by email. No account needed.

Explore

Apps & Tools