GPUStack

GPUStack is an open-source GPU cluster manager developed to provide AI model service and GPU samples. In-house servers manage multiple GPUs in Kubernetes cluster and cloud environments from a single hub; it automatically configures inference engines such as vLLM, SGLang, and TensorRT-LLM. It offers preset modes for low latency or high efficiency and features enterprise operation such as automatic failure recovery, load balancing, authentication.