Xinference

Xinference is a library developed to run language, speech recognition and multi-mode models on distributed systems. It manages the process of serving models in the production environment with features such as automatic batch processing (auto-batch), continuous batch processing support, and shared KV cache. It also offers dynamic planning and autonomous reasoning capabilities with Xagent integration.