llama-swap
Llama-swap is a tool written in the Go language that allows model switching between local AI servers on demand. It has no external dependency and can be run through a configuration file with a single binary file. With OpenAI, Anthropic directs various endpoints that support API standards.
mostlygeek · Local AI / Self-Hosted LLM
Common commercial products in this category: ChatGPT, Claude, Gemini
- You can make instant model switching between your local AI servers.
- You can manage requests through OpenAI and Anthropic compatible endpoint.
- You can list working models and manually remove them from memory.
- ↓ NegativeGitHub Issue
Attempting to connect to upstream llama-server ui via llama-swap results in a 404
“**Describe the bug** I have just updated the llama-swap docker container to the latest cuda version (v159-cuda-b6503) and the upstream web UI no longer works. This is likely caused by the new svelte web UI in llama-server. Looking at the logs, included below, and confirmed in…”
- • NeutralGitHub Issue
Llama-swap is closing the connection after the first request
“Hello partner, When I check you can see both models. You can select a model, and it loads correctly. However, after sending just one message, the model stops responding. The logs show that after the first request, a 404 error appears on /upstream/qwen2.5/v1/chat/completions. I'm…”