Replace OpenAI GPT with another LLM in your app by changing a single line of code. Xinference gives you the freedom to use any LLM you need. With Xinference, you're empowered to run inference with any open-source language models, speech recognition models, and multimodal models, whether in the cloud, on-premises, or even on your laptop.
machine-learning
deployment
inference
pytorch
artificial-intelligence
llama
whisper
gemma
mistral
openai-api
llm
flan-t5
chatglm
llamacpp
vllm
ggml
wizardlm
qwen
llama3
glm4
-
Updated
Nov 22, 2024 - Python