vllm chat
Generate chat completions via the running API server using the OpenAI-compatible REST API. More information: https://docs.vllm.ai/en/latest/serving/openai_compatible_server.html.
- Send a single chat prompt to the API server and get a response:
vllm chat --quick ""
- Chat with a specific model name:
vllm chat --model-name --quick ""
- Chat with a custom system prompt:
vllm chat --system-prompt "" --quick ""
- Chat using a custom API server URL:
vllm chat --url --quick ""
- Chat with an API key for authentication:
vllm chat --api-key --quick ""
- Start an interactive chat session:
vllm chat
来源:tldr-pages contributors · 作者与贡献记录 · CC BY 4.0。离线整理了版式、链接和媒体显示。
