Inference snap
Chat with generative AI Language Models that runs locally without internet. Here inference-snap is one of the supported models:
deepseek-r1,gemma3,gemma4,nemotron3-nano,qwen-vl. More information: https://documentation.ubuntu.com/inference-snaps/.
- Start a chat in terminal (and a chat server in background on first start):
chat
- Show current engine and status of chat server:
status
- Select specific engine (eg.
cudafor NVIDIA GPU). Download and use the appropriate model for the engine:
sudo use-engine
- Show detailed device hardware information (RAM, architecture, etc.):
sudo show-machine
- Get all configuration (
http.host,http.port) of running chat server:
get
来源:tldr-pages contributors · 作者与贡献记录 · CC BY 4.0。离线整理了版式、链接和媒体显示。
