Performance
Make a model fit, and make it fast.
Quantization trades a measurable amount of quality for a large amount of memory. Performance mode sizes each option against your actual hardware, converts to the format your runtime wants, and shows what the trade costs before you commit to it.
- The model you want does not fit in your VRAM
- Generation is too slow to be usable interactively
- You are packaging a model for a specific machine
- You need GGUF, MLX or Safetensors output
- Recovering capability the base model never had
Local coding AI
A completion and review model quantized to sit permanently in VRAM alongside your editor, with your source never leaving the machine.
Qwen2.5 7B Instruct
7.6B · 32k · Apache 2.0
The default starting point for local work on 8–12 GB cards. Strong instruction following and reliable tool-call formatting for its size.
Qwen2.5 14B Instruct
14.8B · 32k · Apache 2.0
The sweet spot for 24 GB cards. Meaningfully stronger reasoning than 7B while still fine-tunable locally with QLoRA.
Qwen2.5 32B Instruct
32.8B · 32k · Apache 2.0
Approaches 70B quality at half the memory. Runs on a single 24 GB card at Q4_K_M with a modest context window.
Qwen2.5 72B Instruct
72.7B · 32k · Qwen License
Frontier-adjacent open weights. Needs a workstation, a multi-GPU rig or a large unified-memory Mac.
Qwen2.5 Coder 7B Instruct
7.6B · 32k · Apache 2.0
The practical local copilot. Supports fill-in-the-middle, so it works as an inline completion model rather than only a chat assistant.
Qwen2.5 Coder 32B Instruct
32.8B · 32k · Apache 2.0
The strongest open code model that still fits one 24 GB card. The reason a lot of people buy a 4090.