Knowledge
Give a model access to your own documents.
Your files are chunked, embedded and stored in a local vector index. At query time the passages that actually matter are retrieved and placed in the prompt, so the model answers from your material instead of its training data — and can cite which document it used.
- The model needs facts it was never trained on
- Source material changes and must stay editable
- Answers need to be traceable back to a document
- You have documents, not input/output examples
- Changing tone, structure or response format
- Teaching a task the model has never seen performed
Private company agent
An internal assistant that knows your documents, answers in your format, can act on internal systems, and never sends a byte outside the building.
Research assistant
Retrieval over a paper library with citations back to the source PDF. No training, no GPU required, minutes to build.
Support responder
Trained on resolved tickets for structure and tone, retrieving live product documentation for the facts.
Qwen2.5 7B Instruct
7.6B · 32k · Apache 2.0
The default starting point for local work on 8–12 GB cards. Strong instruction following and reliable tool-call formatting for its size.
Qwen2.5 14B Instruct
14.8B · 32k · Apache 2.0
The sweet spot for 24 GB cards. Meaningfully stronger reasoning than 7B while still fine-tunable locally with QLoRA.
Qwen2.5 32B Instruct
32.8B · 32k · Apache 2.0
Approaches 70B quality at half the memory. Runs on a single 24 GB card at Q4_K_M with a modest context window.
Qwen2.5 72B Instruct
72.7B · 32k · Qwen License
Frontier-adjacent open weights. Needs a workstation, a multi-GPU rig or a large unified-memory Mac.
Qwen2.5 Coder 7B Instruct
7.6B · 32k · Apache 2.0
The practical local copilot. Supports fill-in-the-middle, so it works as an inline completion model rather than only a chat assistant.
Qwen2.5 Coder 32B Instruct
32.8B · 32k · Apache 2.0
The strongest open code model that still fits one 24 GB card. The reason a lot of people buy a 4090.