Open source
Ollama
Models running on your own machine, with nothing leaving the room.
Where it has an edge
- Client data that cannot go to an outside API
- Offline work, on a plane or a bad connection
- Testing one prompt against several open models
- Learning how a model behaves with no meter running
How I prompt it
Small models need simpler jobs and much more context in the prompt. I keep one file per task with the system prompt in it, and run the same prompt against three or four models before picking one. Low temperature and a strict output format make a small model far more dependable.
Images
Some open models generate images, but the good ones want a serious graphics card. On a laptop I use it for text and code.
Where it slips
A model that fits on a laptop is weaker than the hosted ones, and you become the person who updates it. Budget for real hardware and a little patience.