Back to LaoJin's Picks
Classics · Evergreen projects that stood the test of timeModels & Infra
llama.cpp
Star snapshot：⭐ 121.1k · 2026-07 snapshot
Open GitHub repoWhy it's picked
The engine under local inference: Ollama and countless desktop AI apps are built on it. Unavoidable when you need models on low-spec machines or edge devices, or fine control over quantization and VRAM.
Laojin's advice
Ollama covers daily needs; reach for llama.cpp when you need to squeeze performance. Q4_K_M GGUF is the quality/size sweet spot; its server mode doubles as an OpenAI-compatible endpoint.
Stars are a 2026-07 snapshot for scale reference only; see GitHub for live numbers.