Not a synthetic leaderboard. This index compares local language models against the everyday work of building products: thinking through decisions, writing code, reviewing risk, and handling confidential information.
Every model takes the same two practical tests in each category. The question is not “which model wins?” but “which model is the best fit for this task?”
System design, work-package planning, and commercially grounded critique.
For: decisions before implementationImplementation from clear acceptance criteria and refactoring without avoidable rework.
For: shipping focused codeAsyncTokenBucket implementation with clean lock management, accurate time replenishment without float drift, and types.Race conditions, access-control failures, and resilient remediation.
For: catching costly mistakesStrict structured output and concise synthesis of sensitive source material.
For: local, trustworthy utilitySort by the outcome you care about, filter the kind of model you can run, then select any row to see its full result profile.
Showing the built-in results snapshot.
28 model configurations shown · ranked by overall quality.
Each point uses the same active filters as the ranking. Select a point to open its full result profile.
Quality is the average of four archetype scores. Speed is average streamed completion tokens per second, measured end-to-end; it includes time spent reasoning before the first visible answer.
Each model received the same eight prompts across four everyday engineering archetypes. Evaluation applies deterministic AST syntax checks, strict JSON schema validation, defect pattern matching, and factual extraction alongside focused rubrics for architecture and unit economics. Keyword, defect and fact checks are synonym-aware: common variants such as “dedup” for deduplication or “lock ordering” for lock acquisition order count as a match, so models are scored on the concept rather than one exact phrasing.
Tested with deep chain-of-thought enabled to observe real-world analytical depth and latency trade-offs:
reasoning_effort: "high"max_tokens: 8,192 to eliminate reasoning truncation where models spend 2k+ tokens thinking before writing code.delta.reasoning_content (Qwen/Bonsai/Ornith) and delta.reasoning (GPT-OSS).Tested without internal thinking monologue to measure instantaneous zero-shot synthesis and raw throughput:
reasoning_effort: "none" or standard instruct chat template.lms unload --all) to purge Metal computational graphs and release all Unified Memory.messages: [{role: "user"}]) with non-overlapping prefixes, ensuring each test begins at token index 0 in an empty KV cache.-c 32768), sampling temperature 0.2 across all tests.