LLMBench
LLM general intelligence leaderboard for 16 GB graphics cards
Which local LLM thinks best on a graphics card with 16 GB? Every model here fits completely into 16 GB of VRAM, and all 12 answer the same 662 reasoning questions on the same RTX 5070 Ti. Spatial and physical reasoning, keeping track of objects, altered classic puzzles, logic, math, time and social reasoning: tasks where a model has to think, not recall.
- questions
- 528
- question variants
- 662
- models
- 12
- GPU, 16 GB VRAM
- RTX 5070 Ti
| # | Rank | Model | Strength | Quality per time | Accuracy | Correct | s / answer | Quality | |
|---|---|---|---|---|---|---|---|---|---|
| 1 | 1 | qwen3.8-27bQwen3.8-27B-UD-IQ4_XS | 64×46–89 | 2.6/ min | 92.4 %90–94 % | 612 / 662 | 19.8 | 87.3 % | |
| 2 | 1 | gemma4-26b-a4bgemma-4-26B-A4B-it-UD-Q4_K_XL | 48×35–65 | 9.1/ min | 90.6 %88–93 % | 600 / 662 | 5.5 | 83.2 % | |
| 3 | 3 | gemma4-12bgemma-4-12b-it-Q8_0 | 33×25–43 | 12.3/ min | 88.1 %85–90 % | 583 / 662 | 3.8 | 79.0 % | |
| 4 | 4 | qwen3.5-9bQwen3.5-9B-Q8_0 | 20×16–26 | 3.3/ min | 83.7 %81–86 % | 554 / 662 | 13.5 | 74.0 % | |
| 5 | 4 | muse-glimmer-30b†Muse-Glimmer-30B-IQ4_XS | 15×12–19 | 1.2/ min | 81.3 %78–84 % | 538 / 662 | 39.8 | 79.2 % | |
| 6 | 5 | gpt-oss-20b†gpt-oss-20b-UD-Q8_K_XL | 14×11–18 | 4.2/ min | 79.8 %77–83 % | 528 / 662 | 9.9 | 69.8 % | |
| 7 | 6 | gemma4-e4bgemma-4-E4B-it-UD-Q8_K_XL | 10×8.1–13 | 5.5/ min | 76.1 %73–79 % | 504 / 662 | 6.2 | 57.3 % | |
| 8 | 7 | ministral3-14b†Ministral-3-14B-Instruct-2512-UD-Q6_K_XL | 7.5×6.1–9.3 | 1.9/ min | 72.7 %69–76 % | 481 / 662 | 17.4 | 55.0 % | |
| 9 | 8 | phi-4-14bphi-4-Q6_K_L | 7.4×6.0–9.1 | 6.0/ min | 71.8 %68–75 % | 475 / 662 | 5.0 | 49.2 % | |
| 10 | 10 | granite-4.2-8b†granite-4.2-8b-Q8_0 | 4.9×4.0–6.0 | 1.0/ min | 66.6 %63–70 % | 441 / 662 | 32.1 | 55.5 % | |
| 11 | 11 | llama3.1-8b†Llama-3.1-8B-Instruct-UD-Q8_K_XL | 1.0×0.82–1.2 | 0.2/ min | 41.1 %37–45 % | 272 / 662 | 10.7 | 3.1 % | |
| 12 | 12 | mistral-7b-v0.3†Mistral-7B-Instruct-v0.3-Q8_0 | 0.38×0.31–0.48 | −8.4/ min | 27.5 %24–31 % | 182 / 662 | 1.7 | −23.5 % |
† 5 % or more of the answers were cut off at the output limit or had no readable final answer. They count as wrong, so the model may do better with a larger limit.
Click a column header to sort. Rows with rank 1 are highlighted.
How to read the columns
- Rank
- 1 + the number of models that are significantly better. Two models share a rank when the difference between them could be chance: every pair of models is compared question by question (paired McNemar test, p < 0.05).
- Model
- The model name and below it the exact model file (quantization), so the result can be reproduced.
- Strength
- How much better a model is, as a plain factor: how many times higher its odds (right : wrong) of solving a question are than those of llama3.1-8b (= 1×). Example: on a question that llama3.1-8b gets right half of the time (odds 1 : 1), qwen3.8-27b with 64× has odds of 64 : 1 and is right 98 % of the time; on a question llama3.1-8b gets right only 1 time in 10 (odds 1 : 9), it is right 88 % of the time. Ratios between models are meaningful too: 40× has twice the odds of 20×. The chart uses a log scale; the line shows the 95 % range. Where two lines overlap a lot, the order of the two models is not certain.
- Quality per time
- How much good answering a model delivers per minute: right minus wrong answers per minute of model time on the RTX 5070 Ti. A model that is a little weaker but much faster can be the better choice in daily use; this column shows it. All models run completely in VRAM, so another graphics card mostly makes every model faster or slower by a similar factor.
- Accuracy
- The share of correct answers. Below: the 95 % range (Wilson interval). It puts the models in the same order as the strength, but squeezes the top: near 100 % every further percent is much harder to gain than in the middle, the strength shows that.
- Correct
- Correct answers out of all scored answers.
- s / answer
- Mean time per answer in seconds on the RTX 5070 Ti. Depends on the speed of the model and on how much it writes before it answers.
- Quality
- Right minus wrong answers per 100 answers: what the answers are worth if you trust them. No usable answer counts 0. A wrong multiple-choice answer with k options costs only 1/(k−1), so guessing gains nothing. A model that is often wrong drops further here than in accuracy. Quality per time is the same value per minute instead of per answer.
How the benchmark works
Questions. 528 questions, many of them in two forms (free answer and multiple choice), 662 variants in total. Some are well-known puzzles with a twist that changes the answer, so a memorized solution is wrong. The options of multiple-choice questions are shuffled the same way for every model.
Same conditions for every model. Each model runs locally in llama-server on one RTX 5070 Ti, completely in its 16 GB of VRAM, with temperature 0, seed 42 and at most 7,000 output tokens per answer. Models with a thinking switch run with thinking off, the others as they come. Every question is sent on its own, with this system prompt:
You are a helpful assistant. Answer the question accurately. You may reason first. End your reply with a last line "Final Answer:" followed by your answer.
Scoring. Only the final answer counts: the line starting with “Final Answer:”, compared with the solution (numbers as numbers, options by letter or text). The reasoning before it is not graded. Answers cut off at the output limit count as wrong.
Strength scale. The difficulty of every question was estimated once from the answers of all 12 models (Rasch model with a guessing floor for multiple choice) on 2026-11-14 and is now fixed. Models added later are placed on the same scale without changing the values above, and llama3.1-8b stays the reference (= 1×).