Side by side
Compare models
Pick two or three models. Every figure comes from the test reports; nothing is recalculated here.
| Measure | |||
|---|---|---|---|
| Build and settings | |||
| Maker | Alibaba | Tencent | Tencent |
| Build | Unsloth UD-Q4_K_XL GGUF, revision f5d175d | AngelSlim SPARK-4Bit-Fast GGUF | AngelSlim IQ2 GGUF, about 3.5 bits per weight, revision c65d2c9 |
| File size | 104 GB | 149 GB | 310 GB |
| Machine | NVIDIA DGX Spark, 128 GB | NVIDIA DGX Spark, 128 GB | Mac Studio M3 Ultra, 512 GB |
| Runtime | llama.cpp 2cbca38 with the Hy4 Spark patch | llama.cpp 2cbca38 with the Hy4 Spark patch | llama.cpp PR #32517 at f8d4cc4, Metal |
| Sampling | temperature 1.0, top-p 0.95, top-k 20 (maker thinking-mode settings) | temperature 0.9, top-p 1.0 (maker settings) | temperature 0.9, top-p 1.0 (maker settings) |
| Thinking | on (default) | on (default) | on (default) |
| Speed | |||
| Writing speed, short prompt | 27.7 tok/s | 25.0 tok/s | 12.2 tok/s |
| Writing speed after a 32K prompt | 25.9 tok/s | 23.6 tok/s | 10.9 tok/s |
| Writing speed after a 127K prompt | 23.9 tok/s | 21.7 tok/s | 10.0 tok/s |
| Reading speed, 32K prompt | 970 tok/s | 675 tok/s | 79 tok/s |
| Reading speed, 127K prompt | 933 tok/s | 580 tok/s | 75 tok/s |
| Wait for first token, 32K prompt | 33 s | 47 s | 404 s |
| Wait for first token, 127K prompt | 136 s (2.3 min) | 219 s (3.6 min) | 1,680 s (28 min) |
| Load time | 97 s (1.6 min) | 160 s (2.7 min) | 13 s (warm) |
| Load time, cold | no figure | no figure | 123 s (2.0 min) |
| Least memory left free | 7 GiB | 26 GiB | 139 GiB |
| Time for all 27 attempts | 7.6 hours | 5.1 hours | 17.5 hours |
| Tokens generated, all attempts | 731,765 tokens | 447,477 tokens | 714,824 tokens |
| Test 01: Qwen3.8 Flash Next vs Hy4 Flash | |||
| Pelican on a bicycleThe drawings | |||
| Octopus and penguin bandThe drawings | |||
| Pelican on a bicycleValid drawings, of 3 attemptsOpinion | 3 | 3 | 3 |
| Octopus and penguin bandValid drawings, of 3 attemptsOpinion | 3 | 3 | 3 |
| Playable browser gameComplete gamesAutomatic load test; the rest is opinion | 2 of 3 (1 cut off at the answer limit) | 3 of 3 | 2 of 3 (1 cut off at the answer limit) |
| RoastMy ranking (opinion)Opinion | Place 3 of 3 | Place 2 of 3 | Place 1 of 3 |
| Text adventureComplete gamesRules check plus opinion | 3 of 3 | 3 of 3 (2 without an ending) | 0 of 3: empty answer on turn 2, probably a runtime fault, unconfirmed |
| Bug fixBugs fixed of 3, per attemptHidden tests, automatic | 3, 3, 3 | 3, 3, 3 | 3, 3, 3 |
| Bug fixMinutes per attempt | 26 to 33 | 6 to 9 | over 60 |
| Contract questionsScore of 5, per attemptAnswer key | 4, 5, 5 | 4, 5, 5 | 5, 5, 5 |
| Contract questionsSlip | Attempt 1 read the wrong row of the service credit table: DKK 248 where the answer is DKK 496. | Attempt 1 found the 120-day special term but missed the leap day and gave 31 October 2027. | None |
| 12 steps with a contradictionFlagged the contradictionAnswer key | 1 of 3 | 2 of 3, plus one partial | 2 of 3 |
| 12 steps with a contradictionArithmetic right | 3 of 3 | 3 of 3 | 3 of 3 |
| Network designWords per planOpinion | about 5,000 | 1,600 to 2,300 | 3,200 to 3,800 |
| Network designAccess points proposed | 4 to 12 | 4 to 6 | 6 to 11 |
- The full Hy4 is not a fair third contestant: different machine, different runtime, much heavier compression. Read its column as what the big model does on a Mac, not as a ranking against the other two.
- Short-prompt writing speed is the average of three runs; long-prompt rows are one run each.
- Hy4 Flash speed rows come from the first pilot night (same files and runtime); Qwen rows from the rerun on the current build.
- Speed tests use one fixed sampling setting for every model.
- About nine tenths of what all three models wrote was thinking, not answer.