Orchestra benchmark results
A 14B-parameter model with 4B active per token, on six well-known tests. The numbers below are from our own run, with the settings listed in the method section. We publish the method so your team can repeat it on its own hardware.
Results
| Benchmark | Orchestra 14B MoE, 4B active |
Gemma 4 31B dense |
Qwen3.5-27B dense |
DeepSeek-V3.2 671B MoE |
|---|---|---|---|---|
| MMLU-Pro | 88.3 | 85.2 | 86.1 | 85.0 |
| GPQA Diamond | 85.3 | 84.3 | 85.5 | 82.4 |
| LiveCodeBench v6 | 83.0 | 80.0 | 80.7 | 83.3¹ |
| HLE, no tools | 27.4 | 19.5 | 24.3 | 25.1 |
| Tau2-Bench | 83.9 | 76.9 | 79.0 | 80.3 |
| OALL v2 (Arabic) | 78.41 | — | — | — |
Orchestra figures are NoorAI's own run, November 2026. The other columns are the vendors' published figures (Gemma 4 model card, DeepSeek-V3.2 paper, TII press release). ¹ DeepSeek's LiveCodeBench window is Aug 2024 to Apr 2025. For the Arabic comparison: Falcon-H1-Arabic 34B scored 75.36 on OALL v2 per TII.
How we measured
- Model: Orchestra, 14B parameters, mixture of experts, 4B active per token, served at 8-bit.
- Run: November 2026, on NoorAI hardware. Thinking mode on unless stated; HLE without tools.
- Tests: MMLU-Pro, GPQA Diamond, LiveCodeBench v6, HLE, Tau2-Bench, and OALL v2 for Arabic.
- Comparator figures: the vendors' published numbers, not our run; their settings may differ from ours.
What these numbers mean, and what they do not
Benchmarks measure fixed questions under fixed conditions. They are a fair signal of general capability, not a promise about your own documents. That is why we always show Orchestra on the customer's own data before any decision, and why we welcome your team re-running these tests with the settings above. If you get a different number, tell us.
Size matters too: 4B active parameters per token means the model runs on a fraction of the hardware that the dense models in this comparison need. See the hardware sizing guide, or how to evaluate an Arabic model on your own data.