Knowledge & Reasoning

MMLU Pro Leaderboard

Massive Multitask Language Understanding - Professional Edition

Why This Matters

Tests breadth of knowledge across 57 professional domains - critical for general-purpose AI applications

Good Scores

70%+ is strong, 80%+ is excellent, 85%+ is state-of-the-art

Use Cases

  • •Research assistants
  • •Educational tools
  • •Professional advisory systems
  • •General Q&A chatbots

Peak Score

70.03

Average

25.05

Models Tested

53,438

Median Score

26.31

Efficiency Leaders

Best performance per billion parameters - The smart choices

#1

ChatWaifu_v1.4

100.0M params • Score: 27.50

Efficiency

274.99

#2

ChatWaifu_v1.4

100.0M params • Score: 27.50

Efficiency

274.99

#3

ChatWaifu_v1.4

100.0M params • Score: 27.50

Efficiency

274.99

#4

ChatWaifu_v1.4

100.0M params • Score: 27.50

Efficiency

274.99

#5

ChatWaifu_v1.4

100.0M params • Score: 27.50

Efficiency

274.99

#6

ChatWaifu_v1.4

100.0M params • Score: 27.50

Efficiency

274.99

#7

ChatWaifu_v1.4

100.0M params • Score: 27.50

Efficiency

274.99

#8

ChatWaifu_v1.4

100.0M params • Score: 27.50

Efficiency

274.99

#9

ChatWaifu_v1.4

100.0M params • Score: 27.50

Efficiency

274.99

#10

ChatWaifu_v1.4

100.0M params • Score: 27.50

Efficiency

274.99

Performance by Model Size

How different size classes perform on this benchmark

small

< 1B

Avg Score: 5.83

ChatWaifu_v1.427.50
ChatWaifu_v1.427.50
ChatWaifu_v1.427.50

medium

1B - 10B

Avg Score: 23.79

Luminis-phi-449.15
Luminis-phi-449.15
Luminis-phi-449.15

large

10B - 50B

Avg Score: 34.76

ultiima-32B54.56
ultiima-32B54.56
ultiima-32B54.56

xlarge

> 50B

Avg Score: 45.45

Calme 3.2 Instruct 78b70.03
Calme 3.2 Instruct 78b70.03
Calme 3.2 Instruct 78b70.03

🏆 Open Source Champions

Top permissively licensed models

CalmeRys-78B-Orpo-v0.1

license:mit

66.80

CalmeRys-78B-Orpo-v0.1

license:mit

66.80

CalmeRys-78B-Orpo-v0.1

license:mit

66.80

CalmeRys-78B-Orpo-v0.1

license:mit

66.80

CalmeRys-78B-Orpo-v0.1

license:mit

66.80

📈 Most Downloaded Models

Popularity meets performance

gpt2

11.8M downloads • Rank #49215

1.84

gpt2

11.8M downloads • Rank #49215

1.84

gpt2

11.8M downloads • Rank #49215

1.84

gpt2

11.8M downloads • Rank #49215

1.84

gpt2

11.8M downloads • Rank #49215

1.84

📄 License Analysis

Performance by license type

Unknown70.03
14487 modelsAvg: 24.23
license:mit66.80
2592 modelsAvg: 23.68
license:apache-2.054.56
14019 modelsAvg: 27.31
llama50.39
20447 modelsAvg: 24.52
license:gpl-3.047.69
168 modelsAvg: 13.54
license:cc-by-nc-4.045.46
1019 modelsAvg: 23.78
license:cc-by-nc-sa-4.038.28
46 modelsAvg: 31.79
dataset:HPAI-BSC/pubmedqa-cot-llama3137.27
24 modelsAvg: 37.27

🔧 Framework Analysis

Performance by framework

OTHER70.03
53299 modelsAvg: 25.09
PYTORCH31.09
116 modelsAvg: 12.48
HUGGINGFACE1.27
23 modelsAvg: 1.27

About MMLU Pro

MMLU Pro is an enhanced version of the MMLU benchmark, designed to test language models across 57 professional subjects including STEM, humanities, social sciences, and more. It provides a comprehensive evaluation of a model's knowledge and reasoning capabilities.

Last updated: 11/16/2025

Test These Models Yourself

Run benchmarks on your own data with these platforms

Together.ai

Instant API access to this model

Fastest API

Production-ready inference API. Start free, scale to millions.

Try Free API

Modal

Run this model on serverless GPU

Most Popular

Deploy in seconds with $30 free credits. Pay only for what you use.

Get $30 Free Credits

RunPod

Rent GPU starting at $0.34/hour

Best Value

Deploy on cloud GPU or serverless. 70% cheaper than AWS.

Start from $0.34/hr

Disclosure: We may earn a commission from these partners. This helps keep LLMYourWay free.

Complete Leaderboard

Top 50 models ranked by MMLU Pro performance

Calme 3.2 Instruct 78b

MaziyarPanahi

70.03

Score

Calme 3.2 Instruct 78b

MaziyarPanahi

70.03

Score

Calme 3.2 Instruct 78b

MaziyarPanahi

70.03

Score

#4

Calme 3.2 Instruct 78b

MaziyarPanahi

70.03

Score

#5

Calme 3.2 Instruct 78b

MaziyarPanahi

70.03

Score

#6

Calme 3.2 Instruct 78b

MaziyarPanahi

70.03

Score

#7

Calme 3.2 Instruct 78b

MaziyarPanahi

70.03

Score

#8

Calme 3.2 Instruct 78b

MaziyarPanahi

70.03

Score

#9

Calme 3.2 Instruct 78b

MaziyarPanahi

70.03

Score

#10

Calme 3.2 Instruct 78b

MaziyarPanahi

70.03

Score

#11

Calme 3.2 Instruct 78b

MaziyarPanahi

70.03

Score

#12

Calme 3.2 Instruct 78b

MaziyarPanahi

70.03

Score

#13

Calme 3.2 Instruct 78b

MaziyarPanahi

70.03

Score

#14

Calme 3.2 Instruct 78b

MaziyarPanahi

70.03

Score

#15

Calme 3.2 Instruct 78b

MaziyarPanahi

70.03

Score

#16

Calme 3.2 Instruct 78b

MaziyarPanahi

70.03

Score

#17

Calme 3.2 Instruct 78b

MaziyarPanahi

70.03

Score

#18

Calme 3.2 Instruct 78b

MaziyarPanahi

70.03

Score

#19

Calme 3.2 Instruct 78b

MaziyarPanahi

70.03

Score

#20

Calme 3.2 Instruct 78b

MaziyarPanahi

70.03

Score

#21

Calme 3.2 Instruct 78b

MaziyarPanahi

70.03

Score

#22

Calme 3.2 Instruct 78b

MaziyarPanahi

70.03

Score

#23

Calme 3.2 Instruct 78b

MaziyarPanahi

70.03

Score

#24

Calme 3.2 Instruct 78b

MaziyarPanahi

70.03

Score

#25

calme-3.1-instruct-78b

MaziyarPanahi

68.72

Score

#26

calme-3.1-instruct-78b

MaziyarPanahi

68.72

Score

#27

calme-3.1-instruct-78b

MaziyarPanahi

68.72

Score

#28

calme-3.1-instruct-78b

MaziyarPanahi

68.72

Score

#29

calme-3.1-instruct-78b

MaziyarPanahi

68.72

Score

#30

calme-3.1-instruct-78b

MaziyarPanahi

68.72

Score

#31

calme-3.1-instruct-78b

MaziyarPanahi

68.72

Score

#32

calme-3.1-instruct-78b

MaziyarPanahi

68.72

Score

#33

calme-3.1-instruct-78b

MaziyarPanahi

68.72

Score

#34

calme-3.1-instruct-78b

MaziyarPanahi

68.72

Score

#35

calme-3.1-instruct-78b

MaziyarPanahi

68.72

Score

#36

calme-3.1-instruct-78b

MaziyarPanahi

68.72

Score

#37

calme-3.1-instruct-78b

MaziyarPanahi

68.72

Score

#38

calme-3.1-instruct-78b

MaziyarPanahi

68.72

Score

#39

calme-3.1-instruct-78b

MaziyarPanahi

68.72

Score

#40

calme-3.1-instruct-78b

MaziyarPanahi

68.72

Score

#41

calme-3.1-instruct-78b

MaziyarPanahi

68.72

Score

#42

calme-3.1-instruct-78b

MaziyarPanahi

68.72

Score

#43

calme-3.1-instruct-78b

MaziyarPanahi

68.72

Score

#44

calme-3.1-instruct-78b

MaziyarPanahi

68.72

Score

#45

calme-3.1-instruct-78b

MaziyarPanahi

68.72

Score

#46

calme-3.1-instruct-78b

MaziyarPanahi

68.72

Score

#47

calme-3.1-instruct-78b

MaziyarPanahi

68.72

Score

#48

calme-3.1-instruct-78b

MaziyarPanahi

68.72

Score

#49

CalmeRys-78B-Orpo-v0.1

dfurman• license:mit

66.80

Score

#50

CalmeRys-78B-Orpo-v0.1

dfurman• license:mit

66.80

Score