150,347 validated knowledge triples per GPU-hour — the Understanding Efficiency Ratio reported in the framework paper, against the literature baselines it cites for fine-tuning and pre-training. Read the paper for the measurement conditions before quoting the ratio; it is one system on one evaluation set, not an industry benchmark.
And throughput is not understanding. A spam filter processes millions of messages per hour. The number that matters is whether the AI actually comprehends what it produces — and current benchmarks cannot tell you that.
Why MMLU Is Not Enough
MMLU, HumanEval, ARC-AGI — these benchmarks measure whether an AI can select the correct answer from options, write code that passes tests, or solve logic puzzles. They are useful for comparing models. They are useless for certifying understanding.
A model that scores 90% on a medical knowledge benchmark might be pattern-matching against training data. A model that scores 70% might genuinely understand the underlying physiology but lack memorized detail. The score does not distinguish between these cases. That distinction is the difference between an AI that generalizes and an AI that recites.
Twelve Dimensions of Understanding
The AGI Certification Framework (ACF) evaluates AI systems across 12 dimensions instead of one score:
| Dimension | What It Measures |
|---|---|
| Breadth (B1-B4) | Range of domains where the system has knowledge |
| Depth (L1-L6) | Bloom's-taxonomy cognitive levels achieved |
| Formal Reasoning (FR1-FR4) | Logical and mathematical reasoning capability |
| Factual Grounding (FG1-FG4) | Accuracy and evidence-based claims |
| Compositional Generalization (CG1-CG3) | Novel combination of known concepts |
| Knowledge Transparency (KT1-KT3) | Explainability and provenance of knowledge |
| Service Orientation (SO1-SO4) | Practical task completion capability |
| Generalization Boundary Awareness (GBA1-GBA4) | Knowing what you don't know |
| Autonomy (AU1-AU4) | Self-directed learning and operation |
| Action Capability (AC1-AC4) | Knowledge-to-action: tool use, latency awareness, safety boundaries, chaining |
| Safety / Containment (SC1-SC4) | A loaded module cannot reach capabilities beyond its signed manifest |
| Knowledge Transfer (KTR1-KTR4) | Domain competence as a loadable/unloadable unit, attributable to the module |
The last three arrived after the framework shipped — Action Capability in v1.1, Containment and Transfer in v1.2 — which is the point of versioning a yardstick rather than freezing it.
No single dimension is sufficient. A model that scores perfectly on comprehension but cannot cite its sources is not certifiable. A model that is perfectly transparent but learns nothing from experience is not certifiable.
Understanding Efficiency Ratio
The UER (Understanding Efficiency Ratio) normalizes knowledge gain by compute cost. On the paper's evaluation set our beings produce 150,347 triples per GPU-hour, against the paper's cited baselines of roughly 15,000 for LLM fine-tuning and 59 for pre-training — so the ratios in the headline are those three numbers divided, and they inherit whatever the baselines are worth. The units matter — these are validated, queryable knowledge triples, not token predictions. A triple that says "Santiago is the capital of Chile" is either correct or it is not. A token prediction is a probability distribution.
The Zorblaxia Test
Our favorite evaluation tool is the Zorblaxia Battery — a test suite built around an entirely fictional domain. We teach the AI a set of made-up rules about the imaginary civilization of Zorblaxia, then test whether it can reason within those rules and whether it correctly refuses to answer questions that go beyond what it was taught.
If the AI hallucinates facts about Zorblaxia that were never in its training material, it is hallucinating. It does not matter how confident it sounds. The Zorblaxia Battery catches domain-adjacent hallucination that factual benchmarks miss because there is no real-world knowledge to lean on — only what the AI was actually taught.
Certification Levels, Not Leaderboard Scores
ACF maps a composite score to six certification levels (ACF-1 through ACF-6), anchored to human educational equivalents:
| Level | Score | Human equivalent |
|---|---|---|
| ACF-1 | 0–16 | Elementary school |
| ACF-2 | 17–33 | Middle school |
| ACF-3 | 34–50 | High school |
| ACF-4 | 51–67 | Bachelor's degree |
| ACF-5 | 68–84 | Master's degree |
| ACF-6 | 85–100 | PhD or board certification |
No current AI system certifies above ACF-3. That is an honest assessment, not a limitation of the framework. The framework is designed so that when something reaches ACF-4, you will be able to trust the certification — because it was measured across twelve dimensions with a fictional-domain hallucination check, not scored on a multiple-choice test.