5 hypotheses. 5 failures. 0 out of 5 quantitative targets met.
We are publishing this paper anyway, because the whole point of Hypothesis-Driven Development is that failure is data. And in this case, the data saved us months of work on an approach that was never going to work.
Predictive Processing Was Supposed to Be the Answer
The theory is compelling: the brain operates as a prediction engine, constantly generating expectations and updating when reality deviates. If we could give our AI beings the same mechanism — predict what comes next, learn from prediction errors — they would learn faster and reason better.
So we formulated five hypotheses with specific, quantitative targets:
| Hypothesis | Target | Actual | Result |
|---|---|---|---|
| Topic prediction accuracy | 60% | 54% | Failed |
| Calibration error (ECE) | 15% | 45% | Failed |
| Experience quality score | 75% | 28.6% | Failed |
| Prediction-driven learning speed | 2x baseline | 0.8x baseline | Failed |
| Error-driven knowledge refinement | 50% improvement | 12% improvement | Failed |
Not a single hypothesis met its target. The closest was topic prediction at 54% vs. a 60% target — and that 6-point gap is the difference between useful and noise.
Why It Failed: Token-Level Granularity at Scale
The root cause took us about three weeks to identify, which is exactly the point. With five clearly defined hypotheses and quantitative targets, we did not spend months optimizing hyperparameters hoping for a breakthrough. We ran the experiments, looked at the numbers, and followed the data to the root cause.
The root cause is straightforward: token-level topic granularity at 10,000-vocabulary scale makes prediction nearly impossible. When a being encounters a word, there are thousands of plausible topics it could belong to. The prediction signal — "I expected topic X but got topic Y" — is so noisy that the learning system cannot extract useful updates from it. It is like trying to navigate by stars on a cloudy night. The information is there, but the signal-to-noise ratio defeats you.
This is not a fixable bug. It is a fundamental mismatch between the predictive processing mechanism and the knowledge representation granularity. You would need to either reduce the vocabulary by 100x (losing coverage) or increase the training data by 100x (defeating the efficiency goal). Neither is acceptable.
The Methodology Worked Even Though the Science Didn't
Here is why we consider this paper a success despite the results:
The failed experiments redirected engineering effort to Papers 119 and 120 — both of which succeeded. Paper 119 found that symbolic-first perception (rather than prediction-first) achieved the learning speed gains we were looking for. Paper 120 validated the confidence-routing architecture that replaced the broken prediction mechanism. Both papers were directly informed by what we learned from these five failures.
Without the hypothesis framework, we would have spent months gradually tuning a system that was fundamentally misaligned with its task. With the framework, we spent weeks, identified the root cause, and pivoted.
Publishing Negative Results Should Be Standard
AI research has a publication bias problem. Every paper on arXiv is a success story. Every blog post is a breakthrough. The cumulative effect is that every other team trying predictive processing has to discover the same granularity problem independently, because nobody publishes the failure.
We think that is wasteful. If our five failed experiments save one team from repeating the same dead end, the paper has earned its keep. The most valuable research output is not the answer — it is knowing which questions not to ask.
This is a methodology paper, not an architecture paper. The contribution is not a new system. The contribution is a worked example of Hypothesis-Driven Development doing exactly what it is supposed to do: fail fast, learn fast, redirect fast.