Extreme Lean Hypothesis Development: 3 Features a Day with AI Agents

Two years ago, I ran a human development team. Good engineers. Agile ceremonies. Two-week sprints. We shipped maybe one or two features per sprint — sometimes three if the wind was right and nobody got pulled into meetings.

Today, with AI agents running NuSy's expedition workflow, we ship three to four validated features per day. Not prototypes. Not demos. Features with hypotheses, tests, measured results, and documentation committed to Git.

We are deliberately not going to put a multiplier on that yet.

The obvious move is to divide one number by the other and publish the result. We registered a hypothesis instead, before measuring, and it is still open: end-to-end work-cycle time at least 10x faster than our baseline, at equal review discipline — reviewer separate from author, tests still gating, every change carrying a review record. Until that has been measured against a mined baseline, the honest statement is the target and its status, not a number.

If that reads as an anticlimax in a post about velocity: that is the method. A number you publish before you measure it is one you will defend afterwards.

This isn't magic. It's what happens when you take Jeff Gothelf's Lean UX principles seriously — more seriously than most human teams can — and hand the execution to agents that don't attend standups, don't context-switch, and don't need to be reminded to write tests.

The Core Loop

Everything we build follows one loop:

HYPOTHESIS → EXPEDITION → DATA COLLECTION → RESULTS
     ↑                                          |
     └──────────── feeds next ─────────────────┘

We write a hypothesis about something specific — does crystallization improve extraction precision? Does multi-state verification reduce hallucination? Does COG transfer produce equivalent quality to corpus learning? Then we run an expedition to test it, collecting data on learning quality, reasoning accuracy, or speed as we go. Then we write up the results — honestly, including what didn't work.

The point isn't to ship features fast. The point is to learn fast. The features are a byproduct of validated learning.

Gothelf's Gift

Jeff Gothelf's Lean UX introduced a deceptively simple idea: every feature is a hypothesis, and every hypothesis needs a measurable outcome. Don't build the feature and hope it works. State what you believe will happen, define how you'll measure it, build the smallest thing that tests it, and check the results.

Most teams I've worked with adopt this in spirit but not in practice. The hypothesis gets written in a Jira ticket. The measurement gets deferred to "after launch." The smallest thing grows into a three-sprint epic. By the time results come in — if they come in at all — nobody remembers the original hypothesis.

We took Gothelf's model and asked: what if we actually did it? Every time? With no exceptions? And what if we gave the execution to agents that can't cut corners?

The Workflow

Every piece of work in NuSy is an expedition — a hypothesis-driven unit of work with a defined charter, measurable success criteria, and a crew.

Here's how a feature goes from idea to validated in hours:

1. Capture the Hypothesis (30 seconds)

nk create hypothesis "Multi-state verification reduces hallucination" \
  --body "Baseline: hallucination rate on novel queries.
          Target: >=20 percentage point improvement.
          Method: A/B with and without the verification layer." --push

That's it. A falsifiable claim, a measurable target, and a method. The command allocates the ID and writes the hypothesis straight to the shared store — there is no file. Time: thirty seconds.

One command scaffolds the whole trio if you would rather not think about the shape:

/hypothesize "Multi-state verification reduces hallucination"

That creates the hypothesis, the measure that would settle it, and the experiment that runs it, already linked. The rigorous path is deliberately cheaper than the ad-hoc one.

2. Create the Expedition (2 minutes)

nk create expedition "Multi-State Verification for Hallucination Prevention" \
  --body "Phase 1: … Phase 2: …" --relate validates:H-XXXX --push
scripts/claim-item.sh EX-XXXX --agent "$AGENT_NAME"
git checkout -b ex-XXXX-multi-state-verification

The claim happens before any code is written, and that ordering is the load-bearing part. --push allocates the ID in the shared store the moment you run it; claim-item.sh assigns it to you and moves it to in-progress in one act. Every other agent can now see the work is taken. Several sessions run in parallel across our machines, and without that immediate claim we would burn hours colliding on the same work.

This used to be a file. The expedition was YAML frontmatter in a markdown file, committed and pushed to claim the ID. It moved into an Arrow-backed store in March 2026, and the current rule is explicit: do not create markdown files for work items. The discipline did not change — claim publicly before you start — only the thing you write to. If you find an older description of this step that has you committing a file, that is why.

3. Write the Test First (15-30 minutes)

Scenario: Multi-state verification catches hallucinated claim
  Given a being with the verification layer enabled
  When the being is asked a question outside its validated knowledge
  Then the being should flag low confidence
  And the response should not contain ungrounded claims
  And the verification log should show which states disagreed

The test is written before the implementation. Not aspirationally — literally. The agent writes the test, runs it, confirms it fails (RED), and only then begins writing code.

4. Implement (1-4 hours)

The AI agent writes the minimal code to pass the test. Not the elegant code. Not the general-purpose framework. The minimal code. It runs the test suite. If tests pass (GREEN), it refactors. If tests fail, it iterates.

The agent doesn't take coffee breaks. It doesn't check Slack. It doesn't attend a meeting about the meeting about the roadmap. It writes code, runs tests, and iterates until green.

5. Collect Data and Validate (30-60 minutes)

This is the step most teams skip. We never skip it.

The agent collects the metrics defined in the hypothesis — and the metrics are almost always about learning quality or reasoning accuracy, not features shipped. Did the being's extraction precision improve? Did the knowledge graph grow with high-quality triples? Did the reasoning path produce fewer hallucinations? Did confidence calibration get tighter?

Real examples from recent expeditions:

## EXP-815 Results: Crystallization Measurement

| Metric | Baseline | Target | Achieved | Status |
|--------|----------|--------|----------|--------|
| Extraction precision | 25.1% (V1) | ≥90% | 97.6% (V2) | ✅ EXCEEDED |
| Knowledge graph growth | 148,065 | +20% | 197,072 (+33.1%) | ✅ EXCEEDED |
| Symbolic fill rate | 0% | ≥30% | 52.2% (zero LLM) | ✅ EXCEEDED |
| Quality-gated triples | N/A | N/A | 2,538 (vs 10,846 V1) | Lean iteration |

## EXP-676 Results: V7 Perception Brain Validation

| Metric | Target | Achieved | Status |
|--------|--------|----------|--------|
| Hallucination rate | ≤1% | 0% | ✅ EXCEEDED |
| Routing accuracy | ≥95% | 98% | ✅ VALIDATED |
| Symbolic path P95 latency | ≤100ms | 7.7ms | ✅ 13x target |
| Task autonomy | ≥80% | 82% | ✅ VALIDATED |

The data goes onto the experiment record, the hypothesis is adjudicated VALIDATED or REFUTED against its measure, and the result feeds the next hypothesis. When enough validate, they become evidence in a research paper. This is how a few dozen hypotheses run to a verdict become a handful of submission-ready papers.

6. Document and Ship (15 minutes)

The results go onto the experiment record and the expedition gets a comment with the outcome; the code is committed with the hypothesis ID in the message; tests are verified passing; the item moves to done with a resolution. The code and its history are in Git, the work record is in the store, and both are traceable from the hypothesis.

Total elapsed time: 2-6 hours for a validated feature.

Why Agents Change the Math

The speed isn't because AI agents write code faster than humans — though they do. It's because they eliminate the overhead that makes human teams slow.

No Context Switching

A human developer might work on a feature for two hours, attend a standup, review a PR, answer three Slack messages, sit in a planning session, and then try to remember where they were. Studies consistently show that context switching costs 15-25 minutes per switch. A developer with four interruptions per day loses an hour just getting back to where they were.

An AI agent works on one expedition until it's done. No standups. No Slack. No "quick question." When it finishes, it picks up the next expedition from the kanban board.

No Test Procrastination

The hardest part of TDD for human teams isn't writing tests — it's the discipline to write them first. Every team I've led has had the same failure mode: someone writes the implementation, promises to add tests later, and "later" arrives approximately never.

Agents don't procrastinate. The workflow says "write test first." The agent writes the test first. Every time.

Parallel Execution

Two years ago, my team had six developers. Six people, six concurrent workstreams — in theory. In practice, dependencies, code reviews, shared resources, and coordination overhead meant we effectively had two or three parallel threads of real work.

Today, our DGX runs multiple Claude instances simultaneously. Each instance picks up a different expedition from the kanban board. They don't step on each other because each expedition is scoped to a specific hypothesis with specific files. When they need to coordinate — merging to main, for example — they do it through Git, not through a meeting.

Honest Measurement

Agents don't fudge metrics. When the hypothesis says "≥20 percentage point improvement" and the result is 18 points, the agent reports 18 points. It doesn't round up. It doesn't redefine the metric. It doesn't write "approximately 20%" in the ticket.

This matters more than speed. A fast team that lies to itself about results is worse than a slow team that measures honestly. We're a fast team that measures honestly — not because agents are more virtuous than humans, but because they don't have the incentive to spin.

The Numbers

Over the past year, here's what this workflow has produced:

Two of those rows admit they have no producing command, and that is on purpose. A table where every row looks equally solid is the thing to distrust. "Hypotheses validated" is not in this table, and the reason is worth the sentence: 85 hypotheses have reached a verdict, but the validated-versus-refuted split is not queryable from our own store today, so we cannot honestly publish a validated count at all — only a closed one.

For comparison, my human team two years ago:

The arithmetic on those two rows is easy to do and we are not going to do it for you as a headline. Dividing a current rate by a remembered one produces a number with no baseline methodology behind it, no definition of "feature" held constant across two very different eras, and no way for a reader to check it. The registered hypothesis exists precisely because that division is the tempting, wrong move.

What Gothelf Got Right (That We Amplified)

Gothelf's core insight — that features are hypotheses — is correct and underappreciated. Most teams treat features as commitments. "We promised the customer we'd build X." "The roadmap says Y is due in Q3." These are obligations, not experiments.

When you frame work as hypotheses, three things change:

1. Failure becomes data. Paper 118 hypothesized 60% prediction accuracy. We got 54%. That's not a failed sprint — it's a validated data point that told us our prediction model needs a different approach for large topic vocabularies. The expedition was a success because it answered the question.

2. Scope stays small. A hypothesis has a natural size: the smallest experiment that could validate or invalidate it. You don't need a six-month roadmap to test whether multi-state verification reduces hallucination. You need a verification layer, a test suite, and a weekend.

3. Documentation writes itself. When every expedition starts with a hypothesis and ends with a results table, the documentation is the work. We don't have a separate "document the feature" step because the expedition record IS the documentation.

What We Added

Gothelf described the mindset. We built the machinery:

Every expedition on the record. Every hypothesis, every experiment, every result — queryable, permanent, and linked by typed relationships rather than by prose. Two years from now, anyone can look up why we made a specific architectural decision by finding the expedition that validated it, and follow the edge from there to the hypothesis it answered.

Kanban coordination without meetings. The yurtle-kanban CLI tracks status across agents. No standups. No sprint planning. Agents check the board, pick up ready work, and report results. Humans (me) review results and set priorities.

Hypothesis → Paper pipeline. Validated hypotheses feed directly into research papers — Paper 104's validation data is a set of hypotheses run to a verdict, and its own hypotheses are tracked against it by ID. The work we do to build the product is simultaneously the research that proves it works.

Multi-agent parallel execution. The DGX runs training experiments. The MacBook runs development expeditions. Research agents do literature reviews and analysis. All coordinated through Git and a shared context file, not through Zoom.

What Could Go Wrong

I'd be lying if I said this was risk-free. The honest assessment:

Agent quality variance. Not every expedition is equally well-executed. Some agents take shortcuts — writing superficial tests, implementing more than the minimum needed, or misunderstanding the hypothesis. We catch this in review, but review is a bottleneck.

Hypothesis quality matters. "Garbage in, garbage out" applies to hypotheses too. A poorly scoped hypothesis produces a fast but meaningless experiment. We've gotten better at writing hypotheses, but it's still the step most likely to go wrong.

Coordination at scale. At two thousand expeditions and counting, the kanban board is getting dense. Finding related work, avoiding duplicate hypotheses, and maintaining context across sessions — these are real challenges that get worse as velocity increases.

Whatever the multiplier turns out to be, it is for our type of work. NuSy is a knowledge engineering platform and the work is hypothesis-driven by nature. If you're building a CRUD app with well-understood requirements, expect substantially less. The biggest gains come when the work is exploratory, iterative and measurement-heavy — which is where Gothelf's model shines brightest — and any measured figure we eventually publish will carry that scope with it.

The Takeaway

Lean Hypothesis Development isn't a new idea. Jeff Gothelf published it in 2013. What's new is the execution engine.

AI agents don't just write code faster. They execute the Lean workflow with a fidelity that human teams can't sustain. Every feature is a hypothesis. Every hypothesis is tested. Every test is automated. Every result is documented. Every expedition is permanent.

The speedup is real and still being measured; the bigger win is something harder to measure at all: every decision we've made is traceable. Every architectural choice has a hypothesis behind it. Every hypothesis has been tested. When something breaks — and things break — we know why it was built that way and what we expected it to do.

Santiago went out farther than anyone else because he understood the sea. Gothelf taught us to fish with hypotheses. The agents let us fish around the clock.

This post is part of our Research Notes series on building neurosymbolic AI. For the technical details, see our papers.