Go and Look: Closing Your First Loop

You wrote something down. A claim, a number that would settle it, and a date — committed somewhere you cannot quietly edit it.

That was half the loop. This is the other half, and it takes about fifteen minutes.

If you have not done the first half yet, start with Your First Hypothesis and come back. Everything here assumes you have a prediction sitting in a folder with your name on it.

First, the easy part

Go and look.

Whatever you said you would check — check it. Count the tickets. Open the analytics. Read the twenty emails. If your claim was "most of my support tickets come from one feature", go and sort your support tickets.

Then tell your agent:

Create a file called my-first-hypothesis-result.md next to my hypothesis.
Put in it: today's date, what I found, and whether that supports or does not
support what I predicted. What I found is [WHAT YOU FOUND].
Then commit it with the message "result: checked" and push it.

Push it too — the same reason as last time. A result sitting only on your laptop is one you could still quietly revise, and the prediction and the result are one pair of evidence. Protecting half of it protects none of it.

That is one full turn of the loop. Prediction, then evidence, both dated, both beyond quiet revision.

Most guides stop here. This one does not, because the interesting part is what happens next.

Why the commit matters more than the file

The file is just a file — you could write one any time, including afterwards, and nobody could tell. What makes it evidence is that the commit carries a timestamp you did not type, in an order you cannot rearrange without it showing.

This is the same reason clinical trials register their endpoints before they enrol anyone — and why journals increasingly require it. It is not paperwork. It is the only thing standing between a prediction and a story told afterwards about what you always expected.

There is a second effect, quieter and just as useful: once the target is fixed and visible, you stop negotiating with it. A number you can still adjust is a number some part of you is still adjusting.

We publish our own failures for the same reason →

More technical than this chapter, and written for a different audience. The point above stands without it — skip it freely.

Now the part that makes it a method

Look at how you looked.

Here is what happened when I walked this exact path, on a clean machine, an hour before writing this page.

My claim: most of the posts I have published are short — more than half under 1,500 words. I wrote it down, committed it, and went to count.

posts counted:      32
under 1500 words:   25
that is:            78%
VERDICT: supported

Supported. Comfortably. I could have stopped there — the number agreed with what I had predicted, the loop was closed, the commit was made.

Instead I asked a second question: did I measure what I said I would measure?

I had not. I claimed something about how long a post is to read. What I had counted was how big the file is — and a file contains things a reader never sees. A machine-readable header. My own editorial notes, tucked in comments. One post was carrying 794 words of notes that appear nowhere on the page.

Counting only what a reader actually reads:

posts counted:    32
under 1500 PROSE: 27
that is:          84%
VERDICT: supported

Same verdict. Different number. Two more posts had crossed the line.

Why that near-miss is the whole lesson

The verdict survived. It did not have to.

If the true figure had sat near half — and I had no way of knowing in advance that it did not — a sloppy count would have handed me the opposite answer with exactly the same confidence, in exactly the same format, with a commit hash next to it making it look rigorous.

The record-before-looking would have been perfect. The dates would have been honest. And the conclusion would have been wrong, because the measurement was answering a question I never asked.

This has a name, and it has bitten us in production

We call it a vacuous pass: a check that runs, reports success, and verifies nothing. It is the most expensive kind of test, because it is indistinguishable from one that works. Every dashboard is green, and nobody investigates a green dashboard.

We have hit this in our own systems more than once. A validation gate that passed because it was inspecting a file that no longer existed. A coverage number that counted things nobody had asked it to count. Both were green for months.

The tell is almost always the same: a check that has never been seen to fail. If you cannot remember it going red, you do not know that it can. That is worth five minutes to find out — break the thing on purpose and watch the check notice.

The longer version, from testing in simulation vs testing live →

More technical than this chapter, and written for a different audience. The point above stands without it — skip it freely.

And here is the uncomfortable half.

I only checked because I made myself. The first number agreed with what I already believed.

That is precisely when nobody re-checks. Nobody audits a result that confirms them. If my first count had come back at 40% — refuting my claim — I would have gone over that measurement line by line looking for the error, and I would have found the header words in about ninety seconds.

So the discipline is not "check your work." Everyone says that and nobody does it evenly. The discipline is: check hardest when the answer pleases you.

What to actually do, in about five minutes

Before you close the loop, ask your agent:

Look at how we just measured this. Is there anything included in that count
that should not be, or anything excluded that should be? Answer as if you
were trying to prove the number wrong.

Then read the answer properly. It will often say "looks right", and that is fine — you have spent five minutes buying the difference between a result and a result you can defend.

If you did the counting yourself — in a browser, in a spreadsheet, by reading twenty emails — the agent has nothing to inspect, so ask the question of yourself instead, in writing, in the result file:

My open rate: did it count my own test sends? Bounces? The resend to people who missed it? "Tickets from one feature": did I count duplicates? Tickets I reassigned myself?

Same question, same five minutes. The point is not who runs the check — it is that the check gets run at all, on the thing you are about to believe.

If it finds something, re-run the check and record both numbers. Not just the corrected one. The first number and why it was wrong is the most useful thing in the whole file — it is the part you will recognise next time, before it costs you anything.

That is what my own result file says:

My first count said 78%. Same verdict, different number… My claim was about how long a post is TO READ. My first check measured how big the file is. Those are different questions, and I nearly answered the wrong one without noticing.

When it fails

Sooner or later — usually sooner — you will go and look and find you were wrong.

That is a good outcome, and it is worth being specific about why, because "failure is learning" is the kind of thing people say without believing it.

A refuted hypothesis is worth more than a supported one for a plain, practical reason: it changes what you do next. A confirmed belief leaves you exactly where you were, minus an afternoon. A refuted one hands you a decision you did not know you had to make. The support tickets are not coming from the feature you assumed — so the roadmap you were about to write was aimed at the wrong thing, and now it is not.

The only outcome that teaches you nothing is the belief you never wrote down.

What a week of failures actually looks like

We ran five experiments in one week and all five came back negative. It was one of the most productive weeks we have had, and the arithmetic is worth doing: five directions closed in a week, each of which could have absorbed a month. That is not a bad week dressed up. That is the highest-value week available.

The only thing that made it work was that all five targets were written down first. Otherwise there is no such thing as a negative result — just a thing you tried that you now describe differently.

The write-up, with the numbers →

More technical than this chapter, and written for a different audience. The point above stands without it — skip it freely.

You have now done the whole thing

Look at your folder. Two files, two commits, in order:

result: checked, and corrected how I measured
hypothesis: before looking

That is hypothesis-driven development. Not the tooling, not the vocabulary, not a board with columns on it — a prediction you could not quietly edit, and a look you were honest about.

Everything else in this handbook is that loop, made cheaper to run fifty times.

Where to go next

  • Do it again, on something that matters more. The loop is worth very little run once and quite a lot run habitually. The second one takes about ten minutes now that you know the shape.
  • When one file per hypothesis starts to feel like too many files — and only then — that is when a tool earns its place. That is the next chapter.
If you want to see this run at industrial speed — with a warning

We run this loop three or four times a day across a fleet of AI agents, and we have written that up: Extreme Lean Hypothesis Development.

That is a research note, not a next step in this tutorial, and it reads like one. It is written for people already inside our system, and it uses vocabulary we have not taught you here — expeditions, beings, crews, internal ticket numbers. Nothing in it is required to keep going, and it will not ask you to do anything.

Read it if you are curious what this looks like at scale. Skip it without cost.