AI Self-Improvement, and Why It Is All Arrow

We run two different kanban engines. Both are real, both are open source, and neither one is a downgrade from the other. yurtle-kanban puts a work item in a Markdown file and makes Git the database — and it's explicitly built for AI agents: three of them coordinated a large backlog of work items through it over months of daily use.

⚠ An earlier draft of this paragraph said "with no collision we ever found." That is exactly the shape Go and Look warns about — a check you cannot remember going red is a check you have not shown can go red — and we had no collision detector, so the absence of reports is not evidence of absence. Cut rather than quietly reworded, because it is the more useful example with the reasoning left in. arrow-kanban puts the same kind of work item in an Apache Arrow table backed by Parquet, with typed, directional edges you can query instead of grep. That's the engine our own fleet runs on today. This piece is about why — and it isn't "the other one doesn't work," because it plainly does.

What actually changes between the two

A work item as a Markdown file is something you fetch: an agent shells a command, gets text back, and parses that text into whatever structure it needs before it can reason about it. A work item as an Arrow RecordBatch is something an agent's own process can hold: the table is already in memory, already typed, already queryable, and it doesn't stop being any of those things between one question and the next.

That distinction barely matters at the scale one small team runs at — a few agents, a modest rate of claims and status changes, files small enough that git log and a text parse cost nothing anyone notices. It starts to matter once the thing asking the questions is not a person occasionally checking a board but a fleet of agents making dozens of small coordination decisions an hour: who can claim this, what does it depend on, has that dependency's status changed since the last check. Do that by shelling out and reparsing every time, and the cost isn't the shell call — it's that nothing the agent learned answering question N survives to question N+1. Do it against a resident, typed table, and it does.

arrow-kanban's own description of itself names this directly, and it's a claim anyone can check against the public repo: work items, relations, comments and status history are Arrow RecordBatches, and the relationships between items are typed and directional — "which experiment validates this hypothesis" is a query you can ask the graph, not a string you have to grep out of somebody's prose. That's the same property this whole Handbook keeps coming back to under different names: structured knowledge you can query beats prose you have to re-interpret, and here it shows up one level down, in the tool the fleet uses to track its own work.

Not only fast — checkable

Speed is a real difference, but it's a weak reason on its own to make this call for a whole fleet's coordination layer. The property that actually matters once you're talking about self-improvement — an agent reasoning about its own state, and eventually revising its own operating rules — is different: that state has to be something the agent can prove things against, not merely something it can read fast.

That's the second half of why it's Arrow, and it's an architectural commitment, not a performance one. The governed graph our own runtime is built on doesn't just hold work items — it carries provenance and epistemic status alongside every fact, and a standing rule that an unverified inference is never silently relabeled as a proven one on its way through the system. A cache doesn't buy you that; it buys you speed on top of whatever guarantees the thing behind it already had. "Checkable" is a property of the structure itself, and a structure survives being checked only if nothing has to be serialized out to text and reparsed back in first — the same round trip that costs an agent its resident memory is the round trip that would let an unchecked claim slip through unnoticed.

Try the part that is real today

The argument above is architectural, which makes it easy to nod along to and hard to check. So here is the checkable part, walked on a clean machine at the version you can install right now.

⚠ First trap, and cargo tells you the fix:

$ cargo install --git https://github.com/hankh95/arrow-kanban
error: multiple packages with binaries found: arrow-kanban, arrow-kanban-server. When installing a git repository, cargo will always search the entire repo for any Cargo.toml.
Please specify a package, e.g. `cargo install --git https://github.com/hankh95/arrow-kanban arrow-kanban`.

(That is the URL the public README still gives, which is why it is the one shown failing. It redirects to the Congruentsys org.)

Take its advice and cargo's last line tells you what landed — which is where the version number below comes from, because --version is not a flag on this binary (Ten Minutes to a Board hits the same wall):

   Installed package `arrow-kanban v0.3.0 (...#1b5e988a)` (executable `arrow-kanban`)

Then:

$ arrow-kanban init
$ arrow-kanban create hypothesis "Adding examples improves answers"
Created H-1300: Adding examples improves answers
$ arrow-kanban create experiment "Few-shot vs zero-shot on 200 prompts"
Created EXPR-1301: Few-shot vs zero-shot on 200 prompts

$ arrow-kanban update EXPR-1301 --relate validates:H-1300
  + EXPR-1301 -validates-> H-1300

That edge is typed and directional — not a string in a body, not a link in prose. And the type is enforced, which is the half you can prove in one command:

$ arrow-kanban create chore "A chore"
Created CH-1302: A chore
$ arrow-kanban update CH-1302 --relate validates:H-1300
Error: 'validates' cannot start from a chore (allowed: experiment)

A chore cannot validate a hypothesis, and the tool knows why. That refusal is the whole argument of this chapter in one line: the relationship has a meaning the store understands, so a meaningless one cannot be written. Grep has no opinion about whether the sentence it matched made sense.

And here is what does not work yet, which you would find in about a minute

Having written that edge, try to read it back:

$ arrow-kanban show EXPR-1301          # no relationships section at all
$ arrow-kanban export --format json --board research
  ... "related":[], "depends_on":[] ...        # the validates edge is in neither
$ arrow-kanban query --sparql "SELECT ?s ?o WHERE { ?s <.../validates> ?o }"
  ?s      ?o
  null    null                                  # three null rows

Two read paths miss it. The third does not even look — and that is worth separating, because "three misses" would be overselling a null instrument as corroboration:

$ arrow-kanban query --sparql 'SELECT ?id ?type WHERE { ?item <https://TOTALLY/bogus> ?zzz }'
?id                 ?type
H-1300              hypothesis
EXPR-1301           experiment
CH-1302             chore

A predicate that does not exist returns every row. So --sparql is not evaluating the triple pattern at all; the nulls above appear because ?s and ?o are not item columns, not because the edge is absent. Its own --help is accurate where this chapter's first draft was not — "SPARQL-like filter (subset: SELECT/WHERE/FILTER/ORDER BY/LIMIT)". A filter over columns, not a query over a graph. The edge is accepted, validated, and confirmed on write — and the public CLI at 0.3.0 gives you no way to get it back out. related and dependsOn project into the flat columns that show and export render; validates and the other typed predicates do not. And --sparql cannot stand in for them, because it filters columns rather than matching triples. Filed as #119 while writing this chapter — which is the honest provenance of most of what is in it.

⚠ So the honest status of this chapter's own headline — "a query you can ask the graph, not a string you have to grep" — is that the graph half is real and the query half is not finished in the released binary. The engine our fleet runs against has moved past this; what you can cargo install today has not. Reporting that is cheaper than having you discover it in your third command, and this Handbook is not much use if it only tells you the parts that work.

What we are not claiming yet

The obvious next sentence is a number, and we're not giving you one. Three hypotheses about how much faster this substrate makes an AI fleet — per operation, end to end, and in adopted self-improvement cycles per day — are preregistered and currently unmeasured: targets of ≥100× per operation, ≥10× end to end at equal review discipline, and ≥10× more self-improvement cycles per day, timestamped in our own record before the measuring code existed, specifically so the target can't quietly move once the number comes in. Until those experiments run, the honest version of this piece is the one above: an architectural argument, backed by a public engine you can install and query yourself, with the multiplier stated as a registered target under measurement — not asserted as a result. A book about not doing that would be a strange place to do it anyway.

Where this leaves you

If you're weighing the same choice for your own agents, the test this piece keeps returning to is the one that matters: what does an agent have to do to get its own state back — call out and reparse, or just read what's already resident. Both answers are legitimate engineering, at different scales and for different jobs. We picked resident, typed, and checkable for the layer our own fleet coordinates through, and everything above is the reasoning, not a benchmark.

Read next: the case for the other engine, on its own terms, is yurtle-kanban: A Kanban Board Where Git Is the Database; the graph-vs-prose argument in the abstract is in Graph Engineering Needs Graph Governance; the engine itself is open source and installable — arrow-kanban.