KnowledgeOps: What Happens When You Treat Knowledge Like Code

Here's a question that haunts every organization: where does the knowledge go when people leave?

It goes into Confluence pages nobody reads. SharePoint folders nobody can find. Slack threads that scroll past the free-tier retention limit. The skulls of senior engineers who quit for 30% more at a competitor.

We spent two decades solving this problem for code. Version control, CI/CD, automated testing, infrastructure as code, GitOps — the DevOps revolution made software delivery predictable, repeatable, and auditable. We can deploy a hundred times a day because we treat code as a first-class citizen with a proper lifecycle.

Knowledge doesn't get the same treatment. And it should.

The DevOps Parallel

DevOps didn't just give us better tools. It gave us better principles:

• Everything in version control
• Automate everything you can
• Test before you ship
• Make deployments boring
• Monitor production
• Treat failures as learning opportunities

Now apply those principles to knowledge:

That table isn't aspirational. We've built every row of it.

Inspectable by construction

The foundation of KnowledgeOps is a property, not a storage format: every piece of knowledge is addressable, diffable and attributable — and none of it is smeared across model weights.

We used to state that as "files are the interface, git is the database", and for a long time that was literally true: work items and being knowledge were markdown files with structured frontmatter, and the repository was the database.

That mechanism is gone, and the property survived it. Work items now live in an Arrow store behind a single writer, not in files; being knowledge lives in Arrow tables, not in a directory. The change was made for reasons that had nothing to do with this argument — files could not give us atomic ID allocation across concurrent agents, and a directory scan is not a query engine — and every KnowledgeOps claim below held through it unchanged.

That is worth more than the original formulation. A discipline that only works when your knowledge happens to be in text files is a filing convention. One that survives a substrate change is a discipline. Creating an item is now:

nk create expedition "Crystallization Measurement" \
   --relate implements:VY-42 --tags "extraction,crystallization,precision" --push

--push is what makes the ID atomic; two agents creating at the same moment cannot land on the same one. The record it writes is still diffable, still attributable, still queryable — just not by grep.

Why not a general-purpose database? The same reasons, restated for what we actually run:

• Inspectability is not optional. Any state a being reasons from can be printed, diffed and argued with. If you cannot read it, you cannot review it, and unreviewable knowledge is the thing this whole discipline exists to prevent.
• History is free and non-negotiable. Every change is attributable to an actor and a moment. When knowledge changes, we can say who, when and why — and go back.
• Provenance travels with the fact. A triple carries its source. That is a property of the data, not of the filesystem it happened to be stored on.

Traditional knowledge management hides knowledge behind APIs and proprietary formats. The point was never the file. The point was that nothing about a being's knowledge is hidden from the people responsible for it.

Knowledge as Code

Definitional knowledge — the facts, rules, thresholds and relationships a being reasons with — does not live in application code. It is authored and reviewed as data.

⚠ The example below is the V12/V13 form of this, and it is scoped that way in our own canon: "This section applies to V12/V13 Python code only. V14 Rust crates use Arrow RecordBatches for all knowledge." We are showing it rather than modernising it because the contrast is the lesson, and it is the same contrast either way — the "bad" side is knowledge compiled into application code, and that is still the mistake. Only the "good" side's storage has moved.

Bad:

# Knowledge buried in code
HALLUCINATION_THRESHOLDS = {
    "medical": 0.0,   # zero tolerance
    "legal": 0.05,    # 5% tolerance
    "education": 0.15 # 15% tolerance
}

Good — knowledge authored as data, reviewable on its own terms. This is the real shape, abridged: it is the frontmatter of knowledge/domains/ethics-predicates.md.

---
@prefix domain: <https://nusy.dev/domain/> .

<#ethics-predicates> a domain:ExtractionPatternSet ;
    domain:forDomain "ethics" ;
    domain:riskTier "T1" ;
    domain:inherits <#core-predicates> .
---

# Ethics Extraction Patterns
Ethics is T1: a low but non-zero hallucination budget, and high KBDD coverage...

The difference matters. When knowledge lives in code, only developers can change it. When knowledge lives in Yurtle files, beings can read their own configuration, reason about it, and even suggest improvements. A being that can query its own safety thresholds is a being that can explain why it's refusing to answer a question — not just that it's refusing, but the specific rule that prevents it.

Knowledge Version Control

Code has git diff. Knowledge has it too.

When a being learns something new — extracts a fact from a medical guideline, crystallizes a pattern from interactions, receives a correction from a human reviewer — that change goes through version control. We can:

• Diff knowledge states. What did the being know yesterday that it doesn't know today? What did it learn?
• Review knowledge changes. Before new knowledge gets deployed to a being, it goes through review. Just like a pull request.
• Rollback knowledge. If a being's reasoning degrades after a knowledge update, we can revert to the previous state. git revert works on knowledge just like it works on code.
• Branch knowledge. Want to experiment with a different knowledge structure? Branch it. Test it. If it works, merge it. If it doesn't, delete the branch. No harm done.

This sounds obvious. But think about how knowledge works in most AI systems: you fine-tune a model, and the knowledge is smeared across billions of parameters. You can't diff it. You can't review it. You can't roll it back without retraining from scratch. You can't even tell what the model "knows" versus what it's making up.

In our system, every fact is traceable to a source. Every source is a file. Every file is in Git. Every change is auditable. When a being says "the recommended HbA1c target for type 2 diabetes is less than 7%," you can follow that claim back to the exact guideline section, the exact extraction pipeline run, and the exact commit that added it to the knowledge graph.

KBDD: BDD for Knowledge

This is where it gets fun.

Behavior-Driven Development (BDD) gave software engineering a superpower: the ability to express requirements as executable specifications. Given/When/Then scenarios that both humans and machines can read.

We applied the same pattern to knowledge validation. We call it Knowledge BDD — KBDD.

Scenario: Being identifies HbA1c target for elderly patient
  Given a doctor being with validated diabetes knowledge
  And the patient is over 75 years old
  And the patient has limited life expectancy
  When asked about the HbA1c target
  Then the being should recommend a less stringent target of <8.0%
  And the recommendation should cite the ADA Standards of Care 2026
  And the confidence should be ≥0.95

For a medical being, the unit of testing is the decision point, not the document. Every threshold, every population variant, every special case in a clinical guideline is something the being can get wrong independently, so each one earns its own scenario. The number that falls out of a real guideline is large — large enough that generating them by hand is not the plan.

We are deliberately not putting a figure on it here. An earlier draft of this post said "6,000-20,000 scenarios per guideline"; the only measured scenario figure we have published is 22 BDD scenarios per domain (Paper 113), which counts something different and much narrower. Two numbers that disagree by three orders of magnitude, measuring different things, with a producing command for only one of them, is not a fact — it is two impressions. The measured one stands; the estimate is withdrawn until something produces it.

Why so many? Because a doctor being cannot hallucinate. When someone asks about medication dosing, the answer must be backed by source evidence, validated through KBDD scenarios, and traceable to a specific guideline section. The test isn't "does the being generate plausible text." The test is "does the being generate correct, evidence-based, provenance-tracked recommendations."

We classify domains by risk tier:

Those four names and four numbers are not editorial choices — they are RiskTier in crates/nusy-safety/src/battery.rs, and that file's own tests assert both the labels ("T0-safety-critical" … "T3-creative") and the thresholds.

A T0 medical being doesn't ship until every KBDD scenario passes. Period.

The Knowledge Pipeline

Put it all together and you get a pipeline that would make any DevOps engineer feel right at home:

Source Corpus (guidelines, textbooks, papers)
        ↓
Extraction Pipeline (10-step, authority-weighted)
        ↓
Quality Gate: Provenance Check (every fact traced to source)
        ↓
Knowledge Graph (RDF triples, Yurtle format, Git-backed)
        ↓
KBDD Validation (thousands of scenarios per domain)
        ↓
Competency Questions (can the being answer what it should?)
        ↓
Live Being Test (awaken the being, ask it real questions)
        ↓
Deploy to Production (merge to main)
        ↓
Monitor: Reasoning Accuracy, Confidence Calibration
        ↓
Feedback Loop: New knowledge triggers new extraction cycle

Every stage is automated. Every stage produces artifacts in Git. Every stage has pass/fail criteria. If the KBDD scenarios fail, the knowledge doesn't ship — just like if your unit tests fail, the code doesn't deploy.

Sushi-Grade Knowledge

We use the term "sushi-grade" internally. Sushi-grade fish isn't just fresh — it's been handled with specific care at every step of the supply chain. Temperature controlled. Inspected. Certified safe to eat raw.

Sushi-grade knowledge is the same idea. It's not enough for knowledge to be correct. It must be:

• Extracted from authoritative sources through a validated pipeline
• Traced to its provenance (which guideline, which section, which version)
• Tested against KBDD scenarios
• Reviewed before deployment
• Versioned so changes are trackable
• Rollbackable if something goes wrong

Most AI systems serve knowledge that's more like gas-station sushi. Probably fine. Maybe not. No way to tell until someone gets sick.

Why This Is Exciting

I've been in knowledge engineering for a long time. Long enough to remember expert systems, ontology editors, and knowledge acquisition bottlenecks that killed projects before they started. The field has a history of being tedious, expensive, and fragile.

KnowledgeOps changes the economics. When you can version knowledge like code, test knowledge like code, and deploy knowledge like code, you can iterate on knowledge at software speed. You can experiment. You can fail fast. You can try three different extraction approaches, diff the results, and pick the one that produces better reasoning — all in an afternoon.

The beings in our system treat their own knowledge as a mutable, versionable, testable artifact. They don't just have knowledge — they can inspect it, validate it, explain where it came from, and flag when something doesn't look right. A being that knows what it knows — and knows what it doesn't know — is a being you can trust.

DevOps made software delivery boring. That was the point — boring means reliable.

KnowledgeOps makes knowledge delivery boring. And that's the most exciting thing I've seen in a decade.

This post is part of our Research Notes series on building neurosymbolic AI. For the technical architecture, see our papers.