The next agent should start further ahead

How a local agent system turns corrections and operating experience into context for the next run.

The first experiment runner I built for Voxelbox, the local agent system I use across my projects, kept the hypothesis, result, decision, and a learning extracted from each run. An experiment could produce a bad candidate and a useful correction at the same time. I wanted to discard one without losing the other.

Keeping that learning in the experiment history only solved half the problem. A new agent still had to know the record mattered, find it, and understand when it applied. Otherwise I had built a thorough archive of lessons the system was free to ignore.

I found the same problem in Subjective Truth, which I wrote in 2009 after a Dave Thomas talk about expertise. Extracting a rule strips away the context that an expert uses to decide whether the rule applies. “Check authentication before starting the batch” is useful. The failed runs behind it—which service, which task, and why the failure was misclassified—are what give the instruction meaning.

Jack Dorsey and Roelof Botha describe the same raw material in Block's From Hierarchy to Intelligence:

“Everything we do creates artifacts. Decisions, discussions, code, designs, plans, problems, and progress all exist as recorded actions.”

The artifacts are the easy part. The work is deciding what persists. I use a generate-reflect-curate loop: the work produces evidence; reflection proposes what should change; curation decides whether the lesson deserves a place in future context and keeps its source attached. A correction can become a project instruction. A stable workflow can become a skill. Something narrow stays with the project that produced it.

Over time, those instructions, memories, and skills form a working world model. It records which sources the system trusts, how the projects relate, and why a choice was made the last time it appeared.

I do not want every confident reflection promoted automatically. Repeated behavior is not necessarily correct behavior, and one unusual failure should not become universal advice. A useful memory carries the correction, the experience behind it, and its evidence forward. That gives the next agent a better place to begin without simply accumulating more material.

AI/MLEngineering