Teaching agents to learn

Voxelbox revisits past sessions, extracts candidate lessons and preserves the sources behind them.

Jack Dorsey and Roelof Botha recently described the role of a company world model in From Hierarchy to Intelligence:

“Everything we do creates artifacts. Decisions, discussions, code, designs, plans, problems, and progress all exist as recorded actions.”

That is the part of the thesis I keep coming back to. I want decisions recorded where the next agent can use them, instead of asking someone to reconstruct the context later. Each result can inform the next decision.

An agent runtime needs to remember which sources it trusts, how projects relate, why a decision was made and what a failure taught it. Transcripts, decisions and code provide the raw material, but a future agent benefits only when useful lessons become part of its context.

I had made a related point in Subjective Truth, while calling back to a Dave Thomas talk about expertise:

“Dave asserts that expertise cannot be boiled down to rules because we lose the original context for the rule in the process. Truth, he says, is contextual.”

An instruction without the experience behind it can be applied confidently in exactly the wrong situation.

This may be why the Agentic Context Engineering paper stuck with me. It treats context as an evolving playbook and separates generation, reflection and curation. An agent session leaves evidence behind. Reflection proposes what the system might learn from it, while curation decides what deserves to persist, where it belongs and which sources should travel with it.

I built that cycle into Voxelbox, the local agent system I use across my projects. Meaningful sessions leave summaries behind. A nightly review filters duplicate logs and background jobs, compares new observations with existing memory and proposes candidate lessons. A later pass revisits those candidates. It can keep a narrow lesson with one project or promote a recurring pattern into a shared operating note, with the sources attached.

One scheduled review read the previous day's activity and separated substantive work from duplicate logs and background jobs. It explained why the memory replay process had failed: a list of connections had been passed to a tool as text. The code was already fixed, so the review preserved why it failed and the boundary the next implementation needed to respect.

The idea borrows from research on how humans dream and process experience. A scheduled review runs after the agent has stopped trying to produce an answer. It can revisit what happened, connect events that were separate during execution and notice things the agent missed while finishing the task.

That reminds me a little of playing football. We spent just as many hours in the film room as we did on the field, slowing the game down to see what we missed and what we should carry into the next one. This “dreaming” process gives the agents some of the same space to reflect.

Curation has to remain conservative. Models are very good at turning one event into a plausible general rule, and repeated behavior is not necessarily correct behavior. Some review passes should produce nothing. Future agents should receive the correction and enough of the original experience to understand when it applies.

AI/MLEngineering