When I say an agent system should learn, I do not mean the model is retraining itself or quietly rewriting its own rules.
I mean organizational learning: a completed run captures what changed, someone reviews the pattern, an approved change improves instructions, tools, workflows, topology, or maintained knowledge, and the next run can benefit from it.
Without that process, an agent can finish useful work and leave nothing behind except a transcript. Or it can collect piles of notes that nobody can find, verify, or turn into a better decision. Neither outcome improves the system in a dependable way. A Practical Architecture for Reliable Multi-Agent Systems frames learning as one responsibility in the larger operating model.
This is the improvement layer around that architecture. It turns loop feedback into reviewed change rather than an accidental memory inside one conversation.
The learning loop starts when work ends
The operating sequence is straightforward:
completed run
-> learning closeout
-> manager reflection across the domain
-> small local repair or evolution proposal
-> review and approval
-> changed guidance, knowledge, workflow, or topology
-> better future workThis is a larger version of the feedback loop in Loop Engineering: Goals, Feedback, and Better Agent Prompts. A task can observe and adjust while it works. The learning loop asks a different question after the work: what should change so the organization handles this better next time?
The distinction matters. A task should not get permission to redesign the agent because it had one frustrating run. It should produce evidence that makes a potential change visible to the role or person responsible for the system.
Every meaningful run needs a learning closeout
Completion should not be the last thing a serious agent does.
Each meaningful run should leave a short closeout. This is not a retrospective essay. It answers practical questions:
What became newly known?
What was wrong, missing, or out of date?
What did the work need but could not find?
Where should this information live?
Can a small local repair happen now?
What needs a larger proposal instead?“Nothing learned” is an acceptable answer when the evidence supports it. The goal is not to manufacture a lesson after every task. The goal is to catch changed knowledge before the working context disappears.
A closeout can record a new fact, but corrections are just as valuable. An API guide may be old. A source guide may point to the wrong system. A code path may have a dependency the original request did not reveal. A delivery failure may show that a result reached an owner but not the requester.
The closeout should also name the right home for the lesson. A tool fact may belong in a source guide. A repeatable procedure may belong in a runbook. A role failure may need an AGENTS.md adjustment. A larger design problem may need a new skill, workflow, specialist, or authority decision.
A manager looks for the pattern
One closeout sees one slice of work. A manager needs to see the pattern across the domain.
In my operating model, each domain manager has a weekly reflection. It looks at ordinary work, automations, Runtime behavior, open obligations, closeouts, and anything else that affected the domain’s health that week.
The reflection asks:
- Are the same failures showing up in several runs?
- Did the agents use the right sources and tools?
- Did the system return useful results to the right people?
- Do several closeouts point to an instruction, knowledge, or topology gap?
- Is the agent spending effort on the wrong things or hiding a material risk?
This is active management. It takes judgment and time. A scheduled reflection does not make an agent better by itself.
The manager has limited authority for small, local repairs that are already within its rules: clarify a narrow instruction, repair an obvious source link, update a local note, or schedule a follow-up check. Changes that affect another agent, a cross-domain process, authority, or a live route need a different artifact.
Turn larger lessons into evolution proposals
An evolution proposal is more than a bug report. It should name the evidence, current limitation, proposed change, affected roles or processes, expected benefit, and required approval. It should also say what the proposal does not prove.
Observation: Release reviews keep finding the same dependency late.
Evidence: Several closeouts and reviewer returns point to it.
Proposal: Add dependency mapping to the review workflow and create a
component knowledge note for the affected system.
Decision needed: Approve the workflow change and the new note template.These proposals become an input to a one-on-one with the agent: a separate review of the domain’s operating health, repair needs, and possible expansions. The goal is not only efficiency. A good proposal can ask how the agent might fulfill its charter more completely: notice a risk earlier, give a person a more useful answer, or add a missing source or specialist.
Once a proposal is approved, it becomes a live-system change. How to Change an Agent System Without Breaking Live Work explains why a new workflow, topology, or route should be introduced and proved deliberately.
Shared knowledge needs a useful home
Some lessons matter beyond one role or one task. They need a home people and agents can both navigate.
I use an Obsidian Vault as one implementation of that shared knowledge layer because its material is Markdown. Templates, front matter, links, and typed notes make it possible to capture relationships without trapping them in a proprietary format.
The Vault is not a transcript store. It is a maintained knowledge system. A feature note can link to the components that support it, prior incidents, source documentation, use or risk information, and a review date. A component note can link back to the features it affects, testing limits, known dependencies, and the evidence behind those claims.
Consider a code-change review for a release branch. The immediate job is to assess the change. The work may also reveal a useful relationship among a feature, its components, past failures, telemetry, tests, and release posture. Capturing that relationship once can give the next reviewer a much better starting point than a blank search.
The knowledge system can serve people directly, package a maintained subset beside skills in a repository, or provide a narrow path to more detailed documentation. It should always keep source links, ownership, and review dates so a tidy note does not become a tidy source of stale confidence.
Where an Agent System Keeps Its State explains why shared knowledge has a different job from live task context, working evidence, or a compact request record.
Learning needs evidence and controls
Learning can fail in predictable ways. More notes can create more stale notes. Several similar symptoms can be mistaken for one cause. An automation can make a bad repair faster than a person would. A knowledge graph can make an old relationship look more certain than it is.
That is why the loop needs source links, note types, review dates, limited repair authority, and explicit approval for larger changes. The Proof Ladder for AI Systems is useful here because it keeps a plausible proposal from being mistaken for a proven improvement.
The jobs stay distinct:
- closeouts capture what one run discovered;
- manager reflection interprets a pattern;
- proposals make larger changes reviewable;
- one-on-ones approve and prioritize; and
- the knowledge system preserves what survives that scrutiny.
Improve on purpose
An agent system does not learn because it has a long context window or a folder full of notes. It learns because someone designed a path from work, to evidence, to reflection, to an approved change, to better future work.
That is the part I care about. The system does not change itself freely. It has a disciplined way to decide what should change next.