Self-Improving Agents, But Improving at What?

Self-improving agents have become one of the fastest growing areas of agent research. New papers appear almost weekly describing agents that improve themselves after deployment. The common promise is straightforward: an agent should learn from experience instead of remaining static after it is first built.

The interesting question is what the agent is actually improving. Different systems optimize different parts of the agent stack, and those choices determine both how improvement is measured and how well it transfers to future work.

Conventional Agent Self-Improvement

The majority of self-improving agents optimize the harness that surrounds the foundation model rather than the model itself. Fine-tuning is expensive, slow, and difficult to validate continuously, but notably there have been huge strides in making those cycles faster and cheaper. It is still much easier to modify the surrounding components that influence how the model behaves, like the harness or the parts it manages.

Those components usually include system prompts, task prompts, skills, tool selection policies, planning heuristics, memory, retrieval, and workflow graphs. The common pattern is to observe an agent performing work, identify recurring failures, propose changes to one or more of these components, and validate those changes before adopting them.

The Optimization Loop

Although implementations differ, the optimization loop has become fairly standardized. An agent executes a task while recording its trajectory, including prompts, tool calls, observations, intermediate outputs, errors, retries, execution time, and final results. Production systems often supplement this with human edits, approvals, corrections, or downstream business outcomes.

Those execution traces become the raw material for improvement. The system analyzes successful and unsuccessful trajectories, generates candidate changes to prompts, skills, routing policies, memories, retrieval behavior, or workflows, and then replays historical tasks or runs a held-out evaluation set. The new version is compared with the current one, and changes are promoted only when they improve the selected metrics without causing unacceptable regressions elsewhere.

This creates a fairly conventional optimization pipeline. Collect trajectories, extract patterns, generate candidate modifications, evaluate them against representative work, compare versions, and promote the winner. The agent may participate in every stage, but the process usually still depends on an externally defined evaluation function and promotion policy.

Evals Optimize Tasks

It's the evals that improve the skills, prompts, and models. The system gets better at the tasks represented by its eval suite because that is what it is rewarded for improving.

But tasks are only one possible optimization target. A system could instead evaluate whether it has developed a better understanding of the domain itself. Has it discovered a simpler decomposition? Has it separated durable concepts from task-specific details? Are its intermediate artifacts becoming more reusable? Are its evaluation criteria becoming more general? Has it extracted first principles, guidance, values, or reasoning that can be reused across entirely different kinds of work?

Those questions are much harder to score than task completion, which is one reason they receive less attention today. They are also the kinds of improvements that are far more likely to transfer when the next task is unlike any the system has seen before.

Improving the Structure Behind the Harness

If the optimization target extends beyond task execution, the reusable structure behind the harness becomes the more interesting place to invest. Improving that structure requires a different kind of iteration than replaying benchmark tasks through an eval suite.

Organizations have spent decades building these kinds of assets. They interview experienced practitioners, document decision making, extract first principles, write design guidance, establish values, and refine the way experts think about their work. The goal is not simply to solve today's task more effectively. It is to build reusable ways of thinking that make many future tasks easier.

This matters technically because prompts, skills, and policies are local control surfaces. They influence execution at runtime. But reusable structures, on the other hand, shape the representation of the problem itself, giving every future harness a stronger foundation to build on.

Improving Representations Alongside Behavior

Imagine two agents completing one hundred competitive intelligence projects. The first agent improves its system prompt, adjusts its search strategy, rewrites several skills, and learns which tools produce better summaries. It becomes noticeably better at the next competitive intelligence project because its execution policy has adapted to that work.

The second agent makes those changes as well, but it also reorganizes the reusable structure behind the work. It discovers that market structure, customer segmentation, technical differentiation, ecosystem position, regulatory pressure, and execution maturity repeatedly emerge as independent analytical dimensions. Those dimensions become reusable artifacts with their own inputs, evaluation criteria, dependencies, and synthesis rules.

When the second agent encounters an unfamiliar company or a different industry, it begins with a richer conceptual representation rather than only a better prompt. The improvement transfers because the structure transfers, even when the exact task, company, or market has never appeared in the training traces.

Graphs as Executable Structure

Graphs are especially interesting because they provide a place for higher-order thinking to become durable. At the simplest level, a graph describes how work is decomposed into intermediate artifacts and how those artifacts relate to one another. As that graph evolves, it can become an executable representation of how a domain should be understood rather than simply how one task should be executed.

Instead of optimizing for the shortest path to the answer, graphs make it natural to optimize the thinking itself. A graph can deliberately fan out into multiple perspectives, compare competing explanations, search for dissonance, evaluate alternative decompositions, challenge assumptions, and synthesize the strongest ideas back together. Those intermediate products become reusable assets instead of disappearing into a single prompt.

Current models are rarely rewarded for this kind of exploration because their optimization loop is centered on task completion. If the evaluation only measures the final answer, the journey becomes overhead. Giving higher-order reasoning an explicit structure changes the incentive. The exploration itself becomes part of what is preserved, evaluated, and improved over time.

Harnesses Should Inherit Better Structure

Today's self-improving agents are primarily optimization systems for tasks. They become better at the work represented by their traces and eval suites by refining prompts, skills, planning heuristics, retrieval, and workflows.

The larger opportunity is to let every new harness inherit an improving body of reusable structure, including conceptual models, decomposition patterns, intermediate artifacts, dependency graphs, evaluation frameworks, first principles, guidance, and values. That shifts the optimization target from individual task performance toward the reusable understanding that makes entirely new classes of work easier to tackle.