How AI Agents Learn Know-How by Doing the Work

How Perpetual agents learn from the messy, multi-step work they already do, improving small open-source models inside a company’s own environment.

The rarest thing in a company is not another workflow log. It is judgment: the know-how held by people who can tell which customer signal matters, which exception changes the answer, and which outcome should change the next decision.

Most AI agents today do not acquire that judgment. They can solve tasks, use tools, interact with APIs, and collaborate with people. But once they are deployed, they mostly freeze. The same mistake can appear again tomorrow because nothing in the system truly absorbed what happened yesterday.

Perpetual is built around a different idea: agents should learn from the work they already do. This post shows that loop in practice, with a small open-source model interacting with realistic enterprise environments and measurably improving on long-horizon agent tasks.

Enterprise AI Is a Long-Horizon Learning Problem

AI agents are moving past isolated question-answering. They investigate customer issues, generate reports, search internal documentation, interact with enterprise software, execute workflows, and coordinate across dozens of APIs.

Unlike traditional machine learning benchmarks, enterprise environments constantly evolve.

• Business policies change.
• Products evolve.
• Internal documentation grows.
• Employees discover better workflows.
• Customer expectations shift.

An enterprise agent that never learns slowly drifts out of date. But the obvious alternative is not acceptable either: companies do not want to send proprietary workflows, customer interactions, and hard-won operational know-how back to frontier model providers. Your private know-how is the only intelligence your competitors cannot rent. That is why privacy, security, regulation, and competitive advantage all point to the same need: agents that learn locally from the work they already do.

Why can't today's AI agents learn your business from the actions, outcomes, and feedback they accumulate after deployment?

This is the problem Perpetual solves.

Long-Horizon Work Creates Valuable Learning Data

Every long-horizon task leaves behind more than a final answer. A deployed agent might

• interpret the task state,
• retrieve relevant context,
• call tools and APIs,
• verify intermediate results,
• observe whether the final outcome succeeds or fails.

By the end, there is not just a prompt and a response. There is a trail of reasoning, tool calls, observations, decisions, and observable outcomes.

This complete interaction, often called an agent trajectory, contains much richer supervision than a single training example. A run succeeds or fails. A verifier changes the decision. A tool error blocks the task. A review reveals a missing behavior. Most AI systems throw those signals away. Perpetual turns them into lasting improvement.

Turning Work into Better Agents

At a high level, our continual learning system does three things: capture the signals produced by real work, use Auto-Discover to decide what is worth learning, and route validated patterns into memory, the agent harness, and weights.

After a rollout finishes, the system observes the outcome and can ask a stronger model or domain expert to review the interaction in hindsight. The reviewer is not handed the ground-truth solution. It sees what a knowledgeable employee would see: the original task, the full interaction, and the actions the agent took along the way. The point is not to redo the task. It is to explain what could have gone better. That feedback is combined with the business outcome and converted into new supervision for the deployed model.

Scaling SDPO to Agentic Learning

Many recent continual learning methods, including OPD^[1], OPSD^[2], and SDPO^[3], have demonstrated promising results for adapting language models from user feedback. These methods were primarily developed for single-turn conversations.

Enterprise agents make that problem harder. A typical task may involve dozens of reasoning steps, multiple tool calls, and interactions with external systems. Feedback often arrives only after the task is complete, so the system has to learn from the whole trajectory rather than one response.

To explore this setting, we built the first version of our continual learning system using SDPO. Our implementation begins with Qwen3.5-9B^[4] as the base model. The student observes the original task; the teacher receives additional hindsight information from human feedback, environment feedback, stronger language models, or successful alternative trajectories.

For these experiments, a larger language model reviews the original task and complete trajectory, then produces constructive guidance describing what could have gone better. That resembles how enterprise feedback usually works. Employees rarely know the optimal solution in advance, but they are good at spotting mistakes after the fact.

Under the hood, this first system uses SDPO with a Jensen-Shannon divergence objective, Top-K + Tail distillation, and evaluation through an independent LLM judge. The important result is not the particular recipe. It is the capability: deployed agents can improve directly from their own trajectories.

Continual Learning Across Memory, Agent Harness, and Weights

The optimization algorithm is only one piece of continual learning. A useful agent has to capture experience, separate signal from noise, and improve at the right level of abstraction. Some lessons belong in memory. Some belong in the agent harness: the execution loop around the model, including available tools, retrieval, checks, policies, and evaluators. Others belong in the weights after the pattern has been validated across enough evidence.

The same approach extends to enterprise deployments where agents continuously accumulate new work signals. Patterns transfer. Customer data does not need to. That is the shape of Perpetual’s continual learning system.

A Small Agent That Learns Toward Frontier Performance

To test whether the loop worked, we used τ-Bench^[5], a benchmark for realistic AI agents performing enterprise workflows that require reasoning, memory, and tool use. Unlike conventional language model benchmarks, τ-Bench asks whether an agent can complete an entire business task, not whether it can produce one correct response. We evaluated the first 32 tasks from the retail portion of the benchmark.

Starting from an off-the-shelf Qwen3.5-9B, our continual learning system moves the same small model from BASE 75 to OURS 97+ strict pass@1 on τ-Bench Retail-32.

The blue marks show the base model and the learned model at the same cost scale. The gray marks show larger frontier systems farther right on the cost-per-task axis. Costs are normalized on a log scale using published API list prices where available.

Beyond τ-Bench

One benchmark is never enough. As an early transfer check, we also evaluated on Apex Agents^[6], a substantially different agent benchmark. The base Qwen3.5-9B model achieves roughly 1% accuracy; the learned model reaches approximately 13%. There is plenty left to improve, but the signal is encouraging: a small open-source agent moves from near-zero performance to within 3x of frontier performance at a fraction of the normalized cost.

What the Agent Learns

These experiments show that agents can improve from the work they already do. The companion post, Rethinking Continual Learning, focuses on the harder question underneath that result: how Perpetual turns corrections into tested hypotheses and decides what becomes durable know-how.

Today’s AI agents perform work. Perpetual agents learn know-how by doing the work.