Click, Fail, Learn: Why BEPA Might Be the First GUI Agent That Actually Improves

Clicking is easy. Clicking correctly, after the screen has changed, after a pop-up appears, after the previous attempt failed, and after the agent has only fifteen steps before the evaluator gives up — that is where GUI automation stops looking like a demo and starts looking like work.

This is the problem behind BEPA, short for Bi-Level Expert-to-Policy Assimilation, introduced in the arXiv paper From Off-Policy to On-Policy: Enhancing GUI Agents via Bi-level Expert-to-Policy Assimilation.¹ The paper is about training end-to-end GUI agents, but its practical message is broader: expert workflows are not automatically useful training data. They have to be translated into something the learner can actually perform.

That sounds obvious until one looks at how agent training is often discussed. A strong framework agent completes a desktop task. It leaves behind a successful trace. The trace looks valuable. Surely the smaller end-to-end model can learn from it. Convert the format, fine-tune, mix it into reinforcement learning, and let the benchmark score rise. Clean story. Convenient story. Mostly wrong.

BEPA is interesting because it does not treat expert demonstrations as sacred objects. It treats them as foreign objects. Useful, yes. But foreign. The method asks a more precise question: how can a policy learn from an expert trajectory without being dragged off its own behavioral distribution?

The answer is not “more imitation.” It is assimilation.

The real bottleneck is not missing expertise, but unusable expertise

The paper starts from a split in GUI-agent design.

On one side are framework-based agents. These systems wrap large models with planners, executors, grounding modules, tools, and sometimes multiple cooperating roles. They can be powerful because they do not ask one model to do everything in one breath. Planning can be separated from execution. Grounding can be handled by a specialist. Tool calls can replace fragile mouse actions.

On the other side are end-to-end GUI agents. These map screenshots and instructions directly into action traces. They are simpler to deploy and conceptually cleaner: one policy sees the screen, reasons, and acts. The price is performance. In the paper’s OSWorld-Verified comparison, strong framework-style systems such as Agent S2 and Jedi-7B with o3 sit around or above the low-30s and 40s in success rate, while the base end-to-end UITARS1.5-7B policy is much weaker.

So the obvious move is to use the framework agent as the teacher. Run the strong system, collect successful trajectories, train the end-to-end agent on them, and harvest competence. This is where the paper becomes useful, because it explains why that move is not enough.

The expert trace and the student policy differ in two ways.

First, there is an action-space mismatch. A framework agent may issue tool-level or API-style actions that do not correspond neatly to the primitive click, type, scroll, wait, and drag operations available to the end-to-end policy. Even after conversion, some of the expert’s behavior is not really a native behavior of the student. It is a translation of someone else’s hand movements into a language the student speaks awkwardly.

Second, there is an inference-structure mismatch. Framework traces are often filled with artifacts from their internal architecture: planner outputs, executor reflections, grounding comments, template-like reasoning, and subtask boundaries. The end-to-end policy does not naturally generate this structure. It may be able to copy the words, but copying the words is not the same as learning how to look at the screen and decide what to do.

This distinction matters. A GUI agent does not operate in a static textbook. It operates in a changing interface. The paper explicitly notes that OSWorld-style environments may include stochastic pop-ups, time-sensitive banners, random ads, and altered initial states. A supervised model trained to replay an ideal trace may become better at the trace and worse at the task. Very on-brand for brittle automation: beautiful memory, poor eyesight.

Static imitation solves the wrong version of the problem

The paper evaluates several ways of incorporating expert traces: supervised fine-tuning, reinforcement learning with an auxiliary SFT loss, SFT followed by RL, LUFFY-style off-policy mixing, and trace replacement.

These baselines are not straw men. They represent reasonable instinctive answers to the problem. If demonstrations are useful, train on them. If pure RL is sparse, add imitation. If the expert traces are off-policy, mix them carefully. If a rollout group fails completely, insert a successful trace so the optimizer gets a positive signal.

The results are a useful warning. On OSWorld-Verified, static expert integration does not reliably generalize. SFT performs badly on the reported splits. SFT+RL collapses further. RL+SFT improves some numbers but remains weak on held-out tasks. LUFFY and simple trace replacement preserve entropy better than SFT-style methods, but they still show poor held-out transfer.

The paper’s interpretation is important: the failure is not simply “too little data” or “not enough optimization.” It is non-internalization. The student sees successful behavior but does not turn it into its own robust behavior.

A short table captures the difference.

That last point is the business version of the paper. Enterprise automation teams will eventually collect many successful traces: human workflows, RPA scripts, agent logs, helpdesk resolutions, internal SOPs, QA-approved browser flows. The temptation will be to dump them into a training pipeline and call it institutional memory. BEPA says: not so fast. Memory that the policy cannot inhabit is just a museum.

LEVEL-1 turns expert advice into reachable behavior

BEPA’s first stage, LEVEL-1: Self-Rolled Execution, is a reachability adapter.

The method begins with successful expert trajectories, such as those produced by Agent S2. Instead of directly training the end-to-end policy on the full expert trace, BEPA first extracts a compact natural-language plan. The paper uses GPT-4o as the plan extractor. That plan is then appended to the original task instruction, and the base policy itself rolls out in the real environment.

Only successful self-rolled trajectories are retained.

This is a subtle but important move. LEVEL-1 does not ask the policy to copy the expert’s exact reasoning or action sequence. It gives the policy a scaffold — roughly, “here is the route” — and lets the policy choose its own native actions and step-by-step behavior. If the resulting trajectory succeeds, it is much more likely to be learnable because the student generated it.

The appendix case study makes this concrete. In a Thunderbird file-attachment task, the converted expert trace copies S2-style hotkeys and template reasoning. The self-rolled version receives only a short plan and then acts through its own clicks and observation-grounded thoughts. Both may finish the task, but only one looks like behavior the target policy can naturally reproduce.

LEVEL-1 therefore fixes one problem: expert guidance becomes reachable.

But it introduces another risk. Plan-conditioned success can become too confident and too narrow. The paper reports that LEVEL-1 provides a strong boost on expert-covered tasks but has limited transfer to held-out tasks and causes a pronounced entropy drop. In plain language: the agent learns to follow helpful plans, but if the plan-shaped scaffold remains static, exploration can tighten around the scaffold. The model becomes better, but not flexible enough.

LEVEL-1 is not the full answer. It is the bridge from alien expertise to native behavior.

LEVEL-2 keeps the bridge from becoming stale

The second stage, LEVEL-2: Self-Aligned Off-Policy Assimilation, is where BEPA becomes more than a clever data-cleaning trick.

LEVEL-2 maintains a per-task cache of successful trajectories. Initially, this cache is seeded by LEVEL-1 self-rolled successes. During GRPO training, the policy samples groups of on-policy rollouts. If the current policy succeeds on a task, BEPA refreshes the cache with one of the policy’s own successful trajectories. The default update rule is random sampling among successes.

If the entire rollout group fails and a cached success exists, BEPA injects one cached successful trajectory into the group for the training update.

Two details matter.

First, the cache is updated with the policy’s own emerging successes. Over time, the cache shifts away from “expert-derived guidance” and toward “recent successful behavior by this policy.” This is why the paper calls the process assimilation. The expert does not remain an external tutor forever. The student starts replacing the tutor’s handwriting with its own.

Second, injection happens only upon total exploration failure. BEPA does not flood the training batch with off-policy data. It lets normal on-policy learning proceed when the policy is finding rewards. The cached trajectory appears when the group would otherwise provide no positive signal. This is a practical compromise between pure exploration and supervised replay: do not interrupt learning when the policy is learning; intervene when the learning signal is empty.

In GRPO terms, the injected success creates a contrast inside the rollout group. The successful cached trajectory receives positive advantage, while failed trajectories receive negative advantage after group normalization. The paper’s convergence appendix is careful here. It analyzes the clipped surrogate objective, not a magical guarantee of monotonic improvement in real environment return. Good. Nobody needed another “guaranteed improvement” claim held together by assumptions and vibes.

The practical mechanism is simpler:

1. Let the policy try.
2. If it finds success, store the successful behavior.
3. If it completely fails later, remind it using a success it can increasingly call its own.
4. Keep the reminder close to the policy’s current distribution.

That is the core of BEPA.

The main result is not just a higher score; it is a diagnosis

The paper’s headline number is strong: it reports that BEPA improves UITARS1.5-7B on OSWorld-Verified from 22.87% to 32.13% overall success. The detailed OSWorld table further reports large gains across the training, expert-covered, and held-out splits, including 10.30% on the strictly held-out set versus 5.74% for the base UITARS1.5-7B policy and 5.32% for GRPO.

Those numbers matter, but the deeper value is diagnostic. BEPA is not just “RL plus expert data.” The experiments separate several possible explanations.

The most telling comparison is not BEPA versus the base model. It is BEPA versus the “reasonable” expert-integration baselines.

SFT makes the model more likely to produce converted expert traces, but that does not become robust task success. SFT+RL appears especially fragile. LUFFY and trace replacement avoid the same kind of entropy collapse but still leave static off-policy guidance insufficiently internalized. LEVEL-1 alone helps strongly on expert-covered tasks but transfers poorly. LEVEL-2 alone does better because it keeps the cache aligned with the evolving policy. BEPA combines both and performs best.

The paper’s mechanism-first story holds together because the ablations match the theory. Reachability and dynamic alignment are not decorative labels. They correspond to different failure modes.

The cross-benchmark results are promising, but not a coronation

The generalization tests are encouraging.

On MMBench-GUI L1 content understanding, BEPA improves the UITARS1.5-7B family across easy, medium, and hard levels. The reported weighted overall scores for BEPA are 90.46, 91.91, and 92.60 across the three difficulty bands, compared with 83.69, 87.49, and 90.44 for the base UITARS1.5-7B model. On L2 element grounding, BEPA reports 88.76 on basic and 61.56 on advanced, ahead of the base model’s 83.72 and 58.57.

This supports the idea that BEPA is not merely memorizing OSWorld trajectories. Some of the training appears to improve general screen understanding and grounding behavior.

But the boundary matters. The paper notes that MMBench-GUI L3 and L4 task configurations are not publicly available at the time of writing, so the reported MMBench results cover L1 and L2. That means the evidence is stronger for GUI perception and grounding than for full multi-step task automation across every MMBench level.

Online-Mind2Web is also useful because it evaluates live websites. BEPA reaches 28.52 average success, above UITARS1.5-7B at 23.28, LUFFY at 24.79, and GRPO at 24.81. That is a real gain. It is also far below the strongest listed systems such as Navigator and Google Computer Use. A sober reading: BEPA improves the trained end-to-end policy, but it does not suddenly make a 7B end-to-end agent dominate the entire web-agent ecosystem. Terribly inconsiderate of reality, but there we are.

The domain breakdown adds another important boundary. BEPA does well in several domains and even exceeds Agent S2 on Chrome in the reported case: 22 Chrome successes versus 13 for Agent S2 and 10 for UITARS1.5-7B. But multi-application workflows remain difficult. BEPA reaches only 8 successes where Agent S2 solves 22. That is exactly the kind of limitation enterprise buyers should care about, because real office automation often crosses applications: browser to spreadsheet, email to file system, PDF to CRM. The messy stuff, naturally.

The appendix tests robustness, not a second thesis

The appendix adds two useful pieces of evidence.

First, the extensibility experiments show that BEPA is not locked to one expert or one base policy. With Agent S2 as expert, BEPA improves UITARS1.5-7B. With GUI-Owl as expert, it still works and shows different domain strengths. Combining S2 and GUI-Owl reaches 34.71 overall in the reported OSWorld-Verified table, above either single-expert variant. On OpenCUA-7B, BEPA with Agent S2 reaches 31.98 overall versus the OpenCUA baseline at 24.30.

This suggests the mechanism is portable across some expert and backbone choices. It does not prove that any expert trace pool will work. A weak expert, a badly extracted plan, or a domain where the base policy cannot execute meaningful self-rollouts could still starve the cache.

Second, the sensitivity analysis studies rollout group size and cache update rules. Larger rollout groups improve the chance of discovering rare successes and triggering cache updates, but the paper keeps a default size to balance cost and performance. For cache updates, random selection performs about as well as more engineered rules such as highest log-probability or shortest step, with differences described as marginal and not statistically significant.

This is a good sign. If BEPA depended on a delicate cache-selection heuristic, it would look like another fragile training recipe. Instead, the important variable appears to be dynamic updating itself: once the policy generates a valid success, almost any successful on-manifold trace is useful enough.

The business value is cheaper assimilation, not magic autonomy

For enterprise automation, BEPA’s lesson is not “GUI agents are solved.” They are not. The lesson is that workflow knowledge has to be operationalized into the learner’s action distribution.

A business may already possess valuable traces:

• screen recordings from human operators;
• RPA scripts;
• helpdesk resolutions;
• browser automation logs;
• QA-approved task demonstrations;
• expert agent trajectories from a stronger but expensive framework system.

The naïve approach is to store these as demonstrations and fine-tune a model. BEPA suggests a better pipeline.

The key business inference is this: BEPA makes expert data more reusable by converting it into a policy-compatible improvement loop. That can reduce the cost of collecting ever more demonstrations, because the system can increasingly learn from its own successful executions.

But this is an inference, not something the paper directly proves in a deployment study. The paper evaluates benchmark performance, not enterprise ROI, maintenance cost, security risk, or integration burden. The operational infrastructure is non-trivial. Training GUI agents requires virtualized environments, rollout orchestration, task reset mechanisms, verifiers, caches, and enough compute to sample groups of trajectories. The paper’s Verl-GUI implementation details make the hidden systems work visible: this is not a prompt-engineering trick.

A serious enterprise takeaway is therefore not “install BEPA.” It is: when building computer-use agents, design the learning system so successful behavior can be re-executed, verified, cached, refreshed, and gradually absorbed by the deployed policy. Otherwise, your automation knowledge base may become a graveyard of successful traces nobody can use.

Where BEPA should make readers cautious

BEPA is a strong paper because its limitations are not cosmetic. They directly affect how one should interpret the result.

The first limitation is benchmark scope. The main training environment is OSWorld-Verified, with generalization tested on MMBench-GUI and Online-Mind2Web. That is meaningful, but still not equivalent to uncontrolled enterprise desktops, legacy systems, internal portals, region-specific software, or workflows involving credentials and compliance constraints.

The second limitation is reward structure. BEPA is designed around RLVR: reinforcement learning with verifiable rewards. A task either succeeds or fails based on an evaluator. Many business processes do not have such clean success checks. Was the invoice categorized correctly? Was the customer response compliant? Was the dashboard update materially useful? Preference-based and noisy feedback would require additional machinery.

The third limitation is expert-source dependence. BEPA assumes access to successful expert trajectories. The paper tests Agent S2, GUI-Owl, and a hybrid setup, but the quality, diversity, and coverage of the expert pool still matter. If expert traces do not cover the hard cases, or if plan extraction fails to preserve the right high-level structure, LEVEL-1 has less to work with.

The fourth limitation is multi-application complexity. The domain breakdown shows that multi-app workflows remain difficult. This is not a side note. In business automation, cross-application work is often where the ROI lives. If an agent can handle isolated browser tasks but struggles with browser-plus-spreadsheet-plus-email workflows, it is useful — but not yet the tireless digital employee promised in glossy decks. Those decks should be forced to run OSWorld before being allowed into meetings.

The larger shift: from demonstration storage to competence transfer

BEPA’s best contribution is conceptual. It reframes expert traces as material to be assimilated, not copied.

That distinction is likely to matter as companies move from one-off agent demos to continuously improving automation systems. A demo agent can be manually fixed, prompted, or wrapped with tools. A production agent needs a learning loop that compounds. It needs to preserve successes, learn from failures, and avoid becoming a different broken system every time it is trained.

BEPA offers one such loop:

• Expert traces provide direction.
• Self-rolling turns direction into reachable behavior.
• Dynamic caches keep guidance close to the evolving policy.
• Conditional injection makes complete failure trainable.
• On-policy learning remains the center of gravity.

This is not the first paper to use expert guidance or replay-like ideas in reinforcement learning. The novelty is the fit to GUI-agent mismatch: framework experts and end-to-end policies do not naturally share an action distribution, so the learning system has to build a translation layer and then retire it gradually.

That is why a mechanism-first reading is more useful than a benchmark-first summary. The scores are impressive, but the mechanism is the thing enterprises should remember. In computer-use automation, the scarce asset is not merely data. It is verified successful behavior that the deployed policy can reproduce under changing screens.

BEPA does not make GUI agents fully reliable. It does something more modest and probably more useful: it shows how a policy can stop treating expert traces as alien scripts and start turning them into its own operating memory.

Click. Fail. Learn. Cache the rare success. Replace the tutor slowly. Try again.

For once, “agent improvement” means something more concrete than a larger model and a nicer demo video.

Cognaptus: Automate the Present, Incubate the Future.