CUA-frontier Loading Submit a task

Collaboration · project

Asynchronous integration

Second stage0 tasks

When units publish late against different versions, can they tell which build and evidence is current, recover why the other unit made its choices, and integrate without silently undoing either person's work?

Each person works with their own agent for a long stretch before publishing, so the two units build on different versions of a shared application and look at different builds. When the work meets, the published diff lacks the reasons behind it and old captures show a build that no longer exists. The agents have to line up versions, recover the intent, and keep both features.

Generate a task with your agent Claim a scenarioProject folder

What this project asks

Two units can go through many exchanges with their own people before publishing a branch, a scene or an asset revision. Meanwhile the other unit updates the shared project, so a later request to "resolve and merge" means reconciling both artifacts and the assumptions behind them. Staying in sync here is about versions and knowledge, not clocks. A publication carries the revision, the diff, a description and any captures attached to it. It does not carry the conversation, the alternatives a person rejected, the states the other agent looked at, or the decisions its person made. Seeing the snapshot is not seeing the reasons: the receiving agent may notice a camera offset or a timing value and not know why it is there, and importing an old scene can undo newer behavior that depends on code. Work on asynchronous and concurrent coding (AsynCodeBench, AgentRoom, Passes Alone, Fails Together) judges the code. This project asks whether units can recover intent that shows only in the running application or in the other unit's history, and keep it through the merge.

What tasks can look like

Any task that tests the question fits. Some directions:

  • Late publication. Both units start from one playable revision. One publishes first; the other publishes against the older base and is asked to integrate. A scene-file merge drops the first unit's adjustment, or a clean merge keeps a value whose effect changes under the new hierarchy.
  • Stale evidence. A capture truthfully shows an older build but says nothing about the merge, and tests and isolated previews pass while the integrated build fails in a disclosed state.
  • Missing reasons. The diff and its description leave out why a choice was made; the reason can be had only by asking the other unit or its person, or from that unit's work history.
  • Perspective. An agent has to establish which build its peer inspected, what the peer knows about a later update, and why a choice was kept, and then act on it: ask, preserve the choice, or renegotiate it.
  • One episode under paired conditions: a current or a stale base; an ordinary diff, a rationale summary with linked captures, or access to the observable work history.

What a verifier here should establish

  • Revisions and publication points are fixed, the interaction is real in both publication orders, and a repair that keeps both features exists for each.
  • Preservation through reconciliation: which commitments that held before the merge, and still apply, the merge broke, per owner and by functional or visual effect. A silent overwrite fails even when the merge and the code tests pass, and a correct summary without preservation is not success.
  • Evidence is fresh: every capture and claim names the build, state, view and inputs it came from, and claims about the integrated build were made on it.
  • The affected states of the application were identified and checked again after integration.
  • No harness action overwrites a compatible solution to manufacture a conflict. Publishing early, asking, or fetching current work to prevent divergence counts as success.
  • The record has each branch base, publication, fetch, merge and capture, with the order messages were delivered in, so a reviewer can tell which build existed at each consequential action.

Worth reporting

  • Commitments broken by reconciliation, per owner, functional against visual
  • Current against stale bases, with the same requests and ownership
  • How the reasons reached the other unit: ordinary diffs and descriptions, rationale summaries with linked captures, summaries of matched length, or retrievable histories
  • How often agents checked the peer's current revision before integrating, and whether their claims about what the peer knew were grounded in the record
  • What extra context cost: tokens read and retrievals

For example

Both units start from the playable lift sequence. A reworks the lift scene, its hierarchy and a shoulder camera transition. B adjusts the companion's attachment offset after watching runs from the player's view, and B's person rejected an alternative. B publishes first, with an ordinary description that does not say which camera states led to the offset. A publishes against the older base and is asked to integrate. The merge keeps B's offset, but under A's new hierarchy the warning disappears during the lift transition, while an old capture of B's build still looks fine. A good run checks the integrated player view against B's versioned evidence, asks B why the offset was chosen, and revises the attachment or the transition so that both features hold. Removing A's camera transition to clear the view abandons A's commitment.

Generate a task with your agent

Give this to your coding agent in a clone of the repository. It picks a task that tests this project's question, builds it with a verifier of its own, and checks the package. You run the frontier models and open the pull request; people review it there.

Build a benchmark task for the CUA-frontier project "Asynchronous integration" (async-integration).
Repository: https://github.com/kingofspace0wzz/CUA-frontier

1. Read AGENTS.md and areas/collaboration/async-integration/PROTOCOL.md: what the project asks.
2. Pick a task that tests that question. Start it with:
   python scripts/new_task.py async-integration <task-id>
3. Write task.md, env/ and a verifier that fits this task in verifier/, and
   state in metadata.yaml (intent:) which capability it measures. Every
   criterion the verifier scores must be stated in task.md or the env; the
   facts it checks must be reachable only through the interaction the task
   tests (for computer use, the GUI): tools you offer, such as a shell, must
   not reach them another way.
4. Calibrate the verifier: a reference solution in oracle/ it accepts,
   plausible wrong ones in negatives/ it rejects, and a check: command that
   shows both; run it with python scripts/check_task.py.
5. Have a separate red-team agent, which has not seen oracle/, try to pass
   without doing the task. Prepare its workspace with
   python scripts/red_team.py tasks/async-integration/<task-id>
   and give the prompt it prints to a subagent that does not see your
   conversation (in Codex, fork_turns: "none"). Never launch another agent
   CLI (codex exec, claude -p) for it. Fix what works (a guess that passes
   by chance may stay if its chance per trial is under 1%, shown in
   verifier/README.md), and record every attempt under red_team: in
   results/quality.yaml.
6. Fill in the rest of metadata.yaml, including generated_by, and run
   python scripts/validate.py tasks/async-integration/<task-id>
7. Stop and report what you built, the shortcuts tried, and where your
   trajectory is saved. Do not run the frontier evaluation yourself.

Tasks in this project

Claims, open pull requests and merged tasks, from the board.