CUA-frontier Loading Submit a task

Collaboration · project

Shared game development

Start here0 tasks

Can two units, each a person and their own CUA agent, find and resolve a conflict between their changes to a game's code, scenes and running state by investigating the game itself, without giving up either person's requirements?

Two people develop one game, each through their own computer-use agent. Each person's change passes its own checks, but together they break something that shows in the running game. The agents have to find it by running and watching the game, settle a fix with both people, and keep both features.

What this project asks

In a game studio, people own different parts of one game and know different things, and more of them now work through an agent of their own. In a game, what counts as correct lives in several places at once: the scripts, the scenes and assets, and what a player sees in a given state. Two changes can merge without a single conflict and still hide a warning during one camera move, and a repair that looks right can quietly undo the other person's feature. Prior work covers the pieces: agents acting for separate owners (WeClawArena, Worse Together), one goal split across parallel GUI agents (ParaGUIBench, MACU), patches that pass alone and fail together (Passes Alone, Fails Together), and game development by a single agent (GameDevBench, GameXpert-Bench). This project asks for their intersection. Two units, each a person and their own CUA agent, change the same game; their changes interact; finding and resolving the interaction takes running and watching the game; and each person owns requirements the other must keep. Games come first because scenes, states and play traces can be reproduced and checked.

What tasks can look like

Any task that tests the question fits. Some directions:

  • Two feature requests. Each person asks their own agent for a feature in the same game. The agents clarify, implement and test their own work, and the features interact through code, a scene or asset, and a state of play. Sharing plans early and avoiding the conflict counts as success.
  • Merge day. Both features are finished and each passes alone; merged, they break a shared rule in play. The units reproduce it, find the cause, agree on a fix and release one build.
  • A frozen integrated build with healthy siblings (the baseline, A only, B only), for controlled diagnosis: reproduce the failure, localize the interaction, explain it to both people with matched evidence, and repair it.
  • A person changes their mind. A predeclared update partway through changes one person's commitments. It reaches only that person's agent, and the other unit learns of it through the collaboration.
  • Split knowledge and authority. Each person knows something the other does not, such as which arrival layout a customer fixed or what protection the latest playtest asked for, and only the owner may approve a change to their commitment. A repair that keeps everything needs no approval.
  • Episode families the proposal suggests: an escort encounter on a moving lift (camera transition, companion formation, threat warnings, checkpoint restore); an arena that transforms mid-fight while attacks have to stay distinguishable; checkpoint assistance next to a redesigned route whose prompts and camera framing have to stay readable.

What a verifier here should establish

  • The interaction is real: the baseline and each change alone pass the shared rules, the combination fails them, and a complete reference repair passes. Where a person's commitment is lost while the shared rules still hold, its check is validated before and after integration.
  • Running the game matters. The task is screened against reading the source and running it without looking: a model that only reads the code should not be able to name the trigger, the broken rule and the cause. A case that source reading solves can stay only as a control.
  • Both units act consequentially, and what each person knows, decides or updates changes the outcome.
  • The joint result, part by part: the shared rules that nobody can waive, each person's features (reported per person), authority (every material change to a commitment approved by its owner, for the build actually released) and evidence (claims in handoffs and review packets match the build and state they cite). Joint success needs all of them.
  • Shortcuts fail: removing required behavior, editing evaluation hooks, hiding the failing view, or claiming completion the build does not support. Evaluation runs outside the agents' workspace and does not change how the game behaves.
  • Simulated people answer from a fixed record of what each knows and may decide. Their approval never stands in for correctness, and neither person can waive the other's commitments or a shared rule.
  • Any compatible repair counts, and so does preventing the conflict; no particular dialogue or fix is required.

Worth reporting

  • Joint success and each of its parts, per person rather than averaged
  • Where failed runs broke down: noticing the interaction, getting the context, agreeing, carrying out the fix, or keeping it after an update
  • The kind of conflict: a textual merge conflict, a semantic code conflict, a visual or asset conflict, or one that spans them
  • The same task under controls: two people with two agents, one person with two agents, two people sharing one agent, full shared context, and what the agents could use, from the source alone, through running it without looking and fixed captures, to playing it themselves
  • Messages, questions to people, captures and compute spent
  • An episode card in the task's README: provenance, the starting state, local and published state, both requests, the shared rules, what each person knows and may decide, the reference evidence (baseline, A only, B only, combined, a complete repair), and which models and runs were used while building it

For example

A rescue sequence rides a moving lift. A's agent adds the lift's docking states and a shoulder camera transition; B's agent adds a companion formation and threat warnings. Each passes its checks. Together, after a checkpoint restore, the companion stands in the player's warning region during the camera move and hides a warning that must stay visible. A knows which arrival layout the customer fixed; B knows what protection the latest playtest asked for. A good run finds it by playing, the agents exchange those facts, B approves retiming the formation, and both check the shared build. A run that clears the view by removing the warning breaks a shared rule, and one that drops A's camera transition breaks A's commitment, whatever either agent reports.

Generate a task with your agent

Give this to your coding agent in a clone of the repository. It picks a task that tests this project's question, builds it with a verifier of its own, and checks the package. You run the frontier models and open the pull request; people review it there.

Build a benchmark task for the CUA-frontier project "Shared game development" (multi-user-collab).
Repository: https://github.com/kingofspace0wzz/CUA-frontier

1. Read AGENTS.md and areas/collaboration/multi-user-collab/PROTOCOL.md: what the project asks.
2. Pick a task that tests that question. Start it with:
   python scripts/new_task.py multi-user-collab <task-id>
3. Write task.md, env/ and a verifier that fits this task in verifier/, and
   state in metadata.yaml (intent:) which capability it measures. Every
   criterion the verifier scores must be stated in task.md or the env; the
   facts it checks must be reachable only through the interaction the task
   tests (for computer use, the GUI): tools you offer, such as a shell, must
   not reach them another way.
4. Calibrate the verifier: a reference solution in oracle/ it accepts,
   plausible wrong ones in negatives/ it rejects, and a check: command that
   shows both; run it with python scripts/check_task.py.
5. Have a separate red-team agent, which has not seen oracle/, try to pass
   without doing the task. Prepare its workspace with
   python scripts/red_team.py tasks/multi-user-collab/<task-id>
   and give the prompt it prints to a subagent that does not see your
   conversation (in Codex, fork_turns: "none"). Never launch another agent
   CLI (codex exec, claude -p) for it. Fix what works (a guess that passes
   by chance may stay if its chance per trial is under 1%, shown in
   verifier/README.md), and record every attempt under red_team: in
   results/quality.yaml.
6. Fill in the rest of metadata.yaml, including generated_by, and run
   python scripts/validate.py tasks/multi-user-collab/<task-id>
7. Stop and report what you built, the shortcuts tried, and where your
   trajectory is saved. Do not run the frontier evaluation yourself.

Tasks in this project

Claims, open pull requests and merged tasks, from the board.