CUA-frontier Loading Submit a task

Collaboration · project

Shared 3D design

Second domain0 tasks

Do the coordination problems the game testbed measures carry over to large 3D design, where a valid layout change can obstruct a moving assembly or a required review view?

Two people work on one large 3D model, such as a facility with a modular layout, moving assemblies and a review walkthrough, each through their own CUA agent. One changes the layout, the other the motion and the review views. Together they can block a moving assembly or a required view while every file stays valid.

What this project asks

This is the proposal's second domain, and it is not yet validated. A facility model has a modular layout, moving assemblies and a review walkthrough. Person A changes instances, the hierarchy, spatial relationships and annotations; person B changes motion, dependent transforms, camera paths and review states, through scripts and edits in the application. What each knows in private says which regions a customer fixed and which alternatives are negotiable. A late layout publication can leave every file valid while an assembly now sweeps into a wall or a required review view is blocked, and a script-only repair can restore the motion but move a region the customer fixed. This project asks whether what Shared game development measures holds in a second application: the agents have to inspect the integrated model, exchange versioned evidence and reasons, get decisions from the right person where needed, and verify the result. Large models can still be studied with two units, which keeps the size of the artifact separate from the size of the team.

What tasks can look like

Any task that tests the question fits. Some directions:

  • Layout against motion: a regrouped module and a revised motion path that each check out alone and together cut through an operating envelope.
  • Review views: a camera path or a review state that a layout change silently blocks or reframes.
  • Identity and hierarchy: instances renamed, regrouped or re-parented, so that the other unit's scripts or annotations bind to the wrong objects.
  • Customer-fixed regions: the obvious repair moves something one person promised a customer, so that person has to be consulted or a repair found that keeps it.
  • The same episode with access to the files only, to runs without rendering, and to the model in the application, as in the game testbed.

What a verifier here should establish

  • Public contracts on spatial relationships, supported motion, the visibility of required views and the consistency of object identity, checked in the integrated model, independently of what the agents report.
  • Each person's commitments survive, reported per person, and any change to a commitment was approved by its owner.
  • Evidence claims, such as views, renders and clearance checks, match the model version and the camera they cite.
  • The interaction is real: the baseline and each change pass, the combination fails, and a complete repair passes. Finding it needs looking at the model or running its motion, not only validating files.
  • What the task shows about transfer from games. A claim that a result holds across domains needs an evaluated adapter for this application and coordination effects that recur here, not a description alone.

Worth reporting

  • The same parts as in Shared game development, so the two domains can be compared
  • Whether the conflict needed the rendered model, a run of its motion, or only file validation
  • How the size of the model affects the outcome, with the team held at two units

For example

In a packaging hall, A moves a conveyor module to free a walkway a customer asked for. B retimes a robot arm's pick and sets a review camera for the line's sign-off. Each passes its own checks. Integrated, the arm's swept path clips the moved conveyor's guard in one part of the cycle, and the review view no longer shows the pick. Moving the conveyor back breaks the customer's walkway; slowing the arm misses B's cycle time. A good run finds the clash by running the motion in the integrated model, shows both people the evidence, and agrees on a new approach path or guard that keeps both commitments.

Generate a task with your agent

Give this to your coding agent in a clone of the repository. It picks a task that tests this project's question, builds it with a verifier of its own, and checks the package. You run the frontier models and open the pull request; people review it there.

Build a benchmark task for the CUA-frontier project "Shared 3D design" (shared-3d-design).
Repository: https://github.com/kingofspace0wzz/CUA-frontier

1. Read AGENTS.md and areas/collaboration/shared-3d-design/PROTOCOL.md: what the project asks.
2. Pick a task that tests that question. Start it with:
   python scripts/new_task.py shared-3d-design <task-id>
3. Write task.md, env/ and a verifier that fits this task in verifier/, and
   state in metadata.yaml (intent:) which capability it measures. Every
   criterion the verifier scores must be stated in task.md or the env; the
   facts it checks must be reachable only through the interaction the task
   tests (for computer use, the GUI): tools you offer, such as a shell, must
   not reach them another way.
4. Calibrate the verifier: a reference solution in oracle/ it accepts,
   plausible wrong ones in negatives/ it rejects, and a check: command that
   shows both; run it with python scripts/check_task.py.
5. Have a separate red-team agent, which has not seen oracle/, try to pass
   without doing the task. Prepare its workspace with
   python scripts/red_team.py tasks/shared-3d-design/<task-id>
   and give the prompt it prints to a subagent that does not see your
   conversation (in Codex, fork_turns: "none"). Never launch another agent
   CLI (codex exec, claude -p) for it. Fix what works (a guess that passes
   by chance may stay if its chance per trial is under 1%, shown in
   verifier/README.md), and record every attempt under red_team: in
   results/quality.yaml.
6. Fill in the rest of metadata.yaml, including generated_by, and run
   python scripts/validate.py tasks/shared-3d-design/<task-id>
7. Stop and report what you built, the shortcuts tried, and where your
   trajectory is saved. Do not run the frontier evaluation yourself.

Tasks in this project

Claims, open pull requests and merged tasks, from the board.