CUA-frontier Contribute a task

CUA-frontier

Computer-use agents, tested in game development, physical and embodied AI, and professional tools like CAD.

Most computer-use benchmarks stay inside web and office apps. Here, agents build and playtest games, operate robots, machines and lab instruments, and engineer designs in CAD. Every task grades what the agent actually produces, with a verifier hardened against shortcuts.

Research

Where computer use gets hard

Each area studies something current computer-use benchmarks cannot see: games played on their own clock and next to people, what work done on a screen does in the physical world, chip designs whose checks can be wrong too, and processes whose final state hides the mistake. Two more cut across domains: worlds that do not wait for the agent, and work shared between people and their own agents. Four newer areas, listed first, start the next phase: an agent that controls a robot under real-robot rules, an agent that works a whole shift in a control room, an agent that analyses real materials measurements against a truth known independently, and an agent that operates a spacecraft. Each links a detailed visual proposal.

01 Frontier Embodied When an agent is the controller of a robot and has to live by the rules a real robot lives by, can it get the physical work right, keep within its limits the whole way, and do it again on versions of the task it has not seen? In September and October 2026 several benchmarks started to test coding agents that control robots. Most of them make life easy for the agent: it can read the simulator's internal state, try again after a failure, work in a scene that never changes, and trust that the robot matches its model file. Tasks in this area use real-robot rules by default. The agent sees only what the robot's own sensors report, cannot reset the world, works while the clock keeps running, and may find that the robot differs from its documents. The agent can control the robot by writing code, through an operator's screen, or through a learned policy, and the same checking program judges the result, the limits along the way and unseen versions of the task. This area is also one of the first two where a model is trained on our tasks and improves itself (see the self-improvement pages below).
02 Frontier Operations At an operator's console, can an agent work a whole shift: keep the system inside its safety limits, notice and handle an instrument or a document that is wrong, and hand the shift over truthfully? Power-grid dispatch, air traffic control, water networks, process plants and ship traffic are run from control rooms. The system never stops, the limits always apply, a reading can be wrong and a drawing can be out of date. People in these jobs are trained and assessed on simulators. Tasks in this area put the agent at the console of an open-source simulator for a whole shift, and judge it on the world's own record: whether every limit held at every step, whether each decision was right when it was made, and whether the handover log is true. This area is also one of the first two where a model is trained on our tasks and improves itself (see the self-improvement pages below).
03 Frontier Materials Working in the software materials scientists use, on real measurements whose truth is known independently, can an agent reach the conclusion the physical evidence supports, even when a good-looking fit, a reference structure or a sample label points the other way? Agents for materials science are already tested on running DFT calculations, on optimisation loops under a budget, on question answering and on microscope control. In those benchmarks the answer key is a simulation, a database value or a reference calculation; none judges the agent against what the physical world measured. The field's best-known failure came from exactly that gap: automated XRD analysis in the A-Lab accepted good-looking fits that experts later showed were the wrong phases, and the 2026 correction reduced its claims. Tasks in this area give the agent real raw data whose answer is known by weighing, by certified reference, by the consensus of many laboratories or by a later publication, in open-source software such as GSAS-II, Profex and Fiji, and in a world where a reference structure, a sample label or an instrument calibration can be wrong. A pass means the conclusion matches the physical truth, not that the fit looks good.
04 Frontier Space Can an agent operate a spacecraft through the ground software people use, when contact comes only in short passes, commands take minutes to arrive, and the telemetry or the documents can be wrong, and keep the spacecraft safe while the mission gets done? Spacecraft are run from mission-control software: telemetry displays with limits, command panels, pass schedules and planning timelines. A satellite in low orbit is in view of a ground station for a few minutes at a time, and a command to Mars takes several minutes to arrive. Tasks in this area put the agent at those screens in front of a simulated spacecraft, with the passes and signal delays of a real mission and a spacecraft that may differ from its documents. The simulator's record decides whether the spacecraft stayed inside its flight rules the whole time, whether the objective was met, and whether the same handling works on variants the agent never saw. The area covers civil operations only.
05 Games Can an agent play, test and build games the way people experience them: on the game's clock, next to human players, and without being told the rules? Agents can already generate a playable game and keep improving it. The projects here go after what that loop does not measure. As in CUA-SWE, the agent sees only the screen and uses only keyboard and mouse, and the verifier reads the game's own state.
06 Physical AI Can an agent design, measure, program, operate and simulate physical things through the software people use, with results that hold up when they are built, run or tested? Engineering, lab and plant work runs through software: CAD and slicers, instrument software, teach pendants and PLC editors, teleoperation screens, simulators. Computer-use benchmarks stop at the saved file or the click, and embodied benchmarks score a robot policy, or a coding agent's program, on a goal it is given. Projects here judge the agent's work by what it does once it is built, run or tested in simulated physics: the part holds, the measured value is right, the cell makes good parts at rate, the robot gets there, the simulation predicts the run it stands in for.
07 Chip & EDA When a check fails, can the agent tell whether the design, the testbench, the constraints or the flow is wrong, and fix that one? Agents write RTL and circuits, build their own tests, and push designs through implementation tools. Most prior work runs the tests for the agent or repairs one layer at a time. Projects here give the agent control of both the design and how it is checked, and score recovery against the unchanged specification.
08 Workflows Does the agent get the process right, not just the final state? Business tasks are spread across several systems, and a support ticket can read "resolved" after a refund went out twice. Projects here inject the faults real operations see, such as timeouts with an unknown outcome and duplicate webhooks, and check invariants and the order of operations as well as the end state.
09 Real time Can an agent keep deciding well when the world does not wait for it, and how fast a world can it keep up with? Decision models such as Jev and OpenAI Decisions answer a choice among given options in about a second or less, and they can drive a screen when its candidate actions are offered as that choice. Projects here build the tasks a real-time decision-maker is judged on: screens where decisions come quickly, the options are on the screen, and the best option changes while the agent decides. A task can run at several tempos, and the world's own record grades every decision as it lands, so it tells how fast a world an agent keeps up with, not only whether it finished.
10 Collaboration Can people, each working through their own computer-use agent, reconcile their changes to one application's code, visuals and running behavior, and keep what each of them needs even when their work is out of sync? More people now work through an agent of their own, on products whose parts other people own. Two changes can merge cleanly and still break what a user sees; a fix that looks right can quietly undo someone else's feature; and work published late arrives without the reasons behind it. Projects here give each person their own CUA agent on a shared application, where finding and fixing such a conflict takes running the application and looking at it. Games are the first testbed and large 3D design the second.

Approach

No role grades its own work

Each task is a prompt, a world, a verifier and an intent that have to agree. Agents build a task and attack it, frontier models attempt it, and people read every failure before anyone reviews it. Each step leaves a record that is checked automatically before review.

  1. 01 · Author agentBuilds the taskThe prompt, the world it runs in, a verifier made for it, and what it measures.
  2. 02 · VerifierCalibrated on known answersIt must accept a reference solution and reject plausible wrong ones.
  3. 03 · Red-team agentTries every shortcutA separate agent tries to pass without doing the task. What works gets fixed.
  4. 04 · Solver agentsFrontier runsFrontier models attempt the task end to end, three or more times, and at least one attempt must fail.
  5. 05 · PeopleRead every failureEach failure must come from missing capability, not a broken task.
  6. 06 · Audit agentChecks five agreementsNo hidden requirements, no back doors, no reward hacking, a real request, legitimate failures.
  7. 07 · ReviewersApprove, then mergeSomeone other than the submitter reviews it before it joins the benchmark.

Team

The people behind it

CUA-frontier is built by a growing team of students and researchers, with tasks from contributors across every area.

Organizers

Advisors

Core contributors

Meet the team
Bring a task to the frontier

Pick a project, have your coding agent build a task that tests its question, run the frontier models on it, and open a pull request. People review it once it is merged.