CUA-frontier Loading Submit a task

Area · 5 projects Priority

Physical AI

Can an agent design, measure, program, operate and simulate physical things through the software people use, with results that hold up when they are built, run or tested?

Engineering, lab and plant work runs through software: CAD and slicers, instrument software, teach pendants and PLC editors, teleoperation screens, simulators. Computer-use benchmarks stop at the saved file or the click, and embodied benchmarks score a robot policy, or a coding agent's program, on a goal it is given. Projects here judge the agent's work by what it does once it is built, run or tested in simulated physics: the part holds, the measured value is right, the cell makes good parts at rate, the robot gets there, the simulation predicts the run it stands in for.

Projects

Each project is a direction with room for many kinds of task. Its page says what it asks and what a verifier there should establish.

Design, build, test

 Start here

From a requirement, the agent designs a part, a structure or a mechanism in CAD and the slicer, CAM and analysis tools around it. The benchmark builds and tests the design in simulated physics, and the agent revises it until it meets the requirement at a good trade-off of weight, cost and time to make.

Can an agent turn a requirement into a design that works when it is built and tested, and get it there when the first tests fail?

0 tasksOpen the project →

Lab experiments

 Second stage

From a scientific or quality question, the agent plans experiments, runs them on simulated instruments through the lab's software, analyses the data, decides what to run next, and reports a result that the benchmark checks against hidden truth.

Can an agent run an investigation through lab software, from planning experiments to running instruments and analysing the data, and reach a correct result within its budget and safety limits?

0 tasksOpen the project →

Machine programming

 New

The agent programs a robot cell, a CNC machine or a production line in the tool a technician would use, and the benchmark runs the program on a simulated cell to see whether it makes the right parts, at the rate the job needs, safely.

Can an agent program a robot cell, a CNC machine or a production line in the tools technicians use, so that it does the job (the right parts, at rate, safely), and keep it running when the cell changes?

0 tasksOpen the project →

Teleoperation

 Main bet

The agent drives robots, vehicles or machines through the same screen and controls a person would use, on everyday remote work and on tasks that move faster than it thinks.

Through a human teleoperation interface, which physical tasks can a general CUA do, and what limits it?

0 tasksOpen the project →

Simulated worlds

 Start here

Simulations stand in for the physical world when people decide what to build or how to run it, and AI models now generate them from a sentence. Tasks here build and calibrate a simulation through its GUI, use it to make a decision that is checked in a hidden system, or judge when a simulation, or a learner trained in one, cannot be trusted.

Can an agent build a simulation good enough to decide with, use it to make a physical decision, and notice when a simulation, or a learner in it, cannot be trusted?

0 tasksOpen the project →

Propose a project

Have a question in Physical AI that current benchmarks miss? Propose it as a project: the question, what tasks could look like, and how they could be verified.

opens a GitHub issuePropose →

Tasks in this area

Claims, open pull requests and merged tasks across its projects, from the board.

Coverage

Where this area's tasks sit, merged and in review, and the published benchmarks closest to them. A grey cell holds only prior work, an outlined cell is where we want tasks, and a hatched cell has nothing yet. A task can appear in several cells. Prior work has no project, so the views on the project leave it out.

Loading the tasks…

ProposedIn review ValidatedMerged Prior work onlyFocusWhere we want tasks Nothing yet

Where we want tasks

  • Misleading information × The real software. The real software, and a world that departs from its documents: a reading, a calibration or a drawing that is wrong. No benchmark below has both. Where the software is real (CADWorld) the check stops at the saved file; the instrument benchmark (LabOSBench) grades the procedure on simulators built for it.
  • Misleading information × Machine programming tool. A cell that is not what its drawing says, such as a fixture mounted off, a wrong tool length or a sensor wired inverted, found and handled from the pendant or the PLC editor. PLCWorld grades PLC code in a closed-loop plant; here the program is written in the technician's tool.
  • A change × Machine programming tool. A changeover: a new part, gripper or fixture, so a program that worked has to be changed and proven again on the cell.
  • A change × Engineering software. The requirement, the material or the process changes after the first design works, and the design follows in CAD and the slicer, with the new part made and tested again.
  • Misleading information × Teleoperation screen. One of the operator's instruments is wrong, such as a GPS, a camera or a gauge, and the others are right: the agent has to notice it from the screen and drive on what it trusts.
  • Time pressure × Teleoperation screen. Drive a machine at the world's own speed through a person's controls. Embodied First Exam's real-time mode drives robots through code.

Prior work in the matrix

EFE Embodied First Exam
Oct 2026. Coding agents (Codex, Claude Code) control simulated robots through a robot API, or inside the simulator in a privileged mode: 1,156 tasks and 2,288 tests in 35 suites, 9 of them its own, on 47 robots in MuJoCo, Isaac Sim, OmniGibson, SAPIEN and Genesis. Each task gives the goal in words and a frozen scene. On the 456-task core set the best model on the leaderboard in November 2026 solves 34% (GPT-6 Astra). GPT-6 Luna, run to filter the tasks, solves all 18 LabUtopia tests, where the lab procedure is given.
CW CADWorld
Sep 2026. 200 long-horizon tasks in FreeCAD, through screenshots and GUI actions, across 11 workflows from sketching and assembly to CAM and FEM. Executable checks read the saved FreeCAD project and its outputs; nothing is made or tested. The best agent passes 17.5%, against 87.0% for an expert reference.
CEB CADEngBench
Aug 2026. Models write CadQuery code. 600 part tasks, a new part from a brief or a functional edit, are checked for B-Rep validity, engineering and manufacturability rules, parameter perturbations and a linear-static FEA in CalculiX matched to a reference; 150 assembly tasks ask for the joint between two bodies. Editing supplied CAD is much easier than generating it, and complex edits and the FEA match remain hard.
SV ScienceVision
Nov 2026. 250 expert questions about supplied 3D artifacts in five domains, answered from the native file with code, with viewer tools added (views, hiding parts, slices, measurement), or from six fixed renders, and scored without a model judge. Two of its domains are placed here. The best model scores 0.53 of 1; the native file with code beats the renders by 17 points on average, and the viewer tools change scores by about 2 points either way. The facts are in the file: nothing is built, run or measured.
LOS LabOSBench
Jun 2026. GUI agents operate eight web-based instrument simulators built for the benchmark (SEM, TEM, light and fluorescence microscopy, XRD, EDS, atom probe, FIB and scanning probe): 96 subtasks, from loading and aligning a sample to tuning, acquiring and inspecting. Checkers grade each subtask, with image-quality metrics where the step makes an image. Agents complete many structured subtasks but struggle with adjustments driven by feedback and with long workflows.
PLC PLCWorld
Oct 2026. LLMs write Structured Text that runs against a closed-loop simulated plant: 100 tasks and 473 task-condition pairs in motion control and material handling, scored on task success and safety violations. GPT-5.5 asked directly succeeds on 82.7% of easy cases and 25.1% of hard ones.
ACED ACED-Bench
Oct 2026. Agents investigate scientific scenarios written over hidden probabilistic graphical models: they choose interventions, read finite evidence and commit to a causal answer or decision (180 core tasks). Failures: testing the wrong intervention, losing evidence, and going on after the evidence was enough. The worlds are causal graphs, not physics; placed here for experiment design against hidden truth.
What the tags mean

Physics physics

structures Structures and solids
Strength and stiffness of beams, brackets and printed parts
manufacturing Manufacturing
How printing, machining or assembly changes a part
mechanisms Mechanisms and assemblies
Linkages, gears, threads, snap fits, assembly order
material-handling Material handling
Conveyors, sorting, feeding, filling and palletizing
rigid-body Rigid-body dynamics
Collisions, projectiles, rolling, pendulums
deformables Deformables and granular media
Cloth, rope, paper, piles of small parts
fluids Fluids and soft matter
Flow, rheology, gels
heat Heat
Heat transfer and temperature control
optics Optics
Refraction, dispersion, lenses, gratings
signals Spectra and signals
Peaks, noise and the design of measurements
electrical Circuits and power
Circuits, grids, batteries, motors
chemistry Chemistry and biology
Reagents, reactions, wet-lab work
manipulation Robot manipulation
Arms and hands, including mobile manipulation
locomotion Robots and vehicles in motion
Legged, humanoid, wheeled and flying machines

Interface interface

engineering Engineering software
CAD, slicers, CAM, analysis setup such as FEA or CFD
lab-software Instrument and lab software
Instrument control, methods, bookings, reports
sim-viewer Simulator editor or viewer
A physics simulator's editor or replay, with measuring tools and re-runs
machine-tool Machine programming tool
A teach pendant, offline robot programming, a PLC editor or a CNC control; CAM counts as engineering software
console Control console
Dispatch screens, HMI, SCADA
teleop-screen Teleoperation screen
Camera views with keys, a gamepad or waypoints
notebook Notebook or spreadsheet
Jupyter, a spreadsheet
command-line Command line or API
A shell, an API client or submitted code; a control for the others

How real the software is software

production The real software
The software people use at work, as shipped, such as FreeCAD, PrusaSlicer, Beremiz, Phoebus or Bridge Command; only what sits behind it is simulated
replica A replica
Built for the benchmark to look and behave like one named product, such as a teach pendant modelled on PolyScope
built Built for the task
An app built for the benchmark that stands for no particular product
code Code, no GUI
The agent writes code or calls an API

What makes it hard challenge

the-work The work itself
A large design space, a long job or a tight budget
unknowns Unknown properties
A material, a friction or a machine's behaviour that has to be found out
variation Variation
Parts, loads or conditions that vary, so the work has to hold across them
change A change
The requirement, the material, the instrument or the cell changes, and work that was right stops being right
disturbance Disturbance
A push, an event, a person or an opponent during the run
time-pressure Time pressure
The world keeps moving while the agent decides
misleading Misleading information
A reading, a calibration, a document or a drawing that is wrong
fault A fault to find
Something in the design, the procedure, the program or the setup is wrong
simulator A broken simulator
The simulated world itself breaks physics
exploit An exploit
A high score that comes from a hole in the simulator or the check

What is judged judged

outcome Physical outcome
The design, run, method or program does what was required when the benchmark builds or runs it
quality Quality and cost
Weight, material, time, energy or wear, held to a target or scored against a reference
answer A measured answer
A value or a decision compared with hidden truth
prediction Prediction
A simulation or a model forecasts runs it was not fitted to
diagnosis Diagnosis
Which cause, or which layer, was wrong
audit Audit verdict
Accept or reject a world or a learner, with the root cause
limits Limits held
Safety or process limits held throughout
artifact The saved file
A model, a program or a project checked against its brief without being built, run or tested; where most CAD benchmarks stop
reading A reading of a supplied artifact
An answer read off a model, a scan or a structure handed to the agent, whose facts are already in the file

A task tags itself under coverage: in its metadata.yaml, for example physics: [structures], and can carry several values on an axis. Its project is an axis of its own.