Game development loop

Start here0 tasks

Can an agent find out by playing what is wrong with a game, fix it at the right level, and confirm the fix the way a player would?

Agents change a game, or the engine under it, and check the result by playing it the way a player would.

What this project asks

Agents can already generate a playable game and keep improving it (RSIGame, Play2Code). What that loop does not measure is whether the agent can tell what is wrong by playing, fix it at the right level (the game's code, or the engine underneath), and confirm the fix by playing again. In RSIGame, eighteen autonomous rounds went by without noticing that an opponent never lost health; a person who played two matches found it. (When the game's clock matters to the task, for example reaction speed, timing windows, or telling a game defect from the tester's own delay, the task belongs to Real-time decisions, in the Real time area.)

What tasks can look like

Any task that tests the question fits. Some directions:

Repair or extend a running game when what is wrong only shows in play.
Diagnose a fault from play before reading the source, then repair it and show the repair in the final build.
Change the engine itself (physics, input, rendering, audio, build) when game scripts cannot meet the requirement.
Keep a game working across a series of changes, where a later fix can break an earlier one.

What a verifier here should establish

The game behaves as required in play, judged from its own state or a replay, not from screenshots alone.
Nothing that worked before broke, including other games when the engine changes.
Reported bugs reproduce, and a claimed cause is the actual one.

Worth reporting

Whether the agent played again after its last edit
Where it made the fix (game code or engine) against where the fault was
Agreement with people, where the task involves them

For example

In a brawler, hits land but the opponent's health bar never moves: the damage handler reads a field renamed two commits ago and silently does nothing. The game builds, and its tests, which only check that hits register, pass. An agent that re-reads its own diff calls the game done. One that plays a full match sees the bar stay full, traces it to the handler, fixes it, and plays again until the opponent goes down. The verifier reads the opponent's health from the game state over a replayed match.

Build a benchmark task for the CUA-frontier project "Game development loop" (game-dev-loop). Repository: https://github.com/kingofspace0wzz/CUA-frontier 1. Read AGENTS.md and areas/games/game-dev-loop/PROTOCOL.md: what the project asks. 2. Pick a task that tests that question. Start it with: python scripts/new_task.py game-dev-loop <task-id> 3. Write task.md, env/ and a verifier that fits this task in verifier/, and state in metadata.yaml (intent:) which capability it measures. Every criterion the verifier scores must be stated in task.md or the env; the facts it checks must be reachable only through the interaction the task tests (for computer use, the GUI): tools you offer, such as a shell, must not reach them another way. 4. Calibrate the verifier: a reference solution in oracle/ it accepts, plausible wrong ones in negatives/ it rejects, and a check: command that shows both; run it with python scripts/check_task.py. 5. Have a separate red-team agent, which has not seen oracle/, try to pass without doing the task. Prepare its workspace with python scripts/red_team.py tasks/game-dev-loop/<task-id> and give the prompt it prints to a subagent that does not see your conversation (in Codex, fork_turns: "none"). Never launch another agent CLI (codex exec, claude -p) for it. Fix what works (a guess that passes by chance may stay if its chance per trial is under 1%, shown in verifier/README.md), and record every attempt under red_team: in results/quality.yaml. 6. Fill in the rest of metadata.yaml, including generated_by, and run python scripts/validate.py tasks/game-dev-loop/<task-id> 7. Stop and report what you built, the shortcuts tried, and where your trajectory is saved. Do not run the frontier evaluation yourself.