Area · 1 project
Workflows
Does the agent get the process right, not just the final state?
Business tasks are spread across several systems, and a support ticket can read "resolved" after a refund went out twice. Projects here inject the faults real operations see, such as timeouts with an unknown outcome and duplicate webhooks, and check invariants and the order of operations as well as the end state.
Projects
Each project is a direction with room for many kinds of task. Its page says what it asks and what a verifier there should establish.
Cross-system workflows
Training environmentBusiness tasks spread across several systems, where a side effect can fail or time out and a correct-looking final state can hide a wrong process.
After a side effect with an unknown outcome, does the agent check before it acts again, and keep every system consistent?
0 tasksOpen the project →Propose a project
Have a question in Workflows that current benchmarks miss? Propose it as a project: the question, what tasks could look like, and how they could be verified.
opens a GitHub issuePropose →Tasks in this area
Claims, open pull requests and merged tasks across its projects, from the board.
Proposed projects
Open proposals for this area. Comment on the issue to support or shape one.