AGIBOT open-sources 'Physical Reasoning' dataset to teach robots to decide, not just act
AGIBOT's fourth AGIBOT WORLD 2026 installment grades robot tasks by decision difficulty instead of manipulation difficulty — a dataset built for machines that must judge what to do, not just how to move.
AGIBOT has open-sourced the fourth installment of its AGIBOT WORLD 2026 dataset series, 'Physical Reasoning,' which the company describes as the first real-world dataset built specifically for physical agents. The premise: real tasks rarely have one correct motion sequence, so training data must capture how to decide — what to do first, what to check, and how to recover — rather than only how to execute.
The design pivot is deliberate. Where earlier datasets graded tasks by how hard they are to manipulate, this release grades them by how hard they are to reason about. Nine categories of logical-reasoning scenarios are covered, and each task is captured across multiple real-world states and execution branches. Tasks are organized around four factors — object quantity, occlusion relationships, location occupancy, and task progress — so the same goal under different starting conditions requires a different plan.
The examples are deliberately mundane. A robot told to restock a hotel minibar to three bottles must distinguish empty bottles from full ones, count existing stock, and then decide how many to remove and add. A robot searching drawers for toiletries must remember which drawers it already checked, even when the camera view looks identical. Those judgments — state assessment, step dependencies, process memory — are what the dataset is built to teach.
Annotations pair high-level question-answer labels with low-level action trajectories, linking task goals, decision steps, and real-machine execution. Nine reasoning capabilities are covered: attribute-constrained selection, situational conditional selection, operational feasibility assessment, precondition reasoning, goal-directed search, state and quantity reasoning, long-horizon planning, memory state maintenance, and subtask order reasoning. Tasks are graded Easy, Medium, and Hard, and both successful and failed executions are annotated, giving researchers the error samples needed to study recovery from mistakes.
AGIBOT said the dataset has already been used to train its upcoming agent framework and a new-generation foundation model, and that the longer-term target is robots that can sustain full service workflows — room cleaning, restocking, meal preparation — where every action depends on the last. The release extends the company's open-data strategy: earlier AGIBOT WORLD installments focused on imitation learning from teleoperated demonstrations; this one addresses the gap between knowing how to move and knowing what to do.