RoboArena

10,783 autonomous policy rollouts on real DROID Franka setups across institutions, each paired with double-blind pairwise human preference ratings — an evaluation corpus, not a demonstration corpus.

RoboArena is a living, distributed real-robot benchmark: Chatbot Arena, but the contestants are robot policies and the arena is a network of physical Franka arms. Crucially, every episode is an autonomous policy rollout, not a teleoperated demonstration — a human is present only to reset the scene and score the result.

• 10,783 policy episodes across 3,883 evaluation sessions in the latest snapshot (21.7 GB, 27,148 MP4 videos), up from 4,613 episodes in the first public dump.
• Every site runs the DROID platform: a Franka Panda 7-DoF arm with a Robotiq 2F-85 gripper, a ZED-mini stereo wrist camera, and ZED 2 stereo shoulder cameras (left and right) on a mobile height-adjustable table.
• Evaluators at 7+ academic institutions — Berkeley, Stanford, UW, Montréal, NVIDIA, Penn, UT Austin, Yonsei — each run pairs of competing policies on tasks of their own choosing, then rate which did better. There is no fixed task list and no central lab.
• 21 policies appear in the latest index, including pi0, pi0-fast, pi0.5, several PaliGemma variants, MolmoAct2-DROID and Cosmos3-Nano-Policy, spanning both joint-velocity and joint-position action spaces.
• Each session records a natural-language task instruction, a binary A/B/TIE preference, a 0–100 continuous partial-success score per policy, a binary success flag, episode duration, and free-form evaluator feedback.
• Released as periodic DataDump snapshots on Hugging Face (Aug 2025, Feb 2026, Jul 2026), so the corpus grows over time rather than freezing.
• Published at CoRL 2025. Because it captures how policies actually fail in the wild, it is useful well beyond leaderboards — for failure analysis, reward modelling, and preference learning.