Geppetto tracks 1035 robot training datasets including Open X-Embodiment, DROID, and AgiBot World, each with structured metadata on trajectory count, hours of demonstration, robot platforms, license, commercial use, access model, and file format. Every dataset connects to the robots it trains and the professions it affects.
| Dataset | Year | Category | Trajectories | Hours | Platforms | License | Commercial | Access | Format |
|---|---|---|---|---|---|---|---|---|---|
| OpenLane-V2 | 2023 | research-lab | — | — | Argoverse 2 and nuScenes multi-view images | not stated | No | open | — |
| OpenOccupancy | 2023 | research-lab | — | — | nuScenes | not stated | No | open | — |
| Robo3D | 2023 | research-lab | — | — | corrupted LiDAR benchmarks | not stated | No | open | — |
| RSRD | 2023 | research-lab | — | — | stereo and LiDAR road surface | not stated | No | open | — |
| SSCBench | 2023 | research-lab | — | — | KITTI-360, nuScenes, Waymo | not stated | No | open | — |
| SurroundOcc | 2023 | research-lab | — | — | nuScenes | not stated | No | open | — |
| V2V4Real | 2023 | research-lab | — | — | two vehicles with lidar and cameras | not stated | No | open | — |
| V2X-Seq | 2023 | research-lab | — | — | vehicle and roadside sensors, 28 intersections | not stated | No | gated | — |
| V2X-Traj | 2023 | research-lab | — | — | vehicle and infrastructure trajectories | not stated | No | gated | — |
| Wayve GAIA-1 training dataset | 2023 | research-lab | — | — | Wayve fleet vehicles, model not stated | not stated | No | open | — |
| WiDEVIEW | 2023 | research-lab | — | — | UWB and vision | not stated | No | open | — |
| WOMD-LiDAR | 2023 | research-lab | — | — | Waymo vehicle lidar | Waymo non-commercial licence | No | gated | — |
| Zenseact Open Dataset (ZOD) | 2023 | research-lab | — | — | Zenseact fleet, 8MP camera, lidars, GNSS/IMU | CC BY-SA 4.0 | Yes | gated | — |
| ARGO1M | 2023 | human-video | — | — | Ego4D videos | not stated | No | open | — |
| Aria Pilot Dataset | 2023 | human-video | — | — | Project Aria glasses | Project Aria dataset licence agreement | No | open | — |
| AssemblyHands | 2023 | human-video | — | — | Assembly101 footage | not stated | No | open | — |
| Ego4D-EASG (Egocentric Action Scene Graphs) | 2023 | human-video | — | — | Ego4D videos | not stated | No | open | — |
| EGOFALLS | 2023 | human-video | — | — | egocentric cameras with audio | not stated | No | open | — |
| EgoHumans | 2023 | human-video | — | — | wearable cameras in multi-view capture setup | not stated | No | open | — |
| EgoObjects | 2023 | human-video | — | — | 4 wearable devices (not named in abstract) | MIT (per third-party atlas) | Yes | open | — |
| EgoPlan-Bench / EgoPlan-Bench2 | 2023 | human-video | — | — | existing egocentric videos | not stated | No | open | — |
| EgoThink | 2023 | human-video | — | — | egocentric images from existing video | not stated | No | open | — |
| EPIC-Aff | 2023 | human-video | — | — | EPIC-KITCHENS videos | not stated | No | open | — |
| Epic-Sounding-Object | 2023 | human-video | — | — | EPIC-KITCHENS videos | not stated | No | open | — |
| MMG-Ego4D | 2023 | human-video | — | — | Ego4D videos | not stated | No | open | — |
| PACO | 2023 | human-video | — | — | LVIS images and Ego4D videos | not stated | No | open | — |
| Project Aria open datasets (AEA, ADT, ASE, DTC, AEO, Reading in the Wild) | 2023 | human-video | — | — | Project Aria glasses | not stated | No | open | — |
| SANPO | 2023 | human-video | — | — | stereo cameras worn by pedestrians, plus synthetic | not stated | No | open | — |
| Walking Tours | 2023 | human-video | — | — | first-person walking videos from the web | not stated | No | open | — |
| EgoWholeBody | 2023 | human-video | — | — | simulated head-mounted fisheye camera | not stated | No | open | — |
| POV-Surgery | 2023 | human-video | — | — | simulated head-mounted view | not stated | No | open | — |
| Grasp'D-1M (Fast-Grasp'D) | 2023 | research-lab | — | — | three robot hands, differentiable simulation | not stated | No | open | — |
| UniDexGrasp | 2023 | research-lab | — | — | dexterous hand, tabletop point cloud setting | not stated | No | open | — |
| AffordPose | 2023 | research-lab | — | — | human hand-object interactions with affordance labels | GPL-3.0 (repo) | No | open | — |
| LA-TaskGrasp (GraspGPT) | 2023 | research-lab | — | — | TaskGrasp with language augmentation | not stated | No | open | — |
| NBMOD | 2023 | research-lab | — | — | RGB-D images with oriented grasp boxes | see repo LICENSE | No | open | — |
| OCID-VLG | 2023 | research-lab | — | — | OCID scenes with referring text, masks and grasps | not stated | No | open | — |
| RoboRefIt (VL-Grasp) | 2023 | research-lab | — | — | RGB-D indoor clutter with referring expressions | not stated | No | open | — |
| WIT-UAS (Wildland-fire Infrared Thermal UAS Dataset) | 2023 | real-robot | — | — | drone | custom | No | open | — |
| Ego-Exo4D | 2023 | human-video | — | 1,286.3 | human | custom | No | open | — |
| MimicGen | 2023 | synthetic | — | — | franka_panda, kuka_iiwa, sawyer | CC-BY-4.0 | Yes | open | — |
| RoboSet (RoboAgent) | 2023 | real-robot | — | — | franka_panda | MIT | Yes | open | — |
| RH20T | 2023 | human-video | — | — | franka_panda, kuka_iiwa, multi | custom | No | open | — |
| LIBERO | 2023 | synthetic | — | 13.99 | franka_panda | CC-BY-4.0 | Yes | open | — |
| FurnitureBench | 2023 | real-robot | — | 219.6 | franka_panda | MIT | Yes | open | — |
| ALOHA Static | 2023 | real-robot | — | 2.5 | aloha | MIT | Yes | open | — |
| ManiSkill2 | 2023 | synthetic | — | — | franka_panda | custom | No | open | — |
| DriveLM | 2023 | research-lab | — | — | nuScenes and CARLA | CC BY-NC-SA 4.0 | No | gated | — |
| FEDORA | 2023 | research-lab | — | — | flying event camera | not stated | No | open | — |
| Multi-LiDAR Multi-UAV dataset | 2023 | research-lab | — | — | multiple LiDARs tracking multiple UAVs | not stated | No | open | — |
| MUN-FRL | 2023 | research-lab | — | — | DJI M600 hexacopter and Bell 412 helicopter | not stated | No | open | — |
| GraspNeRF dataset | 2023 | research-lab | — | — | synthetic multi-view cluttered tabletop scenes | not stated | No | open | — |
| LangSHAPE | 2023 | research-lab | — | — | simulated shapes with part-level language and grasps | not stated | No | open | — |
| Sim-MEES | 2023 | research-lab | — | — | mobile manipulator, parallel jaw and suction | not stated | No | open | — |
| Sim-Suction-Dataset | 2023 | research-lab | — | — | synthetic clutter, suction gripper | not stated | No | open | — |
| SLOPER4D | 2023 | research-lab | — | — | human body, head-mounted LiDAR and camera, urban scenes | not stated | No | open | — |
| Hi4D | 2023 | research-lab | — | — | pairs of humans, 4D textured scans | not stated | No | open | — |
| IMHD (I'M HOI) | 2023 | research-lab | — | — | human plus object with object-mounted IMU, RGB camera | not stated | No | open | — |
| MoYo | 2023 | research-lab | — | — | human body yoga poses with pressure and centre of mass | not stated | No | open | — |
| OMOMO (FullBodyManipulation) | 2023 | research-lab | — | — | human full body manipulating large objects | not stated | No | open | — |
| ReMoCap (ReMoS) | 2023 | research-lab | — | — | two interacting humans | not stated | No | open | — |
| HOH (Human-Object-Human Handover) | 2023 | research-lab | — | — | markerless multimodal handovers | not stated | No | open | — |
| PHC processed AMASS | 2023 | research-lab | — | — | SMPL humanoid in Isaac Gym | follows upstream licences | No | open | — |
| YouTube-VLN | 2023 | research-lab | — | — | house tour YouTube videos | not stated | No | open | — |
| RT-Trajectory hindsight trajectory sketches | 2023 | research-lab | — | — | not stated in abstract | not stated | No | open | — |
| TidyBot benchmark | 2023 | research-lab | — | — | household tidying preferences | not stated | No | open | — |
| CityRefer | 2023 | research-lab | — | — | city-scale point clouds | not stated | No | open | — |
| EmbodiedScan | 2023 | research-lab | — | — | ego-centric RGB-D scans | not stated | No | open | — |
| PhysObjects | 2023 | research-lab | — | — | images of common household objects | not stated | No | open | — |
| Driving with LLMs dataset | 2023 | research-lab | — | — | 2D driving simulator with RL agent | not stated | No | open | — |
| REFLECT / RoboFail | 2023 | research-lab | — | — | simulated and real robot failure executions | not stated | No | open | — |
| LingoQA | 2023 | research-lab | — | — | Wayve vehicle front camera | not stated | No | open | — |
| NuPrompt | 2023 | research-lab | — | — | nuScenes | not stated | No | open | — |
| NuScenes-MQA | 2023 | research-lab | — | — | nuScenes | not stated | No | open | — |
| NuScenes-QA | 2023 | research-lab | — | — | nuScenes | not stated | No | open | — |
| Rank2Tell | 2023 | research-lab | — | — | ego vehicle cameras and lidar | research only | No | open | — |
| Reason2Drive | 2023 | research-lab | — | — | nuScenes, Waymo, ONCE | not stated | No | open | — |
| Refer-KITTI | 2023 | research-lab | — | — | KITTI tracking videos | not stated | No | open | — |
| Talk2BEV-Bench | 2023 | research-lab | — | — | nuScenes BEV maps | not stated | No | open | — |
| M3ED | 2023 | research-lab | — | — | multi-robot event camera rig including legged platform | not stated | No | open | — |
| Real-world quadrupedal locomotion offline RL benchmark | 2023 | research-lab | — | — | quadruped robot (real) | not stated | No | open | — |
| AnimalML3D (OmniMotionGPT) | 2023 | research-lab | — | — | 36 animal identities, SMAL-style motion | not stated | No | open | — |
| NTU4DRadLM | 2023 | research-lab | — | — | robot and vehicle platforms with 4D radar | not stated | No | open | — |
| SPOC / CHORES training data | 2023 | research-lab | — | — | Stretch RE-1 in AI2-THOR | not stated | No | open | — |
| 360Loc | 2023 | research-lab | — | — | 360 degree camera with lidar | not stated | No | open | — |
| NUFR-M3F (multi-modal multi-floor SLAM dataset) | 2023 | research-lab | — | — | seven cameras, IMU, ZED stereo, 128-channel lidar | not stated | No | open | — |
| STAR-loc | 2023 | research-lab | — | — | stereo camera and UWB range | not stated | No | open | — |
| TBD Pedestrian Dataset | 2023 | research-lab | — | — | portable multi-camera pedestrian capture rig | not stated | No | open | — |
| FROG | 2023 | research-lab | — | — | knee-high 2D laser range finder | not stated | No | open | — |
| Ground-Challenge | 2023 | research-lab | — | — | ground robot | not stated | No | open | — |
| Hazards&Robots | 2023 | research-lab | — | — | mobile robot visual anomaly detection | cc-by-4.0 | Yes | open | — |
| Kimera-Multi datasets | 2023 | research-lab | — | — | up to 8 ground robots | not stated | No | open | — |
| RoAM | 2023 | research-lab | — | — | custom turtlebot | not stated | No | open | — |
| RTS-GT | 2023 | research-lab | — | — | robotic total stations | not stated | No | open | — |
| SiT (Socially Interactive Trajectory) dataset | 2023 | research-lab | — | — | not stated | not stated | No | open | — |
| SUES-200 | 2022 | research-lab | — | — | drone at four heights plus satellite | not stated | No | open | — |
| VPAIR | 2022 | research-lab | — | — | light aircraft with downward camera | not stated | No | open | — |
| A9 / TUMTraf Intersection | 2022 | research-lab | — | — | roadside cameras and lidars on A9 test bed | registration required | No | gated | — |
| aiMotive Dataset | 2022 | research-lab | — | — | car with lidar, cameras, radars, 360 view | CC BY-NC-SA 4.0 | No | open | — |
| B3D (Berkeley DeepDrive Drone) | 2022 | research-lab | — | — | drone-recorded traffic | not stated | No | open | — |