Geppetto tracks 1035 robot training datasets including Open X-Embodiment, DROID, and AgiBot World, each with structured metadata on trajectory count, hours of demonstration, robot platforms, license, commercial use, access model, and file format. Every dataset connects to the robots it trains and the professions it affects.

DatasetYearCategoryTrajectoriesHoursPlatformsLicenseCommercialAccessFormat
OpenLane-V22023research-lab——Argoverse 2 and nuScenes multi-view imagesnot statedNoopen—
OpenOccupancy2023research-lab——nuScenesnot statedNoopen—
Robo3D2023research-lab——corrupted LiDAR benchmarksnot statedNoopen—
RSRD2023research-lab——stereo and LiDAR road surfacenot statedNoopen—
SSCBench2023research-lab——KITTI-360, nuScenes, Waymonot statedNoopen—
SurroundOcc2023research-lab——nuScenesnot statedNoopen—
V2V4Real2023research-lab——two vehicles with lidar and camerasnot statedNoopen—
V2X-Seq2023research-lab——vehicle and roadside sensors, 28 intersectionsnot statedNogated—
V2X-Traj2023research-lab——vehicle and infrastructure trajectoriesnot statedNogated—
Wayve GAIA-1 training dataset2023research-lab——Wayve fleet vehicles, model not statednot statedNoopen—
WiDEVIEW2023research-lab——UWB and visionnot statedNoopen—
WOMD-LiDAR2023research-lab——Waymo vehicle lidarWaymo non-commercial licenceNogated—
Zenseact Open Dataset (ZOD)2023research-lab——Zenseact fleet, 8MP camera, lidars, GNSS/IMUCC BY-SA 4.0Yesgated—
ARGO1M2023human-video——Ego4D videosnot statedNoopen—
Aria Pilot Dataset2023human-video——Project Aria glassesProject Aria dataset licence agreementNoopen—
AssemblyHands2023human-video——Assembly101 footagenot statedNoopen—
Ego4D-EASG (Egocentric Action Scene Graphs)2023human-video——Ego4D videosnot statedNoopen—
EGOFALLS2023human-video——egocentric cameras with audionot statedNoopen—
EgoHumans2023human-video——wearable cameras in multi-view capture setupnot statedNoopen—
EgoObjects2023human-video——4 wearable devices (not named in abstract)MIT (per third-party atlas)Yesopen—
EgoPlan-Bench / EgoPlan-Bench22023human-video——existing egocentric videosnot statedNoopen—
EgoThink2023human-video——egocentric images from existing videonot statedNoopen—
EPIC-Aff2023human-video——EPIC-KITCHENS videosnot statedNoopen—
Epic-Sounding-Object2023human-video——EPIC-KITCHENS videosnot statedNoopen—
MMG-Ego4D2023human-video——Ego4D videosnot statedNoopen—
PACO2023human-video——LVIS images and Ego4D videosnot statedNoopen—
Project Aria open datasets (AEA, ADT, ASE, DTC, AEO, Reading in the Wild)2023human-video——Project Aria glassesnot statedNoopen—
SANPO2023human-video——stereo cameras worn by pedestrians, plus syntheticnot statedNoopen—
Walking Tours2023human-video——first-person walking videos from the webnot statedNoopen—
EgoWholeBody2023human-video——simulated head-mounted fisheye cameranot statedNoopen—
POV-Surgery2023human-video——simulated head-mounted viewnot statedNoopen—
Grasp'D-1M (Fast-Grasp'D)2023research-lab——three robot hands, differentiable simulationnot statedNoopen—
UniDexGrasp2023research-lab——dexterous hand, tabletop point cloud settingnot statedNoopen—
AffordPose2023research-lab——human hand-object interactions with affordance labelsGPL-3.0 (repo)Noopen—
LA-TaskGrasp (GraspGPT)2023research-lab——TaskGrasp with language augmentationnot statedNoopen—
NBMOD2023research-lab——RGB-D images with oriented grasp boxessee repo LICENSENoopen—
OCID-VLG2023research-lab——OCID scenes with referring text, masks and graspsnot statedNoopen—
RoboRefIt (VL-Grasp)2023research-lab——RGB-D indoor clutter with referring expressionsnot statedNoopen—
WIT-UAS (Wildland-fire Infrared Thermal UAS Dataset)2023real-robot——dronecustomNoopen—
Ego-Exo4D2023human-video—1,286.3humancustomNoopen—
MimicGen2023synthetic——franka_panda, kuka_iiwa, sawyerCC-BY-4.0Yesopen—
RoboSet (RoboAgent)2023real-robot——franka_pandaMITYesopen—
RH20T2023human-video——franka_panda, kuka_iiwa, multicustomNoopen—
LIBERO2023synthetic—13.99franka_pandaCC-BY-4.0Yesopen—
FurnitureBench2023real-robot—219.6franka_pandaMITYesopen—
ALOHA Static2023real-robot—2.5alohaMITYesopen—
ManiSkill22023synthetic——franka_pandacustomNoopen—
DriveLM2023research-lab——nuScenes and CARLACC BY-NC-SA 4.0Nogated—
FEDORA2023research-lab——flying event cameranot statedNoopen—
Multi-LiDAR Multi-UAV dataset2023research-lab——multiple LiDARs tracking multiple UAVsnot statedNoopen—
MUN-FRL2023research-lab——DJI M600 hexacopter and Bell 412 helicopternot statedNoopen—
GraspNeRF dataset2023research-lab——synthetic multi-view cluttered tabletop scenesnot statedNoopen—
LangSHAPE2023research-lab——simulated shapes with part-level language and graspsnot statedNoopen—
Sim-MEES2023research-lab——mobile manipulator, parallel jaw and suctionnot statedNoopen—
Sim-Suction-Dataset2023research-lab——synthetic clutter, suction grippernot statedNoopen—
SLOPER4D2023research-lab——human body, head-mounted LiDAR and camera, urban scenesnot statedNoopen—
Hi4D2023research-lab——pairs of humans, 4D textured scansnot statedNoopen—
IMHD (I'M HOI)2023research-lab——human plus object with object-mounted IMU, RGB cameranot statedNoopen—
MoYo2023research-lab——human body yoga poses with pressure and centre of massnot statedNoopen—
OMOMO (FullBodyManipulation)2023research-lab——human full body manipulating large objectsnot statedNoopen—
ReMoCap (ReMoS)2023research-lab——two interacting humansnot statedNoopen—
HOH (Human-Object-Human Handover)2023research-lab——markerless multimodal handoversnot statedNoopen—
PHC processed AMASS2023research-lab——SMPL humanoid in Isaac Gymfollows upstream licencesNoopen—
YouTube-VLN2023research-lab——house tour YouTube videosnot statedNoopen—
RT-Trajectory hindsight trajectory sketches2023research-lab——not stated in abstractnot statedNoopen—
TidyBot benchmark2023research-lab——household tidying preferencesnot statedNoopen—
CityRefer2023research-lab——city-scale point cloudsnot statedNoopen—
EmbodiedScan2023research-lab——ego-centric RGB-D scansnot statedNoopen—
PhysObjects2023research-lab——images of common household objectsnot statedNoopen—
Driving with LLMs dataset2023research-lab——2D driving simulator with RL agentnot statedNoopen—
REFLECT / RoboFail2023research-lab——simulated and real robot failure executionsnot statedNoopen—
LingoQA2023research-lab——Wayve vehicle front cameranot statedNoopen—
NuPrompt2023research-lab——nuScenesnot statedNoopen—
NuScenes-MQA2023research-lab——nuScenesnot statedNoopen—
NuScenes-QA2023research-lab——nuScenesnot statedNoopen—
Rank2Tell2023research-lab——ego vehicle cameras and lidarresearch onlyNoopen—
Reason2Drive2023research-lab——nuScenes, Waymo, ONCEnot statedNoopen—
Refer-KITTI2023research-lab——KITTI tracking videosnot statedNoopen—
Talk2BEV-Bench2023research-lab——nuScenes BEV mapsnot statedNoopen—
M3ED2023research-lab——multi-robot event camera rig including legged platformnot statedNoopen—
Real-world quadrupedal locomotion offline RL benchmark2023research-lab——quadruped robot (real)not statedNoopen—
AnimalML3D (OmniMotionGPT)2023research-lab——36 animal identities, SMAL-style motionnot statedNoopen—
NTU4DRadLM2023research-lab——robot and vehicle platforms with 4D radarnot statedNoopen—
SPOC / CHORES training data2023research-lab——Stretch RE-1 in AI2-THORnot statedNoopen—
360Loc2023research-lab——360 degree camera with lidarnot statedNoopen—
NUFR-M3F (multi-modal multi-floor SLAM dataset)2023research-lab——seven cameras, IMU, ZED stereo, 128-channel lidarnot statedNoopen—
STAR-loc2023research-lab——stereo camera and UWB rangenot statedNoopen—
TBD Pedestrian Dataset2023research-lab——portable multi-camera pedestrian capture rignot statedNoopen—
FROG2023research-lab——knee-high 2D laser range findernot statedNoopen—
Ground-Challenge2023research-lab——ground robotnot statedNoopen—
Hazards&Robots2023research-lab——mobile robot visual anomaly detectioncc-by-4.0Yesopen—
Kimera-Multi datasets2023research-lab——up to 8 ground robotsnot statedNoopen—
RoAM2023research-lab——custom turtlebotnot statedNoopen—
RTS-GT2023research-lab——robotic total stationsnot statedNoopen—
SiT (Socially Interactive Trajectory) dataset2023research-lab——not statednot statedNoopen—
SUES-2002022research-lab——drone at four heights plus satellitenot statedNoopen—
VPAIR2022research-lab——light aircraft with downward cameranot statedNoopen—
A9 / TUMTraf Intersection2022research-lab——roadside cameras and lidars on A9 test bedregistration requiredNogated—
aiMotive Dataset2022research-lab——car with lidar, cameras, radars, 360 viewCC BY-NC-SA 4.0Noopen—
B3D (Berkeley DeepDrive Drone)2022research-lab——drone-recorded trafficnot statedNoopen—