Geppetto tracks 558 robot training datasets including Open X-Embodiment, DROID, and AgiBot World, each with structured metadata on trajectory count, hours of demonstration, robot platforms, license, commercial use, access model, and file format. Every dataset connects to the robots it trains and the professions it affects.
| Dataset | Year | Category | Trajectories | Hours | Platforms | License | Commercial | Access | Format |
|---|---|---|---|---|---|---|---|---|---|
| MolmoAct Dataset | 2026 | real-robot | — | 22 | franka_panda | Apache-2.0 | Yes | open | — |
| EgoLive | 2026 | human-video | — | 1,680 | — | open-source | Yes | open | — |
| A2RL Vmax | 2026 | research-lab | — | — | A2RL race cars at Yas Marina | not stated | No | open | — |
| EgoTac corpus | 2026 | human-video | — | — | egocentric images with tactile labels | not stated | No | open | — |
| ManipArena | 2026 | real-robot | — | 188 | parallel_jaw, aloha | Apache-2.0 | Yes | open | — |
| SinD 2.0 | 2026 | research-lab | — | — | UAV-recorded signalized intersections | not stated | No | open | — |
| RetailSMV | 2026 | human-video | — | — | synchronised staff-view ego and exo cameras | not stated | No | open | — |
| EgoSound | 2026 | human-video | — | — | Ego4D and EgoBlind videos | not stated | No | open | — |
| FRED (Flooded Road Environments) | 2026 | research-lab | — | — | FLIR Blackfly camera, Ouster OS1-64, RTK IMU | not stated | No | open | — |
| SAW-Bench | 2026 | human-video | — | — | Ray-Ban Meta (Gen 2) smart glasses | CC BY-NC 4.0 (per atlas) | No | open | — |
| SG-Ego | 2026 | human-video | — | — | Ego4D videos | not stated | No | open | — |
| HA-2.2M | 2026 | human-video | — | — | Heterogeneous human videos, hand-centric | not stated | No | open | — |
| CAIP pretraining video | 2026 | human-video | — | — | egocentric human video with extracted 3D hand keypoints | not stated | No | open | — |
| DGS-Net multi-object dexterous grasp dataset | 2026 | research-lab | — | — | dexterous hand in multi-object scenes | not stated | No | open | — |
| Malaga Stereo and Laser Urban Data Set | 2026 | research-lab | — | — | stereo camera and lasers on car | not stated | No | open | — |
| SearchAD | 2026 | research-lab | — | — | frames from 11 driving datasets | not stated | No | open | — |
| IEDD (Interactive Enhanced Driving Dataset) | 2026 | research-lab | — | — | ego-centric interaction segments from driving logs | not stated | No | open | — |
| Industrial Embodied AI Robot Training Dataset 2.0 | 2026 | real-robot | — | — | Estun, Hisense, ZTE | Agreement-required | No | agreement-required | — |
| WT++ | 2026 | human-video | — | — | handheld or body camera on urban walking tours | not stated | No | open | — |
| REAL-I Industrial Humanoid Dataset | 2026 | real-robot | 180,000 | 60,000 | Kuavo 4 Pro, Kuavo 5, Kuavo | Apache-2.0 | Yes | open | — |
| EPIC-Contact | 2026 | human-video | — | — | EPIC-KITCHENS videos | CC BY-NC 4.0 (per atlas) | No | open | — |
| SPARC (Spatial Annotations from Robot Demonstrations at Scale) | 2026 | research-lab | 1,700 | — | Franka, bimanual setups, 12 diverse embodiments | null | No | open | — |
| NVIDIA OmniDreams driving training corpus | 2026 | research-lab | — | — | not stated | not stated | No | open | — |
| BranchDrive | 2026 | research-lab | — | — | driving, action-conditioned intervention branches | not stated | No | open | — |
| RESOLVE | 2026 | research-lab | — | — | roadside multi-resolution cameras and lidar | not stated | No | open | — |
| EgoMemReason | 2026 | human-video | — | — | EgoLife videos | CC BY-NC 4.0 (per atlas) | No | open | — |
| EgoPlay | 2026 | human-video | — | — | mainly Ego4D clips | not stated | No | open | — |
| HT-Bench | 2026 | human-video | — | — | egocentric camera with full-hand tactile sensing | not stated | No | open | — |
| EgoSPT | 2026 | human-video | — | — | egocentric camera, fork pick-and-place | not stated | No | open | — |
| HABIT (Human Aware Behavior and Interaction Training dataset) | 2026 | real-robot | — | 164 | franka_panda | CC-BY-4.0 | Yes | open | — |
| TactiDex | 2026 | human-video | — | — | Human hand with whole-hand tactile sensing; viewpoint not stated in abstract | not stated | No | open | — |
| EgoStream | 2026 | human-video | — | — | several egocentric video sources | not stated | No | open | — |
| KAIST Multispectral Pedestrian Dataset | 2026 | research-lab | — | — | RGB and thermal cameras | not stated | No | open | — |
| LRDDv3 | 2026 | research-lab | — | — | long-range RGB and thermal images of drones | not stated | No | open | — |
| RoboDojo | 2026 | research-lab | — | 38.57 | — | Apache-2.0 (per datasets.bot) | Yes | open | — |
| TuSimple lane and velocity benchmark | 2026 | research-lab | — | — | highway camera | not stated | No | open | — |
| 1X World Model egocentric human video set (1XWM) | 2026 | human-video | — | — | not stated | not stated | No | open | — |
| Transitional Autonomous Vehicle Lane-Changing Dataset | 2026 | research-lab | — | — | transitional autonomous vehicle | not stated | No | open | — |
| OVO-S-Bench | 2026 | human-video | — | — | nine egocentric video sources | not stated | No | open | — |
| Egocentric-100K — Manual Labor Egocentric Video Dataset | 2026 | human-video | — | 100,405 | human (head-mounted fisheye camera) | Apache 2.0 | Yes | gated | mp4, json |
| EgoScale | 2026 | human-video | — | — | egocentric human video | not stated | No | open | — |
| PhysicalAI Robotics GR00T X-Embodiment Sim | 2026 | synthetic | — | — | Franka Panda, Isaac Lab robots, humanoid robots | CC BY 4.0 | Yes | open | parquet, zarr |
| UAV Active Object Detection in the Wild dataset | 2026 | research-lab | — | — | UAV panoramic imagery | not stated | No | open | — |
| EgoEngine | 2026 | research-lab | — | — | RB-Y1 Humanoid Robot | MIT | Yes | open | — |
| EgoExoMem | 2026 | human-video | — | — | synchronised ego-exo videos | not stated | No | open | — |
| Minerva-Ego | 2026 | human-video | — | — | HD-EPIC videos | not stated | No | open | — |
| PalmDex — Embodiment-Agnostic Tactile Manipulation Dataset | 2026 | real-robot | — | — | human hands, dexterous robot hands | CC BY-SA 4.0 | Yes | open | mp4, hdf5, json |
| PanoAir | 2026 | research-lab | — | — | UAV with panoramic camera and IMU | not stated | No | open | — |
| EgoTL | 2026 | human-video | — | — | Egocentric capture with think-aloud speech | not stated | No | open | — |
| BATON | 2026 | research-lab | — | — | production vehicles with driving automation | not stated | No | open | — |
| ReaLiTy and LADS | 2026 | research-lab | — | — | LiDAR across sensors and adverse weather | not stated | No | open | — |
| ArmnetBench v0.1 Dataset | 2026 | real-robot | — | 26 | bimanual_so_101, so_101 | Apache-2.0 | Yes | open | — |
| RoboEdit-14M | 2026 | real-robot | 174,547 | 130.91 | various | arXiv.org perpetual non-exclusive license | No | open | — |
| DarkDriving | 2026 | research-lab | — | — | vehicle cameras on closed test field | not stated | No | open | — |
| Boreas Road Trip (Boreas-RT) | 2026 | research-lab | — | — | Boreas car plus Aeva Aeries II FMCW lidar, Doppler radar | not stated | No | open | — |
| DreamDojo GR-1 Post-Training | 2026 | human-video | — | 88.4 | humanoid_other | CC-BY-NC-4.0 | No | open | — |
| Humanity's Last Machine — Humanoid Robot Supplier Database | 2026 | supply-chain-intelligence | — | — | — | All rights reserved — data sourced from public stock exchange filings and company announcements. Attribution required. | No | open | csv |
| DRIVE-C | 2026 | research-lab | — | — | camera clips with 12 degradation types | not stated | No | open | — |
| SkyDrive | 2026 | research-lab | — | — | traffic monitoring drones | not stated | No | open | — |
| DrivIng | 2026 | research-lab | — | — | multimodal vehicle with geo-referenced digital twin | not stated | No | open | — |
| HiPHI (High-Precision Human Motion Dataset) | 2026 | human-video | — | 617.5 | — | ModalityNet Open Research License v1.0 | No | open | — |
| AgiBot World 2026 | 2026 | real-robot | — | 2,976.4 | AgiBot G2 | CC BY-NC-SA 4.0 | No | gated | parquet, mp4, json |
| DYAD | 2026 | human-video | — | — | HoloLens 2 | not stated | No | open | — |
| Xperience-10M — Large-Scale Egocentric Multimodal Human Experience Dataset | 2026 | human-video | — | 10,000 | human (wearable camera), egocentric rig | Apache 2.0 | Yes | gated | mp4, json, hdf5 |
| Dyna-2 egocentric human video pre-training corpus | 2026 | human-video | — | — | head-mounted first-person cameras | not stated | No | open | — |
| 4DLidarOpen | 2026 | research-lab | — | — | vehicle with forward 4D FMCW lidar, Beijing | not stated | No | open | — |
| EgoMAGIC | 2026 | human-video | — | — | head-mounted stereo camera with audio | CC BY 4.0 (per atlas) | Yes | open | — |
| UniHand-2.0 (Being-H0.5) | 2026 | human-video | — | — | human hands plus 30 robot embodiments | not stated | No | open | — |
| GCA-Bench | 2026 | research-lab | — | — | not stated in abstract | not stated | No | open | — |
| Multi-Vehicle Dataset with scanned 3D vehicle models (RTK-GNSS auto-annotation) | 2026 | research-lab | — | — | camera, LiDAR, radar vehicles | not stated | No | open | — |
| EgoCoT-Bench | 2026 | human-video | — | — | five public egocentric sets | not stated | No | open | — |
| LeRobot Shirt-Folding Dataset | 2026 | real-robot | — | 131 | parallel_jaw | Apache-2.0 | Yes | open | — |
| Universal Navigation Interface (UNI) | 2026 | human-video | — | — | four-wheeled rollator walker with smartphone, pushed by a person | not stated | No | open | — |
| AirGlove | 2026 | human-video | — | — | Egocentric cameras viewing gloved hands | not stated | No | open | — |
| LESS (Local Encoder for Spatial Sensing) | 2026 | human-video | — | 800 | franka_panda | CC-BY-4.0 | Yes | open | — |
| EgoAVU | 2026 | human-video | — | — | existing egocentric videos | not stated | No | open | — |
| EgoSuite-Open100K | 2026 | human-video | — | 10,070 | human | custom | No | open | — |
| SWIFTraj | 2026 | research-lab | — | — | UAV-recorded freeway and urban trajectories | not stated | No | open | — |
| HIW-500: Humanoids In-the-Wild | 2026 | human-video | — | 500 | unitree_humanoid | unknown | No | open | — |
| GR1 Robot Teleoperation Dataset | 2026 | real-robot | — | — | Fourier GR-1, Fourier GR-1T1 | MIT | Yes | open | hdf5, mp4 |
| HiFi-UMI-2K | 2026 | human-video | — | 2,137 | human, umi_gripper | CC-BY-4.0 | Yes | open | — |
| OVPD (OnSite Virtual-Physical Dataset) | 2026 | research-lab | — | — | virtual-physical fusion AV testing | not stated | No | open | — |
| OmniAction — Multimodal Proactive Robot Manipulation Dataset | 2026 | real-robot | — | — | robot arms, humanoid arms | CC BY-NC 4.0 | No | open | mp4, parquet, json |
| Data Pyramid for Embodied Manipulation Survey Dataset | 2026 | research-lab | — | — | — | null | No | open | — |
| HumanNet | 2026 | human-video | — | — | mixed first-person and third-person human video | not stated | No | open | — |
| BiGS (BioTac Grasp Stability) | 2026 | research-lab | — | — | BioTac sensors on robot hand | not stated | No | open | — |
| Online TTM Dataset | 2026 | human-video | — | — | Ego4D social clips | not stated | No | open | — |
| MolmoBot | 2026 | synthetic | — | 5,704 | franka_panda | custom | No | open | — |
| EgoInfinity | 2026 | human-video | — | — | franka_panda, human, unitree_humanoid | custom | No | open | — |
| GraspIT | 2026 | research-lab | — | — | Franka Panda in NVIDIA Isaac Sim tabletop scenes | not stated | No | open | — |
| V2U4Real | 2026 | research-lab | — | — | vehicle plus UAV, lidar and cameras | not stated | No | open | — |
| ABC-130k | 2026 | real-robot | — | 3,598.2 | parallel_jaw, aloha | Apache-2.0 | Yes | open | — |
| RoboReward — Vision-Language Reward Dataset for Robotics | 2026 | real-robot | — | — | Open X-Embodiment robots, RoboArena robots | CC BY 4.0 | Yes | open | mp4, parquet, json |
| UAV3DCrop | 2026 | research-lab | — | — | UAV multi-angle RGB surveys | CC-BY-4.0 | Yes | open | — |
| EgoProceVQA | 2026 | human-video | — | — | egocentric procedural videos | not stated | No | open | — |
| Axis Sim Dataset V1 | 2026 | synthetic | 50,000 | — | Franka Research 3 | open-source | Yes | open | — |
| HandEdit | 2026 | human-video | — | — | Egocentric human hand clips mapped to 26 robot hand URDFs | not stated | No | open | — |
| SlugTrails | 2026 | human-video | — | — | Aria glasses | not stated | No | open | — |
| TouchMoment | 2026 | human-video | — | — | egocentric videos | not stated | No | open | — |
| VRUD | 2026 | research-lab | — | — | drone-recorded vehicle and VRU traffic | not stated | No | open | — |
| Egocentric-1M — The Internet for Physical AI | 2026 | human-video | — | 1,000,000 | human (head-mounted fisheye camera) | Apache 2.0 | Yes | gated | mp4, json, webdataset |
| MolmoAct2 Bimanual YAM Dataset | 2026 | real-robot | — | 720 | yam | Apache-2.0 | Yes | open | — |
| EgoFun3D | 2026 | human-video | — | — | egocentric videos with paired 3D geometry | not stated | No | open | — |
| GrandTour — Legged Robot Dataset in the Wild | 2026 | real-robot | — | — | ANYbotics ANYmal-D | MIT — confirm in repository LICENSE file before activation | Yes | gated | rosbag |
| MyEgo | 2026 | human-video | — | — | egocentric videos | not stated | No | open | — |
| MolmoAct2 SO-100/SO-101 Dataset | 2026 | real-robot | — | — | so_100, so_101 | Apache-2.0 | Yes | open | — |
| FTP-1 Dataset | 2026 | real-robot | — | 3,000 | multi | MIT | Yes | open | — |
| LingBot-VLA 2.0 Pretraining Dataset | 2026 | real-robot | — | 60,000 | Leju, AgiBot, Unitree | Apache-2.0 | Yes | open | — |
| eAP | 2026 | research-lab | — | — | event camera vehicle rig | not stated | No | open | — |
| PhysicalAI Robotics Kitchen Manipulation Demos | 2026 | real-robot | — | 600 | Franka Panda, Omron mobile base | CC BY 4.0 | Yes | open | hdf5, parquet, mp4 |
| EgoEVHands | 2026 | human-video | — | — | stereo event cameras, egocentric | not stated | No | open | — |
| EgoVid-Pro | 2026 | human-video | — | — | in-the-wild egocentric clips | not stated | No | open | — |
| Galaxea Open-World Dataset — Mobile Manipulation | 2026 | real-robot | — | 500 | Galaxea R1 dual-arm mobile robot | CC BY-NC-SA 4.0 | No | open | mp4, parquet, json |
| uniD | 2026 | research-lab | — | — | drone over RWTH campus shared space | not stated | No | open | — |
| EgoIllusion | 2025 | human-video | — | — | egocentric videos | not stated | No | open | — |
| Aria STDR | 2025 | human-video | — | — | Project Aria glasses | not stated | No | open | — |
| Aria Gen 2 Pilot Dataset | 2025 | human-video | — | — | Project Aria Gen 2 glasses | Project Aria dataset terms (per atlas) | No | open | — |
| EgoPrivacy | 2025 | human-video | — | — | existing egocentric videos | not stated | No | open | — |
| UniHand (Being-H0) | 2025 | human-video | — | — | mixed sources: motion capture, VR headsets and RGB-only videos | not stated | No | open | — |
| DexUMI | 2025 | human-video | — | — | wearable hand exoskeleton | not stated | No | open | — |
| MS2 (Multi-Spectral Stereo) | 2025 | research-lab | — | — | stereo RGB, NIR, thermal, LiDAR | not stated | No | open | — |
| EgoBlind | 2025 | human-video | — | — | cameras used by blind people | not stated | No | open | — |
| TeleEgo | 2025 | human-video | — | — | wearable camera with audio | not stated | No | open | — |
| Waymo internal driving dataset (Scaling Laws of Motion Forecasting and Planning) | 2025 | research-lab | — | — | not stated | not stated | No | open | — |
| RFUAV | 2025 | research-lab | — | — | USRP radio frequency recordings of 37 UAVs | not stated | No | open | — |
| TASTE-Rob | 2025 | human-video | — | — | fixed-view ego-centric camera | not stated | No | open | — |
| UrbanV2X | 2025 | research-lab | — | — | vehicle and infrastructure sensors, C-V2X | not stated | No | open | — |
| UrbanIng-V2X | 2025 | research-lab | — | — | multiple vehicles and infrastructure across intersections | not stated | No | open | — |
| EgoExo-Con | 2025 | human-video | — | — | synchronised ego-exo video pairs | not stated | No | open | — |
| UAV-Flow-Sim | 2025 | synthetic | — | — | drone | unknown | No | open | — |
| AGC-Drive | 2025 | research-lab | — | — | vehicles plus UAV | not stated | No | open | — |
| SpikeGrasp | 2025 | research-lab | — | — | stereo spike cameras | not stated | No | open | — |
| TAD-E2E | 2025 | research-lab | — | — | not stated on page opened | not stated | No | open | — |
| CognitiveDrone Dataset | 2025 | synthetic | — | — | drone | unknown | No | open | — |
| STU (Spotting the Unexpected) | 2025 | research-lab | — | — | lidar and camera | not stated | No | open | — |
| PhysicalAI Robotics GR00T Teleop G1 | 2025 | real-robot | — | — | Unitree G1 | CC BY 4.0 | Yes | open | parquet, hdf5, mp4 |
| EgoDex | 2025 | human-video | — | 829 | human | research-only | No | open | — |
| RealSource-World — Dual-Arm Humanoid Manipulation Dataset | 2025 | real-robot | — | — | RS-02 dual-arm humanoid | CC BY-NC-SA 4.0 — non-commercial use only | No | gated | parquet, mp4 |
| NVIDIA GR00T X-Embodiment Sim | 2025 | synthetic | — | — | franka_panda, humanoid_other, unitree_humanoid | CC-BY-4.0 | Yes | open | — |
| RoRaTrack | 2025 | research-lab | — | — | Dallara AV-21 camera | not stated | No | open | — |
| AIRoA Mobile Manipulator Data Engine | 2025 | real-robot | — | 10,000 | Toyota Human Support Robot (HSR) | CC BY-NC-SA 4.0 | No | gated | — |
| RISEE | 2025 | research-lab | — | — | naturalistic trajectories with risk ratings | not stated | No | open | — |
| LSC-ADL | 2025 | human-video | — | — | lifelog wearable camera images (LSC) | not stated | No | open | — |
| U2UData+ | 2025 | synthetic | — | 120 | drone | Apache-2.0 | Yes | open | — |
| AIRoA MoMa Dataset | 2025 | real-robot | — | 94 | — | custom | No | open | — |
| DexWild — In-the-Wild Dexterous Manipulation Dataset | 2025 | human-video | — | — | 6 DoF dexterous hand | MIT | Yes | open | imagefolder, parquet |
| Human2LocoMan — Loco-Manipulation Dataset | 2025 | real-robot | — | — | LocoMan | MIT | Yes | open | hdf5, mp4 |
| DexUMI data | 2025 | human-video | — | — | Human wearing hand exoskeleton matched to Inspire Hand and XHand | not stated | No | open | — |
| RoadSens-4M | 2025 | research-lab | — | — | smartphone sensors and camera | not stated | No | open | — |
| Odyssey | 2025 | research-lab | — | — | automotive LiDAR-inertial rig | not stated | No | open | — |
| AgiBot-Alpha — AgiBot World Dataset | 2025 | real-robot | — | 1,979 | AgiBot A2, AgiBot X1 | AgiBot World Open Academic License — commercial use with restrictions | Yes | open | zarr, mp4 |
| LIBERO-Plus | 2025 | synthetic | — | 31.08 | franka_panda | MIT | Yes | open | — |
| NavINST | 2025 | research-lab | — | — | multi-sensor road test vehicle | not stated | No | open | — |
| RoboArena | 2025 | real-robot | — | — | franka_panda, parallel_jaw | MIT | Yes | open | — |
| EOC-Bench | 2025 | human-video | — | — | Egocentric video | not stated | No | open | — |
| RASMD | 2025 | research-lab | — | — | RGB and SWIR cameras | not stated | No | open | — |
| ExAct | 2025 | human-video | — | — | skilled-activity ego and exo videos | not stated | No | open | — |
| EgoVerse — Egocentric Robot Training Dataset: 8.5M Frames, 74 Countries | 2025 | real-robot | — | 1,213 | aria_bimanual, aria_left_arm, aria_right_arm | CC BY-SA 4.0 | Yes | open | zarr, lerobot, hdf5 |
| UAV-Flow | 2025 | human-video | — | — | drone | unknown | No | open | — |
| Ego-HOIBench | 2025 | human-video | — | — | Egocentric images with hand-verb-object triplets | not stated | No | open | — |
| EMT | 2025 | research-lab | — | — | dash camera, UAE | not stated | No | open | — |
| UniFucGrasp | 2025 | research-lab | — | — | multiple dexterous hand types | not stated | No | open | — |
| EgoTextVQA | 2025 | human-video | — | — | ego-view videos, driving and house-keeping | not stated | No | open | — |
| EgoVLA training mix and Ego Humanoid Manipulation Benchmark | 2025 | human-video | — | — | egocentric human videos; sim benchmark uses Unitree H1 with Inspire hands | MIT (Hub EgoVLA/EgoVLA-Humanoid-Sim) | Yes | open | — |
| ScaleADFG | 2025 | research-lab | — | — | 3D assets with dexterous functional grasps | not stated | No | open | — |
| EgoRe-5M (EgoThinker) | 2025 | human-video | — | — | Egocentric video clips from existing sources | not stated | No | open | — |
| GraspFactory | 2025 | research-lab | — | — | Franka Panda and Robotiq 2F-85 grippers | custom licence file | No | open | — |
| EgoSocial | 2025 | human-video | — | — | Egocentric social interaction video | not stated | No | open | — |
| Learning Gentle Grasping dataset | 2025 | research-lab | — | — | multi-fingered hand with vision, sound and touch | not stated | No | open | — |
| EgoGazeVQA | 2025 | human-video | — | — | Egocentric daily-life video with gaze | not stated | No | open | — |
| EgoEvGesture | 2025 | human-video | — | — | egocentric event camera | not stated | No | open | — |
| Humanoid Everyday | 2025 | real-robot | — | 27.8 | unitree_humanoid | Apache-2.0 | Yes | open | — |
| CATS-V2V | 2025 | research-lab | — | — | two vehicles with lidar and multi-view cameras | not stated | No | open | — |
| PHUMA — Physically-Grounded Humanoid Locomotion Dataset | 2025 | human-video | — | — | Unitree G1, Unitree H1_2 | Apache 2.0 | Yes | open | npy |
| EgoToM | 2025 | human-video | — | — | Ego4D videos | not stated | No | open | — |
| EgoNormia | 2025 | human-video | — | — | egocentric videos of human interactions | not stated | No | open | — |
| OnSiteVRU | 2025 | research-lab | — | — | vulnerable road user trajectories | not stated | No | open | — |
| EvHand-FPV | 2025 | human-video | — | — | Single event camera, first-person view | not stated | No | open | — |
| AevaScenes | 2025 | research-lab | — | — | Aeva FMCW lidar vehicle | not stated | No | open | — |
| DG16M | 2025 | research-lab | — | — | dual-arm grasps | not stated | No | open | — |
| OXE-AugE | 2025 | synthetic | 4,440,000 | — | Franka Emika Panda, UR5e, xArm7 | CC-BY-4.0 | Yes | open | — |
| SimCoachCorpus | 2025 | research-lab | — | — | driving simulator, race track | not stated | No | open | — |
| Dexonomy | 2025 | research-lab | — | — | dexterous hands, 31 GRASP taxonomy types | not stated | No | open | — |
| EgoNight | 2025 | human-video | — | — | egocentric video, day and night pairs | not stated | No | open | — |
| EgoExoBench | 2025 | human-video | — | — | public ego-exo datasets | not stated | No | gated | — |
| REHEARSE-3D | 2025 | research-lab | — | — | LiDAR and radar in emulated rain | not stated | No | open | — |
| EgoTempo | 2025 | human-video | — | — | egocentric videos | not stated | No | open | — |
| EgoEdit (EgoEditData) | 2025 | human-video | — | — | egocentric videos | not stated | No | gated | — |
| R-LiViT | 2025 | research-lab | — | — | roadside lidar, RGB and thermal cameras | not stated | No | open | — |
| Dur360BEV | 2025 | research-lab | — | — | car with spherical camera, 128-channel lidar, RTK GNSS/INS | not stated | No | open | — |
| LeRobot Community Datasets v3 | 2025 | real-robot | — | — | SO-100 arm, SO-101 arm, Franka Panda | Apache 2.0 | Yes | open | parquet, mp4, lerobot |
| Bench-RNR | 2025 | research-lab | — | — | infrastructure LiDARs, repetitive and non-repetitive scanning | not stated | No | open | — |
| EMMA | 2025 | human-video | — | — | Project Aria glasses | not stated | No | open | — |
| V2X-ReaLO | 2025 | research-lab | — | — | V2X-Real vehicles and infrastructure, ROS bags | not stated | No | open | — |
| CoInfra | 2025 | research-lab | — | — | 8-node roadside lidar and camera system at roundabout | not stated | No | open | — |
| U2UData and U2UData+ | 2025 | research-lab | — | — | swarm of up to 15 UAVs | not stated | No | open | — |
| EgoCross | 2025 | human-video | — | — | egocentric video from surgery, industry, extreme sports, animal views | not stated | No | open | — |
| ECBench | 2025 | human-video | — | — | egocentric videos | not stated | No | open | — |
| GuideDog | 2025 | human-video | — | — | pedestrian first-person images | not stated | No | open | — |
| TD-TOG | 2025 | research-lab | — | — | real RGB-D scenes with affordances and planar grasps | not stated | No | open | — |
| DISC | 2025 | research-lab | — | — | simulated vehicle | not stated | No | open | — |
| Digital Twin Catalog (DTC) | 2025 | human-video | — | — | Project Aria glasses and DSLR | dataset licence agreement | No | open | — |
| DexUMI — Dexterous Universal Manipulation Interface | 2025 | human-video | — | — | Inspire Hand | MIT — confirm in dataset directory before activation | Yes | open | zarr |
| HumanPlus | 2024 | human-video | — | 40 | human, unitree_humanoid | unknown | No | open | — |
| Beacon | 2024 | research-lab | — | — | intersection video during blackouts | not stated | No | open | — |
| VidEgoThink | 2024 | human-video | — | — | Ego4D videos | not stated | No | open | — |
| SubT-MRS | 2024 | human-video | — | 2,000 | drone, multi, spot | CC-BY-4.0 | Yes | open | — |
| ARCap | 2024 | human-video | — | — | AR headset with mocap gloves | not stated | No | open | — |
| DexMimicGen | 2024 | real-robot | — | — | franka_panda, humanoid_other, parallel_jaw | CC-BY-NC-4.0 | No | open | — |
| DROID — A Large-Scale In-The-Wild Robot Manipulation Dataset | 2024 | real-robot | — | 350 | Franka Panda | Apache 2.0 | Yes | open | zarr, lerobot, hdf5 |
| V2XPnP Sequential Dataset | 2024 | research-lab | — | — | multiple vehicles and infrastructure | not stated | No | open | — |
| V2X-Real | 2024 | research-lab | — | — | 2 CAVs and 2 smart infrastructure units | not stated | No | open | — |
| Smart-Glasses Engagement dataset | 2024 | human-video | — | — | smart glasses | not stated | No | open | — |
| Ultra-AV | 2024 | research-lab | — | — | longitudinal trajectories of automated vehicles | not stated | No | open | — |
| TUMTraf V2X | 2024 | research-lab | — | — | roadside and vehicle cameras and lidars | not stated | No | gated | — |
| PARSE-Ego4D | 2024 | human-video | — | — | Ego4D videos | not stated | No | open | — |
| SID (Stereo Image Dataset) | 2024 | research-lab | — | — | ZED stereo camera on vehicle | not stated | No | open | — |
| Reasoning grasping benchmark | 2024 | research-lab | — | — | GraspNet-1Billion scenes with implicit instructions | not stated | No | open | — |
| SemanticSpray++ | 2024 | research-lab | — | — | camera, lidar, radar on test track | not stated | No | open | — |
| Ego4D-Sounds | 2024 | human-video | — | — | Ego4D videos | not stated | No | open | — |
| RSUD20K | 2024 | research-lab | — | — | driving-view cameras, Bangladesh | not stated | No | open | — |
| MM-Ego QA data / EgoMemoria | 2024 | human-video | — | — | Ego4D videos | not stated | No | open | — |
| ROAD-Waymo | 2024 | research-lab | — | — | Waymo Open videos | not stated | No | gated | — |
| ROADWork | 2024 | research-lab | — | — | vehicle and phone cameras in work zones | not stated | No | open | — |
| MH-VidQA (Multi-Hop EgoQA) | 2024 | human-video | — | — | Long-form egocentric video | not stated | No | open | — |
| TartanAviation | 2024 | real-robot | — | 3,374 | aircraft | CC-BY-4.0 | Yes | open | — |
| IKEA Ego 3D | 2024 | human-video | — | — | head-mounted device giving ego-view 3D point clouds | not stated | No | open | — |
| ParkingE2E | 2024 | research-lab | — | — | car with surround cameras in garages | not stated | No | open | — |
| HOI-QA (HOI-Ref) | 2024 | human-video | — | — | Egocentric images | not stated | No | open | — |
| HO-Cap | 2024 | human-video | — | — | HoloLens headset plus multiple RGB-D cameras | not stated | No | open | — |
| Open MARS Dataset | 2024 | research-lab | — | — | May Mobility fleet, lidar and surround cameras | not stated | No | open | — |
| HEADS-UP | 2024 | human-video | — | — | head-mounted cameras | not stated | No | open | — |
| OmniHD-Scenes | 2024 | research-lab | — | — | 6 cameras, 6 4D imaging radars, 128-beam lidar | not stated | No | open | — |
| Humanoid Atlas — Global OEM & Supplier Supply Chain Database | 2024 | supply-chain-intelligence | — | — | — | MIT | Yes | open | json, csv |
| NSAVP (Novel Sensors for Autonomous Vehicle Perception) | 2024 | research-lab | — | — | novel sensor suite on vehicle | not stated | No | open | — |
| NAVSIM / NAVSIM v2 | 2024 | research-lab | — | — | nuPlan and OpenScene sensor logs | not stated | No | open | — |
| MUSES | 2024 | research-lab | — | — | camera, lidar, radar, event camera, GNSS | not stated | No | gated | — |
| MSU-4S | 2024 | research-lab | — | — | MSU vehicle with FLIR cameras, lidar, radar | not stated | No | open | — |
| MIPD | 2024 | research-lab | — | — | camera, lidar, 4D radar, sound, vibration | not stated | No | open | — |
| MAN TruckScenes | 2024 | research-lab | — | — | MAN truck, 4 cameras, 6 lidars, 6 radars | not stated | No | open | — |
| EventEgo3D | 2024 | human-video | — | — | head-mounted fisheye event camera | not stated | No | open | — |
| Humanoid Robot Soccer fall prediction | 2024 | research-lab | — | 37.9 | — | apache-2.0 | Yes | open | — |
| InterHub | 2024 | research-lab | — | — | interaction events mined from public driving logs | not stated | No | open | — |
| CapGrasp (SemGrasp) | 2024 | research-lab | — | — | human hand grasps with language | not stated | No | open | — |
| IDD-X | 2024 | research-lab | — | — | multi-view dashcams, India | not stated | No | open | — |
| Claru — Training Data for Physical AI | 2024 | research-lab | — | — | robot arms, egocentric cameras, autonomous vehicles | Commercial — custom licensing per engagement | Yes | proprietary | mp4, json, parquet |
| DROID | 2024 | real-robot | — | 350 | franka_panda | CC-BY-4.0 | Yes | open | — |
| RoboMIND — Multi-Embodiment Intelligence Normative Data | 2024 | real-robot | — | — | Franka Emika Panda, Tien Kung humanoid, AgileX Cobot Magic V2.0 | Apache 2.0 | Yes | open | hdf5, mp4, json |
| HeLiMOS | 2024 | research-lab | — | — | heterogeneous LiDARs | not stated | No | open | — |
| EgoVid-5M | 2024 | human-video | — | — | egocentric video clips, 1080p | not stated | No | open | — |
| EvTTC | 2024 | research-lab | — | — | event camera | not stated | No | open | — |
| DrivingWorld private driving data | 2024 | research-lab | — | — | not stated | not stated | No | open | — |
| DIVA (SimGen) | 2024 | research-lab | — | — | web driving videos plus MetaDrive simulation | CC BY-NC-SA 4.0 for annotations | No | open | — |
| Mobile ALOHA | 2024 | real-robot | — | — | aloha | CC-BY-4.0 | Yes | open | — |
| Grasp'Em (MultiGrasp) | 2024 | research-lab | — | — | Shadow Hand multi-object grasps | not stated | No | open | — |
| CoSEC | 2024 | research-lab | — | — | coaxial stereo event cameras | not stated | No | open | — |
| EgoPoints | 2024 | human-video | — | — | egocentric sequences, EPIC Fields | not stated | No | open | — |
| FMB (Functional Manipulation Benchmark) | 2024 | real-robot | — | 31.3 | franka_panda | CC-BY-4.0 | Yes | open | — |
| Bosch Street Dataset | 2024 | research-lab | — | — | car with imaging radar, lidar, cameras, 360 coverage | not stated | No | open | — |
| TartanDrive 2.0 | 2024 | human-video | — | 7 | offroad_vehicle | MIT | Yes | open | — |
| EgoMimic | 2024 | human-video | — | — | Project Aria glasses (human) plus low-cost bimanual manipulator (robot) | not stated | No | open | — |
| BadODD | 2024 | research-lab | — | — | driving-view cameras, Bangladesh | not stated | No | open | — |
| MultiGripperGrasp | 2024 | research-lab | — | — | 11 grippers from two-finger to five-finger, incl. human hand | not stated | No | open | — |
| RoboCasa — Large-Scale Simulation for Everyday Household Tasks | 2024 | synthetic | — | 2,227 | Franka Panda (simulated) | MIT | Yes | open | json, mp4 |
| Get a Grip | 2024 | research-lab | — | — | multi-finger hand with RGB, point clouds and NeRFs | not stated | No | open | — |
| BODex | 2024 | research-lab | — | — | dexterous hands, bilevel optimization synthesis | not stated | No | open | — |
| Synthetic UAV Flight Trajectories | 2024 | synthetic | — | 21.28 | drone | Apache-2.0 | Yes | open | — |
| EgoCVR | 2024 | human-video | — | — | large-scale egocentric video sets | not stated | No | open | — |
| V2AIX | 2024 | research-lab | — | — | ETSI ITS V2X messages from vehicles and RSUs | not stated | No | open | — |
| TUMTraf Event | 2024 | research-lab | — | — | roadside event and RGB cameras | not stated | No | gated | — |
| RoScenes | 2024 | research-lab | — | — | roadside multi-view cameras on highways | not stated | No | gated | — |
| RCooper | 2024 | research-lab | — | — | roadside cameras and lidars | not stated | No | gated | — |
| Off-road racing datasets (Beyond the Mud) | 2024 | research-lab | — | — | trackside photos of off-road motorcycle races | not stated | No | open | — |
| TartanAir V2 | 2024 | synthetic | — | — | — | CC-BY-4.0 | Yes | open | — |
| CORP | 2024 | research-lab | — | — | 18 cameras and 9 lidars on campus | not stated | No | open | — |
| DexGYSNet (Grasp as You Say) | 2024 | research-lab | — | — | dexterous hand with language guidance | not stated | No | open | — |
| HoloVIC | 2024 | research-lab | — | — | holographic intersections plus vehicle | not stated | No | gated | — |
| EgoGen | 2024 | human-video | — | — | Virtual human with embodied camera | not stated | No | open | — |
| AD4RL | 2024 | research-lab | — | — | real human driving data plus simulator | not stated | No | open | — |
| MMAUD | 2024 | research-lab | — | — | multi-modal anti-UAV sensing | not stated | No | open | — |
| Ego4D-HCap | 2024 | human-video | — | — | Ego4D videos | not stated | No | open | — |
| Fourier ActionNet — Humanoid Manipulation Dataset | 2024 | real-robot | — | — | Fourier GR-2, Fourier GR-1 | Fourier Research License — non-commercial research use | No | gated | hdf5, lerobot |
| UMI (Universal Manipulation Interface) | 2024 | human-video | — | — | handheld gripper with wrist-view camera | not stated | No | open | — |
| UEVAVD | 2024 | research-lab | — | — | UAV-view active detection | not stated | No | open | — |
| EFM3D | 2024 | human-video | — | — | Project Aria glasses | not stated | No | open | — |
| UAV-VisLoc | 2024 | research-lab | — | — | drones plus satellite maps | not stated | No | open | — |
| Data Scaling Laws in Imitation Learning dataset | 2024 | human-video | — | — | UMI handheld gripper | not stated | No | open | — |
| Conversation Partners egocentric dataset | 2024 | human-video | — | — | not stated | not stated | No | open | — |
| Walking Tours | 2023 | human-video | — | — | first-person walking videos from the web | not stated | No | open | — |
| AssemblyHands | 2023 | human-video | — | — | Assembly101 footage | not stated | No | open | — |
| RoboSet — A Diverse Multi-Task Robot Learning Dataset | 2023 | real-robot | — | — | Franka Panda | CC BY 4.0 | Yes | open | hdf5, lerobot |
| Aria Pilot Dataset | 2023 | human-video | — | — | Project Aria glasses | Project Aria dataset licence agreement | No | open | — |
| ARGO1M | 2023 | human-video | — | — | Ego4D videos | not stated | No | open | — |
| ManiSkill2 | 2023 | synthetic | — | — | franka_panda | custom | No | open | — |
| Zenseact Open Dataset (ZOD) | 2023 | research-lab | — | — | Zenseact fleet, 8MP camera, lidars, GNSS/IMU | CC BY-SA 4.0 | Yes | gated | — |
| WOMD-LiDAR | 2023 | research-lab | — | — | Waymo vehicle lidar | Waymo non-commercial licence | No | gated | — |
| WiDEVIEW | 2023 | research-lab | — | — | UWB and vision | not stated | No | open | — |
| Wayve GAIA-1 training dataset | 2023 | research-lab | — | — | Wayve fleet vehicles, model not stated | not stated | No | open | — |
| ALOHA Static | 2023 | real-robot | — | 2.5 | aloha | MIT | Yes | open | — |
| V2X-Traj | 2023 | research-lab | — | — | vehicle and infrastructure trajectories | not stated | No | gated | — |
| V2X-Seq | 2023 | research-lab | — | — | vehicle and roadside sensors, 28 intersections | not stated | No | gated | — |
| V2V4Real | 2023 | research-lab | — | — | two vehicles with lidar and cameras | not stated | No | open | — |
| RflyMAD | 2023 | research-lab | — | — | multicopters, simulated and real | not stated | No | open | — |
| FurnitureBench | 2023 | real-robot | — | 219.6 | franka_panda | MIT | Yes | open | — |
| SANPO | 2023 | human-video | — | — | stereo cameras worn by pedestrians, plus synthetic | not stated | No | open | — |
| Project Aria open datasets (AEA, ADT, ASE, DTC, AEO, Reading in the Wild) | 2023 | human-video | — | — | Project Aria glasses | not stated | No | open | — |
| SurroundOcc | 2023 | research-lab | — | — | nuScenes | not stated | No | open | — |
| RoboRefIt (VL-Grasp) | 2023 | research-lab | — | — | RGB-D indoor clutter with referring expressions | not stated | No | open | — |
| SSCBench | 2023 | research-lab | — | — | KITTI-360, nuScenes, Waymo | not stated | No | open | — |
| LIBERO | 2023 | synthetic | — | 13.99 | franka_panda | CC-BY-4.0 | Yes | open | — |
| PACO | 2023 | human-video | — | — | LVIS images and Ego4D videos | not stated | No | open | — |
| OCID-VLG | 2023 | research-lab | — | — | OCID scenes with referring text, masks and grasps | not stated | No | open | — |
| RSRD | 2023 | research-lab | — | — | stereo and LiDAR road surface | not stated | No | open | — |
| MMG-Ego4D | 2023 | human-video | — | — | Ego4D videos | not stated | No | open | — |
| Robo3D | 2023 | research-lab | — | — | corrupted LiDAR benchmarks | not stated | No | open | — |
| NBMOD | 2023 | research-lab | — | — | RGB-D images with oriented grasp boxes | see repo LICENSE | No | open | — |
| LA-TaskGrasp (GraspGPT) | 2023 | research-lab | — | — | TaskGrasp with language augmentation | not stated | No | open | — |
| ALOHA — Action Chunking with Transformers | 2023 | real-robot | — | — | ALOHA bimanual robot, ViperX 300s, WidowX 250s | MIT | Yes | open | hdf5, lerobot |
| RH20T | 2023 | human-video | — | — | franka_panda, kuka_iiwa, multi | custom | No | open | — |
| BridgeData V2 | 2023 | real-robot | — | 111.1 | WidowX 250s | MIT | Yes | open | hdf5, zarr, mp4 |
| OpenOccupancy | 2023 | research-lab | — | — | nuScenes | not stated | No | open | — |
| OpenLane-V2 | 2023 | research-lab | — | — | Argoverse 2 and nuScenes multi-view images | not stated | No | open | — |
| RoboSet (RoboAgent) | 2023 | real-robot | — | — | franka_panda | MIT | Yes | open | — |
| Occ3D | 2023 | research-lab | — | — | Waymo and nuScenes | not stated | No | open | — |
| MimicGen | 2023 | synthetic | — | — | franka_panda, kuka_iiwa, sawyer | CC-BY-4.0 | Yes | open | — |
| Ego4D-EASG (Egocentric Action Scene Graphs) | 2023 | human-video | — | — | Ego4D videos | not stated | No | open | — |
| Ego-Exo4D | 2023 | human-video | — | 1,286.3 | human | custom | No | open | — |
| Epic-Sounding-Object | 2023 | human-video | — | — | EPIC-KITCHENS videos | not stated | No | open | — |
| EPIC-Aff | 2023 | human-video | — | — | EPIC-KITCHENS videos | not stated | No | open | — |
| IDD-AW | 2023 | research-lab | — | — | RGB and NIR camera pairs | not stated | No | gated | — |
| AffordPose | 2023 | research-lab | — | — | human hand-object interactions with affordance labels | GPL-3.0 (repo) | No | open | — |
| LIBERO — Lifelong Learning for Robot Manipulation | 2023 | synthetic | — | — | Franka Panda (simulated) | MIT | Yes | open | hdf5, lerobot |
| HeLiPR | 2023 | research-lab | — | — | car with heterogeneous lidars | not stated | No | open | — |
| PushT — Diffusion Policy Reference Dataset | 2023 | synthetic | — | — | simulated 2D pusher | Apache 2.0 | Yes | open | lerobot, parquet |
| WIT-UAS (Wildland-fire Infrared Thermal UAS Dataset) | 2023 | real-robot | — | — | drone | custom | No | open | — |
| EgoThink | 2023 | human-video | — | — | egocentric images from existing video | not stated | No | open | — |
| ECMD | 2023 | research-lab | — | — | event cameras with multi-sensor suite | not stated | No | open | — |
| UniDexGrasp | 2023 | research-lab | — | — | dexterous hand, tabletop point cloud setting | not stated | No | open | — |
| Grasp'D-1M (Fast-Grasp'D) | 2023 | research-lab | — | — | three robot hands, differentiable simulation | not stated | No | open | — |
| EgoPlan-Bench / EgoPlan-Bench2 | 2023 | human-video | — | — | existing egocentric videos | not stated | No | open | — |
| EgoObjects | 2023 | human-video | — | — | 4 wearable devices (not named in abstract) | MIT (per third-party atlas) | Yes | open | — |
| AVOID | 2023 | research-lab | — | — | AV incident reports | not stated | No | open | — |
| Aurora Multi-Sensor Dataset | 2023 | research-lab | — | — | Uber ATG vehicles, lidar and cameras, Pittsburgh | not stated | No | open | — |
| AIDE | 2023 | research-lab | — | — | multi-view in-cabin and out-of-cabin cameras | not stated | No | open | — |
| EgoHumans | 2023 | human-video | — | — | wearable cameras in multi-view capture setup | not stated | No | open | — |
| ScenarioNet | 2023 | research-lab | — | — | Waymo, nuScenes, nuPlan scenarios in MetaDrive | not stated | No | open | — |
| EGOFALLS | 2023 | human-video | — | — | egocentric cameras with audio | not stated | No | open | — |
| POV-Surgery | 2023 | human-video | — | — | simulated head-mounted view | not stated | No | open | — |
| I-24 MOTION | 2023 | research-lab | — | — | 276 pole-mounted cameras over 4.2 miles of freeway | not stated | No | gated | — |
| EgoWholeBody | 2023 | human-video | — | — | simulated head-mounted fisheye camera | not stated | No | open | — |
| GRACO | 2023 | research-lab | — | — | UGV and UAV | not stated | No | open | — |
| WiSARD | 2023 | research-lab | — | — | UAV visual and thermal | not stated | No | open | — |
| Dual Radar | 2023 | research-lab | — | — | car with two 4D radars, lidar, camera | not stated | No | open | — |
| Open X-Embodiment — Robotic Learning Datasets | 2023 | real-robot | — | — | Franka Panda, Google RT-2, WidowX 250s | Apache 2.0 | Yes | open | zarr, tfrecord |
| RT-1 Robot Action Dataset | 2022 | human-video | — | 350.6 | — | CC-BY-4.0 | Yes | open | — |
| AssistQ | 2022 | human-video | — | — | newly filmed egocentric instructional videos | not stated | No | open | — |
| VOST | 2022 | human-video | — | — | egocentric videos from existing sets | not stated | No | open | — |
| ARCTIC | 2022 | human-video | — | — | mocap studio with static cameras and an egocentric camera | not stated | No | open | — |
| ActionSense | 2022 | human-video | — | — | eye tracker with first-person camera, EMG, 17-IMU suit, tactile gloves | not stated | No | open | — |
| UmeTrack | 2022 | human-video | — | — | VR headset with 4 fisheye sensors | CC BY-NC 4.0 | No | open | — |
| WildDash 2 | 2022 | research-lab | — | — | dashcam images worldwide | registration required | No | gated | — |
| TransCG | 2022 | research-lab | — | — | RGB-D of transparent objects, robot-collected | CC BY-NC-SA 4.0 | No | open | — |
| View-of-Delft | 2022 | research-lab | — | — | car with 64-layer lidar, stereo camera, 3+1D radar | not stated | No | open | — |
| SceneEgo (EgoGTA, EgoPW-Scene) | 2022 | human-video | — | — | head-mounted fisheye camera, synthetic and real | not stated | No | open | — |
| TJ4DRadSet | 2022 | research-lab | — | — | car with 4D radar, lidar, camera | not stated | No | open | — |
| SceNDD | 2022 | research-lab | — | — | naturalistic driving | not stated | No | open | — |
| PSI | 2022 | research-lab | — | — | dashcam video | not stated | No | open | — |
| TartanDrive | 2022 | human-video | — | 5 | offroad_vehicle | CC-BY-4.0 | Yes | open | — |
| EDINA | 2022 | human-video | — | — | Egocentric RGB with depth and surface normals | not stated | No | open | — |
| DREDS and STD | 2022 | research-lab | — | — | simulated and real RGB-D of specular, transparent, diffuse objects | CC BY-NC 4.0 | No | open | — |
| HOI4D | 2022 | human-video | — | — | head-mounted RGB-D | CC BY-NC 4.0 (per atlas) | No | open | — |
| OpenLane | 2022 | research-lab | — | — | Waymo Open images with 3D lanes | not stated | No | open | — |
| ONCE-3DLanes | 2022 | research-lab | — | — | ONCE vehicle | not stated | No | gated | — |
| SUES-200 | 2022 | research-lab | — | — | drone at four heights plus satellite | not stated | No | open | — |
| EPIC-KITCHENS-100 | 2022 | human-video | — | 100 | human | CC-BY-NC-4.0 | No | open | — |
| GIMO | 2022 | human-video | — | — | head-mounted device with gaze, inertial motion capture | not stated | No | open | — |
| FT-HID | 2022 | human-video | — | — | RGB-D first- and third-person capture | not stated | No | open | — |
| K-Radar | 2022 | research-lab | — | — | car with 4D radar, lidar, stereo cameras | CC BY-NC-ND | No | open | — |
| K-Lane | 2022 | research-lab | — | — | lidar and camera | Apache-2.0 | Yes | open | — |
| Ithaca365 | 2022 | research-lab | — | — | car with cameras and lidar | CC BY-NC-SA 4.0 | No | open | — |
| IDD-3D | 2022 | research-lab | — | — | car with cameras and lidar | not stated | No | gated | — |
| DualCam | 2022 | research-lab | — | — | dual camera traffic lights | not stated | No | open | — |
| Ego4D — Around the World in 3,025 Hours of Egocentric Video | 2022 | human-video | — | 3,025 | human (wearable camera) | Ego4D License — non-commercial research only | No | gated | mp4, json |
| EgoTaskQA | 2022 | human-video | — | — | LEMMA egocentric videos | not stated | No | open | — |
| MultiDex (GenDexGrasp) | 2022 | research-lab | — | — | multiple robotic hands | not stated | No | open | — |
| DAIR-V2X | 2022 | research-lab | — | — | vehicle and roadside cameras and lidar | not stated | No | gated | — |
| Ego4D | 2022 | human-video | — | 3,670 | human | research-only | No | open | — |
| CODA | 2022 | research-lab | — | — | images from KITTI, nuScenes, ONCE | CC BY-NC-SA 4.0 | No | open | — |
| Boreas | 2022 | research-lab | — | — | car with Velodyne Alpha Prime, Navtech radar, FLIR camera | not stated | No | open | — |
| B3D (Berkeley DeepDrive Drone) | 2022 | research-lab | — | — | drone-recorded traffic | not stated | No | open | — |
| DexGraspNet | 2022 | research-lab | — | — | ShadowHand, validated in Isaac Gym | CC BY-NC 4.0 | No | open | — |
| aiMotive Dataset | 2022 | research-lab | — | — | car with lidar, cameras, radars, 360 view | CC BY-NC-SA 4.0 | No | open | — |
| Language-Table | 2022 | real-robot | — | 3,865 | xarm | Apache-2.0 | Yes | open | — |
| EgoHOS | 2022 | human-video | — | — | frames from several egocentric sets | MIT (per atlas) | Yes | open | — |
| Rope3D | 2022 | research-lab | — | — | roadside cameras with lidar labels | registration required (per ad-datasets) | No | gated | — |
| DA2 | 2022 | research-lab | — | — | dual-arm parallel-jaw grasp pairs | not stated | No | open | — |
| UnrealEgo / UnrealEgo2 / UnrealEgo-RW | 2022 | human-video | — | — | virtual eyeglasses with two fisheye cameras; RW adds real device | not stated | No | open | — |
| CALVIN | 2022 | synthetic | — | 24 | franka_panda | MIT | Yes | open | — |
| A9 / TUMTraf Intersection | 2022 | research-lab | — | — | roadside cameras and lidars on A9 test bed | registration required | No | gated | — |
| VPAIR | 2022 | research-lab | — | — | light aircraft with downward camera | not stated | No | open | — |
| OakInk | 2022 | human-video | — | — | multi-camera third-person rig (not egocentric) | CC-BY-NC-SA-3.0 (Hub oakink/OakInk-v1) | No | open | — |
| DexYCB | 2021 | human-video | — | 0.83 | human | CC-BY-NC-4.0 | No | open | — |
| DefGraspSim | 2021 | research-lab | — | — | simulated grasps on deformable objects | see repo LICENSE | No | open | — |
| Contact-GraspNet training scenes | 2021 | research-lab | — | — | simulated tabletop scenes, parallel-jaw | see repo License.pdf | No | open | — |
| TREK-150 | 2021 | human-video | — | — | EPIC-KITCHENS videos | not stated | No | open | — |
| SuctionNet-1Billion | 2021 | research-lab | — | — | GraspNet-1Billion scenes with suction labels | CC BY-NC-SA 4.0 | No | open | — |
| SODA10M | 2021 | research-lab | — | — | dashcam images | CC BY-NC-SA 4.0 | No | gated | — |
| Shifts (vehicle motion prediction) | 2021 | research-lab | — | — | Yandex self-driving fleet tracks | not stated | No | open | — |
| ORBIT | 2021 | human-video | — | — | mobile phones held by blind or low-vision users | not stated | No | open | — |
| TrajAir | 2021 | real-robot | — | 2,442 | drone | CC-BY-4.0 | Yes | open | — |
| SegmentMeIfYouCan (RoadAnomaly21, RoadObstacle21) | 2021 | research-lab | — | — | road images with anomalies | various | No | open | — |
| N-EPIC-Kitchens | 2021 | human-video | — | — | EPIC-KITCHENS videos converted to event streams | not stated | No | open | — |
| OCID-Grasp | 2021 | research-lab | — | — | OCID RGB-D scenes with grasp rectangles | see repo LICENSE | No | open | — |
| ROAD | 2021 | research-lab | — | — | Oxford RobotCar videos | CC BY-SA 4.0 | Yes | open | — |
| RADIal | 2021 | research-lab | — | — | car with HD radar, camera, laser scanner | not stated | No | open | — |
| PVDN (Provident Vehicle Detection at Night) | 2021 | research-lab | — | — | grayscale camera at night | not stated | No | open | — |
| PixSet | 2021 | research-lab | — | — | car with full-waveform flash lidar, cameras, radar | not stated | No | open | — |
| SeaDronesSee | 2021 | research-lab | — | — | UAV over open water | not stated | No | open | — |
| Oxford Road Boundaries | 2021 | research-lab | — | — | Oxford RobotCar | not stated | No | open | — |
| HPS (Human POSEitioning System) | 2021 | research-lab | — | — | human body, body-worn IMUs plus head camera in scanned scenes | not stated | No | open | — |
| H2O | 2021 | human-video | — | — | synchronised multi-view RGB-D including egocentric | not stated | No | open | — |
| Grasps Under Varied Object Orientation Dataset | 2021 | research-lab | — | — | human hand grasps | not stated | No | open | — |
| KITTI-360 | 2021 | research-lab | — | — | car with fisheye and perspective cameras, lidar | CC BY-NC-SA 3.0 | No | open | — |
| Honda LOKI | 2021 | research-lab | — | — | vehicle camera and lidar | research only | No | open | — |
| Robot handle grasping failure data | 2021 | research-lab | — | — | Toyota HSR drawer handle grasping | CC-BY-4.0 | Yes | open | — |
| DurLAR | 2021 | research-lab | — | — | car with 128-channel lidar, stereo camera, lux meter, GNSS/INS | not stated | No | open | — |
| DSEC | 2021 | research-lab | — | — | stereo event cameras, RGB, lidar, GPS | not stated | No | open | — |
| ACDC | 2021 | research-lab | — | — | car camera, Switzerland | not stated | No | gated | — |
| In-flight Positional and Energy-Use Dataset of Package-Delivery Quadcopter UAVs | 2021 | real-robot | — | 10.75 | drone | CC-BY-4.0 | Yes | open | — |
| ARQ-CRISP slip detection dataset 2021 | 2021 | research-lab | — | — | robot hand tactile data during pick and place | not stated | No | open | — |
| Topo-boundary | 2021 | research-lab | — | — | aerial images of road boundaries | not stated | No | open | — |
| IPS300+ | 2021 | research-lab | — | — | roadside lidar and cameras at intersection | CC BY-NC-SA 4.0 | No | open | — |
| Automatum Open Highway | 2021 | research-lab | — | — | drone over German highways | CC BY-ND | No | open | — |
| VID (Visual-Inertial-Dynamical) Dataset | 2021 | research-lab | — | — | multirotor with rotor speed and force sensing | not stated | No | open | — |
| CALVIN — Composing Actions from Language and Vision | 2021 | synthetic | — | — | Franka Panda (simulated) | MIT | Yes | open | npz, pkl |
| RoboMimic — A Study of Imitation Learning from Observations | 2021 | real-robot | — | — | Franka Panda, Sawyer | MIT | Yes | open | hdf5 |
| H2O-3D | 2021 | human-video | — | — | fixed multi-view RGB-D cameras (not egocentric) | not stated | No | open | — |
| TartanAir | 2020 | synthetic | — | 27.8 | drone | CC-BY-4.0 | Yes | open | — |
| ACRONYM | 2020 | research-lab | — | — | simulated parallel-jaw gripper (NVIDIA FleX) | MIT (repo) | No | open | — |
| TaskGrasp | 2020 | research-lab | — | — | RGB-D point clouds of real objects with task labels | not stated | No | open | — |
| Toronto-3D | 2020 | research-lab | — | — | mobile LiDAR urban point cloud | not stated | No | open | — |
| SemanticPOSS | 2020 | research-lab | — | — | LiDAR point clouds | not stated | No | open | — |
| Real Embodied Dataset (RED) | 2020 | research-lab | — | — | hand-mounted RGB-D, upper hemisphere viewpoints | not stated | No | open | — |
| GraspNet-1Billion | 2020 | research-lab | — | — | Kinect Azure and RealSense D435 RGB-D, parallel-jaw grasp labels | CC BY-NC-SA 4.0 | No | open | — |
| PePScenes | 2020 | research-lab | — | — | pedestrian action labels in 3D | not stated | No | open | — |
| Event-Stream grasping dataset | 2020 | research-lab | — | — | neuromorphic event camera | not stated | No | open | — |
| PandaSet | 2020 | research-lab | — | — | car with Hesai Pandar64 and PandarGT lidars, 6 cameras | CC BY 4.0 | Yes | open | — |
| Oxford Radar RobotCar | 2020 | research-lab | — | — | RobotCar with Navtech radar, lidar, cameras | CC BY-NC-SA 4.0 | No | gated | — |
| nuImages | 2020 | research-lab | — | — | nuScenes vehicle cameras | CC BY-NC-SA 4.0 | No | gated | — |
| Robotic Interestingness Dataset (SubTF) | 2020 | human-video | — | 7.8 | — | unknown | No | open | — |
| GRAB | 2020 | research-lab | — | — | human whole body with hands and face, SMPL-X, plus 3D objects | custom, sign-in required | No | gated | — |
| EGAD | 2020 | research-lab | — | — | 3D-printable object meshes | not stated | No | open | — |
| ContactPose | 2020 | research-lab | — | — | human hand grasps, RGB-D and contact maps | not stated | No | open | — |
| LIBRE | 2020 | research-lab | — | — | multiple 3D LiDARs on vehicle | not stated | No | open | — |
| Honda TITAN | 2020 | research-lab | — | — | egocentric vehicle camera, Tokyo | research only | No | open | — |
| FSOCO | 2020 | research-lab | — | — | Formula Student driverless car cameras | not stated | No | open | — |
| Ford Multi-AV Seasonal Dataset | 2020 | research-lab | — | — | Ford AV fleet, lidars, cameras, GPS/IMU | CC BY-NC-SA 4.0 | No | open | — |
| DDAD | 2020 | research-lab | — | — | TRI vehicles, 6 cameras, Luminar lidar | CC BY-NC-SA 4.0 | No | open | — |
| CurveLanes | 2020 | research-lab | — | — | lane images | not stated | No | open | — |
| comma10k | 2020 | research-lab | — | — | comma device road images | MIT | Yes | open | — |
| ALFA (AirLab Failure and Anomaly Dataset) | 2020 | real-robot | — | 1.3 | drone | CC-BY-4.0 | Yes | open | — |
| Cirrus | 2020 | research-lab | — | — | car with two 250 m bi-pattern lidars and camera | not stated | No | open | — |
| CARRADA | 2020 | research-lab | — | — | camera and automotive radar | CC BY-NC-SA 4.0 | No | open | — |
| Car Crash Dataset (CCD) | 2020 | research-lab | — | — | dashcam accident videos | not stated | No | open | — |
| CADC (Canadian Adverse Driving Conditions) | 2020 | research-lab | — | — | Lincoln MKZ with 8 cameras, lidar, GNSS | CC BY-NC 4.0 | No | open | — |
| EgoK360 | 2020 | human-video | — | — | wearable 360 camera | not stated | No | open | — |
| A2D2 | 2020 | research-lab | — | — | Audi Q7 with 6 cameras, 5 lidars | CC BY-ND 4.0 | No | open | — |
| 4Seasons | 2020 | research-lab | — | — | car with stereo camera, IMU, RTK-GNSS | CC BY-NC-SA 4.0 | No | open | — |
| pNEUMA | 2020 | research-lab | — | — | swarm of 10 drones over Athens | not stated | No | open | — |
| EgoCom | 2020 | research-lab | — | 38.5 | — | Modified MIT | Yes | open | — |
| Ego3DHands | 2020 | human-video | — | — | Synthetic two-hand images | not stated | No | open | — |
| Ego2Hands / Ego2HandsPose | 2020 | human-video | — | — | egocentric RGB camera | not stated | No | open | — |
| University-1652 | 2020 | research-lab | — | — | synthetic drone, satellite and ground views | not stated | No | open | — |
| SensatUrban | 2020 | research-lab | — | — | urban-scale photogrammetric point clouds | not stated | No | open | — |
| You2Me | 2019 | human-video | — | — | chest-mounted camera (per awesome list) | not stated | No | open | — |
| A Billion Ways to Grasp | 2019 | research-lab | — | — | simulated parallel-jaw grasps, dense SE(3) sampling | not stated | No | open | — |
| WoodScape | 2019 | research-lab | — | — | 4 fisheye surround cameras, lidar | non-commercial research | No | gated | — |
| 6-DOF GraspNet data | 2019 | research-lab | — | — | simulated parallel-jaw grasps | see repo LICENSE | No | open | — |
| Waymo Open Dataset Perception | 2019 | research-lab | — | — | Waymo vehicle, 5 lidars and 5 cameras | Waymo non-commercial licence | No | gated | — |
| Waymo Open Dataset | 2019 | research-lab | — | — | Waymo vehicles | Waymo terms (sign-in) | No | open | — |
| TEgO | 2019 | human-video | — | — | smartphone camera | not stated | No | open | — |
| UrbanLoco | 2019 | research-lab | — | — | car with full sensor suite, San Francisco and Hong Kong | not stated | No | open | — |
| Unsupervised LLAMAS | 2019 | research-lab | — | — | highway camera with lidar-map lane labels | non-commercial research | No | gated | — |
| SemanticKITTI | 2019 | research-lab | — | — | KITTI odometry lidar sequences | CC BY-NC-SA 4.0 | No | open | — |
| SeeingThroughFog (DENSE) | 2019 | research-lab | — | — | car with gated camera, radar, lidar, FIR | research and teaching | No | gated | — |
| ClearGrasp | 2019 | research-lab | — | — | RGB-D of transparent objects | not stated | No | open | — |
| nuScenes, nuImages, nuPlan family | 2019 | research-lab | — | — | Motional vehicles | not stated in fetched excerpt | No | open | — |
| nuScenes | 2019 | research-lab | — | — | car with 6 cameras, lidar, 5 radars | CC BY-NC-SA 4.0 | No | gated | — |
| MulRan | 2019 | research-lab | — | — | car with radar and lidar | registration required | No | open | — |
| ContactDB | 2019 | research-lab | — | — | human hand contact maps on objects | not stated | No | open | — |
| Honda HEV-I | 2019 | research-lab | — | — | egocentric vehicle camera at intersections | research only | No | open | — |
| Honda H3D | 2019 | research-lab | — | — | HDD vehicle, 360 lidar | research only | No | open | — |
| Fishyscapes | 2019 | research-lab | — | — | Cityscapes and LostAndFound images with anomalies | not stated | No | open | — |
| EU Long-term Dataset | 2019 | research-lab | — | — | multi-sensor vehicle | not stated | No | open | — |
| DrivingStereo | 2019 | research-lab | — | — | car with stereo cameras and lidar | not stated | No | open | — |
| DADA-2000 | 2019 | research-lab | — | — | dashcam accident videos with driver attention | not stated | No | open | — |
| Brno Urban Dataset | 2019 | research-lab | — | — | car with cameras, lidars, thermal camera, GNSS | not stated | No | open | — |
| BLVD | 2019 | research-lab | — | — | vehicle, 5D semantics | not stated | No | open | — |
| Argoverse 1 | 2019 | research-lab | — | — | Argo AI vehicle, ring cameras, lidar, HD maps | CC BY-NC-SA 4.0 | No | open | — |
| A3D (AnAn Accident Detection) | 2019 | research-lab | — | — | dashcam accident videos | not stated | No | open | — |
| NGSIM | 2006 | research-lab | — | — | overhead cameras on US highways | public | No | open | — |
| Columbia Grasp Database | 2009 | research-lab | — | — | precomputed grasps on 3D models (GraspIt) | not stated | No | open | — |
| CMU-MMAC | 2009 | human-video | — | — | wearable camera plus static cameras, mocap, IMUs, audio | not stated | No | open | — |
| Cornell Grasping Dataset | 2011 | research-lab | — | — | RGB-D images with grasp rectangles | not stated | No | open | — |
| GTEA | 2011 | human-video | — | — | camera mounted on a cap | not stated | No | open | — |
| UT Ego | 2012 | human-video | — | — | Looxcie wearable camera | not stated | No | open | — |
| First-Person Social Interactions (Disney) | 2012 | human-video | — | — | cameras mounted on a cap | not stated | No | open | — |
| SHRP2 Naturalistic Driving Study | 2013 | research-lab | — | — | instrumented private cars | data use agreement | No | open | — |
| JPL First-Person Interaction | 2013 | human-video | — | — | Camera on a humanoid observer | not stated | No | open | — |
| BEOID | 2014 | human-video | — | — | gaze-tracked wearable camera | not stated | No | open | — |
| HUJI EgoSeg | 2014 | human-video | — | — | wearable cameras | not stated | No | open | — |
| EgoHands | 2015 | human-video | — | — | Google Glass | not stated | No | open | — |
| EDUB (Ego-Object Discovery) | 2015 | human-video | — | — | lifelogging wearable camera | not stated | No | open | — |
| GUN-71 | 2015 | human-video | — | — | RGB-D camera | not stated | No | open | — |
| Stanford Drone Dataset | 2016 | research-lab | — | — | overhead drone video of campus | CC BY-NC-SA 3.0 | No | open | — |
| Cityscapes | 2016 | research-lab | — | — | car with stereo camera, GPS | non-commercial | No | gated | — |
| Oxford RobotCar | 2016 | research-lab | — | — | Nissan LEAF with cameras, lidar, GPS/INS | CC BY-NC-SA 4.0 | No | gated | — |
| JAAD | 2016 | research-lab | — | — | dashcam | MIT | Yes | open | — |
| Ego2Top | 2016 | human-video | — | — | wearable cameras plus top-view camera | not stated | No | open | — |
| EgoCap | 2016 | human-video | — | — | two fisheye cameras on helmet | not stated | No | open | — |
| LostAndFound | 2016 | research-lab | — | — | stereo camera car | non-commercial | No | open | — |
| Brain4Cars | 2016 | research-lab | — | — | in-cabin and road cameras | not stated | No | open | — |
| BDDV (Berkeley DeepDrive Video) | 2016 | research-lab | — | — | crowd sourced dash cams | not stated | No | open | — |
| EgoDexter | 2017 | human-video | — | — | egocentric RGB-D sensor | not stated | No | open | — |
| Complex Urban (KAIST Urban) | 2017 | research-lab | — | — | car with lidars, stereo camera, GPS/IMU, altimeter | CC BY-NC-SA 4.0 | No | open | — |
| UDRIVE naturalistic driving database | 2017 | research-lab | — | — | instrumented cars, trucks and powered two-wheelers | not stated | No | open | — |
| NEXET | 2017 | research-lab | — | — | Nexar dashcam images | not stated | No | open | — |
| Paris-Lille-3D | 2017 | research-lab | — | — | mobile LiDAR urban point cloud | not stated | No | open | — |
| Mapillary Vistas | 2017 | research-lab | — | — | crowdsourced street-level cameras | CC BY-NC-SA (per ad-datasets) | No | gated | — |
| DDD17 | 2017 | research-lab | — | — | DAVIS event camera on car | not stated | No | open | — |
| Something-Something | 2017 | human-video | — | — | Crowd workers filming their own hands | not stated | No | open | — |
| Wire Detection Dataset | 2017 | human-video | — | — | drone | CC-BY-4.0 | Yes | open | — |
| CityPersons | 2017 | research-lab | — | — | Cityscapes pedestrian boxes | not stated | No | open | — |
| Dex-Net 2.0 | 2017 | research-lab | — | — | synthetic point clouds, parallel-jaw grasps | not stated | No | open | — |
| Zurich Urban MAV Dataset | 2017 | research-lab | — | — | Fotokite MAV with GoPro, GPS, IMU | no restriction | No | open | — |
| EgoGesture | 2017 | human-video | — | — | head-mounted RGB-D camera | not stated | No | open | — |
| THU-READ | 2017 | human-video | — | — | RGB-D sensor mounted on a helmet | not stated | No | open | — |
| EgoYouTubeHands / HandOverFace | 2018 | human-video | — | — | YouTube egocentric videos | not stated | No | open | — |
| Toybox | 2018 | human-video | — | — | Egocentric video of handheld object transformations | not stated | No | open | — |
| Yamaha-CMU Off-Road Dataset (YCOR) | 2018 | human-video | — | — | — | CC-BY-4.0 | Yes | open | — |
| PedX | 2018 | research-lab | — | — | stereo cameras and LiDAR at intersections | not stated | No | open | — |
| VMRD (Visual Manipulation Relationship Dataset) | 2018 | research-lab | — | — | RGB images of stacked objects | not stated | No | open | — |
| 40K naturalistic 6-DoF grasp demonstrations | 2018 | research-lab | — | — | RightHand Robotics three-fingered ReFlex gripper | not stated | No | open | — |
| Robot Learning in Homes grasping dataset | 2018 | research-lab | — | — | low cost mobile robot arm in homes | not stated | No | open | — |
| EuroCity Persons | 2018 | research-lab | — | — | car camera, 31 European cities | research use | No | open | — |
| UAVid | 2018 | research-lab | — | — | UAV 4K slanted-view video | not stated | No | open | — |
| Audi fleet CAN dataset (Drive2Vec) | 2018 | research-lab | — | — | Audi test vehicles with data loggers | not stated | No | open | — |
| CULane | 2018 | research-lab | — | — | cameras on six vehicles, Beijing | non-commercial | No | open | — |
| India Driving Dataset (IDD) | 2018 | research-lab | — | — | car front camera, Hyderabad and Bangalore | not stated | No | gated | — |
| Mo2Cap2 | 2018 | human-video | — | — | cap-mounted fisheye camera | not stated | No | open | — |
| N-CARS | 2018 | research-lab | — | — | event camera car classification | not stated | No | open | — |
| VisDrone | 2018 | research-lab | — | — | drone cameras | not stated | No | open | — |
| RoboNet | 2019 | real-robot | — | — | fetch, franka_panda, kuka_iiwa | CC-BY-4.0 | Yes | open | — |
| A*3D | 2019 | research-lab | — | — | car with cameras and lidar, Singapore | non-commercial academic | No | open | — |
| Mapillary Traffic Sign Dataset (MTSD) | 2019 | research-lab | — | — | crowdsourced street-level cameras | CC BY-NC-SA | No | gated | — |
| Meta-World — Multi-Task and Meta-Reinforcement Learning | 2019 | synthetic | — | — | Sawyer (simulated) | MIT | Yes | open | json, pkl |
| CITR and DUT | 2019 | research-lab | — | — | top-view pedestrian and vehicle trajectories | not stated | No | open | — |
| HO-3D | 2019 | human-video | — | — | one or several fixed RGB-D cameras (not egocentric) | not stated | No | open | — |