Data Pyramid for Embodied Manipulation Survey Dataset
The Data Pyramid for Embodied Manipulation is a comprehensive survey and conceptual framework that organizes the fragmented landscape of robotic learning data into a structured hierarchy. It categorizes the embodied data ecosystem into five complementary sources: real-robot data, UMI-style data, egocentric and exocentric human-video data, simulation data, and general vision-language data. By analyzing the inherent tension between data scalability and robot alignment, the work provides a roadmap for researchers to evaluate data quality, diversity, reusability, and physical fidelity. This survey is designed for practitioners in embodied AI and robotics who seek to understand how mixing these diverse data sources impacts the performance of foundation models. It addresses critical open challenges, including the development of scalable collection pipelines, action alignment across different robot embodiments, and the design of principled data recipes for training robust, general-purpose manipulation agents in complex, real-world environments.
| Year | 2026 |
|---|---|
| Task categories | manipulation, pick-and-place, dexterous-manipulation |
| License | null |
| Access | open |
| Maintainer | Cheng Chi et al. |
| Origin country | CN |
The Data Pyramid for Embodied Manipulation is a systematic survey that addresses the data-scarcity bottleneck in embodied AI. Unlike large language models that benefit from the vast internet, embodied agents require data that couples observations with physical actions. The authors propose a 'pyramid' model to classify data sources based on their scalability and alignment with physical robot hardware. The methodology involves a rigorous review of existing datasets, collection pipelines, and sensing configurations. It highlights the role of UMI-style (Universal Manipulation Interface) data as a bridge between high-fidelity robot data and scalable human-centric videos. The survey serves as a foundational reference for researchers aiming to build large-scale tactile datasets, collect failure-recovery data, and leverage egocentric video for dexterous manipulation tasks. It provides a structured taxonomy that helps the community navigate the trade-offs between simulation-based training and real-world deployment.
Frequently asked questions
What is the Data Pyramid for Embodied Manipulation?
It is a conceptual framework and survey that organizes various sources of robotic learning data into a hierarchy based on scalability and robot alignment.
What are the five sources of data in the pyramid?
The five sources are real-robot data, UMI-style data, egocentric and exocentric data, simulation data, and general vision-language data.
Is this a dataset or a survey paper?
It is a survey paper that reviews and categorizes existing datasets and methodologies rather than being a single downloadable dataset itself.
Who are the primary authors of this work?
The work is authored by a large collaborative team including Yifan Ye, Yankai Fu, Yaoxu Lv, and others from institutions such as Peking University.
Does this work provide a solution for robot-agnostic data?
Yes, it emphasizes UMI-style data collection as a key paradigm for gathering robot-agnostic manipulation data using portable, handheld grippers.