Abstract
As human-robot systems and autonomous robots become increasingly prevalent, the need for task-oriented datasets to study human behaviors in shared spaces has grown significantly. We present a novel dataset focusing on sequential human assembly and disassembly motions in human-robot coexisting environments. It contains over 10,000 samples recorded from multi-view camera setups, each comprising synchronized RGB videos and 2D and 3D human skeletons. Data were collected from 33 participants with diverse physical characteristics and behavior preferences. This dataset highlights practical challenges such as partial occlusions, similar repetitive motions, and varying human behaviors, which are often overlooked in existing datasets and research. Technical validation using benchmarking with state-of-the-art deep learning models reveals significant potential in using this dataset for practical applications. To support diverse research applications, this dataset provides raw and processed data with detailed annotations, including precise timestamps, procedure annotations, and Python codes for reproducibility. It aims to advance research in human motion prediction, task-oriented robotic sequential decision-making, motion and task planning of autonomous robots, and human-robot collaborative policies.