Abstract
With the rapid development of AI chemists, the application of machine-learning methods has brought transformative changes to the field of chemistry. Computer vision technology is also gradually becoming a critical sensory input for AI-driven chemists. However, for a complete AI chemist platform, intelligent robotic arm operation is essential. Our work focuses on researching machine vision-assisted intelligent robotic arm operations for AI chemists. Based on the CViG_II data set, this paper proposes a chemical apparatus object detection and distance estimation model called MODNet. Compared to the original YOLOv8 algorithm, the proposed method improves experimental accuracy to 95.8%, effectively achieving high-precision detection of chemical apparatus. Additionally, the model combined a monocular distance measurement method is used to detect the real distance of the target, with an error margin of less than 5% and real-time inference frame rate greater than 60 fps, enabling rapid and accurate object detection and distance detection in video streams. Moreover, we attempted to utilize the MODNet auxiliary system to provide real-time guidance for robotic arm grasping operations in chemical laboratory environments. Specifically, we conducted validation experiments on unit operations within the Spiro [fluorene-9,9'-xanthene] (SFX) synthesis process, such as chemical reagent preparation procedures. This provides a vision-assisted solution for autonomous chemical synthesis by AI robotic chemists.