Abstract
Precision livestock farming, particularly the collective rearing of animals, remains a pivotal area of focus within agricultural research. However, tracking group-raised animals under conditions of poor lighting, occlusion, and complex outdoor environments continues to pose significant challenges. Due to the intricacies of these conditions, existing methodologies frequently encounter reduced tracking accuracy, decelerated processing rates, and recurrent failures amid occlusion and drift. In response to these challenges, this study introduces SiamCMR, a sophisticated RGB-Thermal (RGBT) object tracking framework tailored for the prolonged observation of group-raised Holstein cows. Constructed upon a dual-stream Siamese network architecture, SiamCMR incorporates innovative feature fusion techniques to deliver robust, real-time tracking capabilities. The framework utilizes a Complementary Coupled Feature Fusion (CCFF) module that merges semi-shared convolutional filters with adaptive sigmoid weighting to efficaciously amalgamate modality-specific features derived from RGB and thermal inputs. To further refine the fusion quality under diverse illumination conditions, we have developed a Multimodal Weight Penalty Module (MWPM), which selectively emphasizes informative channels via batch normalization scaling and feature variance analysis. The framework's resilience to occlusions and drift is enhanced through the integration of reinforcement learning. In experimental evaluations using our proprietary dataset, SiamCMR maintained real-time processing at 135 frames per second (FPS), achieving 81.3% precision (PR) and 58.2% success rate (SR). When compared to the baseline Siamese tracker, SiamFT, which recorded 76.5% PR, 56.2% SR, and 45 FPS, our approach exhibited improvements of 4.8% in PR, 2.0% in SR, and a threefold increase in processing speed, thereby enhancing both tracking accuracy and robustness. Moreover, the system's efficacy has been corroborated through successful implementations on a UAV platform in real-world ranch settings. Results from ablation studies under severe occlusions, light interference, low illumination, and low temperatures validate the effectiveness of the primary components. This research delineates an innovative real-time cattle-tracking solution that augments pasture management by facilitating precise monitoring of cow positions, behaviors, and health, ultimately optimizing feeding strategies and enhancing milk quality and safety.