Abstract
Driver distraction has become a critical factor in traffic accidents, necessitating accurate behavior recognition for road safety. However, existing methods still suffer from limitations such as low accuracy in recognizing drivers' localized actions and difficulties in distinguishing subtle differences between different behaviors. This paper proposes RMTSE, a hybrid attention model, to enhance driver distraction recognition. The framework introduces a Manhattan Self-Attention Squeeze-and-Excitation (MaSA-SE) module that combines spatial self-attention with channel attention mechanisms. This integration enables simultaneous enhancement of discriminative features and suppression of irrelevant characteristics in driving behavior images, improving learning efficiency through focused feature extraction. We also propose to employ a transfer learning strategy utilizing pre-trained weights during the training process, which further accelerates model convergence and enhances feature generalization. The model achieves Top-1 accuracies of 99.82% and 94.95% on SFD3 and 100-Driver datasets, respectively, with minimal parameter increments, outperforming existing state-of-the-art methods.