Abstract
The influence of music genres on children's emotional intelligence is one of the hot topics in the field of multi-modal emotion research. How to fuse multi-modal information has an important impact on children's emotional analysis. Most of the current research is based on transformer, in which the self-attention mechanism module is improved to achieve the fusion effect of multi-modal information. However, it is difficult for these methods to effectively capture the effective information of different modalities. Therefore, for the task of the influence of music genres on children's emotions, this paper proposes a transformer-based multi-modal convolutional neural network. The first is to use the BiLSTM sub-network model to extract the video and audio features and use the BERT sub-network to extract the text features. Secondly, this paper uses the improved transformer cross-modal fusion module to effectively fuse different types of modal information. Finally, the transformer module is used to judge the information of different modalities and analyze the emotion from the multi-modal information. At the same time, a large number of experiments prove that the model based on multi-modal convolutional neural network proposed in this paper surpasses other methods in prediction accuracy and effectively improves the accuracy of sentiment classification tasks.