Abstract
Mobile news classification systems face significant challenges due to their large scale and complexity. In this paper, we perform a comprehensive comparative study between traditional classification models, such as TextCNN and BERT based models and Large Language Models (LLMs), for the purpose of multi-label news categorization in mobile apps about the Chinese mobile news application. We evaluated the performance of conventional techniques, including a BERT model, along with Qwen models that have been tuned with instruction and fine-tuned using the LoRA technique, to optimize their effectiveness while preserving classification accuracy. Our experimental results show that BERT models perform best for multi-label classification with balanced datasets, while textCNN performs better for binary classification tasks. Our results also reveal that the LSTM and MLP classifiers consistently achieve the highest accuracy with text instruction prompts, while random embeddings achieve competitive accuracy. Furthermore, despite the low macro F1 scores due to class imbalance, consistent relative performance confirms the validity of our analysis. Our research reveals crucial information about the classification of automotive news, highlighting the importance of weighing technical prowess against deployment constraints when choosing model architectures.