Abstract
This study aimed to develop an advanced ensemble approach for automated classification of mental health disorders in social media posts. The research question was: can an ensemble of fine-tuned transformer models (XLNet, RoBERTa, and ELECTRA) with Bayesian hyperparameter optimization improve the accuracy of mental health disorder classification in social media text. Three transformer models (XLNet, RoBERTa, and ELECTRA) were fine-tuned on a dataset of social media posts labelled with 15 distinct mental health disorders. Bayesian optimization was employed for hyperparameter tuning, optimizing learning rate, number of epochs, gradient accumulation steps, and weight decay. A voting ensemble approach was then implemented to combine the predictions of the individual models. The proposed voting ensemble achieved the highest accuracy of 0.780, outperforming the individual models: XLNet (0.767), RoBERTa (0.775), and ELECTRA (0.755). The proposed ensemble approach, integrating XLNet, RoBERTa, and ELECTRA with Bayesian hyperparameter optimization, demonstrated improved accuracy in classifying mental health disorders from social media posts. This method shows promise for enhancing digital mental health research and potentially aiding in early detection and intervention strategies. Future work should focus on expanding the dataset, exploring additional ensemble techniques, and investigating the model's performance across different social media platforms and languages.