Abstract
Automatic topic generation is a fundamental tool in unstructured text analysis, yet its application to noisy web-based collections for extracting psychological patterns remains underexplored. This work compares three representative topic models from different families: Latent Dirichlet Allocation (classical probabilistic), BERTopic (embedding-based), and TopClus (deep neural network), evaluating their performance on mental health data from the eRisk initiative. Using posts from individuals with depressive disorders and control groups, we assess topic quality through both automatic coherence metrics and rigorous human evaluation by expert reviewers. This dual approach addresses the limitations of purely automatic evaluation in complex social media datasets where thematic content does not always reveal psychological cues. Our results demonstrate that BERTopic significantly outperforms other models in perceived coherence, identifying clearer and more specific themes, including depression-related topics such as mental health struggles and self-harm. Thematic analysis across user groups revealed that certain topics contained higher proportions of posts from individuals with depression, providing actionable insights for psychological screening. This work underscores the potential of advanced topic models for mental health analysis in noisy social media data and highlights the importance of human evaluation in validating topic quality for sensitive applications.