Automated Routing of Customer Complaints in Financial Services: NMF Topic Discovery Followed by Supervised Text Classification

Abstract

Banks receive thousands of free-text complaints, and someone usually has to read each one before it can be sent to the right team. This study builds and tests a pipeline that does the sorting automatically. The main obstacle was the data: the complaint corpus had no usable labels, because its original product field spread the cases over 17 overlapping categories. The work therefore ran in two phases. In the first, 21,072 non-empty complaints were cleaned with NLTK, reduced to lemmatised nouns and converted into a TF-IDF matrix of 6,104 terms. Non-negative matrix factorisation (NMF) with five components then produced five readable topics: bank account services, credit card or prepaid card, theft or dispute reporting, mortgages and loans, and others. In the second phase, these topics served as labels for six classifiers trained on 75% of the data and tested on the remaining 5,268 complaints. Logistic Regression scored highest, with 95.54% accuracy and a weighted F1-score of 0.96, followed by LinearSVC (95.18%) and LightGBM (94.40%). Random Forest (88.40%), a single Decision Tree (84.55%) and Multinomial Naive Bayes (78.34%) trailed well behind; Naive Bayes found only 28% of the complaints in the "Others" class. The results show that a simple linear model on sparse TF-IDF features can route complaints accurately at very low cost. Because the labels come from NMF on the same features, the scores measure agreement with the topic model rather than with human judgement, and the paper discusses what this means for deployment.