Analysis and Evaluation of Qur’an Translation Topics Using Classical, Neural, and Transformer-Based Topic Modelling

Akhmad Rinaldy Kurnia; Sajarwo Anggai; Murni Handayani

Authors

Akhmad Rinaldy Kurnia Graduate Program of Informatics Engineering, Universitas Pamulang
Sajarwo Anggai Graduate Program of Informatics Engineering, Universitas Pamulang
Murni Handayani Graduate Program of Informatics Engineering, Universitas Pamulang

Keywords:

LDA, BTM, CombinedTM, BERTopic, Topic Modelling, Coherence Score, Topic Diversity, Qur’an Translation

Abstract

Topic modelling is an important approach for extracting latent thematic structures from text corpora, including religious texts that are characterized by dense semantics and short documents. This study aims to compare the performance of several topic modelling methods Latent Dirichlet Allocation (LDA), Biterm Topic Model (BTM), Combined Topic Model (CombinedTM), and BERTopic in extracting topics from the Indonesian translation of the Qur’an. The dataset consists of 6,236 verses, with each verse treated as a single document. Topic quality is evaluated using two main metrics: coherence score (C_v) and topic diversity. The experimental results show that CombinedTM achieves the highest coherence score, with a maximum value of approximately 0.52 at K = 10 topics, followed by BTM, which demonstrates relatively high and stable coherence scores (around 0.50) across certain topic number variations. LDA yields the highest topic diversity, exceeding 0.90, but with lower coherence scores compared to the other models, indicating its limitations in preserving semantic coherence in short texts. Meanwhile, BERTopic exhibits consistently high topic diversity (0.85–0.88) across different numbers of topics, although its bag-of-words–based coherence scores do not always increase significantly. These findings highlight that the choice of topic modelling method should be aligned with the characteristics of the corpus and the objectives of thematic analysis, particularly in the context of short-form religious texts.

References

R. Egger and J. Yu, “A topic modeling comparison between lda, nmf, top2vec, and bertopic to demystify twitter posts,” Front. Sociol., vol. 7, p. 886498, 2022.

X. Yan, J. Guo, Y. Lan, and X. Cheng, “A biterm topic model for short texts,” in Proceedings of the 22nd international conference on World Wide Web, 2013, pp. 1445–1456.

F. Bianchi, S. Terragni, and D. Hovy, “Pre-training is a hot topic: Contextualized document embeddings improve topic coherence,” in Proceedings of the 59th annual meeting of the association for computational linguistics and the 11th international joint conference on natural language processing (volume 2: short papers), 2021, pp. 759–766.

M. Grootendorst, “BERTopic: Neural topic modeling with a class-based TF-IDF procedure,” arXiv Prepr. arXiv2203.05794, 2022.

S. Rahimi, “Saturation in qualitative research: An evolutionary concept analysis,” Int. J. Nurs. Stud. Adv., vol. 6, p. 100174, 2024.

A. B. Dieng, F. J. R. Ruiz, and D. M. Blei, “Topic modeling in embedding spaces,” Trans. Assoc. Comput. Linguist., vol. 8, pp. 439–453, 2020.

A. Zafar, M. Wasim, S. Zulfiqar, T. Waheed, and A. Siddique, “Transformer-based topic modeling for Urdu translations of the Holy Quran,” ACM Trans. Asian Low-Resource Lang. Inf. Process., vol. 23, no. 10, pp. 1–21, 2024.

D. Rolliawati, I. Rozas, and K. Khalid, “Text mining approach for topic modeling of corpus Al Qur’an in Indonesian translation,” in Proceedings of the 2nd International Conference on Quran and Hadith Studies Information Technology and Media in Conjunction with the 1 st International Conference on Islam, Science and Technology, ICONQUHAS & ICONIST, Bandung, October 2–4, 2018, 2020.

S. J. Putra, T. Mantoro, and M. N. Gunawan, “Text mining for Indonesian translation of the Quran: A systematic review,” in 2017 International Conference on Computing, Engineering, and Design (ICCED), IEEE, 2017, pp. 1–5.

K. Stevens, P. Kegelmeyer, D. Andrzejewski, and D. Buttler, “Exploring topic coherence over many models and many topics,” in Proceedings of the 2012 joint conference on empirical methods in natural language processing and computational natural language learning, 2012, pp. 952–961.

Q. Zhu, Z. Feng, and X. Li, “GraphBTM: Graph enhanced autoencoded variational inference for biterm topic model,” in Proceedings of the 2018 conference on empirical methods in natural language processing, 2018, pp. 4663–4672.

X. Zhou, J. Ouyang, and X. Li, “Two time-efficient gibbs sampling inference algorithms for biterm topic model,” Appl. Intell., vol. 48, no. 3, pp. 730–754, 2018.

M. H. Asnawi, A. A. Pravitasari, T. Herawan, and T. Hendrawati, “The combination of contextualized topic model and MPNet for user feedback topic modeling,” IEEE Access, vol. 11, pp. 130272–130286, 2023.

M. de Groot, M. Aliannejadi, and M. R. Haas, “Experiments on generalizability of BERTopic on multi-domain short text,” arXiv Prepr. arXiv2212.08459, 2022.

O. Babalola, B. Ojokoh, and O. Boyinbode, “Comprehensive evaluation of LDA, NMF, and BERTopic’s performance on news headline topic modeling,” J. Comput. Theor. Appl., vol. 2, no. 2, pp. 268–289, 2024.

C. Meaney, X. Wang, J. Guan, and T. A. Stukel, “Comparison of methods for tuning machine learning model hyper-parameters: with application to predicting high-need high-cost health care users,” BMC Med. Res. Methodol., vol. 25, no. 1, p. 134, 2025.

H. Herwinsyah, “Pemodelan Topik dalam Al-Qur’an Menggunakan Library Bertopic pada Model Bahasa Bert,” Simetris J. Tek. Mesin, Elektro dan Ilmu Komput., vol. 14, no. 2, pp. 319–328, 2023.

K. Jamshaid, H. Farooq, and M. T. Siddique, “Topic Modeling of Quranic Verses using Latent Dirichlet Allocation with English Language: Topic Modeling using LDA,” VFAST Trans. Softw. Eng., vol. 12, no. 4, pp. 239–251, 2024.

D. M. Blei, A. Y. Ng, and M. I. Jordan, “Latent dirichlet allocation,” J. Mach. Learn. Res., vol. 3, no. Jan, pp. 993–1022, 2003.