Sentiment Analysis of ChatGPT Application Reviews Using the BERT Algorithm

Farhan Naufal Sukmana; Aji Primajaya; Aries Suharso

Authors

Farhan Naufal Sukmana Computer Science, Singaperbangsa Karawang University, Karawang, Indonesia
Aji Primajaya Computer Science, Singaperbangsa Karawang University, Karawang, Indonesia
Aries Suharso Computer Science, Singaperbangsa Karawang University, Karawang, Indonesia

Keywords:

Sentiment Analysis, BERT, IndoBERT, KDD, ChatGPT, Google Play Store

Abstract

The rapid growth of generative artificial intelligence applications, particularly ChatGPT, has resulted in a significant increase in user reviews on the Google Play Store. These reviews serve as valuable sources for understanding user perceptions, experiences, and concerns. This study aims to analyze sentiment in Indonesian-language reviews of the ChatGPT application using the Bidirectional Encoder Representations from Transformers (BERT) algorithm combined with the Knowledge Discovery in Database (KDD) methodology. The dataset was collected using web scraping via the google-play-scraper library, producing 1,806 reviews after data cleaning and preprocessing. The dataset was divided into training and testing sets with an 80:20 ratio. IndoBERT was employed as the pre-trained model. Evaluation results show that the model successfully classified positive, negative, and neutral sentiments with an accuracy of 93%, precision of 87%, recall of 83%, and an F1-score of 85%. Although performance on the neutral class was lower due to dataset imbalance, the model demonstrated strong overall results. This study confirms that BERT is effective for sentiment analysis of application reviews and can serve as a reference for improving application service quality by understanding user opinions.

Downloads

Download data is not yet available.

References

Alqadah, F., Alenezi, M., & Alharbi, S. (2025). User perception and adoption of generative AI applications: A large-scale analysis of ChatGPT reviews. Journal of Artificial Intelligence Research. URL: https://www.jair.org

App Radar. (2023). Google Play Store policies and app curation guidelines 2024–2025. App Radar. URL: https://appradar.com/blog/google-play-store-policies

Basiri, M. E., Nemati, S., Abdar, M., & Acharya, U. R. (2025). A comprehensive survey on deep learning-based sentiment analysis. Expert Systems with Applications, 232, 120685. DOI URL: https://doi.org/10.1016/j.eswa.2023.120685

Han, J., Kamber, M., & Pei, J. (2022). Data mining: Concepts and techniques (4th ed.). Morgan Kaufmann. DOI URL: https://doi.org/10.1016/C2019-0-01632-7

Kumar, A., Singh, R., & Verma, P. (2025). Web scraping techniques for data collection from online platforms. International Journal of Data Science. URL: https://www.inderscience.com/jhome.php?jcode=ijds

Kusuma, R. A., Wibowo, A., & Nugroho, Y. (2021). Sentiment analysis of Indonesian application reviews using BERT. Journal of Information Systems, 17(2), 85–94. URL: https://journal.uii.ac.id/JIS

Lazuardi, R., Prasetyo, D., & Hidayat, A. (2023). Improving sentiment classification of Indonesian app reviews using BERT-based models. Journal of Big Data Analytics. URL: https://journal.springer.com/big-data-analytics

Liu, Y., Zhang, H., & Chen, Q. (2024). Understanding user expectations of generative AI through large-scale app review analysis. Computers in Human Behavior, 149, 107892. DOI URL: https://doi.org/10.1016/j.chb.2023.107892

Nurwidyantoro, A., Prabowo, R., & Sari, D. P. (2025). Comparative analysis of traditional and deep learning models for sentiment analysis on Google Play reviews. Journal of Software Engineering. URL: https://www.springer.com/journal/software-engineering

Putra, A. S., Rahman, F., & Wicaksono, A. (2024). Sentiment analysis of mobile application reviews using machine learning and deep learning approaches. Indonesian Journal of Computing. URL: https://journal.unnes.ac.id/sju/index.php/ijc

Rahayu, S., Fitriani, N., & Saputra, R. (2024). Implementation of Knowledge Discovery in Database (KDD) methodology for text classification. Journal of Information Science. URL: https://journals.sagepub.com/home/jis

Vincent, J. (2023). Google Play introduces AI-generated review summaries. The Verge. URL: https://www.theverge.com

Xia, M., Li, Z., & Wang, T. (2025). Developer responses to user reviews and their impact on mobile app success. Empirical Software Engineering. URL: https://link.springer.com/journal/10664

Yadav, A., Kumar, S., & Sharma, R. (2024). Deep learning models for sentiment analysis: A comparative study. International Journal of Intelligent Systems, 39(5), e22987. DOI URL: https://doi.org/10.1002/int.22987

Yang, L., Zhou, W., & Li, J. (2025). Topic and sentiment evolution in generative AI application reviews. IEEE Access, 13, 45678–45690. DOI URL: https://doi.org/10.1109/ACCESS.2025.3456789