Comparing the Performance of Specialized BERT Models in Classify-ing Arabic News: AraBERT, CAMeLBERT, and mBERT

Abstract

This study evaluates the comparative performance of advanced Transformer-based models for classifying Arabic news articles into five categories: arts, events and issues, economics, politics, and sports. It further investigates the impact of linguistic specialization on classification accuracy. We utilized a dataset of 173,117 Arabic news articles, which was processed to address class imbalance through under sampling, resulting in a balanced set of 68,677 articles. Three models were trained and evaluated under uniformly tuned hyperparameters for a fair comparison: AraBERT (specialized in Modern Standard Arabic), CAMeLBERT (trained on a mix of MSA and dialects), and mBERT (a multilingual model) as a baseline. The results demonstrate that AraBERT achieved the highest accuracy of 96.04% on the test set, outperforming both CAMeLBERT (94.18%) and mBERT (94.14%). This outcome confirms the "specialization hypothesis," indicating that models specifically pre-trained on formal Arabic yield superior performance on news classification tasks. Furthermore, all Transformer models significantly surpassed traditional methods like CNNs and classical machine learning algorithms (e.g., SVM, Naive Bayes) by a margin of at least ~8.5%. The study concludes that AraBERT is presently the optimal model for this domain and recommends building comprehensive Arabic news datasets, exploring ensemble and hybrid modeling techniques, and expanding research into multi-label classification.