Lightweight transformer models for scalable phishing email detection: A comparative study of ALBERT and TinyBERT on a balanced email corpus

Abstract

Phishing is still a prominent threat to cybersecurity and takes advantage of user trust by sending malicious emails to capture credentials or install malware. Classical machine learning methods have not been able to keep pace with the changing sophistication of phishing content. This work introduces a thorough assessment of two Transformer-based models—ALBERT-base-v2 and TinyBERT—for phishing email classification. Utilizing a real-world dataset downloaded from Kaggle, both models were fine-tuned and compared according to performance measures such as accuracy, precision, recall, F1-score, and ROC-AUC. ALBERT achieved 97.54% in the test and a ROC-AUC of 0.997, whereas TinyBERT achieved 95.42% and a ROC-AUC of 0.992. The results from both models outperform some recent state-of-the-art approaches and validate the practical applicability of lightweight Transformers for cybersecurity use cases. While ALBERT provides better performance for cloud-based applications, TinyBERT provides significant computational efficiency that is suitable for real-time and resource-limited deployments. Recommendations are made for improving adversarial robustness, interpretability, and multilingual robustness. It is shown that Transformer models offer a robust, scalable platform for future phishing detection systems.