Open Access

REVIEW

Natural Language Processing with Transformer-Based Models: A Meta-Analysis

Charles Munyao^*, John Ndia

School of Computing and Information Technology, Murang’a University of Technology, Murang’a, 75-10200, Kenya

* Corresponding Author: Charles Munyao. Email:

Journal on Artificial Intelligence 2025, 7, 329-346. https://doi.org/10.32604/jai.2025.069226

Received 18 June 2025; Accepted 14 August 2025; Issue published 22 September 2025

Abstract

The natural language processing (NLP) domain has witnessed significant advancements with the emergence of transformer-based models, which have reshaped the text understanding and generation landscape. While their capabilities are well recognized, there remains a limited systematic synthesis of how these models perform across tasks, scale efficiently, adapt to domains, and address ethical challenges. Therefore, the aim of this paper was to analyze the performance of transformer-based models across various NLP tasks, their scalability, domain adaptation, and the ethical implications of such models. This meta-analysis paper synthesizes findings from 25 peer-reviewed studies on NLP transformer-based models, adhering to the PRISMA framework. Relevant papers were sourced from electronic databases, including IEEE Xplore, Springer, ACM Digital Library, Elsevier, PubMed, and Google Scholar. The findings highlight the superior performance of transformers over conventional approaches, attributed to self-attention mechanisms and pre-trained language representations. Despite these advantages, challenges such as high computational costs, data bias, and hallucination persist. The study provides new perspectives by underscoring the necessity for future research to optimize transformer architectures for efficiency, address ethical AI concerns, and enhance generalization across languages. This paper contributes valuable insights into the current trends, limitations, and potential improvements in transformer-based models for NLP.

---

Keywords

---

Supplementary Material

Supplementary Material File

---