Open Access iconOpen Access

ARTICLE

crossmark

Classification of Conversational Sentences Using an Ensemble Pre-Trained Language Model with the Fine-Tuned Parameter

R. Sujatha, K. Nimala*

Department of Networking and Communications, SRM Institute of Science and Technology, Kattankulathur, Chengalpattu, Tamilnadu, 603203, India

* Corresponding Author: K. Nimala. Email: email

Computers, Materials & Continua 2024, 78(2), 1669-1686. https://doi.org/10.32604/cmc.2023.046963

Abstract

Sentence classification is the process of categorizing a sentence based on the context of the sentence. Sentence categorization requires more semantic highlights than other tasks, such as dependence parsing, which requires more syntactic elements. Most existing strategies focus on the general semantics of a conversation without involving the context of the sentence, recognizing the progress and comparing impacts. An ensemble pre-trained language model was taken up here to classify the conversation sentences from the conversation corpus. The conversational sentences are classified into four categories: information, question, directive, and commission. These classification label sequences are for analyzing the conversation progress and predicting the pecking order of the conversation. Ensemble of Bidirectional Encoder for Representation of Transformer (BERT), Robustly Optimized BERT pretraining Approach (RoBERTa), Generative Pre‑Trained Transformer (GPT), DistilBERT and Generalized Autoregressive Pretraining for Language Understanding (XLNet) models are trained on conversation corpus with hyperparameters. Hyperparameter tuning approach is carried out for better performance on sentence classification. This Ensemble of Pre-trained Language Models with a Hyperparameter Tuning (EPLM-HT) system is trained on an annotated conversation dataset. The proposed approach outperformed compared to the base BERT, GPT, DistilBERT and XLNet transformer models. The proposed ensemble model with the fine-tuned parameters achieved an F1_score of 0.88.

Keywords


Cite This Article

APA Style
Sujatha, R., Nimala, K. (2024). Classification of conversational sentences using an ensemble pre-trained language model with the fine-tuned parameter. Computers, Materials & Continua, 78(2), 1669-1686. https://doi.org/10.32604/cmc.2023.046963
Vancouver Style
Sujatha R, Nimala K. Classification of conversational sentences using an ensemble pre-trained language model with the fine-tuned parameter. Comput Mater Contin. 2024;78(2):1669-1686 https://doi.org/10.32604/cmc.2023.046963
IEEE Style
R. Sujatha and K. Nimala, "Classification of Conversational Sentences Using an Ensemble Pre-Trained Language Model with the Fine-Tuned Parameter," Comput. Mater. Contin., vol. 78, no. 2, pp. 1669-1686. 2024. https://doi.org/10.32604/cmc.2023.046963



cc This work is licensed under a Creative Commons Attribution 4.0 International License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
  • 421

    View

  • 153

    Download

  • 0

    Like

Share Link