Xiaohui Cui^1,2,#, Chao Song^1,2,#, Dongmei Li^1,2,*, Xiaolong Qu^1,2, Jiao Long^1,2, Yu Yang^1,2, Hanchao Zhang³

CMC-Computers, Materials & Continua, Vol.78, No.3, pp. 3603-3618, 2024, DOI:10.32604/cmc.2024.047321

Abstract Named Entity Recognition (NER) stands as a fundamental task within the field of biomedical text mining, aiming to extract specific types of entities such as genes, proteins, and diseases from complex biomedical texts and categorize them into predefined entity types. This process can provide basic support for the automatic construction of knowledge bases. In contrast to general texts, biomedical texts frequently contain numerous nested entities and local dependencies among these entities, presenting significant challenges to prevailing NER models. To address these issues, we propose a novel Chinese nested biomedical NER model based on RoBERTa and Global Pointer (RoBGP). Our model… More >

Wenfeng Zheng¹, Siyu Lu¹, Zhuohang Cai¹, Ruiyang Wang¹, Lei Wang², Lirong Yin^2,*

CMES-Computer Modeling in Engineering & Sciences, Vol.139, No.3, pp. 2729-2745, 2024, DOI:10.32604/cmes.2023.046692

Abstract In the field of natural language processing (NLP), there have been various pre-training language models in recent years, with question answering systems gaining significant attention. However, as algorithms, data, and computing power advance, the issue of increasingly larger models and a growing number of parameters has surfaced. Consequently, model training has become more costly and less efficient. To enhance the efficiency and accuracy of the training process while reducing the model volume, this paper proposes a first-order pruning model PAL-BERT based on the ALBERT model according to the characteristics of question-answering (QA) system and language model. Firstly, a first-order network… More >

R. Sujatha, K. Nimala^*

CMC-Computers, Materials & Continua, Vol.78, No.2, pp. 1669-1686, 2024, DOI:10.32604/cmc.2023.046963

Abstract Sentence classification is the process of categorizing a sentence based on the context of the sentence. Sentence categorization requires more semantic highlights than other tasks, such as dependence parsing, which requires more syntactic elements. Most existing strategies focus on the general semantics of a conversation without involving the context of the sentence, recognizing the progress and comparing impacts. An ensemble pre-trained language model was taken up here to classify the conversation sentences from the conversation corpus. The conversational sentences are classified into four categories: information, question, directive, and commission. These classification label sequences are for analyzing the conversation progress and… More >

Bashar Alshouha¹, Jesus Serrano-Guerrero^1,*, Francisco Chiclana², Francisco P. Romero¹, Jose A. Olivas¹

CMC-Computers, Materials & Continua, Vol.78, No.2, pp. 1933-1956, 2024, DOI:10.32604/cmc.2023.046711

Abstract Personality recognition plays a pivotal role when developing user-centric solutions such as recommender systems or decision support systems across various domains, including education, e-commerce, or human resources. Traditional machine learning techniques have been broadly employed for personality trait identification; nevertheless, the development of new technologies based on deep learning has led to new opportunities to improve their performance. This study focuses on the capabilities of pre-trained language models such as BERT, RoBERTa, ALBERT, ELECTRA, ERNIE, or XLNet, to deal with the task of personality recognition. These models are able to capture structural features from textual content and comprehend a multitude… More >

Soonshin Seo^1,2, Ji-Hwan Kim^2,*

CMC-Computers, Materials & Continua, Vol.77, No.3, pp. 2833-2856, 2023, DOI:10.32604/cmc.2023.042816

Abstract Automatic speech recognition (ASR) systems have emerged as indispensable tools across a wide spectrum of applications, ranging from transcription services to voice-activated assistants. To enhance the performance of these systems, it is important to deploy efficient models capable of adapting to diverse deployment conditions. In recent years, on-demand pruning methods have obtained significant attention within the ASR domain due to their adaptability in various deployment scenarios. However, these methods often confront substantial trade-offs, particularly in terms of unstable accuracy when reducing the model size. To address challenges, this study introduces two crucial empirical findings. Firstly, it proposes the incorporation of… More >

Asif Khan¹, Huaping Zhang^1,*, Nada Boudjellal², Arshad Ahmad³, Maqbool Khan³

CMC-Computers, Materials & Continua, Vol.76, No.3, pp. 3345-3361, 2023, DOI:10.32604/cmc.2023.041520

Abstract Sentiment analysis plays a vital role in understanding public opinions and sentiments toward various topics. In recent years, the rise of social media platforms (SMPs) has provided a rich source of data for analyzing public opinions, particularly in the context of election-related conversations. Nevertheless, sentiment analysis of election-related tweets presents unique challenges due to the complex language used, including figurative expressions, sarcasm, and the spread of misinformation. To address these challenges, this paper proposes Election-focused Bidirectional Encoder Representations from Transformers (ElecBERT), a new model for sentiment analysis in the context of election-related tweets. Election-related tweets pose unique challenges for sentiment… More >

Guangxia Xu^1,*, Lei Liu², Jingnan Dong³

CMES-Computer Modeling in Engineering & Sciences, Vol.137, No.1, pp. 903-922, 2023, DOI:10.32604/cmes.2023.026627

Abstract In recent years, with the great success of pre-trained language models, the pre-trained BERT model has been gradually applied to the field of source code understanding. However, the time cost of training a language model from zero is very high, and how to transfer the pre-trained language model to the field of smart contract vulnerability detection is a hot research direction at present. In this paper, we propose a hybrid model to detect common vulnerabilities in smart contracts based on a lightweight pre-trained language model BERT and connected to a bidirectional gate recurrent unit model. The downstream neural network adopts… More >

Kuntharrgyal Khysru^1,*, Jianguo Wei^1,2, Jianwu Dang³

CMC-Computers, Materials & Continua, Vol.73, No.3, pp. 4897-4907, 2022, DOI:10.32604/cmc.2022.027591

Abstract In China, Tibetan is usually divided into three major dialects: the Am-do, Khams and Lhasa dialects. The Am-do dialect evolved from ancient Tibetan and is a local variant of modern Tibetan. Although this dialect has its own specific historical and social conditions and development, there have been different degrees of communication with other ethnic groups, but all the abovementioned dialects developed from the same language: Tibetan. This paper uses the particularity of Tibetan suffixes in pronunciation and proposes a lexicon for the Am-do language, which optimizes the problems existing in previous research. Audio data of the Am-do dialect are expanded… More >

Donghyun Lee, Hosung Park, Soonshin Seo, Changmin Kim, Hyunsoo Son, Gyujin Kim, Ji-Hwan Kim^*

CMC-Computers, Materials & Continua, Vol.68, No.1, pp. 537-551, 2021, DOI:10.32604/cmc.2021.015430

Abstract A differentiable neural computer (DNC) is analogous to the Von Neumann machine with a neural network controller that interacts with an external memory through an attention mechanism. Such DNC’s offer a generalized method for task-specific deep learning models and have demonstrated reliability with reasoning problems. In this study, we apply a DNC to a language model (LM) task. The LM task is one of the reasoning problems, because it can predict the next word using the previous word sequence. However, memory deallocation is a problem in DNCs as some information unrelated to the input sequence is not allocated and remains… More >

Zeyu Xiong^1,*, Qiangqiang Shen¹, Yueshan Xiong¹, Yijie Wang¹, Weizi Li²

CMC-Computers, Materials & Continua, Vol.60, No.1, pp. 259-273, 2019, DOI:10.32604/cmc.2019.05155

Abstract Word vector representation is widely used in natural language processing tasks. Most word vectors are generated based on probability model, its bag-of-words features have two major weaknesses: they lose the ordering of the words and they also ignore semantics of the words. Recently, neural-network language models CBOW and Skip-Gram are developed as continuous-space language models for words representation in high dimensional real-valued vectors. These vector representations have recently demonstrated promising results in various NLP tasks because of their superiority in capturing syntactic and contextual regularities in language. In this paper, we propose a new strategy based on optimization in contiguous… More >