Wei Wu^*, Yuan Zhang, Yunpeng Li, Chuanyang Li, Yan Hao

CMES-Computer Modeling in Engineering & Sciences, Vol.140, No.1, pp. 537-555, 2024, DOI:10.32604/cmes.2024.049174

Abstract Fusing hand-based features in multi-modal biometric recognition enhances anti-spoofing capabilities. Additionally, it leverages inter-modal correlation to enhance recognition performance. Concurrently, the robustness and recognition performance of the system can be enhanced through judiciously leveraging the correlation among multimodal features. Nevertheless, two issues persist in multi-modal feature fusion recognition: Firstly, the enhancement of recognition performance in fusion recognition has not comprehensively considered the inter-modality correlations among distinct modalities. Secondly, during modal fusion, improper weight selection diminishes the salience of crucial modal features, thereby diminishing the overall recognition performance. To address these two issues, we introduce an enhanced DenseNet multimodal recognition network… More > Graphic Abstract

Lifang Fu¹, Huanxin Peng^2,*, Changjin Ma², Yuhan Liu²

CMC-Computers, Materials & Continua, Vol.78, No.3, pp. 4399-4416, 2024, DOI:10.32604/cmc.2024.047053

Abstract In recent years, how to efficiently and accurately identify multi-model fake news has become more challenging. First, multi-model data provides more evidence but not all are equally important. Secondly, social structure information has proven to be effective in fake news detection and how to combine it while reducing the noise information is critical. Unfortunately, existing approaches fail to handle these problems. This paper proposes a multi-model fake news detection framework based on Tex-modal Dominance and fusing Multiple Multi-model Cues (TD-MMC), which utilizes three valuable multi-model clues: text-model importance, text-image complementary, and text-image inconsistency. TD-MMC is dominated by textural content and… More >

Yuanle Chen¹, Haobo Wang¹, Chunyu Liu¹, Linyi Wang², Jiaxin Liu¹, Wei Wu^1,*

CMES-Computer Modeling in Engineering & Sciences, Vol.139, No.3, pp. 2985-3009, 2024, DOI:10.32604/cmes.2023.046837

Abstract Recently, there have been significant advancements in the study of semantic communication in single-modal scenarios. However, the ability to process information in multi-modal environments remains limited. Inspired by the research and applications of natural language processing across different modalities, our goal is to accurately extract frame-level semantic information from videos and ultimately transmit high-quality videos. Specifically, we propose a deep learning-based Multi-Modal Mutual Enhancement Video Semantic Communication system, called M3E-VSC. Built upon a Vector Quantized Generative Adversarial Network (VQGAN), our system aims to leverage mutual enhancement among different modalities by using text as the main carrier of transmission. With it,… More >

Mohamed Abdel-Basset¹, Hossam Hawash¹, Mohamed Abouhawwash^2,3,*, S. S. Askar⁴, Alshaimaa A. Tantawy¹

CMC-Computers, Materials & Continua, Vol.78, No.1, pp. 1171-1187, 2024, DOI:10.32604/cmc.2023.044425

Abstract The early implementation of treatment therapies necessitates the swift and precise identification of COVID-19 pneumonia by the analysis of chest CT scans. This study aims to investigate the indispensable need for precise and interpretable diagnostic tools for improving clinical decision-making for COVID-19 diagnosis. This paper proposes a novel deep learning approach, called Conformer Network, for explainable discrimination of viral pneumonia depending on the lung Region of Infections (ROI) within a single modality radiographic CT scan. Firstly, an efficient U-shaped transformer network is integrated for lung image segmentation. Then, a robust transfer learning technique is introduced to design a robust feature… More >

Yuyuan Xiang, Yangli Jia^*, Xiangliang Zhang, Zhenling Zhang

CMC-Computers, Materials & Continua, Vol.77, No.1, pp. 97-114, 2023, DOI:10.32604/cmc.2023.040751

Abstract Event extraction stands as a significant endeavor within the realm of information extraction, aspiring to automatically extract structured event information from vast volumes of unstructured text. Extracting event elements from multi-modal data remains a challenging task due to the presence of a large number of images and overlapping event elements in the data. Although researchers have proposed various methods to accomplish this task, most existing event extraction models cannot address these challenges because they are only applicable to text scenarios. To solve the above issues, this paper proposes a multi-modal event extraction method based on knowledge fusion. Specifically, for event-type… More >

Jiwei Fan, Xiaogang Yang^*, Ruitao Lu, Qingge Li, Siyu Wang

CMC-Computers, Materials & Continua, Vol.77, No.1, pp. 1031-1052, 2023, DOI:10.32604/cmc.2023.039582

Abstract In recent years, many visual positioning algorithms have been proposed based on computer vision and they have achieved good results. However, these algorithms have a single function, cannot perceive the environment, and have poor versatility, and there is a certain mismatch phenomenon, which affects the positioning accuracy. Therefore, this paper proposes a location algorithm that combines a target recognition algorithm with a depth feature matching algorithm to solve the problem of unmanned aerial vehicle (UAV) environment perception and multi-modal image-matching fusion location. This algorithm was based on the single-shot object detector based on multi-level feature pyramid network (M2Det) algorithm and… More >

Leilei Li¹, Yansheng Fu², Dongjie Zhu^2,*, Xiaofang Li³, Yundong Sun², Jianrui Ding², Mingrui Wu², Ning Cao^4,*, Russell Higgs⁵

Intelligent Automation & Soft Computing, Vol.36, No.3, pp. 3295-3307, 2023, DOI:10.32604/iasc.2023.035257

Abstract The knowledge graph with relational abundant information has been widely used as the basic data support for the retrieval platforms. Image and text descriptions added to the knowledge graph enrich the node information, which accounts for the advantage of the multi-modal knowledge graph. In the field of cross-modal retrieval platforms, multi-modal knowledge graphs can help to improve retrieval accuracy and efficiency because of the abundant relational information provided by knowledge graphs. The representation learning method is significant to the application of multi-modal knowledge graphs. This paper proposes a distributed collaborative vector retrieval platform (DCRL-KG) using the multimodal knowledge graph VisualSem… More >

H. M. Shahzad^1,3, Sohail Masood Bhatti^1,3,*, Arfan Jaffar^1,3, Muhammad Rashid²

Intelligent Automation & Soft Computing, Vol.36, No.2, pp. 1561-1570, 2023, DOI:10.32604/iasc.2023.032525

Abstract In recent years, research on facial expression recognition (FER) under mask is trending. Wearing a mask for protection from Covid 19 has become a compulsion and it hides the facial expressions that is why FER under the mask is a difficult task. The prevailing unimodal techniques for facial recognition are not up to the mark in terms of good results for the masked face, however, a multimodal technique can be employed to generate better results. We proposed a multimodal methodology based on deep learning for facial recognition under a masked face using facial and vocal expressions. The multimodal has been… More >

Junhua Xi¹, Kouquan Zheng¹, Yifan Zhong², Longjiang Li³, Zhiping Cai^1,*, Jinjing Chen⁴

Intelligent Automation & Soft Computing, Vol.35, No.3, pp. 3099-3111, 2023, DOI:10.32604/iasc.2023.030298

Abstract In geometry processing, symmetry research benefits from global geometric features of complete shapes, but the shape of an object captured in real-world applications is often incomplete due to the limited sensor resolution, single viewpoint, and occlusion. Different from the existing works predicting symmetry from the complete shape, we propose a learning approach for symmetry prediction based on a single RGB-D image. Instead of directly predicting the symmetry from incomplete shapes, our method consists of two modules, i.e., the multi-modal feature fusion module and the detection-by-reconstruction module. Firstly, we build a channel-transformer network (CTN) to extract cross-fusion features from the RGB-D… More >

Noureen Fatima¹, Rashid Jahangir², Ghulam Mujtaba¹, Adnan Akhunzada^3,*, Zahid Hussain Shaikh⁴, Faiza Qureshi¹

CMC-Computers, Materials & Continua, Vol.72, No.3, pp. 4357-4374, 2022, DOI:10.32604/cmc.2022.023830

Abstract The Coronavirus Disease 2019 (COVID-19) pandemic poses the worldwide challenges surpassing the boundaries of country, religion, race, and economy. The current benchmark method for the detection of COVID-19 is the reverse transcription polymerase chain reaction (RT-PCR) testing. Nevertheless, this testing method is accurate enough for the diagnosis of COVID-19. However, it is time-consuming, expensive, expert-dependent, and violates social distancing. In this paper, this research proposed an effective multi-modality-based and feature fusion-based (MMFF) COVID-19 detection technique through deep neural networks. In multi-modality, we have utilized the cough samples, breathe samples and sound samples of healthy as well as COVID-19 patients from… More >