Open Access iconOpen Access



Research on Sarcasm Detection Technology Based on Image-Text Fusion

Xiaofang Jin1, Yuying Yang1,*, Yinan Wu1, Ying Xu2

1 School of Information and Communication Engineering, Communication University of China, Beijing, 100024, China
2 Cable Television Technology Research Institute, Academy of Broadcasting Science, Beijing, 100053, China

* Corresponding Author: Yuying Yang. Email: email

Computers, Materials & Continua 2024, 79(3), 5225-5242.


The emergence of new media in various fields has continuously strengthened the social aspect of social media. Netizens tend to express emotions in social interactions, and many people even use satire, metaphors, and other techniques to express some negative emotions, it is necessary to detect sarcasm in social comment data. For sarcasm, the more reference data modalities used, the better the experimental effect. This paper conducts research on sarcasm detection technology based on image-text fusion data. To effectively utilize the features of each modality, a feature reconstruction output algorithm is proposed. This algorithm is based on the attention mechanism, learns the low-rank features of another modality through cross-modality, the eigenvectors are reconstructed for the corresponding modality through weighted averaging. When only the image modality in the dataset is used, the preprocessed data has outstanding performance in reconstructing the output model, with an accuracy rate of 87.6%. When using only the text modality data in the dataset, the reconstructed output model is optimal, with an accuracy rate of 85.2%. To improve feature fusion between modalities for effective classification, a weight adaptive learning algorithm is used. This algorithm uses a neural network combined with an attention mechanism to calculate the attention weight of each modality to achieve weight adaptive learning purposes, with an accuracy rate of 87.9%. Extensive experiments on a benchmark dataset demonstrate the superiority of our proposed model.


Cite This Article

APA Style
Jin, X., Yang, Y., Wu, Y., Xu, Y. (2024). Research on sarcasm detection technology based on image-text fusion. Computers, Materials & Continua, 79(3), 5225-5242.
Vancouver Style
Jin X, Yang Y, Wu Y, Xu Y. Research on sarcasm detection technology based on image-text fusion. Comput Mater Contin. 2024;79(3):5225-5242
IEEE Style
X. Jin, Y. Yang, Y. Wu, and Y. Xu "Research on Sarcasm Detection Technology Based on Image-Text Fusion," Comput. Mater. Contin., vol. 79, no. 3, pp. 5225-5242. 2024.

cc This work is licensed under a Creative Commons Attribution 4.0 International License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
  • 199


  • 82


  • 0


Share Link