Open Access iconOpen Access

ARTICLE

crossmark

A Semantic Supervision Method for Abstractive Summarization

Sunqiang Hu1, Xiaoyu Li1, Yu Deng1,*, Yu Peng1, Bin Lin2, Shan Yang3

1 School of Information and Software Engineering, University of Electronic Science and Technology of China, Chengdu, 610054, China
2 School of Engineering, Sichuan Normal University, Chengdu, 610066, China
3 Department of Chemistry, Physics, and Atmospheric Sciences, Jackson State University, Jackson, MS, 39217, USA

* Corresponding Author: Yu Deng. Email: email

Computers, Materials & Continua 2021, 69(1), 145-158. https://doi.org/10.32604/cmc.2021.017441

Abstract

In recent years, many text summarization models based on pre-training methods have achieved very good results. However, in these text summarization models, semantic deviations are easy to occur between the original input representation and the representation that passed multi-layer encoder, which may result in inconsistencies between the generated summary and the source text content. The Bidirectional Encoder Representations from Transformers (BERT) improves the performance of many tasks in Natural Language Processing (NLP). Although BERT has a strong capability to encode context, it lacks the fine-grained semantic representation. To solve these two problems, we proposed a semantic supervision method based on Capsule Network. Firstly, we extracted the fine-grained semantic representation of the input and encoded result in BERT by Capsule Network. Secondly, we used the fine-grained semantic representation of the input to supervise the fine-grained semantic representation of the encoded result. Then we evaluated our model on a popular Chinese social media dataset (LCSTS), and the result showed that our model achieved higher ROUGE scores (including R-1, R-2), and our model outperformed baseline systems. Finally, we conducted a comparative study on the stability of the model, and the experimental results showed that our model was more stable.

Keywords


Cite This Article

APA Style
Hu, S., Li, X., Deng, Y., Peng, Y., Lin, B. et al. (2021). A semantic supervision method for abstractive summarization. Computers, Materials & Continua, 69(1), 145-158. https://doi.org/10.32604/cmc.2021.017441
Vancouver Style
Hu S, Li X, Deng Y, Peng Y, Lin B, Yang S. A semantic supervision method for abstractive summarization. Comput Mater Contin. 2021;69(1):145-158 https://doi.org/10.32604/cmc.2021.017441
IEEE Style
S. Hu, X. Li, Y. Deng, Y. Peng, B. Lin, and S. Yang, “A Semantic Supervision Method for Abstractive Summarization,” Comput. Mater. Contin., vol. 69, no. 1, pp. 145-158, 2021. https://doi.org/10.32604/cmc.2021.017441



cc Copyright © 2021 The Author(s). Published by Tech Science Press.
This work is licensed under a Creative Commons Attribution 4.0 International License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
  • 2049

    View

  • 1615

    Download

  • 0

    Like

Share Link