Vol.69, No.1, 2021, pp.145-158, doi:10.32604/cmc.2021.017441
OPEN ACCESS
ARTICLE
A Semantic Supervision Method for Abstractive Summarization
  • Sunqiang Hu1, Xiaoyu Li1, Yu Deng1,*, Yu Peng1, Bin Lin2, Shan Yang3
1 School of Information and Software Engineering, University of Electronic Science and Technology of China, Chengdu, 610054, China
2 School of Engineering, Sichuan Normal University, Chengdu, 610066, China
3 Department of Chemistry, Physics, and Atmospheric Sciences, Jackson State University, Jackson, MS, 39217, USA
* Corresponding Author: Yu Deng. Email:
Received 30 January 2021; Accepted 17 March 2021; Issue published 04 June 2021
Abstract
In recent years, many text summarization models based on pre-training methods have achieved very good results. However, in these text summarization models, semantic deviations are easy to occur between the original input representation and the representation that passed multi-layer encoder, which may result in inconsistencies between the generated summary and the source text content. The Bidirectional Encoder Representations from Transformers (BERT) improves the performance of many tasks in Natural Language Processing (NLP). Although BERT has a strong capability to encode context, it lacks the fine-grained semantic representation. To solve these two problems, we proposed a semantic supervision method based on Capsule Network. Firstly, we extracted the fine-grained semantic representation of the input and encoded result in BERT by Capsule Network. Secondly, we used the fine-grained semantic representation of the input to supervise the fine-grained semantic representation of the encoded result. Then we evaluated our model on a popular Chinese social media dataset (LCSTS), and the result showed that our model achieved higher ROUGE scores (including R-1, R-2), and our model outperformed baseline systems. Finally, we conducted a comparative study on the stability of the model, and the experimental results showed that our model was more stable.
Keywords
Text summarization; semantic supervision; capsule network
Cite This Article
S. Hu, X. Li, Y. Deng, Y. Peng, B. Lin et al., "A semantic supervision method for abstractive summarization," Computers, Materials & Continua, vol. 69, no.1, pp. 145–158, 2021.
This work is licensed under a Creative Commons Attribution 4.0 International License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.