Open Access iconOpen Access



A Semantic Supervision Method for Abstractive Summarization

Sunqiang Hu1, Xiaoyu Li1, Yu Deng1,*, Yu Peng1, Bin Lin2, Shan Yang3

1 School of Information and Software Engineering, University of Electronic Science and Technology of China, Chengdu, 610054, China
2 School of Engineering, Sichuan Normal University, Chengdu, 610066, China
3 Department of Chemistry, Physics, and Atmospheric Sciences, Jackson State University, Jackson, MS, 39217, USA

* Corresponding Author: Yu Deng. Email: email

Computers, Materials & Continua 2021, 69(1), 145-158.


In recent years, many text summarization models based on pre-training methods have achieved very good results. However, in these text summarization models, semantic deviations are easy to occur between the original input representation and the representation that passed multi-layer encoder, which may result in inconsistencies between the generated summary and the source text content. The Bidirectional Encoder Representations from Transformers (BERT) improves the performance of many tasks in Natural Language Processing (NLP). Although BERT has a strong capability to encode context, it lacks the fine-grained semantic representation. To solve these two problems, we proposed a semantic supervision method based on Capsule Network. Firstly, we extracted the fine-grained semantic representation of the input and encoded result in BERT by Capsule Network. Secondly, we used the fine-grained semantic representation of the input to supervise the fine-grained semantic representation of the encoded result. Then we evaluated our model on a popular Chinese social media dataset (LCSTS), and the result showed that our model achieved higher ROUGE scores (including R-1, R-2), and our model outperformed baseline systems. Finally, we conducted a comparative study on the stability of the model, and the experimental results showed that our model was more stable.


Cite This Article

APA Style
Hu, S., Li, X., Deng, Y., Peng, Y., Lin, B. et al. (2021). A semantic supervision method for abstractive summarization. Computers, Materials & Continua, 69(1), 145-158.
Vancouver Style
Hu S, Li X, Deng Y, Peng Y, Lin B, Yang S. A semantic supervision method for abstractive summarization. Comput Mater Contin. 2021;69(1):145-158
IEEE Style
S. Hu, X. Li, Y. Deng, Y. Peng, B. Lin, and S. Yang "A Semantic Supervision Method for Abstractive Summarization," Comput. Mater. Contin., vol. 69, no. 1, pp. 145-158. 2021.

cc This work is licensed under a Creative Commons Attribution 4.0 International License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
  • 1878


  • 1518


  • 0


Share Link