Open Access iconOpen Access

ARTICLE

crossmark

A Method of Integrating Length Constraints into Encoder-Decoder Transformer for Abstractive Text Summarization

Ngoc-Khuong Nguyen1,2, Dac-Nhuong Le1, Viet-Ha Nguyen2, Anh-Cuong Le3,*

1 Faculty of Information Technology, Haiphong University, Haiphong, 180000, Vietnam
2 Faculty of Information Technology, VNU University of Engineering and Technology, Hanoi, 10000, Vietnam
3 Natural Language Processing and Knowledge Discovery Laboratory, Faculty of Information Technology, Ton Duc Thang University, Ho Chi Minh City, Vietnam

* Corresponding Author: Anh-Cuong Le. Email: email

Intelligent Automation & Soft Computing 2023, 38(1), 1-18. https://doi.org/10.32604/iasc.2023.037083

Abstract

Text summarization aims to generate a concise version of the original text. The longer the summary text is, the more detailed it will be from the original text, and this depends on the intended use. Therefore, the problem of generating summary texts with desired lengths is a vital task to put the research into practice. To solve this problem, in this paper, we propose a new method to integrate the desired length of the summarized text into the encoder-decoder model for the abstractive text summarization problem. This length parameter is integrated into the encoding phase at each self-attention step and the decoding process by preserving the remaining length for calculating head-attention in the generation process and using it as length embeddings added to the word embeddings. We conducted experiments for the proposed model on the two data sets, Cable News Network (CNN) Daily and NEWSROOM, with different desired output lengths. The obtained results show the proposed model’s effectiveness compared with related studies.

Keywords


Cite This Article

N. Nguyen, D. Le, V. Nguyen and A. Le, "A method of integrating length constraints into encoder-decoder transformer for abstractive text summarization," Intelligent Automation & Soft Computing, vol. 38, no.1, pp. 1–18, 2023.



cc This work is licensed under a Creative Commons Attribution 4.0 International License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
  • 547

    View

  • 121

    Download

  • 0

    Like

Share Link