About the journal

• Home page
• Contact information
• Aims and scope
• Indexing information
• Editorial policies
• ComSIS consortium
• Journal boards
• Managing board

For authors

• Information for contributors
• Paper submission
• Article submission through OJS
• Copyright transfer form
• Download section

For readers

• Forthcoming articles
• Current issue
• Archive

For reviewers

• View and review submissions

News

• Journal's Facebook page
• Calls for special issues
• New issue notification

Variational Neural Decoder for Abstractive Text Summarization

Huan Zhao¹, Jie Cao¹, Mingquan Xu¹ and Jian Lu¹

1. College of Computer Science and Electronic Engineering, Hunan University
   Changsha, Hunan, P.R. China, 410000
   {hzhao,jiecao,hb xmq,jianlu}@hnu.edu.cn

Abstract

In the conventional sequence-to-sequence (seq2seq) model for abstractive summarization, the internal transformation structure of recurrent neural networks (RNNs) is completely determined. Therefore, the learned semantic information is far from enough to represent all semantic details and context dependencies, resulting in a redundant summary and poor consistency. In this paper, we propose a variational neural decoder text summarization model (VND). The model introduces a series of implicit variables by combining variational RNN and variational autoencoder, which is used to capture complex semantic representation at each step of decoding. It includes a standard RNN layer and a variational RNN layer [5]. These two network layers respectively generate a deterministic hidden state and a random hidden state. We use these two RNN layers to establish the dependence between implicit variables between adjacent time steps. In this way, the model structure can better capture the complex semantics and the strong dependence between the adjacent time steps when outputting the summary, thereby improving the performance of generating the summary. The experimental results show that, on the text summary LCSTS and English Gigaword dataset, our model has a significant improvement over the baseline model.

Key words

abstractive summarization, sequence-to-sequence, variational auto-encoder, variation neural inferer

Digital Object Identifier (DOI)

https://doi.org/10.2298/CSIS200131012Z

Publication information

Volume 17, Issue 2 (June 2020)
Year of Publication: 2020
ISSN: 2406-1018 (Online)
Publisher: ComSIS Consortium

Full text

Available in PDF
Portable Document Format

How to cite

Zhao, H., Cao, J., Xu, M., Lu, J.: Variational Neural Decoder for Abstractive Text Summarization. Computer Science and Information Systems, Vol. 17, No. 2, 537–552. (2020), https://doi.org/10.2298/CSIS200131012Z