Chain-of-event prompting for multi-document summarization by large language models
International Journal of Web Information Systems
ISSN: 1744-0084
Article publication date: 15 February 2024
Issue publication date: 30 April 2024
Abstract
Purpose
In the era of big data, various industries are generating large amounts of text data every day. Simplifying and summarizing these data can effectively serve users and improve efficiency. Recently, zero-shot prompting in large language models (LLMs) has demonstrated remarkable performance on various language tasks. However, generating a very “concise” multi-document summary is a difficult task for it. When conciseness is specified in the zero-shot prompting, the generated multi-document summary still contains some unimportant information, even with the few-shot prompting. This paper aims to propose a LLMs prompting for multi-document summarization task.
Design/methodology/approach
To overcome this challenge, the authors propose chain-of-event (CoE) prompting for multi-document summarization (MDS) task. In this prompting, the authors take events as the center and propose a four-step summary reasoning process: specific event extraction; event abstraction and generalization; common event statistics; and summary generation. To further improve the performance of LLMs, the authors extend CoE prompting with the example of summary reasoning.
Findings
Summaries generated by CoE prompting are more abstractive, concise and accurate. The authors evaluate the authors’ proposed prompting on two data sets. The experimental results over ChatGLM2-6b show that the authors’ proposed CoE prompting consistently outperforms other typical promptings across all data sets.
Originality/value
This paper proposes CoE prompting to solve MDS tasks by the LLMs. CoE prompting can not only identify the key events but also ensure the conciseness of the summary. By this method, users can access the most relevant and important information quickly, improving their decision-making processes.
Keywords
Acknowledgements
The work described in this paper was partially supported by Zhejiang Provincial Natural Science Foundation of China (No. LQ21F020019), National Natural Science Foundation of China (No. 62276233) and Key Research Project of Zhejiang Province (2023C01048).
Citation
Bao, S., Li, T. and Cao, B. (2024), "Chain-of-event prompting for multi-document summarization by large language models", International Journal of Web Information Systems, Vol. 20 No. 3, pp. 229-247. https://doi.org/10.1108/IJWIS-12-2023-0249
Publisher
:Emerald Publishing Limited
Copyright © 2024, Emerald Publishing Limited