Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2511.06183 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915606918332416 |
|---|---|
| author | Miyazato, Ryuhei Wei, Ting-Ruen Wu, Xuyang Wu, Hsin-Tai Harada, Kei |
| author_facet | Miyazato, Ryuhei Wei, Ting-Ruen Wu, Xuyang Wu, Hsin-Tai Harada, Kei |
| contents | Aspect-based summarization aims to generate summaries that highlight specific aspects of a text, enabling more personalized and targeted summaries. However, its application to books remains unexplored due to the difficulty of constructing reference summaries for long text. To address this challenge, we propose BookAsSumQA, a QA-based evaluation framework for aspect-based book summarization. BookAsSumQA automatically generates aspect-specific QA pairs from a narrative knowledge graph to evaluate summary quality based on its question-answering performance. Our experiments using BookAsSumQA revealed that while LLM-based approaches showed higher accuracy on shorter texts, RAG-based methods become more effective as document length increases, making them more efficient and practical for aspect-based book summarization. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_06183 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | BookAsSumQA: An Evaluation Framework for Aspect-Based Book Summarization via Question Answering Miyazato, Ryuhei Wei, Ting-Ruen Wu, Xuyang Wu, Hsin-Tai Harada, Kei Computation and Language Aspect-based summarization aims to generate summaries that highlight specific aspects of a text, enabling more personalized and targeted summaries. However, its application to books remains unexplored due to the difficulty of constructing reference summaries for long text. To address this challenge, we propose BookAsSumQA, a QA-based evaluation framework for aspect-based book summarization. BookAsSumQA automatically generates aspect-specific QA pairs from a narrative knowledge graph to evaluate summary quality based on its question-answering performance. Our experiments using BookAsSumQA revealed that while LLM-based approaches showed higher accuracy on shorter texts, RAG-based methods become more effective as document length increases, making them more efficient and practical for aspect-based book summarization. |
| title | BookAsSumQA: An Evaluation Framework for Aspect-Based Book Summarization via Question Answering |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2511.06183 |