Cover Image

Improving Long Text Understanding with Knowledge Distilled from Summarization Model

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Liu, Yan, Yang, Yazheng, Chen, Xiaokang
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Towards Understanding and Improving Knowledge Distillation for Neural Machine Translation
by: Zhang, Songming, et al.
Published: (2023)

ConTextual: Improving Clinical Text Summarization in LLMs with Context-preserving Token Filtering and Knowledge Graphs
by: Piya, Fahmida Liza, et al.
Published: (2025)

LiveLongBench: Tackling Long-Context Understanding for Spoken Texts from Live Streams
by: Wu, Yongxuan, et al.
Published: (2025)

UniTabE: A Universal Pretraining Protocol for Tabular Foundation Model in Data Science
by: Yang, Yazheng, et al.
Published: (2023)

Leveraging Long-Context Large Language Models for Multi-Document Understanding and Summarization in Enterprise Applications
by: Godbole, Aditi, et al.
Published: (2024)

Survey on Knowledge Distillation for Large Language Models: Methods, Evaluation, and Application
by: Yang, Chuanpeng, et al.
Published: (2024)

Contextualization Distillation from Large Language Model for Knowledge Graph Completion
by: Li, Dawei, et al.
Published: (2024)

Multi-Stage Balanced Distillation: Addressing Long-Tail Challenges in Sequence-Level Knowledge Distillation
by: Zhou, Yuhang, et al.
Published: (2024)

Information-Theoretic Distillation for Reference-less Summarization
by: Jung, Jaehun, et al.
Published: (2024)

WAND: Windowed Attention and Knowledge Distillation for Efficient Autoregressive Text-to-Speech Models
by: Lee, Hanna, et al.
Published: (2026)

CNNSum: Exploring Long-Context Summarization with Large Language Models in Chinese Novels
by: Wei, Lingxiao, et al.
Published: (2024)

S^2tory: Story Spine Distillation for Movie Script Summarization
by: Lu, Mingzhe, et al.
Published: (2026)

Automatic Summarization of Long Documents
by: Chhibbar, Naman, et al.
Published: (2024)

EasyDistill: A Comprehensive Toolkit for Effective Knowledge Distillation of Large Language Models
by: Wang, Chengyu, et al.
Published: (2025)

Pedagogically-Inspired Data Synthesis for Language Model Knowledge Distillation
by: He, Bowei, et al.
Published: (2026)

Improving Factual Consistency of News Summarization by Contrastive Preference Optimization
by: Feng, Huawen, et al.
Published: (2023)

MemLong: Memory-Augmented Retrieval for Long Text Modeling
by: Liu, Weijie, et al.
Published: (2024)

Prompt Chaining or Stepwise Prompt? Refinement in Text Summarization
by: Sun, Shichao, et al.
Published: (2024)

Language-Independent Representations Improve Zero-Shot Summarization
by: Solovyev, Vladimir, et al.
Published: (2024)

A Comparative Study of Quality Evaluation Methods for Text Summarization
by: Nguyen, Huyen, et al.
Published: (2024)

Knowledge Distillation with Structured Chain-of-Thought for Text-to-SQL
by: Thaker, Khushboo, et al.
Published: (2025)

Teaching-Assistant-in-the-Loop: Improving Knowledge Distillation from Imperfect Teacher Models in Low-Budget Scenarios
by: Zhou, Yuhang, et al.
Published: (2024)

Discourse-Driven Evaluation: Unveiling Factual Inconsistency in Long Document Summarization
by: Zhong, Yang, et al.
Published: (2025)

ChallengeMe: An Adversarial Learning-enabled Text Summarization Framework
by: Deng, Xiaoyu, et al.
Published: (2025)

Dual-Space Knowledge Distillation for Large Language Models
by: Zhang, Songming, et al.
Published: (2024)

CNsum:Automatic Summarization for Chinese News Text
by: Zhao, Yu, et al.
Published: (2025)

InforME: Improving Informativeness of Abstractive Text Summarization With Informative Attention Guided by Named Entity Salience
by: Shen, Jianbin, et al.
Published: (2025)

DistillSpec: Improving Speculative Decoding via Knowledge Distillation
by: Zhou, Yongchao, et al.
Published: (2023)

Understanding LLM Behavior in Multi-Target Cross-Lingual Summarization
by: Ryu, Sangwon, et al.
Published: (2026)

Improving Automatic Summarization of Radiology Reports through Mid-Training of Large Language Models
by: Lyu, Mengxian, et al.
Published: (2026)

Extracting and Understanding the Superficial Knowledge in Alignment
by: Chen, Runjin, et al.
Published: (2025)

Reliability Gated Multi-Teacher Distillation for Low Resource Abstractive Summarization
by: Sumit, Dipto, et al.
Published: (2026)

OPSDL: On-Policy Self-Distillation for Long-Context Language Models
by: Zhang, Xinsen, et al.
Published: (2026)

Recursively Summarizing Enables Long-Term Dialogue Memory in Large Language Models
by: Wang, Qingyue, et al.
Published: (2023)

Gecko: Versatile Text Embeddings Distilled from Large Language Models
by: Lee, Jinhyuk, et al.
Published: (2024)

LIFT: Improving Long Context Understanding Through Long Input Fine-Tuning
by: Mao, Yansheng, et al.
Published: (2024)

Balancing Rewards in Text Summarization: Multi-Objective Reinforcement Learning via HyperVolume Optimization
by: Song, Junjie, et al.
Published: (2025)

Segmenting Text and Learning Their Rewards for Improved RLHF in Language Model
by: Yin, Yueqin, et al.
Published: (2025)

sui-1: Grounded and Verifiable Long-Form Summarization
by: Droste, Benedikt, et al.
Published: (2026)

Confidence-aware Self-Semantic Distillation on Knowledge Graph Embedding
by: Liu, Yichen, et al.
Published: (2022)