Cover Image

Chunk-Distilled Language Modeling

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Li, Yanhong, Livescu, Karen, Zhou, Jiawei
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

On the Predictive Power of Representation Dispersion in Language Models
by: Li, Yanhong, et al.
Published: (2025)

OKBench: Democratizing LLM Evaluation with Fully Automated, On-Demand, Open Knowledge Benchmarking
by: Li, Yanhong, et al.
Published: (2025)

Distilling to Hybrid Attention Models via KL-Guided Layer Selection
by: Li, Yanhong, et al.
Published: (2025)

Cross-Modal Taxonomic Generalization in (Vision-) Language Models
by: Xu, Tianyang, et al.
Published: (2026)

Text or Pixels? It Takes Half: On the Token Efficiency of Visual Text Inputs in Multimodal LLMs
by: Li, Yanhong, et al.
Published: (2025)

Grounding Language Model with Chunking-Free In-Context Retrieval
by: Qian, Hongjin, et al.
Published: (2024)

SignMusketeers: An Efficient Multi-Stream Approach for Sign Language Translation at Scale
by: Gueuwou, Shester, et al.
Published: (2024)

Flow-SLM: Joint Learning of Linguistic and Acoustic Information for Spoken Language Modeling
by: Chou, Ju-Chieh, et al.
Published: (2025)

H-Net++: Hierarchical Dynamic Chunking for Tokenizer-Free Language Modelling in Morphologically-Rich Languages
by: Zakershahrak, Mehrdad, et al.
Published: (2025)

HiChunk: Evaluating and Enhancing Retrieval-Augmented Generation with Hierarchical Chunking
by: Lu, Wensheng, et al.
Published: (2025)

Chunking German Legal Code
by: Prior, Max, et al.
Published: (2026)

BiLD: Bi-directional Logits Difference Loss for Large Language Model Distillation
by: Li, Minchong, et al.
Published: (2024)

Adaptive Chunking: Optimizing Chunking-Method Selection for RAG
by: Júnior, Paulo Roberto de Moura, et al.
Published: (2026)

QCRD: Quality-guided Contrastive Rationale Distillation for Large Language Models
by: Wang, Wei, et al.
Published: (2024)

ReAD: Reinforcement-Guided Capability Distillation for Large Language Models
by: Cheng, Xueqi, et al.
Published: (2026)

Mixed Distillation Helps Smaller Language Model Better Reasoning
by: Li, Chenglin, et al.
Published: (2023)

MiniLLM: On-Policy Distillation of Large Language Models
by: Gu, Yuxian, et al.
Published: (2023)

Distilling Mathematical Reasoning Capabilities into Small Language Models
by: Zhu, Xunyu, et al.
Published: (2024)

Black-Box On-Policy Distillation of Large Language Models
by: Ye, Tianzhu, et al.
Published: (2025)

HintMR: Eliciting Stronger Mathematical Reasoning in Small Language Models
by: Hossain, Jawad, et al.
Published: (2026)

Cross-Modal Knowledge Distillation for Speech Large Language Models
by: Wang, Enzhi, et al.
Published: (2025)

Capturing Nuanced Preferences: Preference-Aligned Distillation for Small Language Models
by: Gu, Yanggan, et al.
Published: (2025)

LLMR: Knowledge Distillation with a Large Language Model-Induced Reward
by: Li, Dongheng, et al.
Published: (2024)

Language Model Distillation: A Temporal Difference Imitation Learning Perspective
by: Yu, Zishun, et al.
Published: (2025)

Key-Point-Driven Mathematical Reasoning Distillation of Large Language Model
by: Zhu, Xunyu, et al.
Published: (2024)

Contextualization Distillation from Large Language Model for Knowledge Graph Completion
by: Li, Dawei, et al.
Published: (2024)

ChunkNorris: A High-Performance and Low-Energy Approach to PDF Parsing and Chunking
by: Ciancone, Mathieu, et al.
Published: (2025)

EasyDistill: A Comprehensive Toolkit for Effective Knowledge Distillation of Large Language Models
by: Wang, Chengyu, et al.
Published: (2025)

EmoDistill: Offline Emotion Skill Distillation for Language Model Agents in Adversarial Negotiation
by: Long, Yunbo, et al.
Published: (2026)

DynamicRAG: Leveraging Outputs of Large Language Model as Feedback for Dynamic Reranking in Retrieval-Augmented Generation
by: Sun, Jiashuo, et al.
Published: (2025)

Enhancing Retrieval Augmented Generation with Hierarchical Text Segmentation Chunking
by: Nguyen, Hai Toan, et al.
Published: (2025)

A Systematic Investigation of Document Chunking Strategies and Embedding Sensitivity
by: Shaukat, Muhammad Arslan, et al.
Published: (2026)

What Happened in LLMs Layers when Trained for Fast vs. Slow Thinking: A Gradient Perspective
by: Li, Ming, et al.
Published: (2024)

Dual-Space Knowledge Distillation for Large Language Models
by: Zhang, Songming, et al.
Published: (2024)

Debate, Reflect, and Distill: Multi-Agent Feedback with Tree-Structured Preference Optimization for Efficient Language Model Enhancement
by: Zhou, Xiaofeng, et al.
Published: (2025)

Large Language Models Explore by Latent Distilling
by: Zeng, Yuanhao, et al.
Published: (2026)

A New HOPE: Domain-agnostic Automatic Evaluation of Text Chunking
by: Brådland, Henrik, et al.
Published: (2025)

ChunkLLM: A Lightweight Pluggable Framework for Accelerating LLMs Inference
by: Ouyang, Haojie, et al.
Published: (2025)

Adaptive Token Boundaries: Integrating Human Chunking Mechanisms into Multimodal LLMs
by: Yu, Dongxing
Published: (2025)

Pre-training Distillation for Large Language Models: A Design Space Exploration
by: Peng, Hao, et al.
Published: (2024)