Copertina

ILRe: Intermediate Layer Retrieval for Context Compression in Causal Language Models

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Liang, Manlai, Liu, Mandi, Ji, Jiangzhou, Li, Huaijun, Yang, Haobo, He, Yaohan, Li, Jinlong
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Computation and Language Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Lag-Relative Sparse Attention In Long Context Training
di: Liang, Manlai, et al.
Pubblicazione: (2025)

LagKV: Lag-Relative Information of the KV Cache Tells Which Tokens Are Important
di: Liang, Manlai, et al.
Pubblicazione: (2025)

Basis Sharing: Cross-Layer Parameter Sharing for Large Language Model Compression
di: Wang, Jingcun, et al.
Pubblicazione: (2024)

LongEmbed: Extending Embedding Models for Long Context Retrieval
di: Zhu, Dawei, et al.
Pubblicazione: (2024)

Does Representation Matter? Exploring Intermediate Layers in Large Language Models
di: Skean, Oscar, et al.
Pubblicazione: (2024)

Proxy Compression for Language Modeling
di: Zheng, Lin, et al.
Pubblicazione: (2026)

Compressed Context Memory For Online Language Model Interaction
di: Kim, Jang-Hyun, et al.
Pubblicazione: (2023)

MeTHanol: Modularized Thinking Language Models with Intermediate Layer Thinking, Decoding and Bootstrapping Reasoning
di: Xi, Ningyuan, et al.
Pubblicazione: (2024)

Sorted LLaMA: Unlocking the Potential of Intermediate Layers of Large Language Models for Dynamic Inference
di: Kavehzadeh, Parsa, et al.
Pubblicazione: (2023)

GRASP: Replace Redundant Layers with Adaptive Singular Parameters for Efficient Model Compression
di: Liu, Kainan, et al.
Pubblicazione: (2024)

Iterative Layer-wise Distillation for Efficient Compression of Large Language Models
di: Kovalev, Grigory, et al.
Pubblicazione: (2025)

Unveiling Causal Reasoning in Large Language Models: Reality or Mirage?
di: Chi, Haoang, et al.
Pubblicazione: (2025)

Can Language Models Compose Skills In-Context?
di: Liu, Zidong, et al.
Pubblicazione: (2025)

Context-Guided Dynamic Retrieval for Improving Generation Quality in RAG Models
di: He, Jacky, et al.
Pubblicazione: (2025)

Diagnosing Retrieval Bias Under Multiple In-Context Knowledge Updates in Large Language Models
di: Qiao, Boyu, et al.
Pubblicazione: (2026)

LongAlign: A Recipe for Long Context Alignment of Large Language Models
di: Bai, Yushi, et al.
Pubblicazione: (2024)

LoRAP: Transformer Sub-Layers Deserve Differentiated Structured Compression for Large Language Models
di: Li, Guangyan, et al.
Pubblicazione: (2024)

Causal Reflection with Language Models
di: Aryan, Abi, et al.
Pubblicazione: (2025)

Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling
di: Ren, Liliang, et al.
Pubblicazione: (2024)

Core Context Aware Transformers for Long Context Language Modeling
di: Chen, Yaofo, et al.
Pubblicazione: (2024)

A Comprehensive Survey on Long Context Language Modeling
di: Liu, Jiaheng, et al.
Pubblicazione: (2025)

Uncertainty Quantification for In-Context Learning of Large Language Models
di: Ling, Chen, et al.
Pubblicazione: (2024)

Adaptive Feature-based Low-Rank Compression of Large Language Models via Bayesian Optimization
di: Ji, Yixin, et al.
Pubblicazione: (2024)

Prompting Fairness: Integrating Causality to Debias Large Language Models
di: Li, Jingling, et al.
Pubblicazione: (2024)

In-context Autoencoder for Context Compression in a Large Language Model
di: Ge, Tao, et al.
Pubblicazione: (2023)

MiniDisc: Minimal Distillation Schedule for Language Model Compression
di: Zhang, Chen, et al.
Pubblicazione: (2022)

Causal Graph Discovery with Retrieval-Augmented Generation based Large Language Models
di: Zhang, Yuzhe, et al.
Pubblicazione: (2024)

APB: Accelerating Distributed Long-Context Inference by Passing Compressed Context Blocks across GPUs
di: Huang, Yuxiang, et al.
Pubblicazione: (2025)

FoldGPT: Simple and Effective Large Language Model Compression Scheme
di: Liu, Songwei, et al.
Pubblicazione: (2024)

Unlocking the Potentials of Retrieval-Augmented Generation for Diffusion Language Models
di: Yu, Chuanyue, et al.
Pubblicazione: (2026)

RetrievalAttention: Accelerating Long-Context LLM Inference via Vector Retrieval
di: Liu, Di, et al.
Pubblicazione: (2024)

Compress the Context, Keep the Commitments: A Formal Framework for Verifiable LLM Context Compression
di: Trukhina, Natalia, et al.
Pubblicazione: (2026)

Retrieval-Augmented and Knowledge-Grounded Language Models for Faithful Clinical Medicine
di: Liu, Fenglin, et al.
Pubblicazione: (2022)

Beyond Hard and Soft: Hybrid Context Compression for Balancing Local and Global Information Retention
di: Liao, Huanxuan, et al.
Pubblicazione: (2025)

Retrieval meets Long Context Large Language Models
di: Xu, Peng, et al.
Pubblicazione: (2023)

Exploring Context Window of Large Language Models via Decomposed Positional Vectors
di: Dong, Zican, et al.
Pubblicazione: (2024)

Spectral Insights into Data-Oblivious Critical Layers in Large Language Models
di: Liu, Xuyuan, et al.
Pubblicazione: (2025)

Investigating Layer Importance in Large Language Models
di: Zhang, Yang, et al.
Pubblicazione: (2024)

LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compression
di: Jiang, Huiqiang, et al.
Pubblicazione: (2023)

RAVEN: In-Context Learning with Retrieval-Augmented Encoder-Decoder Language Models
di: Huang, Jie, et al.
Pubblicazione: (2023)