Cover Image

Beyond Hard and Soft: Hybrid Context Compression for Balancing Local and Global Information Retention

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Liao, Huanxuan, Hu, Wen, Xu, Yao, He, Shizhu, Zhao, Jun, Liu, Kang
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

DATA: Decomposed Attention-based Task Adaptation for Rehearsal-Free Continual Learning
by: Liao, Huanxuan, et al.
Published: (2025)

Neural-Symbolic Collaborative Distillation: Advancing Small Language Models for Complex Reasoning Tasks
by: Liao, Huanxuan, et al.
Published: (2024)

Shuttle Between the Instructions and the Parameters of Large Language Models
by: Sun, Wangtao, et al.
Published: (2025)

Dynamic Parametric Retrieval Augmented Generation for Test-time Knowledge Enhancement
by: Tan, Yuqiao, et al.
Published: (2025)

Awakening Augmented Generation: Learning to Awaken Internal Knowledge of Large Language Models for Question Answering
by: Liao, Huanxuan, et al.
Published: (2024)

SparK: Query-Aware Unstructured Sparsity with Recoverable KV Cache Channel Pruning
by: Liao, Huanxuan, et al.
Published: (2025)

From Instance Training to Instruction Learning: Task Adapters Generation from Instructions
by: Liao, Huanxuan, et al.
Published: (2024)

$\textit{SKIntern}$: Internalizing Symbolic Knowledge for Distilling Better CoT Capabilities into Small Language Models
by: Liao, Huanxuan, et al.
Published: (2024)

ResAdapt: Adaptive Resolution for Efficient Multimodal Reasoning
by: Liao, Huanxuan, et al.
Published: (2026)

LLaSA: Large Language and Structured Data Assistant
by: Xu, Yao, et al.
Published: (2024)

From $P(y|x)$ to $P(y)$: Investigating Reinforcement Learning in Pre-train Space
by: Tan, Yuqiao, et al.
Published: (2026)

The Zero-Step Thinking: An Empirical Study of Mode Selection as Harder Early Exit in Reasoning Models
by: Tan, Yuqiao, et al.
Published: (2025)

Neural Incompatibility: The Unbridgeable Gap of Cross-Scale Parametric Knowledge Transfer in Large Language Models
by: Tan, Yuqiao, et al.
Published: (2025)

ControlLM: Crafting Diverse Personalities for Language Models
by: Weng, Yixuan, et al.
Published: (2024)

Active Continual Learning: On Balancing Knowledge Retention and Learnability
by: Vu, Thuy-Trang, et al.
Published: (2023)

S$^3$HQA: A Three-Stage Approach for Multi-hop Text-Table Hybrid Question Answering
by: Lei, Fangyu, et al.
Published: (2023)

SC-Taxo: Hierarchical Taxonomy Generation under Semantic Consistency Constraints using Large Language Models
by: Cai, Shiqiang, et al.
Published: (2026)

CITI: Enhancing Tool Utilizing Ability in Large Language Models without Sacrificing General Performance
by: Hao, Yupu, et al.
Published: (2024)

Evaluating Personalized Tool-Augmented LLMs from the Perspectives of Personalization and Proactivity
by: Hao, Yupu, et al.
Published: (2025)

Soft Tokens, Hard Truths
by: Butt, Natasha, et al.
Published: (2025)

Adapting LLMs for Efficient Context Processing through Soft Prompt Compression
by: Wang, Cangqing, et al.
Published: (2024)

Simulating Hard Attention Using Soft Attention
by: Yang, Andy, et al.
Published: (2024)

SoftLMs: Efficient Adaptive Low-Rank Approximation of Language Models using Soft-Thresholding Mechanism
by: Bhatnagar, Priyansh, et al.
Published: (2024)

DynSplit-KV: Dynamic Semantic Splitting for KVCache Compression in Efficient Long-Context LLM Inference
by: Ye, Jiancai, et al.
Published: (2026)

ExpNote: Black-box Large Language Models are Better Task Solvers with Experience Notebook
by: Sun, Wangtao, et al.
Published: (2023)

Find Parent then Label Children: A Two-stage Taxonomy Completion Method with Pre-trained Language Model
by: Xia, Fei, et al.
Published: (2024)

A Comprehensive Survey on Long Context Language Modeling
by: Liu, Jiaheng, et al.
Published: (2025)

Learning to Translate from Soft to Hard LLM Prompts
by: Kongsomjit, Pitipat, et al.
Published: (2026)

APB: Accelerating Distributed Long-Context Inference by Passing Compressed Context Blocks across GPUs
by: Huang, Yuxiang, et al.
Published: (2025)

S3Eval: A Synthetic, Scalable, Systematic Evaluation Suite for Large Language Models
by: Lei, Fangyu, et al.
Published: (2023)

Efficient Data Learning for Open Information Extraction with Pre-trained Language Models
by: Fan, Zhiyuan, et al.
Published: (2023)

ILRe: Intermediate Layer Retrieval for Context Compression in Causal Language Models
by: Liang, Manlai, et al.
Published: (2025)

Accurate LoRA-Finetuning Quantization of LLMs via Information Retention
by: Qin, Haotong, et al.
Published: (2024)

Compress the Context, Keep the Commitments: A Formal Framework for Verifiable LLM Context Compression
by: Trukhina, Natalia, et al.
Published: (2026)

ItD: Large Language Models Can Teach Themselves Induction through Deduction
by: Sun, Wangtao, et al.
Published: (2024)

Mitigating Degree Bias Adaptively with Hard-to-Learn Nodes in Graph Contrastive Learning
by: Hu, Jingyu, et al.
Published: (2025)

Beyond Instruction Following: Evaluating Inferential Rule Following of Large Language Models
by: Sun, Wangtao, et al.
Published: (2024)

MoELoRA: Contrastive Learning Guided Mixture of Experts on Parameter-Efficient Fine-Tuning for Large Language Models
by: Luo, Tongxu, et al.
Published: (2024)

Generate-on-Graph: Treat LLM as both Agent and KG in Incomplete Knowledge Graph Question Answering
by: Xu, Yao, et al.
Published: (2024)

Easy-to-Hard Generalization: Scalable Alignment Beyond Human Supervision
by: Sun, Zhiqing, et al.
Published: (2024)