Imagen de Portada

Compression Represents Intelligence Linearly

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Huang, Yuzhen, Zhang, Jinghan, Shan, Zifei, He, Junxian
Formato:	Preprint
Publicado:	2024
Materias:	Computation and Language Artificial Intelligence Information Theory Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

B-STaR: Monitoring and Balancing Exploration and Exploitation in Self-Taught Reasoners
por: Zeng, Weihao, et al.
Publicado: (2024)

Language Modeling Is Compression
por: Delétang, Grégoire, et al.
Publicado: (2023)

Memorization-Compression Cycles Improve Generalization
por: Yu, Fangyuan
Publicado: (2025)

Learning is Forgetting: LLM Training As Lossy Compression
por: Conklin, Henry C., et al.
Publicado: (2026)

Know Your Limits: Entropy Estimation Modeling for Compression and Generalization
por: Badger, Benjamin L., et al.
Publicado: (2025)

From Accuracy to Robustness: A Study of Rule- and Model-based Verifiers in Mathematical Reasoning
por: Huang, Yuzhen, et al.
Publicado: (2025)

SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild
por: Zeng, Weihao, et al.
Publicado: (2025)

Self-Play Only Evolves When Self-Synthetic Pipeline Ensures Learnable Information Gain
por: Liu, Wei, et al.
Publicado: (2026)

The Information of Large Language Model Geometry
por: Tan, Zhiquan, et al.
Publicado: (2024)

How Many Features Can a Language Model Store Under the Linear Representation Hypothesis?
por: Garg, Nikhil, et al.
Publicado: (2026)

An Information Theoretic Perspective on Agentic System Design
por: He, Shizhe, et al.
Publicado: (2025)

Diff-eRank: A Novel Rank-Based Metric for Evaluating Large Language Models
por: Wei, Lai, et al.
Publicado: (2024)

Learn to Reason Efficiently with Adaptive Length-based Reward Shaping
por: Liu, Wei, et al.
Publicado: (2025)

LASER: Linear Compression in Wireless Distributed Optimization
por: Makkuva, Ashok Vardhan, et al.
Publicado: (2023)

Optimal Quantization for Matrix Multiplication
por: Ordentlich, Or, et al.
Publicado: (2024)

A Survey on Large Language Models from Concept to Implementation
por: Wang, Chen, et al.
Publicado: (2024)

Geometric Signatures of Compositionality Across a Language Model's Lifetime
por: Lee, Jin Hwa, et al.
Publicado: (2024)

A Training-free Method for LLM Text Attribution
por: Radvand, Tara, et al.
Publicado: (2025)

Analyzing and Improving Chain-of-Thought Monitorability Through Information Theory
por: Anwar, Usman, et al.
Publicado: (2026)

Measuring Uncertainty in Transformer Circuits with Effective Information Consistency
por: Krasnovsky, Anatoly A.
Publicado: (2025)

SPEX: Scaling Feature Interaction Explanations for LLMs
por: Kang, Justin Singh, et al.
Publicado: (2025)

A Communication-Theoretic Framework for LLM Agents: Cost-Aware Adaptive Reliability
por: Omidvar, Hamed, et al.
Publicado: (2026)

The Stepwise Informativeness Assumption: Why are Entropy Dynamics and Reasoning Correlated in LLMs?
por: Català, Mar Gonzàlez I, et al.
Publicado: (2026)

Subjective Depth and Timescale Transformers: Learning Where and When to Compute
por: Wieser, Frederico, et al.
Publicado: (2025)

SQuat: Subspace-orthogonal KV Cache Quantization
por: Wang, Hao, et al.
Publicado: (2025)

The Detection-Extraction Gap: Models Know the Answer Before They Can Say It
por: Wang, Hanyang, et al.
Publicado: (2026)

Familiarity-Aware Evidence Compression for Retrieval-Augmented Generation
por: Jung, Dongwon, et al.
Publicado: (2024)

OD-Stega: LLM-Based Relatively Secure Steganography via Optimized Distributions
por: Huang, Yu-Shin, et al.
Publicado: (2024)

SmartChunk Retrieval: Query-Aware Chunk Compression with Planning for Efficient Document RAG
por: Zhang, Xuechen, et al.
Publicado: (2025)

Semantic Faithfulness and Entropy Production Measures to Tame Your LLM Demons and Manage Hallucinations
por: Halperin, Igor
Publicado: (2025)

LightThinker: Thinking Step-by-Step Compression
por: Zhang, Jintian, et al.
Publicado: (2025)

DIVE: Embedding Compression via Self-Limiting Gradient Updates
por: Zhao, Dongfang
Publicado: (2026)

LLM-Enhanced Linear Autoencoders for Recommendation
por: Moon, Jaewan, et al.
Publicado: (2025)

LightThinker++: From Reasoning Compression to Memory Management
por: Zhu, Yuqi, et al.
Publicado: (2026)

ComMer: a Framework for Compressing and Merging User Data for Personalization
por: Zeldes, Yoel, et al.
Publicado: (2025)

Beyond RAG: Task-Aware KV Cache Compression for Comprehensive Knowledge Reasoning
por: Corallo, Giulio, et al.
Publicado: (2025)

Transformers Can Represent $n$-gram Language Models
por: Svete, Anej, et al.
Publicado: (2024)

MacRAG: Compress, Slice, and Scale-up for Multi-Scale Adaptive Context RAG
por: Lim, Woosang, et al.
Publicado: (2025)

SemanticZip: A Pilot Framework for Lossy Text Compression with LLMs as Semantic Decompressors
por: Trukhina, Natalia, et al.
Publicado: (2026)

Agentic Entropy-Balanced Policy Optimization
por: Dong, Guanting, et al.
Publicado: (2025)