Copertina

Bridging Kolmogorov Complexity and Deep Learning: Asymptotically Optimal Description Length Objectives for Transformers

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Shaw, Peter, Cohan, James, Eisenstein, Jacob, Toutanova, Kristina
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Machine Learning Artificial Intelligence Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

ALTA: Compiler-Based Analysis of Transformers
di: Shaw, Peter, et al.
Pubblicazione: (2024)

BgGPT 1.0: Extending English-centric LLMs to other languages
di: Alexandrov, Anton, et al.
Pubblicazione: (2024)

The Role of Sparsity for Length Generalization in Transformers
di: Golowich, Noah, et al.
Pubblicazione: (2025)

On the Optimal Reasoning Length for RL-Trained Language Models
di: Nohara, Daisuke, et al.
Pubblicazione: (2026)

Transformers Can Achieve Length Generalization But Not Robustly
di: Zhou, Yongchao, et al.
Pubblicazione: (2024)

On Evaluating LLM Alignment by Evaluating LLMs as Judges
di: Liu, Yixin, et al.
Pubblicazione: (2025)

Reward-free Alignment for Conflicting Objectives
di: Chen, Peter, et al.
Pubblicazione: (2026)

Algorithmic Capabilities of Random Transformers
di: Zhong, Ziqian, et al.
Pubblicazione: (2024)

Position Coupling: Improving Length Generalization of Arithmetic Transformers Using Task Structure
di: Cho, Hanseul, et al.
Pubblicazione: (2024)

Observable Propagation: Uncovering Feature Vectors in Transformers
di: Dunefsky, Jacob, et al.
Pubblicazione: (2023)

On the Hidden Objective Biases of Group-based Reinforcement Learning
di: Fontana, Aleksandar, et al.
Pubblicazione: (2026)

Calibrating Long-form Generations from Large Language Models
di: Huang, Yukun, et al.
Pubblicazione: (2024)

Language and Experience: A Computational Model of Social Learning in Complex Tasks
di: Colas, Cédric, et al.
Pubblicazione: (2025)

Learn to Reason Efficiently with Adaptive Length-based Reward Shaping
di: Liu, Wei, et al.
Pubblicazione: (2025)

How Does Response Length Affect Long-Form Factuality
di: Zhao, James Xu, et al.
Pubblicazione: (2025)

Optimizing Pre-Training Data Mixtures with Mixtures of Data Expert Models
di: Belenki, Lior, et al.
Pubblicazione: (2025)

Re-evaluating Automatic LLM System Ranking for Alignment with Human Preference
di: Gao, Mingqi, et al.
Pubblicazione: (2024)

BIRCO: A Benchmark of Information Retrieval Tasks with Complex Objectives
di: Wang, Xiaoyue, et al.
Pubblicazione: (2024)

References Improve LLM Alignment in Non-Verifiable Domains
di: Shi, Kejian, et al.
Pubblicazione: (2026)

Length-MAX Tokenizer for Language Models
di: Dong, Dong, et al.
Pubblicazione: (2025)

EMORL: Ensemble Multi-Objective Reinforcement Learning for Efficient and Flexible LLM Fine-Tuning
di: Kong, Lingxiao, et al.
Pubblicazione: (2025)

More is not always better? Enhancing Many-Shot In-Context Learning with Differentiated and Reweighting Objectives
di: Zhang, Xiaoqing, et al.
Pubblicazione: (2025)

Effective Reasoning Chains Reduce Intrinsic Dimensionality
di: Prasad, Archiki, et al.
Pubblicazione: (2026)

Which Attention Heads Matter for In-Context Learning?
di: Yin, Kayo, et al.
Pubblicazione: (2025)

RLP: Reinforcement as a Pretraining Objective
di: Hatamizadeh, Ali, et al.
Pubblicazione: (2025)

SpanNorm: Reconciling Training Stability and Performance in Deep Transformers
di: Wang, Chao, et al.
Pubblicazione: (2026)

Learn Globally, Speak Locally: Bridging the Gaps in Multilingual Reasoning
di: Hwang, Jaedong, et al.
Pubblicazione: (2025)

The Anxiety of Influence: Bloom Filters in Transformer Attention Heads
di: Balogh, Peter
Pubblicazione: (2026)

ComplexityNet: Increasing LLM Inference Efficiency by Learning Task Complexity
di: Bae, Henry, et al.
Pubblicazione: (2023)

Multi-Objective Large Language Model Unlearning
di: Pan, Zibin, et al.
Pubblicazione: (2024)

Pareto Multi-Objective Alignment for Language Models
di: He, Qiang, et al.
Pubblicazione: (2025)

DLER: Doing Length pEnalty Right - Incentivizing More Intelligence per Token via Reinforcement Learning
di: Liu, Shih-Yang, et al.
Pubblicazione: (2025)

Stepwise Penalization for Length-Efficient Chain-of-Thought Reasoning
di: Li, Xintong, et al.
Pubblicazione: (2026)

Position as Probability: Self-Supervised Transformers that Think Past Their Training for Length Extrapolation
di: Lee, Philip Heejun
Pubblicazione: (2025)

Bridging the Gap: Dynamic Learning Strategies for Improving Multilingual Performance in LLMs
di: Kumar, Somnath, et al.
Pubblicazione: (2024)

Transformers Struggle to Learn to Search
di: Saparov, Abulhair, et al.
Pubblicazione: (2024)

When More is Less: Understanding Chain-of-Thought Length in LLMs
di: Wu, Yuyang, et al.
Pubblicazione: (2025)

Understanding and Improving Length Generalization in Hierarchical Sparse Attention Models
di: Leng, Jiaqi, et al.
Pubblicazione: (2025)

An Empirical Study on Context Length for Open-Domain Dialog Generation
di: Shen, Xinyi, et al.
Pubblicazione: (2024)

Provable Length Generalization in Sequence Prediction via Spectral Filtering
di: Marsden, Annie, et al.
Pubblicazione: (2024)