Imagen de Portada

Intrinsic Entropy of Context Length Scaling in LLMs

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Shi, Jingzhe, Ma, Qinwei, Liu, Hongyi, Zhao, Hang, Hwang, Jeng-Neng, Li, Lei
Formato:	Preprint
Publicado:	2025
Materias:	Machine Learning Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Scaling Law for Time Series Forecasting
por: Shi, Jingzhe, et al.
Publicado: (2024)

Gradient Imbalance in Direct Preference Optimization
por: Ma, Qinwei, et al.
Publicado: (2025)

Entropy Centroids as Intrinsic Rewards for Test-Time Scaling
por: Zhao, Wenshuo, et al.
Publicado: (2026)

CHOPS: CHat with custOmer Profile Systems for Customer Service with LLMs
por: Shi, Jingzhe, et al.
Publicado: (2024)

UT-ACA: Uncertainty-Triggered Adaptive Context Allocation for Long-Context Inference
por: Zhou, Lang, et al.
Publicado: (2026)

Megalodon: Efficient LLM Pretraining and Inference with Unlimited Context Length
por: Ma, Xuezhe, et al.
Publicado: (2024)

Position: The Inevitable End of One-Architecture-Fits-All-Domains in Time Series Forecasting
por: Ma, Qinwei, et al.
Publicado: (2026)

Demystifying OPD: Length Inflation and Stabilization Strategies for Large Language Models
por: Luo, Feng, et al.
Publicado: (2026)

LANCET: Neural Intervention via Structural Entropy for Mitigating Faithfulness Hallucinations in LLMs
por: Wang, Chenxu, et al.
Publicado: (2026)

The Role of Deductive and Inductive Reasoning in Large Language Models
por: Cai, Chengkun, et al.
Publicado: (2024)

FinGPT: Enhancing Sentiment-Based Stock Movement Prediction with Dissemination-Aware and Context-Enriched LLMs
por: Liang, Yixuan, et al.
Publicado: (2024)

LongAct: Harnessing Intrinsic Activation Patterns for Long-Context Reinforcement Learning
por: Ping, Bowen, et al.
Publicado: (2026)

SELF: Self-Extend the Context Length With Logistic Growth Function
por: Dang, Phat Thanh, et al.
Publicado: (2025)

Improving Variable-Length Generation in Diffusion Language Models via Length Regularization
por: Cheng, Zicong, et al.
Publicado: (2026)

Does Alignment Tuning Really Break LLMs' Internal Confidence?
por: Oh, Hongseok, et al.
Publicado: (2024)

Rethinking Perplexity: Revealing the Impact of Input Length on Perplexity Evaluation in LLMs
por: Cheng, Letian, et al.
Publicado: (2026)

Diffusion LMs Can Approximate Optimal Infilling Lengths Implicitly
por: Liu, Hengchang, et al.
Publicado: (2026)

What Scales in Cross-Entropy Scaling Law?
por: Yan, Junxi, et al.
Publicado: (2025)

LongWriter: Unleashing 10,000+ Word Generation from Long Context LLMs
por: Bai, Yushi, et al.
Publicado: (2024)

Sculptor: Empowering LLMs with Cognitive Agency via Active Context Management
por: Li, Mo, et al.
Publicado: (2025)

Flux Attention: Context-Aware Hybrid Attention for Efficient LLMs Inference
por: Qiu, Quantong, et al.
Publicado: (2026)

Beyond the Limits: A Survey of Techniques to Extend the Context Length in Large Language Models
por: Wang, Xindi, et al.
Publicado: (2024)

A Controlled Study on Long Context Extension and Generalization in LLMs
por: Lu, Yi, et al.
Publicado: (2024)

Scaling Efficient LLMs
por: Kausik, B. N.
Publicado: (2024)

Training-Inference Consistent Segmented Execution for Long-Context LLMs
por: Shang, Xianpeng, et al.
Publicado: (2026)

ALIEN: Aligned Entropy Head for Improving Uncertainty Estimation of LLMs
por: Zabolotnyi, Artem, et al.
Publicado: (2025)

From Interpolation to Extrapolation: Complete Length Generalization for Arithmetic Transformers
por: Duan, Shaoxiong, et al.
Publicado: (2023)

Reward Is Enough: LLMs Are In-Context Reinforcement Learners
por: Song, Kefan, et al.
Publicado: (2025)

TokenSelect: Efficient Long-Context Inference and Length Extrapolation for LLMs via Dynamic Token-Level KV Cache Selection
por: Wu, Wei, et al.
Publicado: (2024)

Towards Self-Robust LLMs: Intrinsic Prompt Noise Resistance via CoIPO
por: Yang, Xin, et al.
Publicado: (2026)

Explore-Execute Chain: Towards an Efficient Structured Reasoning Paradigm
por: Yang, Kaisen, et al.
Publicado: (2025)

ProxSparse: Regularized Learning of Semi-Structured Sparsity Masks for Pretrained LLMs
por: Liu, Hongyi, et al.
Publicado: (2025)

Is In-Context Learning Sufficient for Instruction Following in LLMs?
por: Zhao, Hao, et al.
Publicado: (2024)

Brewing Knowledge in Context: Distillation Perspectives on In-Context Learning
por: Li, Chengye, et al.
Publicado: (2025)

Towards Robust and Parameter-Efficient Knowledge Unlearning for LLMs
por: Cha, Sungmin, et al.
Publicado: (2024)

Let's (not) just put things in Context: Test-Time Training for Long-Context LLMs
por: Bansal, Rachit, et al.
Publicado: (2025)

Scaling Long-Horizon LLM Agent via Context-Folding
por: Sun, Weiwei, et al.
Publicado: (2025)

Why Semantic Entropy Fails: Geometry-Aware and Calibrated Uncertainty for Policy Optimization
por: Zhang, Zheyuan, et al.
Publicado: (2026)

An Empirical Study on Context Length for Open-Domain Dialog Generation
por: Shen, Xinyi, et al.
Publicado: (2024)

LongSafety: Enhance Safety for Long-Context LLMs
por: Huang, Mianqiu, et al.
Publicado: (2024)