Imagen de Portada

LZ Penalty: An information-theoretic repetition penalty for autoregressive language models

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Ginart, Antonio A., Kodali, Naveen, Lee, Jason, Xiong, Caiming, Savarese, Silvio, Emmons, John R.
Formato:	Preprint
Publicado:	2025
Materias:	Machine Learning Artificial Intelligence Information Theory
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Asynchronous Tool Usage for Real-Time Agents
por: Ginart, Antonio A., et al.
Publicado: (2024)

An information-theoretic model of shallow and deep language comprehension
por: Li, Jiaxuan, et al.
Publicado: (2024)

The LZ78 Source
por: Sagan, Naomi, et al.
Publicado: (2025)

An information-theoretic lower bound in time-uniform estimation
por: Duchi, John C., et al.
Publicado: (2024)

Enabling High Data Throughput Reinforcement Learning on GPUs: A Domain Agnostic Framework for Data-Driven Scientific Research
por: Lan, Tian, et al.
Publicado: (2024)

Order-theoretic models for decision-making: Learning, optimization, complexity and computation
por: Hack, Pedro
Publicado: (2024)

DNA storage approaching the information-theoretic ceiling
por: Banal, James L.
Publicado: (2026)

Information-theoretic Bayesian Optimization: Survey and Tutorial
por: Garrido-Merchán, Eduardo C.
Publicado: (2025)

Unifying concepts in information-theoretic time-series analysis
por: Bryant, Annie G., et al.
Publicado: (2025)

Reasoning Curriculum: Bootstrapping Broad LLM Reasoning from Math
por: Pang, Bo, et al.
Publicado: (2025)

Assembly Theory is an approximation to algorithmic complexity based on LZ compression that does not explain selection or evolution
por: Abrahão, Felipe S., et al.
Publicado: (2024)

A quantum semantic framework for natural language processing
por: Agostino, Christopher J., et al.
Publicado: (2025)

Common information in well-mixing graphs and applications to information-theoretic cryptography
por: Caillat-Grenier, Geoffroy, et al.
Publicado: (2024)

Neural Networks Learn Generic Multi-Index Models Near Information-Theoretic Limit
por: Zhang, Bohan, et al.
Publicado: (2025)

Geometric constellation shaping for wireless optical intensity channels: An information-theoretic approach
por: Zhou, Suhua, et al.
Publicado: (2024)

An information-theoretic proof of the Erdős-Kac theorem
por: Rocke, Aidan
Publicado: (2022)

INDICT: Code Generation with Internal Dialogues of Critiques for Both Security and Helpfulness
por: Le, Hung, et al.
Publicado: (2024)

Informational Embodiment: Computational role of information structure in codes and robots
por: Pitti, Alexandre, et al.
Publicado: (2024)

WaterSIC: information-theoretically (near) optimal linear layer quantization
por: Lifar, Egor, et al.
Publicado: (2026)

Unified Training of Universal Time Series Forecasting Transformers
por: Woo, Gerald, et al.
Publicado: (2024)

Non-autoregressive Generative Models for Reranking Recommendation
por: Ren, Yuxin, et al.
Publicado: (2024)

Lossless data compression by large models
por: Li, Ziguang, et al.
Publicado: (2024)

The information-theoretic complexity of differentiable functions
por: Ruijgrok, Matthijs
Publicado: (2026)

Optimal Configuration of Reconfigurable Intelligent Surfaces With Non-uniform Phase Quantization
por: Lu, Jialong, et al.
Publicado: (2024)

Entropy-informed Decoding: Adaptive Information-Driven Branching
por: Evans, Benjamin Patrick, et al.
Publicado: (2026)

Clique-Based Deletion-Correcting Codes via Penalty-Guided Clique Search
por: Pandav, Aniruddh, et al.
Publicado: (2026)

The Derivation Penalty in Premise-Erasure Caching: Capacity, Strong Converse, and Dispersion Dichotomy
por: Xu, Jianfeng
Publicado: (2026)

DyMU: Dynamic Merging and Virtual Unmerging for Efficient VLMs
por: Wang, Zhenhailong, et al.
Publicado: (2025)

Transport information Bregman divergences
por: Li, Wuchen
Publicado: (2021)

Overlapped-repetition Shor codes achieving fourfold asymptotic rate
por: Chang, En-Jui
Publicado: (2025)

FAdam: Adam is a natural gradient optimizer using diagonal empirical Fisher information
por: Hwang, Dongseong
Publicado: (2024)

The topology of synergy: linking topological and information-theoretic approaches to higher-order interactions in complex systems
por: Varley, Thomas F., et al.
Publicado: (2025)

On the Optimal Source Key Size of Secure Gradient Coding
por: Zhou, Yang, et al.
Publicado: (2025)

R-NeRF: Neural Radiance Fields for Modeling RIS-enabled Wireless Environments
por: Yang, Huiying, et al.
Publicado: (2024)

Flexible Multi-Beam Synthesis and Directional Suppression Through Transmissive RIS
por: Xiong, Rujing, et al.
Publicado: (2024)

Rateless Joint Source-Channel Coding, and a Blueprint for 6G Semantic Communications System Design
por: Khosravirad, Saeed R.
Publicado: (2025)

Information-theoretic language of proteinoid gels: Boolean gates and QR codes
por: Sharma, Saksham, et al.
Publicado: (2024)

Distributed satellite information networks: Architecture, enabling technologies, and trends
por: Zhang, Qinyu, et al.
Publicado: (2024)

Metasurface-Enabled Superheterodyne Transmitter With Decoupled Harmonic-Free Signal Generation and Precoding
por: Dong, Xuehui, et al.
Publicado: (2025)

CodeXEmbed: A Generalist Embedding Model Family for Multiligual and Multi-task Code Retrieval
por: Liu, Ye, et al.
Publicado: (2024)