Cover Image

Establishing a Scale for Kullback-Leibler Divergence in Language Models Across Various Settings

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Kishino, Ryo, Takase, Yusuke, Oyama, Momose, Yamagiwa, Hiroaki, Shimodaira, Hidetoshi
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Likelihood Variance as Text Importance for Resampling Texts to Map Language Models
by: Oyama, Momose, et al.
Published: (2025)

Mapping 1,000+ Language Models via the Log-Likelihood Vector
by: Oyama, Momose, et al.
Published: (2025)

Domain Mixture Design via Log-Likelihood Differences for Aligning Language Models with a Target Model
by: Kishino, Ryo, et al.
Published: (2026)

Revisiting Cosine Similarity via Normalized ICA-transformed Embeddings
by: Yamagiwa, Hiroaki, et al.
Published: (2024)

Understanding Higher-Order Correlations Among Semantic Components in Embeddings
by: Oyama, Momose, et al.
Published: (2024)

Language Model Maps for Prompt-Response Distributions via Log-Likelihood Vectors
by: Takase, Yusuke, et al.
Published: (2026)

Measuring Affinity between Attention-Head Weight Subspaces via the Projection Kernel
by: Yamagiwa, Hiroaki, et al.
Published: (2026)

Axis Tour: Word Tour Determines the Order of Axes in ICA-transformed Embeddings
by: Yamagiwa, Hiroaki, et al.
Published: (2024)

Quantifying Lexical Semantic Shift via Unbalanced Optimal Transport
by: Kishino, Ryo, et al.
Published: (2024)

Norm of Mean Contextualized Embeddings Determines their Variance
by: Yamagiwa, Hiroaki, et al.
Published: (2024)

Shimo Lab at "Discharge Me!": Discharge Summarization by Prompt-Driven Concatenation of Electronic Health Record Sections
by: He, Yunzhen, et al.
Published: (2024)

Predicting drug-gene relations via analogy tasks with word embeddings
by: Yamagiwa, Hiroaki, et al.
Published: (2024)

Better Estimation of the Kullback--Leibler Divergence Between Language Models
by: Amini, Afra, et al.
Published: (2025)

Rethinking Kullback-Leibler Divergence in Knowledge Distillation for Large Language Models
by: Wu, Taiqiang, et al.
Published: (2024)

DeLTa: A Decoding Strategy based on Logit Trajectory Prediction Improves Factuality and Reasoning Ability
by: He, Yunzhen, et al.
Published: (2025)

Knowledge Sanitization of Large Language Models
by: Ishibashi, Yoichi, et al.
Published: (2023)

3D Rotation and Translation for Hyperbolic Knowledge Graph Embedding
by: Zhu, Yihua, et al.
Published: (2023)

Decoupled Kullback-Leibler Divergence Loss
by: Cui, Jiequan, et al.
Published: (2023)

Generalized Kullback-Leibler Divergence Loss
by: Cui, Jiequan, et al.
Published: (2025)

Vocabulary Expansion of Large Language Models via Kullback-Leibler-Based Self-Distillation
by: Linder, Max Rehman
Published: (2025)

Block-Diagonal Orthogonal Relation and Matrix Entity for Knowledge Graph Embedding
by: Zhu, Yihua, et al.
Published: (2024)

Beyond Chains: Bridging Large Language Models and Knowledge Bases in Complex Question Answering
by: Zhu, Yihua, et al.
Published: (2025)

Fast Kd-trees for the Kullback--Leibler Divergence and other Decomposable Bregman Divergences
by: Pham, Tuyen, et al.
Published: (2025)

Wasserstein Distance Rivals Kullback-Leibler Divergence for Knowledge Distillation
by: Lv, Jiaming, et al.
Published: (2024)

Zipfian Whitening
by: Yokoi, Sho, et al.
Published: (2024)

Evaluating Earth-Observing Satellite Sampling Effectiveness Using Kullback-Leibler Divergence
by: Esmaeili, Negin, et al.
Published: (2025)

Entropy and the Kullback-Leibler Divergence for Bayesian Networks: Computational Complexity and Efficient Implementation
by: Scutari, Marco
Published: (2023)

Parallelizing MCMC with Machine Learning Classifier and Its Criterion Based on Kullback-Leibler Divergence
by: Matsumoto, Tomoki
Published: (2024)

Diversity-Aware Reverse Kullback-Leibler Divergence for Large Language Model Distillation
by: Luong, Hoang-Chau, et al.
Published: (2026)

The Hellinger Bounds on the Kullback-Leibler Divergence and the Bernstein Norm
by: Kaji, Tetsuya
Published: (2026)

Fractional Programming for Kullback-Leibler Divergence in Hypothesis Testing
by: Park, Jeongwoo, et al.
Published: (2026)

Scaling Laws for Upcycling Mixture-of-Experts Language Models
by: Liew, Seng Pei, et al.
Published: (2025)

A Hierarchical Decomposition of Kullback-Leibler Divergence: Disentangling Marginal Mismatches from Statistical Dependencies
by: Cook, William
Published: (2025)

Robust Phase Retrieval via Reverse Kullback-Leibler Divergence
by: Choudhury, Nazia Afroz, et al.
Published: (2022)

Central Limit Theorem on Symmetric Kullback-Leibler (KL) Divergence
by: Rojas, Helder, et al.
Published: (2024)

Natural Fingerprints of Large Language Models
by: Suzuki, Teppei, et al.
Published: (2025)

Unifying computational entropies via Kullback-Leibler divergence
by: Agrawal, Rohit, et al.
Published: (2019)

Distributionally Robust LQG with Kullback-Leibler Ambiguity Sets
by: Fochesato, Marta, et al.
Published: (2025)

Quantum Causal Discovery via Amplitude Estimation of Kullback-Leibler Divergence
by: Sodagari, Shabnam
Published: (2026)

A New Estimator of Kullback--Leibler Divergence via Shannon Entropy
by: Cadirci, Mehmet Siddik, et al.
Published: (2026)